SDA - Sensitive Data Archive

Repository: neicnordic/sensitive-data-archive

sda repository consists of a suite of services which are part of NeIC Sensitive Data Archive and implements the components required for data submission. It can be used as part of a Federated EGA or as a stand-alone (isolated) Sensitive Data Archive. sda was built with support for both S3 and POSIX storage.

The SDA submission pipeline has four main steps:

1. Ingest splits file headers from files, moving the header to the database and the file content to the archive storage.
2. Verify verifies that the header is encrypted with the correct key, and that the checksums match the user-provided checksums.
3. Finalize associates a stable accessionID with each archive file and backups the file.
4. Mapper maps file accessionIDs to a datasetID.

There are also additional support services:

1. Auth authentication service used in conjunction with the s3inbox.
2. Intercept relays messages from CentralEGA to the system.
3. ReEncrypt reencrypts a given file header with a given public key.
4. s3inbox proxies uploads to the an S3 compatible storage backend.
5. sync mirrors ingested data between sites in the Bigpicture project.
6. syncapi is used in the Bigpicture project for mirroring data between two installations of SDA.