Recs.
Updated
MyTardis is an open-source data management platform aimed at researchers. MyTardis handles the underlying storage to ensure that data is securely archived and provides access to the data through a web portal. Data hosted in MyTardis can also be accessed via SFTP.
The Monash University implementation of MyTardis ingested more than 17 million data files with a total size of 1.3PB arranged in 70,000 datasets, which were acquired during 7,600 experiments and collected from 117 instruments.
Whilst the reporting tool is lighting fast, it does not affect MyTardis performance at all. Initial data population and its daily amendment is organised using a custom script, which connects to MyTardis database directly, fetches minimum required information and passes it in bulk to Elasticsearch for indexing and storage. Kibana service is interacting with Elasticsearch, independently from MyTardis installation.