- Training data: raw and cleaned datasets, often tens or hundreds of terabytes, read in full on every epoch.
- Checkpoints: large files written during long runs and read back when a job restarts or resumes.
- Model weights: the files your inference servers load each time they start or scale up.
- Retrieval data: the documents, chunks and embeddings behind an assistant that answers from your own files.
- Logs, evaluations and outputs: prompts, responses and scores you keep to measure and improve a model.
- Backups of all of it: with object lock on, so a bad script or ransomware can’t delete them.
All of it sits behind the S3 API, so code that reads from S3 today, using boto3, rclone or a data loader built on them, points at our endpoint with new keys.