Machine Learning Engineering Open Book
The AI Compute Platform for frontier teams. SkyPilot turns fragmented AI compute into one AI supercomputer, so frontier AI teams build custom intelligence faster.
Slurm: A Highly Scalable Workload Manager
A DSL for data-driven computational pipelines
Vendor-agnostic orchestration for training, inference and agentic workloads across NVIDIA, AMD, TPU, and Tenstorrent on clouds, Kubernetes, and bare metal.
A WDL, CWL and Python API supporting easy-to-use workflow engine. It is scalable, efficient and cross-platform (Linux/macOS).
Best practices & guides on how to write distributed pytorch training code
A Slurm cluster using docker-compose
Lightweight fast function pipeline (DAG) creation in pure Python for scientific (HPC) workflows 🕸️🧪
Collection of best practices, reference architectures, model training examples and utilities to train large models on AWS.
A scheduler for GPU/CPU tasks
TorchX is a universal job launcher for PyTorch applications. TorchX is designed to have fast iteration time for training/research and support for E2E production ML pipelines when you're ready.
Run Slurm in Kubernetes
Run Slurm on Kubernetes. A Slinky project.
An open-source toolkit for deploying and managing high performance clusters for HPC, AI, and data analytics workloads.
Add a description, image, and links to the slurm topic page so that developers can more easily learn about it.
To associate your repository with the slurm topic, visit your repo's landing page and select "manage topics."