Source Overview#

This page provides a curated, high-level overview of the most relevant source files in the repository with a short description for each. Use the sidebar to navigate to the corresponding full source views.

Kubernetes (k8s)#

  • k8s/deployment.yaml: Deployment for core services (e.g., Postgres), sets replicas and pod template.

  • k8s/service.yaml: ClusterIP Service exposing in-cluster networking endpoints.

  • k8s/ingress.yaml: Ingress rules for external routing via Traefik.

  • k8s/pvc.yaml: PersistentVolumeClaim for durable storage.

Theme Overrides (overrides)#

  • overrides/main.html: Material template override that injects a right-hand comments panel for annotated reading.

SLURM Configuration#

  • slurm-config/slurm.conf: SLURM scheduler configuration used by master and workers.

  • slurm-image/Dockerfile: Image used to run SLURM components.

  • slurm-image/entrypoint.sh: Entry script configuring and starting SLURM services.

FastAPI Application (src/app)#

  • main.py: FastAPI app entrypoint, routes and integration of services.

  • config.py: Central configuration constants and settings used across the app.

  • database.py: SQLAlchemy session management and database utilities.

  • models.py: SQLAlchemy ORM models (Coordinate, WeatherObservation, TrainingStatus, TrainingLog).

  • schemas.py: Pydantic models for request/response validation.

  • utils.py: Helper utilities (time formatting, startup checks, status formatting).

  • weather_ingest.py: Data ingestion from external weather APIs into the database.

  • ml_train.py: Training orchestration logic triggered by background jobs/Slurm.

  • training_helpers.py: Shared helpers used during training.

  • training_jobs.py: Job dispatch logic coordinating with the scheduler.

  • coordinates_manager.py: Business logic to manage tracked coordinates in DB.

  • coordinates_setup.py: Initialization routines for coordinate datasets.

  • coordinates_utils.py: Coordinate utilities and validation.

  • imputation.py: Missing-data handling for time-series.

  • init_db.py: Database initialization routines and fixtures.

  • templates/index.html: Landing page template.

  • templates/train_status.html: Training status and results view template.

  • requirements.txt: Python dependencies for the application environment.

Root Tooling and Config#

  • Dockerfile: Container image for the FastAPI application.

  • docker-compose.yml: Local stack for FastAPI, Postgres, and SLURM cluster (env-driven).

  • Makefile: Linting and security tasks.

  • menu.sh: Developer convenience script.

  • mkdocs.yml: Documentation site configuration and navigation.

  • mypy.ini: Type checker configuration.

  • pyproject.toml: Tooling configuration (formatters/linters) and metadata.

  • security_check.sh: Security scanning helper.