AreteusML

Overview

AreteusML is an ML pipeline for Banking77 intent classification. It covers the whole path: data validation, training, export, serving, monitoring and explanation. I built it to do ML engineering beyond calling a model API, and trained the model on Kaggle's free T4 GPU.

Key Features

ModernBERT fine-tuning: an encoder-only transformer fine-tuned on 13,083 banking queries across 77 intent classes.
ONNX INT8 quantization: CPU inference of about 10 ms per prediction (10.11 ms mean over 100 samples) against 45.66 ms for the PyTorch model, and a model that shrank from 571MB to 146MB.
Monitoring: Prometheus metrics and Grafana dashboards, plus Python alert rules for confidence, latency and error rate.
Drift checks: Evidently compares incoming data and predictions against the training data.
Explainability: SHAP feature importance and attention heatmaps show which tokens drive a prediction.

What I Measured

91.3% accuracy and 91.4% macro F1 on Banking77.
Classical baselines trained on TF-IDF features for comparison: SVM 87.9%, Logistic Regression 84.5% and Random Forest 83.9%.
Latency was measured on CPU for three formats: PyTorch FP32 at 45.66 ms, ONNX FP32 at 19.86 ms and ONNX INT8 at 10.11 ms.

Training on a Free GPU

The T4 on Kaggle's free tier has 16GB of VRAM, so the training setup had to be frugal. I used the Adafactor optimizer instead of AdamW, gradient checkpointing and FP16 mixed precision, and a batch size of 2 with gradient accumulation up to an effective batch of 32. The loss is class-weighted cross-entropy, and early stopping watches macro F1 with a patience of 3 epochs.

Serving and Pipeline

FastAPI service with Redis caching, rate limiting and a flag on low-confidence predictions so a human can review them.
Dagster orchestrates the pipeline steps, and MLflow tracks the experiments.
Docker Compose runs five services: the API, Redis, a Streamlit dashboard, Prometheus and Grafana.
Feedback loop: users can submit a correction tied to a specific prediction by its UUID.
91 pytest tests (86 run by default, 5 need a GPU or are slow and are skipped).

What I Learned

I had integrated plenty of AI APIs before this, and training, shrinking and serving my own model turned out to be a different problem. Most of the work sat around the model: validating the data with Pandera, tracking experiments, checking what quantization costs and what it saves, and watching for drift.

The memory limits of the free GPU forced the optimizer, checkpointing and precision choices. I understand those trade-offs better for having been made to use them.

Tech Stack

Model: ModernBERT-base, PyTorch, Hugging Face Transformers Inference: ONNX Runtime (INT8 dynamic quantization), FastAPI Data: Pandera validation, DVC versioning, Banking77 dataset Tracking: MLflow experiments, Dagster orchestration Monitoring: Prometheus, Grafana, Python alert rules, Evidently drift checks Explainability: SHAP, attention heatmaps Dashboard: Streamlit with Plotly Infrastructure: Docker Compose, Redis, uv

Key Features

ModernBERT fine-tuning: an encoder-only transformer fine-tuned on 13,083 banking queries across 77 intent classes.

ONNX INT8 quantization: CPU inference of about 10 ms per prediction (10.11 ms mean over 100 samples) against 45.66 ms for the PyTorch model, and a model that shrank from 571MB to 146MB.

Monitoring: Prometheus metrics and Grafana dashboards, plus Python alert rules for confidence, latency and error rate.

Drift checks: Evidently compares incoming data and predictions against the training data.

Explainability: SHAP feature importance and attention heatmaps show which tokens drive a prediction.

Training on a Free GPU

Serving and Pipeline

FastAPI service with Redis caching, rate limiting and a flag on low-confidence predictions so a human can review them.

Dagster orchestrates the pipeline steps, and MLflow tracks the experiments.

Docker Compose runs five services: the API, Redis, a Streamlit dashboard, Prometheus and Grafana.

Feedback loop: users can submit a correction tied to a specific prediction by its UUID.

91 pytest tests (86 run by default, 5 need a GPU or are slow and are skipped).

What I Learned

The memory limits of the free GPU forced the optimizer, checkpointing and precision choices. I understand those trade-offs better for having been made to use them.

Tech Stack