SA
MLE Discipline 03 / 04

Machine Learning Engineering

Models are easy; pipelines that keep them honest are the work. Medical imaging, RLHF research, and MLOps that survives contact with production.

Machine Learning Engineering abstract artwork

The story

The CBCT scan validator at Hybridge is the project that taught me what production ML actually costs. The model is the small part: a self-supervised spatiotemporal masked autoencoder that runs inference in 58 milliseconds on half a gigabyte of VRAM. The big part is everything that keeps it honest: a fully automated, versioned training data pipeline where training and inference share one preprocessing path, and a weekly CI/CT job that evaluates against a deterministic holdout and refuses to promote a model that fails the gate.

I benchmarked six architectures behind one shared interface before picking the production model: a 2D ConvNeXt baseline, a hybrid transformer, a 3D CNN, an nnU-Net adapter, a VAE anomaly detector, and the ST-MAE that won. When the model is wrong, confidence thresholds route the scan to a human instead of pretending.

Before Hybridge, I spent a year and a half at NYU researching whether RLHF methods (PPO, GRPO) could make Llama a measurably better persuader. Reward modeling and preference optimization, the same machinery behind modern aligned models, pointed at argument quality. That training-side depth is why I trust myself to debug the serving side.

What I bring

No train/serve skew

One ETL path feeds both training and inference. Versioned tensors, idempotent processing, object-lock concurrency control.

Gated promotion

Weekly CI/CT evaluation on a deterministic holdout with an accuracy gate. A model that can't prove itself doesn't ship.

Training-side depth

RLHF, GRPO, PPO, reward modeling at NYU; self-supervised pretraining (DINO, ST-MAE) at Hybridge. Not just an API consumer.

Cost-conscious serving

Scale-to-zero Cloud Run, event-driven inference, ~$25/month for a system a vendor quoted at $98K up front.

The projects

Built, shipped, and used.

CBCT Scan Validator preview

CBCT Scan Validator

In production

An in-house medical-imaging quality gate that catches bad dental CT scans at upload time across 13 artifact classes, routing corrective actions to the front desk before a designer wastes hours.

  • Six architectures benchmarked behind one interface; ST-MAE won on accuracy-per-watt
  • Event-driven inference: scan lands in storage, verdict email in ~20 seconds
  • Fully automated versioned training pipeline; new labeled scans flow in continuously
  • ~$125K year-one cost displacement vs the vendor proposal
Read the full story
LLM Persuasion (NYU research) preview

Eighteen months of research on making Llama argue better: reward modeling over argument quality, then PPO and GRPO fine-tuning against it. The full RLHF stack, hands on.

  • Preference data collection and reward-model training from scratch
  • Policy optimization with PPO and GRPO; the tradeoffs between them, learned the hard way
Read the full story
Loan Radar preview

Loan Radar

Academic

End-to-end MLOps for a loan-default model: MLflow experiment tracking, Airflow orchestration, Kubernetes serving, and CI/CD that retrains and redeploys without a human.

  • The full lifecycle: data validation, training, registry, deployment, monitoring
Read the full story

The writing

Notes from this side of the work.