June 2026 GenAI / LLM Fine-Tuning / QLoRA / Text-to-SQL / NLP / Model Evaluation
Analytics Copilot (Text-to-SQL).
Mistral-7B fine-tuned with QLoRA to turn natural language into schema-conditioned SQL, with a from-scratch evaluation harness and dual-provider inference failover.
Overview
A full fine-tuning pipeline — dataset curation, QLoRA training, custom dual-benchmark evaluation, Hub publishing — wrapped in a production-shaped serving layer. Mistral-7B-Instruct is fine-tuned with 4-bit NF4 quantization and LoRA adapters (r=16) across all seven attention and MLP projections, then served through a Hugging Face Inference Endpoint with adapter routing and an OpenAI fallback. The Streamlit front end carries zero GPU or ML runtime dependencies: training and serving are fully decoupled.
Highlights
- Fine-tuned a 7B model with under 1% trainable parameters via 4-bit NF4 quantization and LoRA r=16 adapters across seven projection matrices
- Built a from-scratch evaluation harness — Exact Match, literal-masked structural EM, sqlglot parse-validity, and AST-based schema-adherence checking that catches hallucinated tables and columns string similarity would miss
- Dual-benchmark evaluation quantifying the generalization gap: 35.0% exact match / 100% parse success / 97.5% schema adherence in-domain, vs 76% schema adherence zero-shot on Spider
- Multi-tier inference failover (HF Inference Endpoint with adapter routing → OpenAI) plus a proactive endpoint wake-up ping, deliberately separated so warm-ups never trigger the failure path
- Root-caused a 271-violation CI lint break as upstream linter drift rather than new code, fixing it with a 9-line config pin instead of a repo-wide reformat
- ~6,419 LOC, 36 tests across 14 files
Stack
Python · Mistral-7B · QLoRA · PEFT · Unsloth · TRL · bitsandbytes · Transformers · sqlglot · Hugging Face Endpoints · Streamlit · GitHub Actions