Overview
Developed an AI system for assessing students' English presentations. Designed, deployed, and operated an Azure ML inference system used for presentation assessment by approximately 30,000 students monthly.
Architecture
Presentation Attitude Assessment Demo
Uses MediaPipe landmarks to analyze gaze, posture, and movement for presentation-attitude assessment.
Gaze Direction Tracking
Hand Gesture Recognition
Key Achievements
- [MLOps] Designed, deployed, and operated an Azure ML Endpoint and Blob Storage inference pipeline used for presentation assessment by approximately 30K students monthly
- [LLM Agent] Built student-and-evaluator agents to generate and review grade- and proficiency-conditioned synthetic English writing for international assessment exploration
- [Audio] Fine-tuned NVIDIA Parakeet STT on non-native children's speech, reducing WER from 9.6% to 4.4%
- [Vision] Built MediaPipe-based gaze and posture assessment with 98.32% validation accuracy and 98.77% F1, configured for practical CPU inference
- [Vision] Used a quantized image-generation model during development to reduce GPU cost; added AWS image-safety detection and initial human review for a children's service
- [Search] Combined BM25 and vector retrieval; internal log-based evaluation showed up to ~20% fewer LLM calls at the same quality threshold
Tech Stack
- Cloud: Azure ML, Blob Storage, AWS Image Moderation
- Frameworks: PyTorch, HuggingFace, LangGraph
- Models: Quantized Diffusion, NVIDIA Parakeet, MediaPipe
- Serving: FastAPI, ONNX
- Search: Hybrid Search (BM25 + Vector)