Live Demo · Medical Clinic AI Receptionist

Talk to the
AI Phone Agent

A live call to a production AI voice agent — custom Java SIP server, VoIP media engine, Aliyun ASR/TTS, and a local RAG pipeline. Ask it anything a clinic patient would ask.

Try asking (QUERY):

Hours
"What are your clinic hours?"
Walk-in
"Do you accept walk-ins?"
Prescription
"Can I get a refill over the phone?"
Sick Note
"I need a doctor's note for work."
Late
"I'm 15 minutes late for my appointment."
Referral
"Checking on my specialist referral."
Urgent
"My mother collapsed and is unconscious."
New Patient
"Can I register as a new patient?"
Implicit inheritance
1. "I'm running about 15 minutes late for my appointment."
2. "What if I can't make it at all?"
3. "How much is the cancellation fee?"
Entity Normalization
1. "I need a sick note for work."
2. "Can I get a repeat prescription?"
3. "I need to check my blood work results."
Boundary / Out of Scope
1. "What are your clinic hours?"
2. "Can you book me a flight to Toronto?"
3. "What's the weather like today?"
Urgent Fast-Track
1. "my mother has a headache.she is unconscious."
2. "My chest hurts and I can't breathe."
3. "I feel a bit dizzy."
Initializing...
Clinic AI Line
sip:20008005@172.31.19.141:27801

Tested on PC Chrome / Firefox / Edge · iOS Safari (iPhone / iPad)

More intents to try:

Greeting
"Hello, good morning!"
"Hey, is anyone there?"
Command
"Transfer me to a human agent."
"I want to talk to a representive."
"Can you repeat that?"
"byebye","hangup"
"I beg your pardon"
""Turn up the volume"
"Can you speak louder please"
"Turn down the volume"
"Please lower your voice"
Feedback
"That's very helpful, thank you."
Inform
"My name is John, I'm a registered patient."
Chitchat
"How are you today?"
"Tell me a joke"

Please ask me a question about the clinic knowledge base ▾

About This System

Intelligent Voice Agent for Medical Clinics

A production-grade AI phone assistant built from the ground up — custom Java SIP signalling server, VoIP media engine, Aliyun ASR/TTS, local RAG pipeline with BGE embeddings and pgvector, real-time 7-class intent classification, and sub-2s end-to-end latency.

Python / FastAPI pgvector RAG BGE Embeddings Qwen LLM Intent Classifier Rerank Pipeline Java SIP Server VoIP Java Media Aliyun ASR / TTS Janus WebRTC gpt-4o Redis Ubuntu Docker
Architecture Diagram

Key Technical Highlights

What Makes It Work

🧠
Intent Classification
7-class classifier (QUERY, COMMAND, INFORM, FEEDBACK, ACK, GREETING, CHITCHAT) with entity normalization and category routing.
🔍
Two-Stage RAG
BGE-large embedding → pgvector ANN search → BGE-reranker cross-encoder. Fast-track for high-confidence hits. Sub-400ms retrieval.
⚡
Dynamic LLM Routing
Simple queries auto-downgrade to a faster model. Complex answers use a higher-capability model (qwen-plus or gpt-4o). Supports both Aliyun Qwen and OpenAI backends. Saves 500–800ms per turn.
📞
Custom SIP Stack
Self-developed Java SIP signalling server + VoIP media engine. WebRTC bridged via Janus gateway.
🚨
Instant Repeat Response
L1,L2 Two-tier Redis cache answers repeated and paraphrased questions instantly — exact match <5ms, semantic match <300ms, bypassing RAG.
🌐
Multilingual Ready
Same pipeline runs Chinese education (bge-zh) and English clinic (bge-en). Config-driven, no code changes.
🔄
Session Management
Multi-turn conversation with automatic session lifecycle. Context inherited across turns — caller never needs to repeat themselves. Sessions auto-expire after 30 minutes.
🏗️
Full-Stack AI Telephony System
Sole architect and developer of the AI agent layer, and deployed end-to-end on AWS EC2 — from database to browser demo.

Changelog

Update History

June 5, 2026

L1 / L2 Redis Semantic Cache

Two-tier Redis cache intercepts repeat and paraphrased queries before RAG — exact match under 5ms, semantic match under 300ms. Includes FAQ warmup, fallback detection, and cache export tool.

Copyright & Attribution

Intellectual Property Notice

RAG Pipeline · AI Agent · Intent Classifier
Designed and developed by Harvey Liu. Originally built in Java, refactored in Python (2026). Released as open-source — source code available at github.com/lhw300.
SIP Server (Java)
Designed and developed under Harvey Liu's technical lead at LCall Technology. The compiled software is freely available for download and use. Source code is proprietary.
VoIP / PBX Media Engine (Java)
Designed and developed under Harvey Liu's technical lead at LCall Technology. Copyright © LCall Technology. Used with permission for personal portfolio demonstration, job interviews, and technical discussion purposes only. Not for commercial use.
WebRTC Browser Demo (JavaScript)
The browser-side SIP demo is built upon the open-source Janus Gateway demo code (janus.js, sip.js) by Meetecho s.r.l., licensed under the MIT License. Modified and integrated for portfolio demonstration purposes.