Stack & Architecture
Built with FreeSWITCH, gRPC, Triton, C++, SIP, Asterisk, AWS. Delivered for Tomato AI, 2023-05–2023-12.
<!-- Auto-drafted from profile data. Add problem/approach detail as needed. -->← all projectsview live back to projects
Tomato AI - Real-Time Voice Accent Modification
Built the complete infrastructure for real-time voice accent modification on live SIP calls over 7 months. Created a custom FreeSWITCH plugin (mod_tomato) that intercepts the caller's audio stream, sends it to an ML inference model via bidirectional gRPC, and re-inserts the accent-modified audio back into the live call. Integrated NVIDIA Triton Inference Server via a C++ client for model serving. Built SIP proxy handling both signaling and media traffic, deployed Asterisk-based call center with multi-agent support, IVR menus, and inbound DID. Verified selective voice modification on incoming and outgoing call legs per agent. System accepted live calls from mobile phones on public networks. Produced full restoration documentation and AWS asset mapping at project closure.
Outcomes
- Built custom FreeSWITCH plugin (mod_tomato) for real-time audio stream interception
- Integrated NVIDIA Triton Inference Server via C++ gRPC client
- Deployed full SIP proxy and Asterisk call center with multi-agent support
- Live calls from mobile phones on public networks with selective accent modification
Tech Stack
FreeSWITCHgRPCTritonC++SIPAsteriskAWS
