← All projects

Project

Walter — An Experimental Virtual Assistant

I built Walter as an experimental virtual assistant before the widespread use of large language models. It combined listening, speech recognition, punctuation restoration, and synthesized cues, alongside experiments in language parsing and facial recognition.

Walter explored speech and vision interfaces using individual Python libraries. I used a microphone and webcam on ordinary hardware to experiment with how an assistant could listen, process spoken language, and recognize a face.

Approach

  • Speech recognition: speech_recognition sent captured audio to Google’s Web Speech API for transcription.
  • Punctuation restoration: a bidirectional-RNN model from the punctuator library restored sentence boundaries before further text processing.
  • Language parsing: nltk handled tokenization and part-of-speech tagging, with Stanford NER for extracting names from an utterance.
  • Speech synthesis: pyttsx3 produced spoken responses and listening cues.
  • Facial recognition: a separate OpenCV pipeline used Haar-cascade detection, a labeled training set, and an LBPH recognizer.

Outcome

The speech pipeline greeted the user, listened, captured audio, transcribed it, and restored punctuation. Separate language-parsing and facial-recognition experiments let me explore additional ways for the assistant to interpret input. This work preceded Jarvis, which connected a voice interface to language models and tools.

Skills & tools