Overview

This is diagram of the basic architecture for our version of Reachy. It is base on a Pub/Sub design pattern. Mosquitto is being used for the MQTT message server. The major components for the system are the following:

• Main Controller - the "brains" of Reachy
• Sound - handles speech synthesis and speech recognition
• Computer Vision - the camera, OpenCV and the Coral Edge TPU
• Body - handles movement of the antennas in the head and movement of the arm

Messages

The components in the system communicate by sending messages on the message queue. The Sound, Computer Vision and Body components will not communicate directly to each other. They will only talk to the Main Controller. When the Main Controller receives a message from one of the other components, it will decide what messages need to be sent out, if any.

Message Types by Component

Sound

Speech Synthesis

• Say - request to say a specific piece of text
• Said - response that the say request was completed

Speech Recognition

• Sound Threshold - message sent when some sound threshold is reached
• Listen - request to listen for what someone is saying
• Heard - the response for "Listen"; the text that it heard someone say ****

Computer Vision

Facial Recognition