Audio streaming and recording

Introduction

In this tutorial, we will walk through the steps needed to take advantage of the audio feed feature available for Furhat robots designated for research. See Audio Feed section for more info.

After following this tutorial you will be able to stream and record Furhat's input and output audio.

You will need a research designated Furhat robot connected to your local network and a computer connected to the same network that will be used for the external audio feed processing.

Download the code

On your workstation, clone the tutorials repository from GitHub:

git clone https://github.com/FurhatRobotics/tutorials.git

In this repository you will find the audiofeed-demo folder with:

• An example source code for audio streaming.
• A README.md file with short instructions.

Read the audio

The audio is published through a ZMQ socket, which link will be provided in the web interface. The input audio (microphone) is in the left channel, and the output audio (speech synthesis) is in the right channel. This means that out of every 4 bytes, the first 2 will be the input audio and the other 2 will be the output.

You can test the example code by running main.kt in the source code, trying the playback or record audio functions.

The FurhatAudioFeedStreamer class provides the logic to connect to that socket and stream the data to whatever interface implementations are provided (in this case one of the following two).

Playback

The FurhatAudioFeedPlayback class is an example of how to play the audio live. It redirects the audio to a SourceDataLine.

Record

Similarly, you can use the FurhatAudioFeedRecorder class to write wav files to the root of your project. Choose between the startRecordAll() or startRecordSeparate() functions to capture all audio together or in separate files.