Realtime API

The Realtime API allows your application, runnning on a separate computer, to control the robot through an asynchronous protocol, using WebSockets.

A WebSocket is a communication protocol that allows a persistent, two-way connection between a client and a server, enabling them to send data to each other in real time without repeatedly reopening connections. In this case, Furhat will host the websocket server and your program (client) can connect to it.

The requests can be sent from any programming language, including javascript in web browsers. Most programming languages provide easy-to-use WebSocket client libraries. To make it even easier to use from Python, we are providing a dedicated Python client.

NOTE

By default, the Realtime API is disabled. You need to choose the security level that you want in order to start using the Realtime API.

Security

You can configure the extent to which clients are allowed to connect and use the WebSocket API in the Furhat web interface settings. The settings are specific to the use case (scope):

1. Clients running on the robot (skills developed in Kotlin, packaged and uploaded to the robot), or on the local computer in case of the SDK.
   • Always allowed, without any authentication.
2. Clients connecting from the same local network as the robot or SDK.
   • Can be configured to always be allowed, or requiring authentication.
3. Clients connecting from a different network as the robot or SDK. This can be done if the robot or SDK is exposed with a global IP, or if NAT has been configured in your router.
   • Authentication is always required.

In case authentication is required, you can generate or specify an API key in the Furhat web interface that must be used by the WebSocket client.

Playground

When your robot is running, you can go to http://<ROBOT_IP>:9000/ where you will find a playground. Here, you can test out the different requests you can send using the websocket API. This lets you easily test and understand what the different methods do.

If you want to access the same playground but use it for your virtual furhat, you go to http://127.0.0.1:9000/.

Connection

The client should connect to the WebSocket on:

ws://<ROBOT_IP>:9000/v1/events

Authentication

If authentication is required (as configured in the robot's web interface), an authentication message must be sent from the client to the server as the first message:

{
    "type": "request.auth", 
    "key": "AUTH_KEY"
}

The server will then respond with a message saying whether the connection was successful, and the scope of the connection:

{
    "type": "response.auth", 
    "access": true,
    "scope": "localhost"
}

Sending and receiving events

The client can send and receive messages in json format. Every message has a type field. Messages types from the client to the server always start with request.*. Message types from the server to the client always start with response.*

A full specification of events can be found here.

If you want to know which response is related to which request, it is also possible to attach a request_id to any request event. The corresponding response will then also carry this id. For example:

{
    "type": "request.speak.text", 
    "text": "I am a speaking robot",
    "request_id": "my_event_123"
}

Would give you this response once the robot starts speaking:

{
    "type": "response.speak.start", 
    "text": "I am a speaking robot",
    "request_id": "my_event_123"
}