Graphical programming of skills using Blockly

Note! Blockly was recently added to the Furhat Platform. Blockly flows that were created with an earlier beta version of Blockly may not be compatible. The beta version of Blockly can be downloaded from furhat.io/downloads

Blockly is one of three ways to program the robot. The other two being Remote API and Kotlin Skill API.

Blockly is intended to make it easy for anyone (skill developers, designers, marketing staff, students) to quickly create an interaction with the Furhat Robot and explore the world of social robotics and digital agents.

Blockly and the Kotlin Skill API share the same general skill framework described in: skills. If your are familiar with the framework you will understand the components of Blockly easily. Vice versa, testing out Blockly is a good introduction to creating skills with the more powerful Kotlin Skill API and regular programming IDE. Note however, that you can't use Blockly and the Kotlin Skill API interchangeably to seamlessly switch between the two interfaces.

Blockly can be used by to:

  • quickly prototype an interaction
  • create simple example interactions
  • create complex wizarded interactions
  • learn/teach about social robot interactions
  • learn/teach the skill framework of the Furhat Platform

Blockly works both for the Furhat Robot and the Virtual Furhat.

Any graphical block programming interface restricts the complexity of interactions you can build and maintain, Blockly is no exception. For autonomous skills that are anything more than an example or prototype we don't recommend to use Blockly, however creating entirely wizarded interactions in Blockly is more doable.

Getting started with Blockly

Please see the (video) tutorial on how to build your first Blockly flow.

Example flows

  • HowAreYou - Hello, how are you-interaction.
  • CreateSkill - Basic createSkill template.
  • Name - Simple user name question.

How the Blockly works

Technically Blockly run in the platform as a skill. To run older beta versions of Blockly, you can download it as a .skill file from furhat.io/downloads and run them on Furhat the same way you run any skill.

Blockly outputs an .xml flow file. This xml can be imported in Blockly and then run on the Furhat robot (or Virtual Furhat) from the Blockly GUI.

Current functionality and limitations of Blockly

The Blockly graphical programming tool already allows you to build quite complex flows and many of the concepts documented on this site are supported.

Functionality allowed (currently) exclusively in the Blockly tool:

  • Extended, built-in GUI support for
    • showing content (images, videos, websites)
    • showing a selector with 1-8 options
    • showing a form on screen, with a custom webhook allowing you to for example add an entry to a google sheet on form submission

Current limitations of interactions built using the Blockly tool:

  • Only built-in NLU Entities supported. You can however create your own intents.
  • Only one Blockly canvas supported which limits the possibility to overview a large amount of blocks created.
  • When using the Show website block to display a web page in the GUI, make sure to use a HTTP(S) link, e.g. 'https://furhatrobotics.com', instead of 'furhatrobotics.com'.
  • Interactions (.xml flows) created in blockly can only be run from the Blockly GUI (or set to autostart), they can not be run as .skill files.