Advanced Speaking and Listening control (Beta)

Per default, Furhat is either speaking or listening. Thus, when Furhat is speaking, the user cannot interrupt the robot.

It is possible, however, to allow for interruptions and to make Furhat pause and resume speaking.

Allowing for interruptions

NB: To allow interruptions, you must either use a headset, so that the microphone doesn't pick up the robot's own speech, or Furhat has to be configured to ignore its own speech when listening for the user. Currently, when this feature is enabled, Furhat will hear that the user is speaking and is trying to interrupt the robot, but will miss the initial part of the user's utterance at the point of interruption.

You can configure Furhat to allow for interruptions in general:

// Make Furhat interruptable during all furhat.ask(...)
furhat.param.interruptableOnAsk = true
// Make Furhat interruptable during all furhat.say(...)
furhat.param.interruptableOnSay = true

Alternatively (or in addition), you can enable or disable interruptable for each individual command:

furhat.ask(interruptable=true) {
    +"Could you please tell me your name?"
}
furhat.say(interruptable=true) {
    +"Hi there, I am a speaking robot"
}

Per default, even if these parameters are set to allow for interruption, interruption is not allowed if there are no onResponse handlers with intents registered for the current state, since it doesn't make sense to allow the user to interrupt if the robot has no way of interpreting the user's response. However, you can override this rule if you want (for example if you are using your own NLU):

// This parameters is set to false per default
furhat.param.interruptableWithoutIntents = true

Note that when you allow for interruptions, this might cause confusion in case the user interrupts the robot by accident. Thus, you should carefully consider when this is a good design choice and not.

When interruption is enabled, there is a small delay after the robot starts to speak before it starts to listen for incoming speech (to avoid confusion). The default is 1000ms, but this can be changed:

furhat.param.interruptableDelay = 1500

Interruptions can be handled just like other responses. However, it is also possible to check whether the response was an interruption or not. This is very useful if you use an open microphone solution, since Furhat is very likely to have missed the first part of the user's utterance (as explained above), in which case you can ask the user to repeat:

onResponse(cond={it.interrupted}) {
    furhat.ask("Sorry, what did you say?")
}

This behaviour is implemented in the default dialog state (which is implicitly inherited by all states), but you can of course override it and implement your own handling of it, if you want.

Pause and resume speaking

When Furhat is speaking and a non-instant trigger is triggered, or if you call furhat.stopSpeaking(), Furhat will stop speaking, and there is no way of resuming the utterance.

If you want to pause (or resume) an utterance, this has to be done from an instant trigger. You do this by calling furhat.pauseSpeaking() and furhat.resumeSpeaking(). Let's say we would like to pause speaking when the user looks away from Furhat, and then resume speaking when the user looks back:

// Note that the trigger has to be declared to be instant
onUserAttend(instant=true) {user->
    if (!user.isAttendingFurhat) {
        furhat.pauseSpeaking()
    } else {
        // We resume the turn from where we left off, inserting the utterance "Let's see"
        furhat.resumeSpeaking { +"Let's see" }
    }
}

When resuming the turn, you can also specify where you want the resumption to take place:

// Resume at the last punctuation, e.g. comma or period (default):
furhat.resumeSpeaking(at = UtterancePoint.PUNCTUATION) 
// Resume at the last word
furhat.resumeSpeaking(at = UtterancePoint.WORD) 
// Resume at the beginning of the current utterance segment
furhat.resumeSpeaking(at = UtterancePoint.SEGMENT)
// Resume at the beginning of the current utterance 
furhat.resumeSpeaking(at = UtterancePoint.BEGINNING)

Utterance segments are naturally created when you insert blocking calls or audio in the utterance (but not for inline gestures or other asynchronous behaviours). You can also insert new segment manually in the utterance builder by calling +newSegment.

Resume speaking after interruptions

When the user interrupts Furhat while speaking (if that is enabled), the turn is automatically paused, which means that it is also possible to resume speaking. For example, we could decide to resume speaking if the user says something which didn't trigger any intent:

// Note that the trigger has to be declared to be instant
onResponse(instant=true, intent=NullIntent, cond={it.interrupted}) {
    furhat.resumeSpeaking {
        +"Let's see"
    }
}