Snippets

Note: this is an experimental feature that will likely change going forward.

Snippets allow you to quickly create chat-oriented dialog with Furhat. You specify the dialog similar to how a movie script lists the utterances of the two speakers. Each snippet specifies only a small part of the dialog, but they can be connected with each other so that the dialog can follow different paths. Snippets allow for mixed initative dialog where the system might sometimes give responses to the user's requests, and the other way around. The general idea is that the snippets are defined in a declarative way. The exact execution of the dialog (e.g. how often the system should take the initiative) is defined outside of the snippets.

Two main use-cases of Snippets are;

1. As a chit-chat enhancement to a task-oriented skill: Adding chit-chat functionality to a goal-oriented skill, for example a Fruit seller where you might also want to support the robot chit-chatting about the weather, about sports, music or similar topics. Once a snippet has been depleted, you will likely want to lead the user back to the task-oriented dialog.
2. As a substitute to ordinary flows of states if you skill is mainly focused on chit-chat, i.e no real task-oriented dialog.

Of course, you may combine the two in any way you see fit for your use-case.

User-initiated snippets

Snippets are defined using Kotlin and grouped in a collection of snippets like this:

val mySnippets = snippets {
    snippet {
        user("How are you")
        respond("I am fine")
    }
}

In this simple example, we have only defined one user-initiated snippet. Such snippets should start with a "user" utterance ("How are you?") and can continue in different ways, here we just follow up with a system response ("I am fine"). User-initiated snippets are always active in the dialog, which means that the system always listens and reacts to them.

Typically, you want to provide several examples of what the user might say. It is also possible to add several possible ways of responding (but which should have the same meaning):

snippet {
    user("How are you?", "How are you doing?")
    respond("I am fine") alt "I am doing just fine"
}   

You can also utilize the speech utterance functionality for the system response, which make it possible to for example add gestures to the system's behavior:

snippet {
    user("How are you?", "How are you doing?")
    respond {+Gestures.Smile; +"I am fine"}
}   

System-initiated snippets

When the system takes the initiative (for example starts the dialog), it searches for the first system-initiated snippet (one that starts with a "request"):

snippet {
    request("Hi there, how are you doing?")
    user("I am fine", "I am doing just fine", "great, thanks for asking")
    respond("That's good to hear")
}   

After the system has said "Hi there, how are you doing?", it will listen for the user responding. If the user's next utterance matches the specified user response (e.g. "I am fine"), the system will continue the snippet with the response ("That's good to hear"). Remember that the system always also listens for all user-initatied snippets, so the user could also say "How are you?", which would match the snippet defined above.

Note that the snippet could also end with a new system request, thus it is perfectly fine to have several system utterances after each other (but not several user utterances):

snippet {
    request("Hi there, how are you doing?")
    user("I am fine", "I am doing just fine", "great, thanks for asking")
    respond("That's good to hear")
    request("What have you been up to?")
}   

The system keeps track of which requests it has made. Per default, the system is allowed to make the same respond several times, but it cannot make the same request more than once (but this can be changed, as explained further down).
Thus, if it would end up in the same snippet a second time, it would say "That's good to hear", but not follow up with the last request.

Proceed

You can also end a snippet with a proceed() command. This will force the robot to immediately take the initiative (searching for another robot-initative snippet):

snippet {
    request("Hi there, how are you doing?")
    user("I am fine", "I am doing just fine", "great, thanks for asking")
    respond("That's good to hear")
    proceed()

Ending the conversation

You can end the conversation with a terminate() command:

snippet {
    user("I don't want to talk anymore")
    respond("Okay, goodbye")
    terminate()
}

Labels

To be able to refer to system and user utterances, you can assign labels to them:

object UserCheckStatus : UserLabel()
object RobotCheckStatus : RobotLabel()

val mySnippets = snippets {

    snippet {
        user("How are you?", "How are you doing?") label UserCheckStatus
        respond("I am fine") alt "I am doing just fine"
        request(RobotCheckStatus)
    }   

    snippet {
        request("How are you doing?") label RobotCheckStatus
        user("I am fine", "I am doing just fine", "great, thanks for asking")
        respond("That's good to hear")
    }  

}

When the same label is applied to several utterances, it means that these utterances have the same meaning.

In this case, the label RobotCheckStatus is used to refer to a request specified in another place, which also handles the user's response (e.g. "I am fine"). Thus, it could be thought of as a jump to that snippet.

If the same UserLabel is applied to several user utterances which provide different examples, these examples would all be collected and form a common intent. In the same way, if the same RobotLabel is applied to several utterances with provided texts or speech utterances, these would all be seen as valid system utterances for that label, and the system would randomly pick from them.

Bridges

To make the dialog more fluent, it can help to bridge responses and requests:

snippet {
    user("How are you?", "How are you doing?") label UserCheckStatus
    respond("I am fine") alt "I am doing just fine"
    bridge("and you")
    request(RobotCheckStatus)
}   

Remember that the system never issues the same request twice, so if it has already done a RobotCheckStatus, it would not do it again. The bridge() is only used if the following request() is used. This makes them different from respond().

Branching dialog with Context snippets

Labels are very useful when you want to branch the dialog. Let's say that we want the user to be able to answer the RobotCheckStatus request in a different way:

snippet {
    context(RobotCheckStatus)
    user("Not so well actually", "I had a terrible day")
    respond("That's sad to hear")
}   

These snippets are similar to the system-initiated snippets above (since they start with a system utterance), but unlike snippets starting with a request(), they will not be used as an initiative by the system.

Specifying UserLabel with intent

You can also provide the UserLabel with an Intent, in which case you don't have to give it any examples.

For example, you can provide it with a built-in intent:

object Yes : UserLabel(furhatos.nlu.common.Yes())

You can also provide example utterances for an intent directly in the UserLabel definition:

object UA_Why : UserLabel(listOf("why", "how come", "why is that"))

Implied labels

Short utterances like "red", "yes", "no" and "why?" are very common in dialog. However, their meaning is typically dependent on the preceding context. Thus, it would not make sense to start a user-initiated snippet with such a short utterance, since these snippets should be relevant att all points in the dialog.

When using them as a follow-up to a system utterance, it often helps to give them an implied label:

snippet {
    request("Do you have pets?")
    user(Yes) implies UserHasPets
}   

snippet {
    user("I have pets") label UserHasPets
    respond("How nice. I would love to have a pet")
}

In this case, we say that the reply "yes" to the question "Do you have pets?" implies (means the same thing as) "I have pets". We do this by implying another UserLabel. This also makes it possible for the user to answer the request with "I have pets", or actually to say that phrase at any point in the dialog and get a valid response from the system.

Alternative user responses

You can also provide several alternative user responses to branch of the dialog in different directions, using the "switch" command:

snippet {
    request("Do you have pets?")
    switch {
        user(Yes) implies UserHasPets
        user(No) implies UserHasNoPets
    }
}   

Inline branching

Another way of branching is to do it inline (with arbitrary depth):

snippet {
    request("Do you have pets?")
    switch {
        user(Yes) branch {
            respond("How nice. I would love to have a pet")
        }
        user(No) branch {
            request("I see, would you like to have a pet?")         
            switch {
                user(Yes) branch {
                    respond("Me too!")
                }
                user(No) branch {
                    request("Of course, not everyone likes pets")   
                }
            }     
        }
    }
}   

Although inline branching is more compact, it can quickly get complex if it is too deep. Also it does not allow for more flexible dialog, as was the case with UserHasPets above. Use it with care.

Entities and Variables

Sometimes you want to allow semantic entities in the user's response:

object entities {
    var name : PersonName? = null
}

val mySnippets = snippets {

    entities(entities)

    snippet {
        request("What is your name?")
        user("My name is @name")
        respond {+"Nice to meet you ${entities.name.value}"}
    }

}

As can be seen, the entities need to be defined as properties in an object, which is then passed to the snippets builder with the entities(...) command.

The name of these properties can then be used with a @-symbol in the user utterances examples. The value is then stored in the provided property object, and can be accessed for example in the next system response. Note that the response in this case has to be defined as a speech utterance, otherwise the variable will be evaluated at initiation (returning null) and not at runtime.

Effects and accessing the User object

Each user utterance can have an associated "effect", which is a closure that is executed if the user utterance is matched. This could for example be used to store the user's name in the user object:

snippet {
    request("What is your name?")
    user("My name is @name") effect {
        user.name = entities.name.value
    }
    respond {+"Nice to meet you ${user.name}"}
}

For this to work, you need to extend the user object with a "name" field:

var User.name : String? by UserDataDelegate()

Conditioning system utterances

You can add conditions to system utterances with "cond". The condition must be met in order for the utterance to be used. For example, you can ensure that the system doesn't ask for the user's name if it is already known:

snippet {
    snippet {
        request("What is your name?") cond {user.name == null}
        user("My name is @name") effect {
            user.name = entities.name.value
        }
        respond {+"Nice to meet you ${user.name}"}
    }
}

As said earlier, there is an implicit condition for system requests that they can only be used once in one session. You can override this requirement. To override this implicit condition, you should use "newCond". Here we specify that the request can be used up to three times in one session:

snippet {
    request("Do you want to talk about something else?") newCond sessionLimit(3)
    user("yes")
    respond("You can ask me about anything")
}

Branching system utterances

You can make a random choice of system utterances at any point:

snippet {
    user("How are you today?")
    random {
        respond("Not very well, actually")
        respond("I am doing just fine")
    }
}

You can also make a conditioned choice (assuming there is a variable called "robotFeeling" defined):

snippet {
    user("How are you today?")
    choice {
        respond("Not very well, actually") cond {robotFeeling < 0.2}
        respond("I am alright") cond {robotFeeling < 0.7}
        respond("I feel great")
    }
}

If you need to choose between deeper branches of dialog, you can use the "branch" command:

snippet {
    user("How are you today?")
    choice {
        respond("Not very well, actually") cond {robotFeeling < 0.2}
        respond("I am alright") cond {robotFeeling < 0.7} branch {
            request("How are you?")
        }
        respond("I feel great")
    }
}

Special user labels: NoMatch and NoInput

There are two special pre-defined user labels for handling miscommunication:

• NoMatch: Representing user utterances where no intent matched
• NoInput: Representing no input from the user

Examples:

snippet {
    user(NoMatch)
    respond("Sorry, I didn't understand that")
}

Handling repetitions

In some cases you want the system to repeat what it has said before, for example when there is miscommunication or the user asks the system to repeat.

snippet {
    user(NoMatch)
    respond("Sorry, I didn't understand that")
    // 2 means that we should repeat the second-to-last robot utterance
    repeat(2)
}

snippet {
    user(NoInput)
    respond("Sorry, I didn't hear anything")
    repeat(2)
}

snippet {
    user("what did you say", "could you repeat that")
    // 1 means that we should repeat the last robot utterance
    repeat(1)
}

If there were alternatives in repeated robot utterance, the system will pick another alternative (effectively rephrasing the utterance).

Generic contexts: Request and Response

While context() has been used above to condition a snippet on a certain RobotLabel, it can also be used to condition it on the generic type of the preceding robot utterance:

snippet {
    // This matches a previous utterance which was made with request()
    context(UtteranceType.Request)
    user(NoInput)
    // If we get silence here, we repeat the last request
    repeat(1)
}

snippet {
    // This matches a previous utterance which was made with respond()
    context(UtteranceType.Response)
    user(NoInput)
    // If we get silence here, we just proceed with a new initiative
    proceed()
}

Sub-dialogs

In dialog, it is quite common that people ask counter-questions, which are not intended to switch topic. In such cases, it should still be possible to continue the dialog from before the counter-question was initiated. This is an example:

Robot: What is your name?
User: First or last?
Robot: First
User: Gabriel
Robot: Nice to meet you Gabriel

To handle this, user-initiated snippets are either treated as a sub-dialog, or a topic shift, depending on what it contains. If it only contains one user turn and no request() commands , the ongoing snippet is kept in memory. Otherwise, it is treated as a topic shift and the ongoing snippet will be abandoned.

Thus, the following snippets would allow for the dialog example above:

snippet {
    request("What is your name?") label RobotRequestName
    user("@name")
    respond {+"Nice to meet you ${entities.name.value}"}
}

snippet {
    context(RobotRequestName)
    user("First or last?")
    respond("First")
}

As you can see, the user response ("Gabriel") to the robot's initial request ("What is your name?") is still valid, even after the sub-dialog has been completed. Notice also that the sub-dialog started with a context() command, which makes it only being valid in that context (although that is not mandatory, if you want it to be more general).

Importing snippets

Sometimes you want to import snippets from another snippet collection:

val mySnippets = snippets {

    snippet {
        user("How are you")
        respond("I am fine")
    }

    // Importing snippets where we talk about music
    import(musicSnippets)

}

This is a great way of packaging snippets into different collections and reuse them. Notice that the order of the snippets and the imports is of importance: When the system takes the initiative, it will search for the first snippet that has not been used before, starting from the top.

Topics

You can also assign a topic to a snippet collection:

val MusicTopic = Topic()

val musicSnippets = snippets(MusicTopic) {

    snippet {
        user("Let's talk about music")
        respond("Okay, let's talk about music")
        proceed()
    }

    snippet {
        user("What is your favorite artist?")
        respond("I like Kraftwerk")
    }

    snippet {
        request("Do you like Krafwerk?")
        user(Yes)
        respond("I like them too")
    }

}

This makes all snippets in the collection associated with the topic. When a snippet is triggered that is associated with a topic, the robot keeps track of the current topic. When the system takes the initiative (like after the proceed() command), it will first search for robot-initiative snippets in the current topic. If these are exhausted, it will search through all snippets.

Testing snippets from the command-line

You can interact with your snippets from the command-line, by running it like this:

SnippetRunner(mySnippets, { User.NOBODY }).runInConsole()

Using snippets in the flow

There are two ways of using snippets in the flow, either as the main form of interaction, or as a general "fallback" in your application, for example to handle simple questions from the user.

Here is an example of how you can run the snippets as the main form of interaction (where the snippets are contained in the chatSnippets variable). As you can see, the ChatState inherits a special SnippetState. By raising the special TakeInitiative event on entry, you trigger the snippets to take initiative. You can also see how we handle if the user leaves and if the snippets terminate.

val Start = state {

    onEntry {
        if (users.count > 0) {
            furhat.attend(users.random)
            goto(ChatState)
        } else {
            goto(Idle)
        }
    }

}

val Idle = state {

    onEntry {
        furhat.attendNobody()
    }

    onUserEnter {
        furhat.attend(it)
        goto(ChatState)
    }

}

val ChatState = state(parent=SnippetState(chatSnippets)) {

    onEntry {
        raise(TakeInitiative())
    }

    onUserLeave {
        goto(Idle)
    }

    onEvent<TerminateSnippetState> {
        goto(Idle)
    }

}

If you want to use the snippets as a fallback, you simply create a SnippetFallbackState that you use as parent state in your general state that all your other states inherit (for example called "Interaction").

val ChitChatFallback = SnippetFallbackState(chatSnippets)

val Interaction = state(parent=ChitChatFallback) {

}