Intent Handling

After a voice command has been transcribed and your intent has been successfully recognized, Rhasspy is ready to send a JSON event to another system like Home Assistant or a remote Rhasspy server.

You can also handle intents by:

  • Listening for hermes/intent/<intentName> messages over MQTT (details)
  • Connecting a websocket to /api/events/intent (details)

Available intent handling systems are:

Home Assistant

Add to your profile:

"handle": {
  "system": "hass"
},

"home_assistant": {
  "access_token": "",
  "api_password": "",
  "event_type_format": "rhasspy_{0}",
  "url": "http://hassio/homeassistant/"
}

If you're running Rhasspy as an add-on inside Hass.io, the access token is automatically provided. Otherwise, you'll need to create a long-lived access token and set home_assistant.access_token manually.

See the Home Assistant Example for sentences that use Home Assistant's built-in intents and a slot program can automatically download the names of your entities.

When an intent is triggered, any speech received from Home Assistant is automatically forwarded to your text to speech system.

Intents

Home Assistant can accept intents directly from Rhasspy using an HTTP endpoint. Add the following to your configuration.yaml file:

intent:

This will enable intents over the HTTP endponit. Next, write intent scripts to handle each Rhasspy intent:

intent_script:
  ChangeLightColor:
    action:
      ...

The possible actions are the same as in automations.

Events

Rhasspy can also send Home Assistant an event every time an intent is recognized through its REST API. The type of the event is determined by the name of the intent, and the event data comes from the tagged words in your sentences.

For example, if you have an intent like:

[ChangeLightColor]
set the (bedroom light){name} to (red | green | blue){color}

and you say something like "set the bedroom light to blue", Rhasspy will POST to the /api/events/rhasspy_ChangeLightColor endpoint of your Home Assistant server with the following data:

{
  "name": "bedroom light",
  "color": "blue"
}

Rhasspy also provides a few extra fields besides the intent's slots:

In order to do something with the rhasspy_ChangeLightColor event, create an automation with an event trigger. For example, add the following to your automation.yaml file:

- alias: "Set bedroom light color (blue)"
  trigger:
    platform: event
    event_type: rhasspy_ChangeLightColor
    event_data:
      name: 'bedroom light'
      color: 'blue'
  action:
    ...

Implemented by rhasspy-homeassistant-hermes

MQTT

Rhasspy automatically publishes intents over MQTT (Hermes protocol). This allows Rhasspy to send interoperate with Snips.AI compatible systems.

Home Assistant can listen directly to these intents using the snips plugin. To use it, add to your Home Assistant's configuration.yaml file:

snips:

intent_script:
  ...

See the intent script documentation for details on how to handle the intents.

Self-Signed Certificate

If your Home Assistant uses a self-signed certificate, you'll need to give Rhasspy some extra information.

Add to your profile:

"home_assistant": {
  ...
  "pem_file": "/path/to/certfile",
  "key_file": "/path/to/keyfile"
}

Set home_assistant.pem_file to the full path to your PEM certificate file. If your key is separate, set home_assistant.key_file as well.

Use the environment variable RHASSPY_PROFILE_DIR to reference your current profile's directory. For example, $RHASSPY_PROFILE_DIR/my.pem will tell Rhasspy to use a file named my.pem in your profile directory when verifying your self-signed certificate.

Remote Server

Rhasspy can POST the intent JSON to a remote URL.

Add to your profile:

"handle": {
  "system": "remote",
  "remote": {
      "url": "http://<address>:<port>/path/to/endpoint"
  }
}

When an intent is recognized, Rhasspy will POST to handle.remote.url with the intent JSON. Your server should return JSON back, optionally with additional information (see below).

Implemented by rhasspy-remote-http-hermes

Speech

If the returned JSON contains a "speech" key like this:

{
  ...
  "speech": {
    "text": "Some text to speak."
  }
}

then Rhasspy will forward speech.text to the configured text to speech system using a hermes/tts/say message.

Command

Once an intent is successfully recognized, can call a custom program to handle it.

Add to your profile:

"handle": {
  "system": "command",
  "command": {
      "program": "/path/to/program",
      "arguments": []
  }
}

When an intent arrives, Rhasspy will call your custom program with the intent JSON printed to standard in. You should return JSON to standard out, optionally with additional information (see below).

The following environment variables are available to your program:

  • $RHASSPY_BASE_DIR - path to the directory where Rhasspy is running from
  • $RHASSPY_PROFILE - name of the current profile (e.g., "en")
  • $RHASSPY_PROFILE_DIR - directory of the current profile (where profile.json is)

See handle.sh or handle.py for example programs.

Speech

If the returned JSON contains a "speech" key like this:

{
  ...
  "speech": {
    "text": "Some text to speak."
  }
}

then Rhasspy will forward speech.text to the configured text to speech system using a hermes/tts/say message.

Implemented by rhasspy-remote-http-hermes

Dummy

Disables intent handling.

Add to your profile:

"handle": {
  "system": "dummy"
}