Check the full documentation with tutorials and references on https://wit.ai/docs

Why Wit

Purpose

More and more app users want to express their intent with natural language, instead of clicking buttons, filling forms, or having to follow complex steps. They would like to just say or type what they want the app to perform.

Wit enables developers to add a modern natural language interface to their app or device with minimal effort. Precisely, Wit turns sentences into structured information that the app can use.

Developers don’t need to worry about Natural Language Processing algorithms, configuration data, performance and tuning. Wit encapsulates all this and lets you focus on the core features of your apps and devices.

Use cases

Known use cases include:

• SMS-based information or remote control services

• IM-based information or remote control services

• “Quick add” features a la Google Calendar (replacing a form with free text input)

• Natural Language querying a la Facebook Graph Search (turning a sentence into a database query)

• Personal Assistants a la Apple’s Siri

• App voice control with natural commands as opposed to strict-syntax commands (using Wit on top of Android’s voice recognition for instance)

• Robot dialog interfaces

Web console and API

There are two ways to interact with Wit:

• A web console (https://console.wit.ai) where you (the developer) configure what you want Wit to do for you
  

• An API (https://api.wit.ai) that your app will call to send natural language sentences (text) and get structured information (JSON) in return.

→ curl \
>     -H 'Authorization: Bearer B2QHJX7UFGF74O4MHX7UC54SSI4NLM5P' \
>     'https://api.wit.ai/message?q=i%20need%20a%20bud%20right%20now%21'

{
  "msg_id": "d953bd6c-c620-4dae-a3fc-7634b4330073",
  "msg_body": "i need a bud right now!",
  "outcome": {
    "intent": "grab_me_something",
    "entities": {
      "object_to_grab": {
        "value": "beer",
        "start": 9,
        "end": 12,
        "body": "bud"
      }
    },
    "confidence": 0.6310633902098893
  }
}

Determining user intent

Intents

Wit’s first goal is to determine the user intent. An intent is “something the user wants to do”, like:

• Ask about the weather at a certain place

• Ask about the result of a sports game

• Create a reminder in their agenda

• Ask about their schedule for a certain day

• Order their house to turn on the lights

• Ask how many friends posted a status update the day before

• Say hello

In the context of your app, you’ll focus on a finite list of possible user intents. You define this list. Typically, each intent corresponds to one action in your app.

Expressions

Each intent may be expressed in an infinite number of ways by the end-user (actually, even with thousands of occurrences, it’s very rare to observe the exact same sentence twice). We call them expressions.

For instance, the intent Set Alarm may have the following expressions:

• “Wake me up at 5 tomorrow”

• “Set the alarm at 6am”

• “I’d like you to wake me up by midnight”

Given a user expression, Wit tries to determine the user intent.

Teaching by example

Wit learns from examples. So for each intent, you must provide Wit with a few expressions. Based on this (and on all the things Wit already know in the background), Wit will be able to determine if a never-seen-before expression maps to the same intent.

Click on an intent in the Intents screen to see or add expressions.

Once you have defined a few intents (if you only set one intent, Wit will map all expressions to this intent, so you must set at least two intents if you want to try) and a few expressions for each intent, you can check if Wit already understands never seen before expressions.

Trying out new sentences

Type a test sentence in the “Try out sentence…” field in the top banner, press Enter and see the result.

Did Wit find the right intent?

• If it did, great! Reward it by clicking “Validate”. Wit will remember this expression as another example for the intent, which will improve its accuracy.

• If it’s wrong, no worry! Wit needs to learn a few more examples. Select the right intent in the dropdown and validate, Wit will remember that.

Protip — Actually, you can even create a new intent right from the dropdown. Just type the new intent name and press Enter.

Training status

You’ll also notice a (normally) green dot in the top banner.

The color code is:

• Green: Wit is “clean”, indicating that all the examples you provided have been taken into account.

• Orange: Wit is “dirty”. The latest examples you provided will be taken into account in a few minutes (click on the dot to see exactly when). Don’t be mad at Wit if it does not understand something until then.

Intents with entities

What’s an entity?

So far, we’ve had simple intents. In the real world, most intents bear some kind of additional information. Examples:

Expression	Intent	Entities
“Set the temperature to 68°”	Set Thermostat	temperature = 68
“Wake me up tomorrow at 5am”	Set Alarm	datetime = 2013-07-04T050000Z

In addition to determining the user intent, Wit tries to capture and normalize this additional information for you.

Declaring entities

When you create an intent, you explicitely indicate which entities you want to capture. Let’s say we have a Set Alarm intent for which we want to capture the date/time the user wants to set the alarm at.

1. Type a sentence like “I need to wake up at midnight” in the “Try out…” field in the top banner. Press Enter.
   

2. Make sure to select (or create) an intent in the Intent dropdown.

3. Using your mouse cursor, select “at midnight” in the sentence (the span that corresponds to the value of the entity you want to capture). You’ll see the list of built-in entities that Wit knows how to capture.
   

4. Type “date” to filter the list and click on “wit/datetime”.
   

5. You’ll notice that Wit should have properly resolved the date/time interval corresponding to “at midnight”
   

6. Click “Validate”.

You’ve just instructed Wit to try to capture a wit/datetime entity when it recognizes a Set Alarm intent. Note that, at the same time, you also provided a new expression for this intent.

If you provide a few more expressions, and once Wit is trained, you’ll notice that when you try out “wake up next Thursday at noon” Wit should automatically capture the wit/datetime entity with the right value.

Entities are always bound to an intent. That means that if Wit has detected a date, for instance, it won’t assign it to an entity unless the expression is mapped to an intent that bears a wit/datetime entity.

Built-in entities

Whenever possible, always use built-in entities (prefixed by wit/). These entities are trained on large datasets and should perform better than you own entities, which are trained just on the expressions you provide.

Choose your entities carefully (don’t hesitate to ask us if you are not sure). Read the documentation string in the right column. Here are the most important built-in entities:

wit/datetime

Wit recognizes and resolves many date and time expressions, like:

• “today”

• “tomorrow at 8am”

• “next Monday”

• “in ten days in the afternoon”

• “4 minutes ago”

Wit works with intervals:

• “9am” is resolved as ‘from 09:00:00 to 10:00:00’

• “9:00am” is resolved as ‘from 09:00:00 to 09:01:00’

Our motivation is that when the user says 9:00, they really mean that precision is important. This is not the same as 9am alone. We don’t want to drop this information.

If you don’t agree with Wit’s interpretation of the value of your datetime, you can change the value in the form and validate. As of this alpha version, Wit won’t use this information during automatic training. But we’ll look at it manually and improve it. In a later version, this information will feed the training.

If some dates seem to be off by one day, please read the section about time zones and local dates.

wit/number

Use this entity to capture a number or a quantity in an expression.

Example:

• “Book a table for three.” => wit/number = 3

Currently, only integers are recognized. Decimal numbers are not recognized.

wit/phrase_to_translate, wit/*_search_query, wit/message_body, wit/reminder (and more free text entities)

Use these entities to capture a raw string of characters from within an expression. That’s useful when you want to capture a message the user wants to write, a phrase they want to translate, an agenda entry text, etc.

Examples:

• “Translate I love you into Chinese” => wit/phrase_to_translate = “I love you”

• “Remind me to buy the milk tomorrow at 8am” => wit/reminder = “buy the milk”

Protip — These “free text” entities needs more examples than other entities to be trained properly, because there is not simple intrinsic patterns, like for dates or numbers, helping Wit to recognize the entity. Wit has to rely mostly on context (words and features before and after the span) in order to recognize these entities.

wit/location

Examples:

• “Go to Market & 3rd” => wit/location = “Market Street and 3rd Street, SF, CA, USA”

We are currently working on the location type. Wit will not only recognize locations in your sentences, but also resolve them to precise geographical data you can leverage. If you plan to use Wit for a location-centric service, we would be glad to hear from you to better serve your requirements. Please get in touch with us!

Developer-defined enum-based entities

Enum-based entities

In addition to the built-in entities, you can create your own entities. As of this alpha version, this is limited to enum-based entities.

Examples:

• “paint the cube in green” => color = “green”

• “did the giants win?” => MLB_team = "SF Giants”

Values and expressions

An enum-based entity is composed of values. For instance, the values of the MLB_teams entity are the teams: SF Giants, NY Mets, etc.

The value SF Giants corresponds to one or several expressions, which are different synonymical ways to refer to the value: “Giants”, “SF Giants”, “San Francisco Giants”, etc.

When an enum-based entity is captured, its value is always one of the values, as opposed to an expression:

• “did the San Francisco Giants win?” => MLB_team = “SF Giants”

• “did the giants win?” => MLB_team = “SF Giants”

Creating and using your own enum-based entities

Let’s say you want to create a “Grab me something” intent for your robot to bring you beer. For that, you’ll need an enum-based entity corresponding to the list of things your robot can grab.

1. Type a first expression like “grab me a beer” in the try out field. Since your robot (and Wit) knows nothing about this intent yet, it thinks it’s a “Set Alarm” intent (notice the info message though).
   

2. Let’s create our intent: type “Grab me something” and press Enter in the intent dropdown.
   

3. Now we need to tell Wit about the entity. Select “beer” in the expression.
   

4. There is no built-in entity for object_to_grab, so let’s just type this and press Enter.
   
   

5. It looks good! Click Validate. Your object_to_grab entity has been created, and it has one value, beer, with one expression, “beer”.

6. Let’s wait a minute until our instance gets trained. Once the status light is green again, type “grab me a bud” in the try out field. You’ll realize that the Grab me something intent has been recognized, but not the object_to_grab entity. Because Wit doesn’t know (yet) that bud is a beer (well, there could be a debate about that “fact”, but that’s out of the scope of this introduction).
   

7. Select the span “bud” in the expression, and then object_to_grab as the entity.
   

8. Now you have bud as the value. If you want your robot to set just one canonical value beer for all beers, select beer in the dropdown. It means that a new expression “bud” will be mapped to our existing value beer.
   

Open enumerations and suggested values

Even if enum-based entities are based on a list of values, Wit can make some guesses and find “new” values. For instance, if you have a Set TV channel intent that rely on a TV_channel entity, you might have trained Wit with a few dozens well-known TV channel names like “CNN”, “PBS”, etc. But one day, a user says “I want to watch KXXX”, and “KXXX” is not in your list. If Wit has been trained with enough expressions, it should be able to guess that “KXXX” is probably a TV channel. The API response will contain the right entity:

{
  ...
  "entities": {
    "TV_channel": {
      "value": "KXXX",
      "body": "KXXX",
      "suggested": true
    }
  }
}

Please note the suggested: true part. It means that Wit did not find “KXXX” in its list of TV channels, but guessed it was probably one anyway. Whether you accept or not this suggestion, it’s your decision. For instance, you could ask the user to confirm their intent in this case.

Finally, if you find the expression “I want to watch KXXX” in your Inbox and validate it, “KXXX” will be added to the TV_channel list. So next time, it won’t be “suggested”, and Wit will be a little bit smarter.

Developer-defined general entities

We are currently working on developer-defined non-enum-based entities. Unlike enum-based entities, you’ll be able to customize Wit’s both recognition and resolution strategies for your entities.

If you need special entities right now, please get in touch with us.

API

Now that you’ve trained your Wit instance, it’s time to integrate it into your app, get your feet wet and see how it behaves in the real world.

Check out the full API reference.

For now, if you just want to see the API in action:

1. Click on Settings in the nav bar on the left
   

2. Type a sentence and copy the curl command to your terminal
   

→ curl \
>     -H 'Authorization: Bearer B2QHJX7UFGF74O4MHX7UC54SSI4NLM5P' \
>     'https://api.wit.ai/message?q=i%20need%20a%20bud%20right%20now%21'

{
  "msg_id": "d953bd6c-c620-4dae-a3fc-7634b4330073",
  "msg_body": "i need a bud right now!",
  "outcome": {
    "intent": "grab_me_something",
    "entities": {
      "object_to_grab": {
        "value": "beer",
        "start": 9,
        "end": 12,
        "body": "bud"
      }
    },
    "confidence": 0.6310633902098893
  }
}

Continuous improvement: the Inbox

Wit works bottom-up. Let your end-users express their intents freely, and have your app learn from that and adapt. Not the other way around.

Here comes Wit’s Inbox.

Each time Wit receives an API call and processes a sentence, it adds it in your inbox. You can read each item there as “Hey, dear developer, I received this sentence and answered with this intent and these entities, was it right?”. As a developer and (in this context) educator, your job is to either reward (click Validate) or correct (edit the outcome, and Validate). Either way, Wit learns from your action.

Protip — If your inbox is empty and you want to see what it looks like, make a curl call to the API

This is the preferred way to improve your Wit configuration. Try to keep an empty Wit inbox. Actually you can even create intents and entities straight from your Inbox.

Sometimes, you may get irrelevant sentences in your inbox. Just click “Dismiss” to make them disappear.

Advanced topics

Time zone and local time

When Wit resolves certain wit/datetime entities, it needs a local time. For instance at midnight UTC, “today” won’t be resolved in the same way for a user based in UTC-01 and another based in UTC+01.

By default, Wit uses the time zone defined in your instance settings (click Settings in the nav bar in Wit console). If an API call comes without any additional information about time zone or local time, a local time is inferred from this default time zone (and the UTC time given by our server).

If you want to provide Wit with a specific time zone or local time for a given API call, use the context field. Check out the full reference in the API documentation.