Check the full documentation with tutorials and references on [https://wit.ai/docs](https://wit.ai/docs) # Why Wit ## Purpose More and more app users want to express their intent with natural language, instead of clicking buttons, filling forms, or having to follow complex steps. They would like to just say or type what they want the app to perform. Wit enables developers to add a modern natural language interface to their app or device with minimal effort. Precisely, Wit turns sentences into structured information that the app can use. Developers don’t need to worry about Natural Language Processing algorithms, configuration data, performance and tuning. Wit encapsulates all this and lets you focus on the core features of your apps and devices. ## Use cases Known use cases include: * SMS-based information or remote control services * IM-based information or remote control services * "Quick add" features a la Google Calendar (replacing a form with free text input) * Natural Language querying a la Facebook Graph Search (turning a sentence into a database query) * Personal Assistants a la Apple’s Siri * App voice control with natural commands as opposed to strict-syntax commands (using Wit on top of Android’s voice recognition for instance) * Robot dialog interfaces ## Web console and API There are two ways to interact with Wit: * A web console ([https://console.wit.ai](https://console.wit.ai)) where you (the developer) configure what you want Wit to do for you ![image alt text](https://wit.ai/docs/images/intro/console.png) * An API ([https://api.wit.ai](https://api.wit.ai)) that your app will call to send natural language sentences (text) and get structured information (JSON) in return. ```bash → curl \ > -H 'Authorization: Bearer B2QHJX7UFGF74O4MHX7UC54SSI4NLM5P' \ > 'https://api.wit.ai/message?q=i%20need%20a%20bud%20right%20now%21' ``` ```json { "msg_id": "d953bd6c-c620-4dae-a3fc-7634b4330073", "msg_body": "i need a bud right now!", "outcome": { "intent": "grab_me_something", "entities": { "object_to_grab": { "value": "beer", "start": 9, "end": 12, "body": "bud" } }, "confidence": 0.6310633902098893 } } ``` # Determining user intent ## Intents Wit’s first goal is to determine the user intent. An intent is "something the user wants to do", like: * Ask about the weather at a certain place * Ask about the result of a sports game * Create a reminder in their agenda * Ask about their schedule for a certain day * Order their house to turn on the lights * Ask how many friends posted a status update the day before * Say hello In the context of your app, you’ll focus on a finite list of possible user intents. You define this list. Typically, each intent corresponds to one action in your app. ![image alt text](https://wit.ai/docs/images/intro/intents.png) ## Expressions Each intent may be expressed in an infinite number of ways by the end-user (actually, even with thousands of occurrences, it’s very rare to observe the exact same sentence twice). We call them *expressions*. For instance, the intent `Set Alarm` may have the following expressions: * "Wake me up at 5 tomorrow" * "Set the alarm at 6am" * "I'd like you to wake me up by midnight" Given a user *expression*, Wit tries to determine the user *intent*. **Teaching by example** Wit learns from examples. So for each intent, you must provide Wit with a few expressions. Based on this (and on all the things Wit already know in the background), Wit will be able to determine if a never-seen-before expression maps to the same intent. Click on an intent in the Intents screen to see or add expressions. ![image alt text](https://wit.ai/docs/images/intro/intent-setalarm.png) Once you have defined a few intents (if you only set one intent, Wit will map all expressions to this intent, so you must set at least two intents if you want to try) and a few expressions for each intent, you can check if Wit already understands never seen before expressions. ## Trying out new sentences Type a test sentence in the "Try out sentence..." field in the top banner, press Enter and see the result. ![image alt text](https://wit.ai/docs/images/intro/eval.png) Did Wit find the right intent? * If it did, great! Reward it by clicking "Validate". Wit will remember this expression as another example for the intent, which will improve its accuracy. * If it’s wrong, no worry! Wit needs to learn a few more examples. Select the right intent in the dropdown and validate, Wit will remember that. **Protip** — Actually, you can even create a new intent right from the dropdown. Just type the new intent name and press Enter. ## Training status You’ll also notice a (normally) green dot in the top banner. ![image alt text](https://wit.ai/docs/images/intro/statusgreen.png) The color code is: * Green: Wit is "clean", indicating that all the examples you provided have been taken into account. * Orange: Wit is "dirty". The latest examples you provided will be taken into account in a few minutes (click on the dot to see exactly when). Don’t be mad at Wit if it does not understand something until then. # Intents with entities ## What’s an entity? So far, we’ve had simple intents. In the real world, most intents bear some kind of additional information. Examples: Expression | Intent | **Entities** -----------------------------|-------------------|------------- "Set the temperature to 68°" | Set Thermostat | **temperature = 68** "Wake me up tomorrow at 5am" | Set Alarm | **datetime = 2013-07-04T050000Z** In addition to determining the user intent, Wit tries to capture and normalize this additional information for you. ## Declaring entities When you create an intent, you explicitely indicate which entities you want to capture. Let's say we have a `Set Alarm` intent for which we want to capture the date/time the user wants to set the alarm at. 1. Type a sentence like "I need to wake up at midnight" in the “Try out...” field in the top banner. Press Enter. ![image alt text](https://wit.ai/docs/images/intro/eval.png) 2. Make sure to select (or create) an intent in the Intent dropdown. 3. Using your mouse cursor, select "at midnight" in the sentence (the span that corresponds to the value of the entity you want to capture). You'll see the list of built-in entities that Wit knows how to capture. ![image alt text](https://wit.ai/docs/images/intro/ent1.png) 4. Type "date" to filter the list and click on "wit/datetime". ![image alt text](https://wit.ai/docs/images/intro/ent2.png) 5. You’ll notice that Wit should have properly resolved the date/time interval corresponding to "at midnight" ![image alt text](https://wit.ai/docs/images/intro/ent3.png) 6. Click "Validate". You’ve just instructed Wit to try to capture a `wit/datetime` entity when it recognizes a `Set Alarm` intent. Note that, at the same time, you also provided a new expression for this intent. ![image alt text](https://wit.ai/docs/images/intro/ent4.png) If you provide a few more expressions, and once Wit is trained, you’ll notice that when you try out "wake up next Thursday at noon" Wit should automatically capture the `wit/datetime` entity with the right value. ![image alt text](https://wit.ai/docs/images/intro/ent5.png) Entities are always bound to an intent. That means that if Wit has detected a date, for instance, it won’t assign it to an entity unless the expression is mapped to an intent that bears a `wit/datetime` entity. ## Built-in entities Whenever possible, always use built-in entities (prefixed by **wit/**). These entities are trained on large datasets and should perform better than you own entities, which are trained just on the expressions you provide. Choose your entities carefully (don't hesitate to ask us if you are not sure). Read the documentation string in the right column. Here are the most important built-in entities: ### wit/datetime Wit recognizes and resolves many date and time expressions, like: * "today" * "tomorrow at 8am" * "next Monday" * "in ten days in the afternoon" * "4 minutes ago" Wit works with intervals: * "9am" is resolved as ‘from 09:00:00 to 10:00:00’ * "9:00am" is resolved as ‘from 09:00:00 to 09:01:00’ Our motivation is that when the user says 9:00, they really mean that precision is important. This is not the same as 9am alone. We don’t want to drop this information. If you don’t agree with Wit’s interpretation of the value of your datetime, you can change the value in the form and validate. As of this alpha version, Wit won’t use this information during automatic training. But we’ll look at it manually and improve it. In a later version, this information will feed the training. If some dates seem to be off by one day, please read [the section about time zones and local dates](#time-zone-and-local-time). ### wit/number Use this entity to capture a number or a quantity in an expression. Example: * "Book a table for three." => **wit/number = 3** Currently, only integers are recognized. Decimal numbers are not recognized. ### wit/phrase\_to\_translate, wit/\*\_search\_query, wit/message\_body, wit/reminder (and more *free text* entities) Use these entities to capture a raw string of characters from within an expression. That's useful when you want to capture a message the user wants to write, a phrase they want to translate, an agenda entry text, etc. Examples: * "Translate I love you into Chinese" => **wit/phrase\_to\_translate = "I love you"** * "Remind me to buy the milk tomorrow at 8am" => **wit/reminder = "buy the milk"** **Protip** — These "free text" entities needs more examples than other entities to be trained properly, because there is not simple intrinsic patterns, like for dates or numbers, helping Wit to recognize the entity. Wit has to rely mostly on context (words and features before and after the span) in order to recognize these entities. ### wit/location Examples: * "Go to Market & 3rd" => **wit/location = "Market Street and 3rd Street, SF, CA, USA"** We are currently working on the location type. Wit will not only recognize locations in your sentences, but also resolve them to precise geographical data you can leverage. If you plan to use Wit for a location-centric service, we would be glad to hear from you to better serve your requirements. Please [get in touch with us](mailto:contact@wit.ai)! ## Developer-defined enum-based entities ### Enum-based entities In addition to the built-in entities, you can create your own entities. As of this alpha version, this is limited to enum-based entities. Examples: * "paint the cube in green" => **color = "green"** * "did the giants win?" => **MLB_team = "SF Giants”** ### Values and expressions An *enum-based entity* is composed of *values*. For instance, the values of the `MLB_teams` entity are the teams: `SF Giants`, `NY Mets`, etc. The value `SF Giants` corresponds to one or several *expressions*, which are different synonymical ways to refer to the value: “Giants”, “SF Giants”, “San Francisco Giants”, etc. When an enum-based entity is captured, its value is always one of the **values**, as opposed to an expression: * "did the San Francisco Giants win?" => **MLB_team = "SF Giants"** * "did the giants win?" => **MLB_team = "SF Giants"** ### Creating and using your own enum-based entities Let's say you want to create a "Grab me something" intent for your robot to bring you beer. For that, you'll need an enum-based entity corresponding to the list of things your robot can grab. 1. Type a first expression like "grab me a beer" in the try out field. Since your robot (and Wit) knows nothing about this intent yet, it thinks it's a "Set Alarm" intent (notice the info message though). ![image alt text](https://wit.ai/docs/images/intro/beer1.png) 2. Let's create our intent: type "Grab me something" and press Enter in the intent dropdown. ![image alt text](https://wit.ai/docs/images/intro/beer2.png) 3. Now we need to tell Wit about the entity. Select "beer" in the expression. ![image alt text](https://wit.ai/docs/images/intro/beer3.png) 4. There is no built-in entity for `object_to_grab`, so let's just type this and press Enter. ![image alt text](https://wit.ai/docs/images/intro/beer4.png) ![image alt text](https://wit.ai/docs/images/intro/beer5.png) 5. It looks good! Click Validate. Your `object_to_grab` entity has been created, and it has one value, `beer`, with one expression, "beer". 6. Let's wait a minute until our instance gets trained. Once the status light is green again, type "grab me a bud" in the try out field. You'll realize that the `Grab me something` intent has been recognized, but not the `object_to_grab` entity. Because Wit doesn't know (yet) that bud is a beer (well, there could be a debate about that "fact", but that's out of the scope of this introduction). ![image alt text](https://wit.ai/docs/images/intro/bud1.png) 7. Select the span "bud" in the expression, and then `object_to_grab` as the entity. ![image alt text](https://wit.ai/docs/images/intro/bud2.png) 8. Now you have `bud` as the value. If you want your robot to set just one canonical value `beer` for all beers, select `beer` in the dropdown. It means that a new expression "bud" will be mapped to our existing value `beer`. ![image alt text](https://wit.ai/docs/images/intro/bud3.png) ### Open enumerations and suggested values Even if enum-based entities are based on a list of values, Wit can make some guesses and find "new" values. For instance, if you have a `Set TV channel` intent that rely on a `TV_channel` entity, you might have trained Wit with a few dozens well-known TV channel names like "CNN", "PBS", etc. But one day, a user says "I want to watch KXXX", and "KXXX" is not in your list. If Wit has been trained with enough expressions, it should be able to guess that "KXXX" is probably a TV channel. The API response will contain the right entity: ```json { ... "entities": { "TV_channel": { "value": "KXXX", "body": "KXXX", "suggested": true } } } ``` Please note the `suggested: true` part. It means that Wit did not find "KXXX" in its list of TV channels, but guessed it was probably one anyway. Whether you accept or not this suggestion, it's your decision. For instance, you could ask the user to confirm their intent in this case. Finally, if you find the expression "I want to watch KXXX" in your Inbox and validate it, "KXXX" will be added to the `TV_channel` list. So next time, it won't be "suggested", and Wit will be a little bit smarter. ## Developer-defined general entities We are currently working on developer-defined non-enum-based entities. Unlike enum-based entities, you’ll be able to customize Wit’s both recognition and resolution strategies for your entities. If you need special entities right now, please [get in touch with us](mailto:contact@wit.ai). # API Now that you’ve trained your Wit instance, it’s time to integrate it into your app, get your feet wet and see how it behaves in the real world. Check out the [full API reference](https://wit.ai/docs/api). For now, if you just want to see the API in action: 1. Click on Settings in the nav bar on the left ![image alt text](https://wit.ai/docs/images/intro/settings.png) 2. Type a sentence and copy the curl command to your terminal ![image alt text](https://wit.ai/docs/images/intro/curl.png) ```bash → curl \ > -H 'Authorization: Bearer B2QHJX7UFGF74O4MHX7UC54SSI4NLM5P' \ > 'https://api.wit.ai/message?q=i%20need%20a%20bud%20right%20now%21' ``` ```json { "msg_id": "d953bd6c-c620-4dae-a3fc-7634b4330073", "msg_body": "i need a bud right now!", "outcome": { "intent": "grab_me_something", "entities": { "object_to_grab": { "value": "beer", "start": 9, "end": 12, "body": "bud" } }, "confidence": 0.6310633902098893 } } ``` # Continuous improvement: the Inbox Wit works bottom-up. Let your end-users express their intents freely, and have your app learn from that and adapt. Not the other way around. Here comes Wit’s Inbox. ![image alt text](https://wit.ai/docs/images/intro/inbox.png) Each time Wit receives an API call and processes a sentence, it adds it in your inbox. You can read each item there as "Hey, dear developer, I received this sentence and answered with this intent and these entities, was it right?". As a developer and (in this context) educator, your job is to either reward (click Validate) or correct (edit the outcome, and Validate). Either way, Wit learns from your action. **Protip** — If your inbox is empty and you want to see what it looks like, [make a curl call to the API](#api) This is the preferred way to improve your Wit configuration. Try to keep an empty Wit inbox. Actually you can even create intents and entities straight from your Inbox. Sometimes, you may get irrelevant sentences in your inbox. Just click "Dismiss" to make them disappear. # Advanced topics ## Time zone and local time When Wit resolves certain `wit/datetime` entities, it needs a local time. For instance at midnight UTC, "today" won’t be resolved in the same way for a user based in UTC-01 and another based in UTC+01. By default, Wit uses the time zone defined in your instance settings (click Settings in the nav bar in Wit console). If an API call comes without any additional information about time zone or local time, a local time is inferred from this default time zone (and the UTC time given by our server). If you want to provide Wit with a specific time zone or local time for a given API call, use the `context` field. Check out [the full reference in the API documentation](https://wit.ai/docs/api#context).