How To Build a Sentiment Analysis App

It is often necessary to analyze texts and detect whether they have a positive, negative, or neutral tone. This task is called sentiment analysis in natural language processing. It is especially important, for example, for analyzing a customer’s feedback automatically. Luckily, there is an API that can perform sentiment analysis for your applications out of the box.

How to get access to the Sentiment Analysis API

The Sentiment Analysis API (by Twinword) is available on the RapidAPI platform. To start using the API, you should do the following:

1. Create a RapidAPI user account.

RapidAPI is the largest API marketplace. It simplifies searching through all the necessary APIs, exploring them, and using them in the development and production. It is absolutely free to sign up for RapidAPI.

2. Navigate to the Sentiment Analysis API page on the RapidAPI platform.

To do this, you can use the search bar, or browse the categories menu:

You can also use the direct link to the API.

3. Subscribe to the Sentiment Analysis API.

To start using the API, you need to choose a suitable pricing plan. To do this, click on the Pricing tab and select the plan that best suits your needs. If you want to explore the API’s features first, you can subscribe to the Basic plan that provides 500 free requests/month.

Now you are ready to use the Sentiment Analysis API.

How to work with the Sentiment Analysis API

Every API on the RapidAPI platform has a unique structure. Let’s explore it a little bit before jumping into real coding.

Go to the Endpoints tab. It is probably the most important place on every API’s page. On the left side of the screen, you should see the list with the available endpoints. There are only two endpoints supported by the Sentiment Analysis API:

Actually, both endpoints do the same thing, but the first is for the POST HTTP requests, and the second processes GET requests.

When you click on the Analyze (get) endpoint, the following picture should appear in the central part of the screen:

There you can see which headers and required parameters you should include in your requests (as well as optional parameters, if any). Pay attention to X-RapidAPI-Host and X-RapidAPI-Key header parameters. The first one points to the particular API on the RapidAPI platform. The second is unique for each user’s account. This is important for billing and statistics (you can view interesting dashboards about your API usage patterns in the My Apps section of the website).

The only required body parameter for the Sentiment Analysis API is text. There you should place the text that you want to analyze and detect the sentiment of. 

The right side of the screen contains the code snippet and the response example (see below). You can select different programming languages and HTTP libraries to see a sample. If you want to change the text you want to analyze, you should enter it in the corresponding field in the central part of the screen and then press the Test Endpoint button to update the code snippet and response example.

You can see that the response is the JSON object. The most interesting fields are type, score, and keywords. The type field contains information about sentiment (positive, negative, or neutral). It is detected based on the score value that varies from -1 to 1. If it falls in a range between -0.05 to 0.05, then the API concludes that the sentiment is negative. All larger score values are considered as positive sentiment and all smaller values are seen as negative. The keywords field contains a list with the most important words for the sentiment analysis also includes and the sentiment score for every single word.

How to use the Sentiment Analysis API with Python & Django

Now we are going to show you how to create a basic website that will use the sentiment analysis feature of the API. We will use a well-known Django web framework and Python 3.6.

The idea of the web application is the following: Users will leave their feedback (reviews) on the website. It is expected that the number of user comments will be vast. But our company has a limited number of specialists in the user support team. So they need to decide which user requests they should process first. The reasonable tactic is to first communicate with customers who had some issues with our products and therefore left negative feedback. So, in the section of our website designated for internal support staff, we want to show the reviews with the most negative sentiment at the top of the page.

As we are going to show only the example of using the Sentiment Analysis API in your application, we will not develop a full-featured website. There will be a page with the web form that users will use to leave their feedback. The API will detect the sentiment of the comments before they are saved. The feedback data will be stored in the database along with the sentiment scores. Finally, we will show the comments sorted by the sentiment score on the page that is accessible for our support team employees.

To start a new project in Python it is recommended to create a virtual environment:

python -m venv venv

Then activate the created virtual environment and install Django. After you install it, create a skeleton of the project using the startproject command of django-admin.

python -m pip install Django
django-admin startproject website

Django projects consist of applications. We want to create an application with the name sentiment inside the website project (run this command from the directory where the manage.py file is located):

python manage.py startapp sentiment

Now we can write the code. The first thing you need to do is to create a Django model, which is the way you interact with the database (object-relational mapping – ORM). With our specific aim in mind, we need to create a database table Review, where the user comments will be stored. In the Django app, models should be placed in the file models.py. Here is the code we placed in this file (it is located in the sentiment directory, as well as all further files if otherwise is not specified):

from django.db import models


class Review(models.Model):
   author = models.CharField(max_length=150)
   author_email = models.EmailField()
   review_title = models.CharField(max_length=300, null=True)
   review_body = models.TextField(blank=True)
   creation_date = models.DateTimeField(auto_now_add=True, blank=True)
   sentiment_score = models.DecimalField(max_digits=5, decimal_places=4)

   def is_critical(self):
       return True if self.sentiment_score < -0.2 else False

   def __str__(self):
       return self.review_title

Python’s class Review represents the table in the database. Inside the class, we have created the range of needed fields for reviews: is_critical() which is the function for determining whether the review should be processed with priority, and the representation __str__() function (not necessary, but useful in the development and debugging).

We want to create reviews via the web form, so let’s create the form in the forms.py file:

from django.forms import ModelForm, EmailField, CharField, Textarea

from .models import Review


class ReviewForm(ModelForm):
   author_email = EmailField(max_length=80, help_text='Required', required=True, label="Email")
   author = CharField(max_length=150, label='Your name')
   review_title = CharField(max_length=300, label='Title', required=True)
   review_body = CharField(widget=Textarea({}), label='Feedback', required=True)

   class Meta:
       model = Review
       fields = ['author', 'author_email', 'review_title', 'review_body', 'sentiment_score']

Our form is based on the Review model (we defined this in the Meta class). Also, we tuned some field parameters by adding labels, max length setting, etc.

The next step is to create a urls.py file and specify web addresses that are accessible on the website. Each web path should have a corresponding view (the backend logic). Each route also needs to be given a name. Those names can be used to conveniently refer to the specific route. See the urls.py file below:

from django.urls import path

from . import views

urlpatterns = [
   path('', views.IndexView.as_view(), name='index'),
   path('review_sent/', views.FeedbackCreateView.as_view(), name='review_sent'),
   path('reviews/', views.ReviewsListView.as_view(), name='reviews'),
   path('reviews/<int:pk>/', views.ReviewDetailsView.as_view(), name='review_details'),
]

We created four routes (index, review_sent, reviews, and review_details). In order to make our routes active, we should also modify the website/urls.py file. Here is its content:

from django.contrib import admin
from django.urls import path, include

urlpatterns = [
   path('admin/', admin.site.urls),
   path('', include('sentiment.urls')),
]

As you can see, we have included the sentiment.urls element to the urlpatterns variable. Another component is the Django admin panel (this component was preconfigured).

Now let’s explore the views.py file. It is quite large, so we will explain it section-by-section. At the top of the file, we import all the libraries and objects we need to work with. This is also where we define IndexView:

import requests

from django.shortcuts import render, redirect
from django.contrib import messages
from django.views.generic.base import View
from django.views.generic import DetailView, ListView

from .models import Review
from .forms import ReviewForm


class IndexView(View):
   def get(self, request):
       return render(request, 'sentiment/index.html')

IndexView has simple logic: when receiving the GET request, it renders the index.html page. Here is how this HTML file looks:

<h1>Home page</h1>

<hr>

<p>What do you want to do?</p>

<br>

<a href="{% url 'review_sent' %}">Leave a review about our products</a>

<p></p>

<a href="{% url 'reviews' %}">Check pending reviews</a>

It is not a complete HTML code, but for the purpose of this demonstration, it should be enough. The home page asks users what they want to do. There are two options: go to the page with the feedback form or see the already existing reviews. The second option is for the support team members and it should be accessible only for the users with special permission. Nevertheless, Django authentication features are not the scope of our tutorial, so we will not check users that want to visit this page. Note how the href attribute is defined. It uses the power of the template: we specify just the name of the path (from the urls.py file), not the absolute URL.

The next view class is the FeedbackCreateView. It is responsible for displaying the form for review creation (when the request is GET) and for processing the POST request after the form is submitted. The template you’ll need for this is the feedback_create.html.

class FeedbackCreateView(View):
   api_url = "https://twinword-sentiment-analysis.p.rapidapi.com/analyze/"
   api_headers = {
       'x-rapidapi-host': "twinword-sentiment-analysis.p.rapidapi.com",
       'x-rapidapi-key': "<YOUR-RAPIDAPI-KEY>",
       'content-type': "application/x-www-form-urlencoded"
       }

   def get(self, request):
       review_form = ReviewForm(initial={'sentiment_score': -2.0})
       return render(request, 'sentiment/feedback_create.html', {'review_form': review_form})

   def post(self, request):
       review_form = ReviewForm(request.POST)

       if review_form.is_valid():
           review_instance = review_form.save(commit=False)

           text = review_instance.review_body
           payload = {'text': text}

           response = requests.request("POST", self.api_url, data=payload, headers=self.api_headers)

           review_instance.sentiment_score = response.json()['score']
           review_instance.save()

           messages.success(request, 'Form submission was successful')
           return redirect('review_sent')

       review_form = ReviewForm()
       return render(request, 'sentiment/feedback_create.html', {'review_form': review_form})

The post() method is the most interesting and also the most important part of our application. If the submitted form is valid, we generate an API query using the text from the review_body field. Then we fetch the score field of the response, write this score to the sentiment_score field of the review_instance object, and save this object. After this, the user is shown a success message. When creating the query to the API, we use the api_url and api_headers variables that we defined at the top of the class. Make sure that you specify your own RapidAPI key there.

Here is the feedback_create.html template:

<a href="{% url 'index' %}">Home</a>

<h1>Send us your feedback!</h1>

{% if messages %}
<ul>
   {% for message in messages %}
   <li style="color: green"> {{ message }} </li>
   {% endfor %}
</ul>
{% endif %}

<hr>

<form method="post" action="{% url 'review_sent' %}" >

{% csrf_token %}

<table>
   <tr>
       <td>{{ review_form.sentiment_score.label_tag.as_hidden }}</td>
       <td>{{ review_form.sentiment_score.as_hidden }}</td>
   </tr>

   <tr>
       <td>{{ review_form.author.label_tag }}</td>
       <td>{{ review_form.author }}</td>
   </tr>

   <tr>
       <td>{{ review_form.author_email.label_tag }}</td>
       <td>{{ review_form.author_email }}</td>
   </tr>

   <tr>
       <td>{{ review_form.review_title.label_tag }}</td>
       <td>{{ review_form.review_title }}</td>
   </tr>

   <tr>
       <td>{{ review_form.review_body.label_tag }}</td>
       <td>{{ review_form.review_body }}</td>
   </tr>
</table>

<input type="submit" value="Send">

</form>

At the top of the page, we display the messages (if any) and then create the form. The sentiment_score field is presented, but hidden. We want to compute the value for this field in the view using the Sentiment Analysis API.

Below you can see the remaining two views: ReviewsListView and ReviewDetailsView. The first is responsible for creating the reviews_list.html template. It shows the reviews that need to be processed by the support team. The second view creates the review_details.html and displays the details about an individual review (like the body of the review, title, creation date, author, etc.

class ReviewsListView(ListView):
   template_name = 'sentiment/reviews_list.html'
   context_object_name = 'reviews_list'

   def get_queryset(self):
       return Review.objects.order_by('sentiment_score')


class ReviewDetailsView(DetailView):
   model = Review
   template_name = 'sentiment/review_details.html'

The reviews_list.html template:

<a href="{% url 'index' %}">Home</a>

<h1>The reviews list</h1>

{% if reviews_list %}

   <p>Critical reviews are highlighted with red. Please, process them first.</p>

   <ul>
   {% for review in reviews_list %}
       <li><a {% if review.is_critical %} style="color: red"  {% endif %}
              href="{% url 'review_details' pk=review.pk %}">{{ review.review_title }}</a></li>
   {% endfor %}
   </ul>
{% else %}
   <p>No reviews are available.</p>
{% endif %}

The peculiarity of the template is that if the review is critical it should be highlighted in red. 

The review_details.html template:

<a href="{% url 'index' %}">Home</a>

<h1>Review details</h1>

<p>Author: <i>{{review.author}}</i></p>
<p>Email: <i>{{review.author_email}}</i></p>
<p>Created at: <i>{{review.creation_date}}</i></p>
<p>Sentiment score: <i>{{review.sentiment_score}}</i></p>

{% if review.is_critical %}
<p style="color: red"> CRITICAL </p>
{% endif %}

<h3>Title: {{ review.review_title }} </h3>

<p> {{ review.review_body }} </p>

Now we have all the code we need to run our app. But before running, we need to create the database and the Review table inside it. In Django, we can do this using migration. 

First, go to the settings.py file and make sure that the INSTALLED_APPS variable has the element ‘sentiment.apps.SentimentConfig’. Save the file.

Then, create migrations by issuing the following command from the Terminal (when you are in the directory where the manage.py file is located):

python manage.py makemigrations sentiment

Next, apply the migrations:

python manage.py migrate

This will create all internal tables in the database, as well as the Review table. You don’t need to worry about the database because Django uses SQLite database by default and it is enough for our project. 

To start running the application, use this command:

python manage.py runserver

You should see something like this in the Terminal:

Let’s test our application now.

Sentiment Analysis App

In your browser, follow the following link http://localhost:8000/. You should see the home page:

Click the first link to go to the review creation form. Let’s create a positive review:

Click the Send button. A success message should appear:

After this, we created two more reviews. Then we went to the home page, and then clicked on the second link. Here is what we have there:

As you can see, the second review is highlighted with red and it is at the top of the list. The third review is in the middle and not highlighted, and the positive review is at the bottom.

Here is the most negative review:

Note that the sentiment score detected by the Sentiment Analysis API is around -0.23. When the score is less than 0.2, we highlight this review with red and put the CRITICAL mark in the review details. The third review is rather neutral (the sentiment score is 0.009) and the first review is positive (with the sentiment score around 0.42). Reviews are also ordered by the sentiment score in ascending order, with the most negative reviews on top of the list.

Now the customer support team might find it easier to understand which reviews they need to process first!

Conclusion

Sentiment analysis can be a valuable feature for a wide range of companies, applications, and use cases. In fact, there should be a place for sentiment analysis in most businesses that work with people as their customers (hotels, cinemas, travel companies, airlines, retail and e-commerce shops, etc.).

We hope that our tutorial was interesting and put some light on the topic of how you can seamlessly embed sentiment analysis into your application. Also, please note that the RapidAPI platform has many other powerful APIs for natural language processing that could be helpful for you.