Table of Contents
If you are that person who likes to track all the latest trends happening across the world, then what is your primary source of information? Undoubtedly, you would consider Twitter trends.
Here at RapidAPI, we have many APIs that sift through tons of data from Twitter and other social media platforms to get you the most relevant information. So why not leverage one of these APIs to build an app for you? In this blog post, we show you how to build a Twitter trend analysis app. This is a web app, powered by Node.js to display the top ten Twitter trends from a country on a given day.
For building this app, we have picked the Twitter Trending Topics Archive API. So let’s get started with exploring this API.
Getting Access to the Twitter Trending Topics Archive API
The Twitter Trending Topics Archive API is a great way to explore historical trends. It returns the data in the form on a CSV file, making it easy to store and process data for future use.
Follow the steps below to activate this API with your RapidAPI account.
1. Sign Up for RapidAPI Account
To begin using the Twitter Trending Topics Archive API, you’ll first need to sign up for a free RapidAPI developer account. With this account, you get a universal API Key to access all APIs hosted on RapidAPI.
RapidAPI is the world’s largest API marketplace, with over 10,000 APIs and a community of over 1,000,000 developers. Our goal is to help developers find and connect to APIs to help them build amazing apps.
2. Subscribe to Twitter Trending Topics Archive API
Once signed up, log on to your RapidAPI account and access the API Console.
Now click on the “Pricing” tab and opt-in for the basic subscription. It gives you a limited quota of 10 free API calls per month.
3. Test Your API Subscription
Once subscribed, come back to the “Endpoints” tab on the API console. You can see that there is only one endpoint ‘GET /download’, for Twitter Trending Topics Archives API.
It accepts two optional parameters for date and country. Triggering the API with default parameters values will fetch you a response something like:
Try it out and click on the link in the API response. The link points to a CSV file containing the archived trends.
The trends are arranged in columns starting with the trend string, place name, place category, date, time, and the impression count.
That is it! Your API subscription is working. Now you are ready to take this to the next level for building an app.
How To Use Twitter Trending Topics Archive API with Node.js
Node.js provides some excellent REST API frameworks. Express.js is one of the most popular frameworks for building Node.js based RESTful web interfaces. We will leverage the capabilities of Express.js and Angular to build a demo web app that shows the top ten Twitter trends.
Overview of the Twitters Trends App
This app has a server and client component. The server component is a Node.js/Express.js based REST API that accepts a client’s request to fetch the top ten Twitter trends for a given country and date. The client is a web app, hosted on Angular, that serves as the user interface (UI). It displays the trends in a graphical format.
The server component acts as an intermediary between the client and the Twitter Trending Topics Archives API. The CSV file returned from the Twitter Trending Topics Archive API response is processed in the server to extract only the top ten trends, based on their impression count.
Let’s jump in and get started with programming the app.
Before writing the code, you have to follow a few prerequisites to set up the development environment for Node.js and Angular.
Prerequisites
You must install the Node.js and Angular platform tools for development as follows:
- Node.js: Download and install Node.js from https://nodejs.org/en/. It is recommended to download and install the LTS version ( currently 12.18.2 LTS). You can refer to https://docs.npmjs.com/downloading-and-installing-node-js-and-npm for the steps to install Node.js along with npm (node package manager)
- Angular: Download and install Angular CLI by following instructions at https://angular.io/guide/setup-local
It is a good practice to also set up a project directory for the code. Based on the component-based segregation between the Node.js server and the Angular client, it makes sense to create a root project directory, and create subdirectories within that, named ‘server’ and ‘client.’
Now you are all set. Follow along with the subsequent sections to add code for programming the server and client.
Programming the Server
First, you will code the server.
Open a terminal, change to the ‘server’ subdirectory, and follow the steps below to write the server program.
Step 1: Initialize a new Node project and install the dependencies
Using the npm tool, create a new Node.js project by running the following command.
npm init
This command will prompt you for some inputs. You can safely ignore them and hit ENTER to take the default values. Subsequently, run the following commands to install the library dependencies.
npm install express -save
npm install cors -save
npm install request -save
Step 2: Create the index.js file
Create a new file named ‘index.js’ within the ‘server’ subdirectory. Open it in a code editor and add the following code:
// using express frame work to run server
var express = require("express");
// used for cross origin scripting
const cors = require('cors');
// used to make http request to
const nodeRequest = require('request');
var app = express();
app.use(cors());
var pickedUpArray = {};
// Rapid API key here
const rapidApiTrendsTwitterApikey = '<YOUR_RAPIDAPI_KEY>';
// Rapid api host
const rapidApiTrendsTwitterApiHost = 'onurmatik-twitter-trends-archive-v1.p.rapidapi.com';
// Rapid api url
const rapidApiTrendsTwitterApiUrl = 'https://onurmatik-twitter-trends-archive-v1.p.rapidapi.com/download';
// Port to run server
const port = 3000;
const numberOfUniqueHashtagsKeyToBeAccessed = 10;
//csv columns names and indexes for the twitter trend returned
const idxKey = 0, idxPlaceName = 1, idxPlaceType = 2, idxDate = 3 , idxTime = 4, idxHits = 5;
// start the server
app.listen(port, () => {
console.log("Server running on port " + port );
});
|
This code sets up a Node.js server to run as an express.js app to listen for requests on port 3000. It also defines a few global variables used later for parsing and processing the CSV file.
Make sure to replace the placeholder < YOUR_RAPIDAPI_KEY> with the actual API key assigned to your account by RapidAPI.
Step 3: Define the handler for /getTrends API endpoint
The server defines a single API endpoint /getTrends, used to fetch the top ten Twitter trends. Add the following function below the previous code to implement this endpoint.
app.get("/getTrends", (request, response, next) => {
console.log("Received Get trends request for Country : " + request.query.country + " Date : " + request.query.date)
getDataUrl( request.query.country , request.query.date ).then( dataUrl => {
getTrendsData(dataUrl).then( topTrends => {
response.json( topTrends ) ;
pickedUpArray = {};
});
});
});
|
This handler calls two internal functions. The getDataURL( ) function is invoked to fetch the URL of the CSV file returned from the Twitter Trending Topics Archive API. Subsequently, the getTrendsData( ) function gets invoked to process the data and produce the top 10 trends.
Let’s define these two internal functions.
Step 4: Define the function getDataUrl( )
Append the ‘index.js‘ with the following code for defining the getDataUrl( ) function.
function getDataUrl( country, date ){
return new Promise( function (resolve, reject) {
var options = {
method: 'GET',
url: rapidApiTrendsTwitterApiUrl,
qs: { country: country, date: date },
headers: {
'x-rapidapi-host': rapidApiTrendsTwitterApiHost,
'x-rapidapi-key': rapidApiTrendsTwitterApikey,
useQueryString: true
}
};
nodeRequest(options, function (error, response, body) {
if (error) throw new Error(error);
resolve( JSON.parse( body).url );
});
});
}
|
This function makes the API call to Twitter Trending Topics Archive API by using the Node requests module. It returns a Promise object that resolves the URL of the CSV file produced by the API.
Step 5: Define the function getTrendsData( )
Append the ‘index.js‘ with the following code for defining the getTrendsData( ) function.
function getTrendsData( dataUrl ){
var options = {
method: 'GET',
url: dataUrl,
headers: {
useQueryString: true
}
};
return new Promise( function (resolve, reject){
nodeRequest(options, function (error, res, body) {
if (error) { console.log("error in response") ;throw new Error(error);}
var trendsData = body.toString()
.split('\n') // split string to lines
.map(e => e.trim()) // remove white spaces for each line
.map(e => e.split(',').map(e => e.trim())); // split each line to array
var sortedArray = trendsData.sort(function(a, b) { return Number(b[idxHits]) - Number(a[idxHits]) });
// Sorted array has trends for same key at different time slots during the day.
// so need to iterate the array till we get top ten
var i = 0, j = 0;
var key = '';
var arrayLength = sortedArray.length;
//KEYS are case sensitive.
while( i < numberOfUniqueHashtagsKeyToBeAccessed && j < arrayLength ){
if( sortedArray[j][idxKey] != key && !pickedUpArray[sortedArray[j][idxKey]]
&& sortedArray[j][idxKey].length != 0
&& sortedArray[j][idxHits].length != 0 ){
key = sortedArray[j][idxKey];
pickedUpArray[sortedArray[j][idxKey]] = {
placeName : sortedArray[j][idxPlaceName],
placeType : sortedArray[j][idxPlaceType],
date : sortedArray[j][idxDate],
time : sortedArray[j][idxTime],
hits : sortedArray[j][idxHits],
}
i++;
}
j++;
}
resolve(pickedUpArray);
});
});
} |
This function retrieves the CSV file from the URL contained in dataURL argument. It then proceeds to split the CSV records to extract the trends and their impression count. Since the CSV has the same trends across multiple cities, there is a need to iterate over the entire list of trends and aggregate them to find the final list based on the top ten impression counts.
With this step, the ‘index.js‘ file is complete. You can now save it and close the file.
Programming the Client
Now, you will code the frontend UI of the app. On the terminal, change the directory to the ‘client’ subdirectory and follow the steps below to create a new Angular app.
Step 1: Create a new Angular project
Using the Angular CLI, create an empty project named twitter-trends.
ng new twitter-trends
This command will prompt you for a few questions. Select “No” for “Would you like to add Angular routing(y/N)?” and select “CSS” for “Which stylesheet format would you like to use?”.
Angular CLI will initialize a new project from scratch and create a new subdirectory named ‘twitter-trends’. Inside it, there is a subdirectory hierarchy of an empty Angular project with the skeleton code. This subdirectory structure looks like this:
Step 2: Install the app dependencies
The Angular app is dependent on Bootstrap, JQuery and Chart.js.
Change the current directory on the terminal to the ‘twitter-trends’ subdirectory. Now using the npm tool, install all the dependencies as follows.
npm install bootstrap –save
npm install jquery –save
npm install ng2-charts@2.4.1 –save
npm install chart.js –save
Step 3: Configure Angular project file to include dependencies
Open the file ‘angular.json’ under the path ‘client/twitter-trends/’. Update the styles and scripts object within the projects -> architect -> build object path as follows.
"styles": [
"./node_modules/bootstrap/dist/css/bootstrap.css",
"src/styles.css"
],
"scripts": [
"node_modules/jquery/dist/jquery.min.js",
"./node_modules/bootstrap/dist/js/bootstrap.js"
]
|
Now open the file ‘app.module.ts’ under ‘client/twitter-trends/src/app’. This is where all the dependent libraries are imported. Replace the default code within the file and replace with:
‘client/twitter-trends/src/app/app.module.ts’
import { BrowserModule } from '@angular/platform-browser';
import { NgModule } from '@angular/core';
import { FormsModule } from '@angular/forms';
import { ChartsModule } from 'ng2-charts';
import { HttpClientModule } from '@angular/common/http';
import { AppComponent } from './app.component';
@NgModule({
declarations: [
AppComponent
],
imports: [
BrowserModule,
FormsModule,
HttpClientModule,
ChartsModule,
],
providers: [],
bootstrap: [AppComponent]
})
export class AppModule { }
|
Step 4: Add the HTML code
To define the UI for frontend, you have to update the ‘app.component.html’ file. It is the default HTML component generated as part of the Angular project. Replace the default code with the following:
‘client/twitter-trends/src/app/app.component.html’
<div class="container ">
<div class="row">
<div class="col-md-4 form-1">
<h3>Get Historical Twitter Trends!</h3>
<label for="countries">Select the country you want the trends for</label>
<form class="form-group">
<select name="countries" [(ngModel)]='model.country' [ngModelOptions]="{standalone: true}" id="countries">
<option *ngFor="let country of countriesToCheckTwitterTrends; let i = index" [value]="country.code">{{country.name}}</option>
</select>
<br><br>
<label for="countries">Select the date</label>
<br>
<div class="row">
<input [(ngModel)]='model.date' [ngModelOptions]="{standalone: true}" id="datefield" type='date'
min='{{dateFromWhereTwitterTrendsCanBePickedUp}}' max='2118-03-09' value="{{todaysDateToGetTwitterTrends}}">
<input class="ml-2" type="submit"(click)="getTwitterTrends()">
</div>
</form>
</div>
<div class="col-md-8 form-2">
<h3>Twitter trends</h3>
<div *ngIf="showSpinner" class="loader">Loading...</div>
<div *ngIf="showChart" class="chart-wrapper">
<canvas baseChart
[datasets]="barChartData"
[labels]="barChartLabels"
[options]="barChartOptions"
[plugins]="barChartPlugins"
[legend]="barChartLegend"
[chartType]="barChartType">
</canvas>
</div>
</div>
</div>
</div> |
This file has a standard HTML structure. It creates a Bootstrap based two-column layout. The left column has the form for country and date selection, along with a submit button. The dropdown data for country selection and the click event handler function getTwitterTrends( ) are part of the TypeScript file that you will define in the next step.
The right column contains a <canvas> element for displaying the top ten trends graph.
Step 5: Add the TypeScript code for defining the frontend logic
Angular uses TypeScript as the programming language. The HTML component has an accompanying TypeScript file ‘app.component.ts’ in the same path. Open this file and replace the default code with the following:
‘client/twitter-trends/src/app/app.component.ts’
import { Component, OnInit } from '@angular/core';
import { HttpClient } from '@angular/common/http';
import { ChartOptions, ChartType, ChartDataSets } from 'chart.js';
import { Label } from 'ng2-charts';
@Component({
selector: 'app-root',
templateUrl: './app.component.html',
styleUrls: ['./app.component.css']
})
export class AppComponent implements OnInit {
//api base url
API : any = 'http://localhost:3000/getTrends';
dateFromWhereTwitterTrendsCanBePickedUp = '2017-09-03';
// Random 100 years span, we are setting max to current date in ngOnInit
todaysDateToGetTwitterTrends: any = '2118-03-09';
model: any = {};
//two digit country codes
countriesToCheckTwitterTrends = [
{ name : 'United States Of Ameria', code : 'US' },
{ name : 'India', code : 'IN' },
{ name : 'Japan', code : 'JP' },
{ name : 'Canada', code : 'CA' }
];
//Chart options and parameters
barChartOptions: ChartOptions = {
responsive: true,
};
barChartLabels: Label[] = [];
barChartType: ChartType = 'bar';
barChartLegend = true;
barChartPlugins = [];
barChartData: ChartDataSets[] = [ { } ];
showChart : boolean = false;
showSpinner : boolean = false;
showError : boolean = false;
constructor( private http: HttpClient ){
this.model.country = 'none';
}
ngOnInit(): void {
//To Set right date - max use can use is yesterday.
// Dissecting the date for day, month, year
let yesterday: any = new Date();
yesterday.setDate( yesterday.getDate() - 1);
let dd: any = yesterday.getDate();
let mm: any = yesterday.getMonth() + 1; // January is 0!
let yyyy: any = yesterday.getFullYear();
// For leading 0
if (dd < 10) { dd = '0' + dd; }
if( mm < 10 ) { mm = '0' + mm; }
//setting today as the max user can set for twitter trends.
this.todaysDateToGetTwitterTrends = yyyy + '-' + mm + '-' + dd;
document.getElementById('datefield').setAttribute( 'max', this.todaysDateToGetTwitterTrends );
//Set the initial values for date and country.
this.model.date = this.todaysDateToGetTwitterTrends;
this.model.country = this.countriesToCheckTwitterTrends[0].code;
}
getTwitterTrends(){
this.showError = false;
this.showChart = false;
this.showSpinner = true;
//Reset previous data for charts
this.barChartLabels = [];
this.barChartData = [{ }];
this.http.get( this.API
+ '?country=' + this.model.country
+ '&date=' + this.model.date, {
headers : {
"Content-type": "application/json",
"Accept": "application/json",
}
} ).toPromise().then( trends => {
// Stop spinner here.
console.log( trends );
this.barChartLabels = Object.keys(trends);
let data = [];
data = Object.keys(trends).map( key => {
return Number(trends [key]['hits']);
})
this.barChartData = [{ data: data, label: 'Twitter Trends', backgroundColor: 'rgba(0,98,204, 1)' }];
this.showSpinner = false;
this.showChart = true;
},
error => {
//Stop spinner here.
this.showSpinner = false;
this.showError = true;
// Put an alert here
console.log("There was error in getting information, please try again.");
console.log(error);
});
}
}//End of app component.
|
The variable countriesToCheckTwitterTrends contains the four countries along with their two-digit country code as required by the Twitter Trending Topics Archive API. This is populated in the UI as per the HTML code you added in the previous step.
The main logic for this app is contained within the getTwitterTrends( ) method. It invokes the /getTrends API endpoint of the server to retrieve the top ten Twitter trends and sets up the chart data for display.
Step 6: Add the CSS styles for the UI
The CSS for the HTML component is defined in ‘app.component.css’. Replace the default file content as follows:
‘client/twitter-trends/src/app/app.component.css’
.container{
margin-top: 5%;
margin-bottom: 5%;
}
.form-1{
padding: 5%;
background: #0062cc;
box-shadow: 0 5px 8px 0 rgba(0, 0, 0, 0.2), 0 9px 26px 0 rgba(0, 0, 0, 0.19);
}
.form-1 h3{
text-align: center;
color: #fff;
}
.form-1 label{
color:#fff;
}
.form-2{
padding: 5%;
box-shadow: 0 5px 8px 0 rgba(0, 0, 0, 0.2), 0 9px 26px 0 rgba(0, 0, 0, 0.19);
}
.form-2 h3{
text-align: center;
color: #000;
}
.container form{
padding: 10%;
}
.btnSubmit
{
width: 50%;
border-radius: 1rem;
padding: 1.5%;
border: none;
cursor: pointer;
}
.form-1 .btnSubmit{
font-weight: 600;
color: #0062cc;
background-color: #0062cc;
}
.form-2 .btnSubmit{
font-weight: 600;
color: #0062cc;
background-color: #fff;
}
.loader,
.loader:after {
border-radius: 50%;
width: 10em;
height: 10em;
}
.loader {
margin: 60px auto;
font-size: 10px;
position: relative;
text-indent: -9999em;
border-top: 1.1em solid rgba(0,98,204, 0.2);
border-right: 1.1em solid rgba(0,98,204, 0.2);
border-bottom: 1.1em solid rgba(0,98,204, 0.2);
border-left: 1.1em solid #0062cc;
-webkit-transform: translateZ(0);
-ms-transform: translateZ(0);
transform: translateZ(0);
-webkit-animation: load8 1.1s infinite linear;
animation: load8 1.1s infinite linear;
}
@-webkit-keyframes load8 {
0% {
-webkit-transform: rotate(0deg);
transform: rotate(0deg);
}
100% {
-webkit-transform: rotate(360deg);
transform: rotate(360deg);
}
}
@keyframes load8 {
0% {
-webkit-transform: rotate(0deg);
transform: rotate(0deg);
}
100% {
-webkit-transform: rotate(360deg);
transform: rotate(360deg);
}
}
|
The CSS definition mainly formats the contents of HTML UI elements. It also displays the spinner while waiting for the response from the server.
With this, the client application programming is complete.
Testing the Twitter Trends App
To test this app, you have to run the Node.js server and Angular client app on two separate terminal windows.
On the first terminal, change to the ‘server’ subdirectory and run the following command.
node index.js
This command launches the /getTrends API on localhost:3000.
On the second terminal, change to the ‘client/twitter-trends’ subdirectory and run the following command.
ng serve
This command compiles the Angular app and launches an internal development server to host the web app. It may take a few minutes to compile for the first time.
Now launch the web app on a browser with http://localhost:4200. (Make sure that you enable CORS on the browser using a CORS plugin, like Moesif extension for Chrome)
You should see the below screen.
Go ahead and pick a country and date. Submit the request to see the top ten Twitter trends.
Conclusion
We hope this post gives you a good baseline to use the Twitter Trending Topics Archive API with Node.js.
The API has tons of historical trends data waiting to be mined. As always, there are more possibilities for what you can visualize with this data then just having a top ten trend graph. We can’t wait to see what you build with it.
Signing off now, until next time, we get back with yet another interesting Node.js demo app with RapidAPI.
How to get historical Twitter trends?
You can get the historical twitter trends for a country by using the Twitter Trending Topics Archive API. This API is available as part of RapidAPI's API Marketplace
How to get historical Twitter trends for free?
You can subscribe to Twitter Trending Topics Archive API via RapidAPI. This API offers a limited quota of 10 free API calls per month. With $25/m.o you can get unlimited access to the API.
How to access Twitter data using API?
You can look up third party APIs on RapidAPI, that provide Twitter related data. Use the search features on RapidAPI website to search for 'Twitter'. You will get a lot of APIs related to the Twitter feed, trends, influencers, hashtags, and more.
Leave a Reply