How To Use a News API with Ruby on Rails

RapidAPI provides a good deal of News APIs. From technology news to movie articles, you can most likely find what you need for your application here. In this article, we’ll be focusing on the Hacker News API. This is an API for the well renowned Hacker News website, where technology and other different types of articles get submitted, voted and commented on. We’ll be building a simple clone of the website using Ruby on Rails and the API. This should give you a quick idea of how to use the different APIs provided, plus how to integrate them into a Ruby on Rails application.

Related: How to Convert an Existing Rails App into an API

The Hacker News API

The Hacker News API is rather simple and clever. Almost everything on the site is considered an Item. This means that both articles and comments use the same endpoint. Moreover, Items have descendants or kids. So, for example, a submitted article will have comments as to its descendants, and each comment will have responses to that comment as its descendants. Let’s play around with it so we can better understand how it works. For this, go to the Hacker News API page on RapidAPI. Note the endpoints list on the bottom left:

We’ll be using the “Top Stories” and “Items” endpoints. The “Users” endpoint is something we’ll leave for you to explore at the end of this tutorial. Select the Top Stories endpoint and then click on the “Test Endpoint” button on the right. After a few moments you should see a list of numbers on the bottom right of the page:

These are Item IDs. The Top Stories endpoint only returns the IDs of the top stories, instead of returning them directly. This leaves the client with the responsibility of loading each item independently. Now, select the first item id from the list, select the Items endpoint on the left of the page, and paste the Item ID into the id parameter box in the middle of the page, like in the screenshot below. Then click on the “Test Endpoint” button.

This should load and return a detailed view of the item. Notice we have a title, time, score, the submitter’s username and also kids. These are the descendants of this item, in this case, comments.

To get the detail of each comment, you can just repeat the process. Copy the comment’s ID, paste it in the id parameter box like before, and click the “Test Endpoint” button. You should get the comment’s details.

Connect to the Hacker News API
The Hacker News API is also available in GraphQL.

How to use a News API with Ruby On Rails

Requirements

Now that you have a little understanding of how the API works, we can jump into building our Hacker News clone. For this, you’ll be needing Ruby installed in your system. Check out the official installation page, since the installation instructions vary by platform. After you’re done with this, make sure you have Rails installed. Just run gem install rails and wait for the command to finish. If you’re not at all familiar with Rails, perhaps give this article a try, which should give you a quick introduction to how it works.

Now, we can create a new app. For this, run the following command on your terminal:

rails new hackernews --skip-action-mailer --skip-action-mailbox --skip-action-text --skip-active-record --skip-active-storage

Notice that we’re not installing a bunch of modules since we don’t need them. You can check out what each of them does by googling them, but in summary, it’s mostly email and database/storage related things this project won’t be needing. Once you’re done, you should be able to switch to the newly created hackernews folder, and run rails s to start the server. Once that command is running, you can use your browser to go to http://localhost:3000, where you should be greeted with this:

You’ve successfully created and ran your Rails app. We’ll be adding Bootstrap as well, to make wireframing the app simpler. Open your Gemfile and add this at the bottom:

gem 'bootstrap', '~> 4.4.1'

Run bundle install and once that’s done, rename the app/assets/stylesheets/application.css to app/assets/stylesheets/application.scss. Remove the lines that start with require and append this line to the bottom:

@import "bootstrap";

We will also need to install the Excon gem. Open the Gemfile again and append this:

gem 'excon'

And run bundle install.

Browse Top News APIs

Create our API Client

For our Controllers to be able to interact with the API, we’ll build a client to make it easier. This way we avoid duplicating code everywhere. Create a new folder in the lib folder called hackernews, then create a new file called client.rb. Paste this in there:

module Hackernews
  class Client
    def initialize
      @host = 'YOUR_HOST'
      @key = 'YOUR_API_KEY'
    end

    def item(id)
      get("item/#{id}")
    end

    def topstories
      get('topstories')
    end

    private

    def get(path)
      response = Excon.get(
        'https://' + @host + '/' + path + '.json?print=pretty',
        headers: {
          'x-rapidapi-host' => @host,
          'x-rapidapi-key' => @key,
        }
      )

      return false if response.status != 200

      JSON.parse(response.body)
    end
  end
end

You can get the Host and API Key from the Hacker News API page on RapidAPI. If you’re not logged in, you’ll need to do that. You can see them in the middle of the page, after selecting one of the endpoints on the left. We have structured this client as simple as possible. We initialize it using the API and Host, then there are two methods: item and topstories. Both make use of the private get method. This method does most of the heavy lifting, that is, requesting the path it gets and converting it to JSON. We’ll come back to this file later to make some improvements.

For Rails to pick up this file automatically, we need to add some configuration code. Open config/application.rb and add these two lines inside the Application class:

config.autoload_paths << Rails.root.join('lib')
config.eager_load_paths << Rails.root.join('lib')

Also, make sure you initialize the client in the application controller, so all the controllers can have access to it. Make sure the file in app/controllers/application_controller.rb looks like this:

class ApplicationController < ActionController::Base
def client
@client ||= Hackernews::Client.new
end
end

 

Create the Homepage

Our homepage will be pretty similar to the actual Hacker News homepage. For this, we’ll need to create a new Controller, add some code to the router, and add some templates. Let’s get right to it. Start by creating a new controller: app/controllers/stories_controller.rb. Add this to the file:

class StoriesController < ApplicationController
  def top
    @stories = client.topstories(0, 10)
  end
end

You’ll notice we’re passing two parameters to the topstories method. This will help us with pagination later. Let’s actually modify our client to reflect this:

def topstories(start = 0, per_page = 10, expand = true)
  stories = get('topstories')[start...start + per_page]

  if expand
    stories.map! do |story|
      item(story)
    end
  end

  stories
end

The new method takes three parameters (with sensible defaults). A start parameter, to tell our method where to start showing topstories from, a per_page parameter, to calculate how many items per page to show, and an expand parameter. This last one is crucial, since, as you may recall, the topstories endpoint only returns the IDs of the stories, not the stories themselves. Our new method will first get only the IDs that correspond to the page we need (by slicing the array of returned IDs), then expand each story using the item method.

Now, let’s create a new template to show the stories. Create a new folder in the app/views folder called stories. In there, create a new file called top.html.erb, and add this to it:

<% @stories.each do |story| %>
  <div class="media mb-3">
    <div class="media-body">
      <h5 class="mt-0 mb-1">
        <a href="<%= story['url'] %>"><%= story['title'] %></a>
      </h5>

      <%= story['score'] %> points
      by <%= story['by'] %>
      <%= time_ago_in_words(story['time']) %> ago
      | <%= story['descendants'] %> comments
    </div>
  </div>
<% end %>

This template will loop through all the stories in the current page and create a row with the URL, title, amount of comments, score and author. Also, let’s tweak our application layout a bit. Open the file app/views/layouts/application.html.erb change the body to look something more like this:

<body>
  <div class="container">
    <h1><%= link_to 'Hacker News', root_path %></h1>

    <%= yield %>
  </div>
</body>

Which brings us to the final piece of this section. We need to add a route to our router, so Rails can know which controller to use. Open config/routes.rb and add this route:

root to: 'stories#top'

You should now be able to go to http://localhost:3000, and see a list of stories:

Browse Top News APIs

Add pagination

To add pagination, we need to change our controller a bit, and also add some code to our template. Let’s do the controller first. Change the contents of the top method in the StoriesController to look more like this:

def top
  @start = (params[:start] || 0).to_i
  @per_page = (params[:per_page] || 10).to_i
  @per_page = [@per_page, 20].min # max 20 per page

  @stories = client.topstories(@start, @per_page)
end

First, we get the start parameter, defaulting to zero, then convert it to an integer (parameters are strings by default). We do the same with the per_page parameter, but also, we make sure a user can’t set it to more than 20 per page since this would make the website too slow and would be a potential DDoS vulnerability (if the users’ sets 1000 we’d be making 1000 requests, one per each item). We then pass these two parameters to the topstories method in our client.

Now, let’s add some pagination controls to our template. Append this to the top.html.erb template:

<nav>
  <ul class="pagination justify-content-center">
    <li class="page-item <%= 'disabled' if @start == 0 %>">
      <%= link_to 'Previous', root_path(start: @start - @per_page), class: 'page-link' %>
    </li>
    <li class="page-item">
      <%= link_to 'Next', root_path(start: @start + @per_page), class: 'page-link' %>
    </li>
  </ul>
</nav>

This is a simple Bootstrap pagination control that shows a Previous and a Next button. The Previous button will be disabled if we’re already on the first page (ie. @start == 0), and pagination is simply adding or subtracting the @per_page amount to the current @start value. You should now be able to see this at the bottom of each page and use it to control it.

Show the comments

Each story in Hacker News has comments. Remember when we were exploring the API before? They are embedded as the comment IDs in each story as the kids. Since they’re also items, we don’t need to change our client. We’ll just create a new method in our stories controller and use that to render the comments. But first, to avoid duplicating code, we’ll extract the rendering of the story details to a partial. Create a new file in app/views/stories called _story.html.erb. Here, just paste the code we used to render the story details:

<div class="media mb-3">
  <div class="media-body">
    <h5 class="mt-0 mb-1">
      <a href="<%= story['url'] %>"><%= story['title'] %></a>
    </h5>

    <%= story['score'] %> points
    by <%= story['by'] %>
    <%= time_ago_in_words(story['time']) %> ago
    | <%= story['descendants'] %> comments
  </div>
</div>

In the file top.html.erb change the story code with this:

<%= render partial: 'story', collection: @stories %>

This will loop over the @stories array and render the partial template for each. Go ahead and try it out. The website should be working exactly the same so far.

Now, create a new method in the StoriesController:

def show
  @story = client.item(params[:id])
  @comments = (@story['kids'] || []).map do |comment|
    client.item(comment)
  end
end

This gets the story details and then gets the comment details (remember, we only get the comment IDs for each story). Notice that we do (@story['kids'] || []). This is because in case a story doesn’t have any comments, the kids attribute won’t exist, so we need to default to an empty array to avoid an error. Now, create a new view in app/views/stories called show.html.erb and add this to it:

<%= render partial: 'story', object: @story %>

<%= render partial: 'comments/comment', collection: @comments %>

Notice how we’re using partials here as well. One partial for the story details, and a partial for all the comments. Let’s create the comment partial. Create a new folder in the app/views directory called comments. Then, create a new file called _comment.html.erb and add this to it:

<div class="media mb-3">
  <div class="media-body">
    <span class="text-muted">
      <%= comment['by'] %> <%= time_ago_in_words(comment['time']) %> ago
      | <%= comment.fetch('kids', []).count %> responses
    </span>

    <p><%= sanitize comment['text'] %></p>
  </div>
</div>

<hr>

We need to create a new route for this. So, open config/routes.rb and add this route:

get 'stories/:id', to: 'stories#show', as: :story

Finally, edit the _story.html.erb partial to create a link to the details. Change the comments part so it becomes a link:

  <%= time_ago_in_words(story['time']) %> ago
  | <%= link_to "#{story['descendants']} comments", story_path(story['id']) %>
</div>

Try it out. You should be able to refresh the home page and click on the comments link, to get a detailed view of the comments:

Browse Top News APIs

Dive deep into comments

As you may have noticed before, comments can have responses, and they can have responses, and so own. In the original Hacker News website, they are shown in a tree view. To make things a bit simpler for this tutorial, we’ll make this into a separate page the user can click through to dive deep into each thread. First, change the partial for comments (_comment.html.erb), so that there’s a link to the thread view:

  <%= comment['by'] %> <%= time_ago_in_words(comment['time']) %> ago
  | <%= link_to "#{comment.fetch('kids', []).count} responses", comment_path(comment['id']) %>
</span>

With that, create a new controller called comments_controller.rb, and add this to it:

class CommentsController < ApplicationController
  def show
    @comment = client.item(params[:id])
    @comments = (@comment['kids'] || []).map do |kid|
      client.item(kid)
    end
  end
end

We’re doing pretty much the same as we did for the story detail, where we get the parent comment and then fetch its children (in case there are any). We also need to create the template for this new route. Create a new file in app/views/comments called show.html.erb and add this:

<%= render partial: 'comment', object: @comment %>

<h4>Responses</h4>

<%= render partial: 'comment', collection: @comments %>

To make this work we’ll need to add a new route to our routes.rb file:

get 'comments/:id', to: 'comments#show', as: :comment

If you refresh your website, you should be able to click through each comment and dive into each thread.

Browse Top News APIs

Conclusion

I hope this tutorial helped show how to use the Hacker News API but also showed you how powerful Rails can be when building an application. Even though we didn’t use a database, using Rails to build an API client can be just as beneficial and simple. As a further exercise, I’d recommend building a profile view for a user. The Hacker News API has a user endpoint that gives you some user information plus stories and comments they have submitted. You should be able to leverage the partial templates we created to make this with little effort.