The Benefits of Asynchronous Programming

The Benefits of Asynchronous Programming

Christian Carijutan June 19, 2023 Team Blog

A deep dive into Asynchronous Processing

Dealing with long time processing for a single transaction or waiting too long when downloading something from an application and causing an upstream timeout error due to heavy loads makes the server slow to respond to requests. 

We decide to use asynchronous processing.

Our existing process downloads CSV files from the server, including parsing, extracting records from the database, generating it to CSV, and running on a single thread process. In that process, we encountered an upstream timeout error every time the records got bigger. Our first approach is still to do it all synchronously but optimize the parsing and extracting of records, but still, an upstream timeout error occurs when the records get bigger. 

In that situation, we tried to find another solution to solve the problem, then found the asynchronous processing and started looking at it. 

Flow diagram before the asynchronous process

Dive into asynchronous programming

There are many different options for asynchronous programming, and this is one of the subsets of libraries out there.

Redis

  1. https://github.com/resque/resque
  2. https://github.com/sidekiq/sidekiq

AWS S3 bucket

  1. https://github.com/aws/aws-sdk-rails

To start, let’s set up our library to implement asynchronous processing:

Setting up Resque

Setup Redis

You’ll need Redis installed. Assuming you’re on MacOS and using Homebrew:

brew redis install

Follow the instructions in the notes to start Redis on boot, or start it manually with redis-server

Starting and stopping Redis in the foreground

To test your Redis installation, you can run the redis-server executable from the command line:

redis server


Setup Resque

Install Resque by adding gem ‘resque’ to the Gemfile and running bundle install

gem 'resque'

Setting up AWS S3

gem 'aws-sdk-s3'

S3 Configuration

To set it up, go to config/storage.yml First, uncomment the section headed by Amazon. Next, fill in your bucket’s name and region. For my application, the Amazon section of config/storage.yml looks like this:

For details on configuring region and credentials, see the developer guide.

def client
  Aws::S3::Client.new(    region: 'us-east-2',    credentials: credentials,  )
end
def upload
  client.put_object(bucket: 'bucket_name', key: 'file-name', body:'data')
end
def download
  client.get_object(bucket: 'bucket_name', key: 'file-name')
end

Now we can perform asynchronous programming by creating the controller and job for the export CSV controller. 

Writing Jobs

Each job should be a separate Ruby class that extends ApplicationJob and defines perform method.

class GenerateCSVJob < ApplicationJob
  queue_as :default

  def perform      
     # define dummy text to seed the CSV records
     text =<<-EOS    
     id,first name,last name,age    
     1,taro,tanaka,20    
     2,jiro,suzuki,18    
     3,ami,sato,19    
     4,yumi,adachi,21   
     EOS
   csv = CSV.generate(text, headers: true) do |csv|     csv.add_row(["5", "saburo", "kondo", "34"])   
end         
client.put_object(bucket: 'bucket_name', key: 'file-name', body:data)  
end
  private
  def client
    Aws::S3::Client.new(
      region: 'us-east-2',
      credentials: credentials,
     )
  end
end

Writing Controller

To run the execution of the job, initialize the instance class and call the method performed to the controller.

class ExportCSVController < ApplicationController
  # This method will upload the csv file to s3 through background job
  def upload    GenerateCSVJob.new.perform_now()
  end
  def download    client.get_object(bucket: exports_bucket, key: @filename)  end
end

To understand asynchronous programming, see the flow diagram from request to response.

An application can handle heavy tasks without waiting for the result. When the result is ready, a separate call will request the server download or present it to the user and this is how asynchronous programming works in background processes. 

Conclusions

If your application is designed for high/heavy load requests, it will require more than synchronous processes. There are several benefits associated with asynchronous programming, including:

  • It provides an improved user experience. Asynchronous programming can improve the overall user experience since it helps systems run more efficiently. It helps reduce wait times, which often inconvenience the users.
  • It helps improve an application’s performance. This type of programming can also help improve an application’s speed, the user interface of an application and make it easier for users to do their target.
  • It’s possible to apply it to many programming languages. While asynchronous programming can make coding more complex, using the technique in various languages with different syntaxes is possible.

More Team Blog Posts