Adapting Your Rails App: Sidekiq Adjustments for Overcoming Restrictions

Introduction

One day, a customer informed us that their system’s data wasn’t syncing with a third-party service in their solution. In this blog post, I’ll share how we resolved this issue through straightforward Sidekiq adjustments and a comprehensive integration review.

Identifying the Problem

The solution relied on background jobs to synchronise data with the third-party service. However, these jobs unexpectedly stopped working without any code changes in the project. We identified the issue: we were hitting a rate limit. After contacting third-party tech support, they confirmed a limit of 5 API calls per second. 😱

The Solution

After the analysis, we determined the need for adjustments:

Dedicated Queue: Implement precise control over background jobs by creating a dedicated queue for the third-party service.
Uniqueness and Job Execution: Ensure uniqueness for this specific queue and prevent parallel job execution.
Fine-Tuning: Fine-tune the job logic to manage API calls effectively and avoid exceeding the limit.

Implementing Sidekiq Adjustments

Their system used Sidekiq, and I’ll now explain the settings we implemented for the first two solution components: a dedicated queue and parallel job execution. It can be helpful if you need to do something similar for your background tasks. Here are the steps we took:

1. Add the ‘sidekiq-limit_fetch’ gem


# A Sidekiq plugin to support advanced queue control (limiting, pausing, blocking, querying)
gem 'sidekiq-limit_fetch'

2. Reconfigure the queue settings.

Since they were using Heroku, the worker definitions were located in the Procfile. We changed it from this:


worker: bundle exec sidekiq -c 2 -v -q default -q mailers

To this:


worker: bundle exec sidekiq config/sidekiq.yml

We created the config/sidekiq.yml file, retaining the original settings but including the new queue and established a different concurrency limit:


:concurrency: 2
:queues:
  - default
  - mailer
  - new_queue
:limits:
  :new_queue: 1

Finally, we updated all related jobs to specify the newly created queue:


class MyImportantJob
  include Sidekiq::Worker

  queue_as :new_queue
end

Fine-Tuning

After implementing these changes, we adapted the solution to accommodate the constraints of the API rate limit. This involved:

Refactoring the application fine-tuning unnecessary calls.
Extending the synchronisation intervals and implementing precise code controls to manage the rated pace effectively.
Exploring bulk alternative endpoints to replace the one-by-one calls, conducting numerous tests to ensure the reliability of any substituted bulk requests.

Conclusion

We maintained all of the core features of the application, with the primary goal being to ensure it operates smoothly throughout this period of restrictions, minimising the impact of errors and ensuring up-to-date data.

Thanks for reading, if you encounter challenges in your Ruby applications, don’t hesitate to contact reinteractive for expert assistance.