Setting Priorities For Sidekiq Workers
I love Sidekiq, and Medstro is a Sidekiq Pro customer. We started out with pretty basic usage, using no custom workers, only using the ActiveRecord extension which enables .delay
that you might be familiar with in other queueing systems. We also use devise-async to automatically catch all our outgoing devise email.
We've gradually replaced more and more of our use of .delay
with custom workers, which gives us more control and configurability over what work gets delayed, how it behaves when it fails, etcetera.
Recently, we for the first time wanted to give different jobs different priorities. When a new user registers, there is some non-urgent work that is done in the background. When several users register around the same time, the queue can get backed up. Meanwhile, other types of jobs can't be done, such as sending out email notifications. Email notifications should go out immediately, so we wanted to start specifying different priorities for different types of jobs.
Sidekiq makes this easy as pie. First, in your sidekiq configuration, list your queues in the priority you want:
# ...
:queues:
- default
- low
Then, in the workers that shouldn't go into the default queue, specify a different queue:
class NewUserSetupWorker
include Sidekiq::Worker
sidekiq_options queue: :low
def perform(user_id)
# ...
end
end
And that's it! Now all those jobs will go into the "low" queue and everything else will go into the default queue. No matter how many jobs are in the "low" queue, anything put into the "default" queue will be executed next.