Batches are only available in Sidekiq Pro
Church360º Members lets users send end-of-year statement emails to their entire congregation. Some of our users could send be sending upwards of 3,000 emails at one time. We wanted a way to give the user a progress bar for the jobs instead of just notifying them once they are all sent.
From the docs, a batch is
a collection of jobs which can be monitored as a group. You can create a set of jobs to execute in parallel and then execute a callback when all the jobs are finished.
This is our current code for creating jobs:
Turning this into a batch is as simple as:
Batches come with two types of callbacks:
- success - when all jobs in the batch have completed successfully.
- complete - when all jobs in the batch have run once, successful or not.
We enable inline mode in
sidekiq.rb to avoid having to run Redis and a Sidekiq process at all times in development.
inline mode runs the job immediately instead of enqueuing it
Unfortunately, batch callbacks have to be called manually in test modes. Instead of creating a bunch of conditionals, we can wrap our previous code so batches are created when Sidekiq is running and run immediately in inline mode.
Optionally add an environment variable to easily toggle inline mode.
Now you can have your jobs run with a Sidekiq instance if you start your server with
ENABLE_SIDEKIQ=1 bundle exec rails s.
The other half is giving the user a progress bar. Create a route that points to either a controller or interactor that can look up batch status. Could be something similar to
/api/batch/status/:bid. The important part is to return percentage finished to the browser.
Our app has a
Message model that the browser polls for every minute. Polling ramps up to every 3 seconds when any batches are in progress.
Here is the piece of the
coffescript file that updates progress:
This post is a quick snapshot of how we use batches to improve our customer’s experience when sending emails. This just scratches the surface of what’s possible.