Turning off ActiveRecord query cache to improve memory consumption in background jobs
October 17, 2021
The ActiveRecord query cache can defeat effective garbage collection when working on big batches of data. Disabling it entirely in background jobs can be a solid memory consumption win. Here's how to do it:
Sidekiq
app/lib/sidekiq_ar_cache.rb
class SidekiqArCache
def call(worker, msg, queue)
ActiveRecord::Base.uncached do
yield
end
end
end
config/initializers/sidekiq.rb
Sidekiq.configure_server do |config|
config.server_middleware do |chain|
chain.add SidekiqArCache
end
end
Que
Que.job_middleware.push(
-> (job, &block) {
ActiveRecord::Base.uncached do
block.call
end
}
)
further reading
John Bachir's Code Blog