Rails in_batches API
At Sariina, we have a daily update job that affects millions of rows.
On this table, we also have an
ON UPDATE trigger.
The update takes a long time and a lot of disk IO.
It makes the database virtually useless until the job is done.
I thought it would be a good idea to try and do the update in smaller batches. I implemented a few solutions and the best one is now merged into Rails core. DHH suggested we make a new API for this. The API looks good in my opinion. Here are a few examples:
Person.in_batches.each_record(&:party_all_night!) Person.in_batches.update_all(awesome: true) Person.in_batches(of: 2000).delete_all Person.in_batches.map do |relation| relation.delete_all sleep 10 # Throttles delete queries end
This solved our update problem and we are using it in production for our app.
in_batches API, the updates are done in smaller batches.
The database can process other queries as well without any problem.
To try this feature upgrade to the newest Rails commit or use this gem in your existing app.