Sending emails in web application to users is very often process. We send emails related to registration, new changes in application, advertising, last activities or friends’ invitations. This is very common. Even we create that functionality in ours apps frequently, we still have bugs there. I would like to share with you one of the bugs examples.
In example, which I would like to show you, is one of this parallel examples. I found this bug some time ago. And it was there before I even started working in this project. In this application, there was possible to invite a friend. When the user wanted to invite his friend to the app, two things happened in parallel. One, the application created invitation object and saved it into the database. Second, application was sending email to the user’s friend. When application was small and everything was in one place invitation process looks OK. Everything was fine. But then, the application started to grow. Infrastructure changed. Database, workers, core application and other services went to dockers containers. Many queues were created for workers. Service for handling emails had also its own queue. And something was not right.
When a user sends invitation to his friend, we get information from Sidekiq (a tool for background processing in Ruby), that this invitation is not found in database. I checked that based on what we had in logs and everything looks fine. Invitation in database exist. After some time Sidekiq retry to run process for sending emails with success. At first I thought, This happened only one time, maybe this is some kind of a mistake? I decided not to investigate this problem and just monitor the application. Record in DB exists, mail was sent to the user. Nothing to worry. But the problem came back. This time I am known, that this was not a mistake and something is wrong. I started debugging this problem. At first, I looked to code and I get:
class Invitation < ActiveRecord::Base after_save :send_invitation_email # (...) def send_invitation_email AppMailer.invitation_email( user_id: user.id, recipient_email: email, invitation_id: self.id ).deliver_later end end
The code was ok. Not great, but nothing very complex. I had
after_save callback and simple method using mailers. When I try to reproduce this problem locally, never happened. So for debugging this problem only staging environment was helpful.
Finding the reason
After some time I understood, what was the problem. First, very important is the order of callback in Rails models:
before_validation after_validation before_save around_save before_create around_create after_create after_save after_commit/after_rollback
As you see
after_save isn’t the last one. I already had id for our invitation, but this invitation is still not persist and in the database. Second, this problem with
Invitation not found happened only when our Sidekiq had no work to do. So, when queue is empty. Our process related to sending email was so fast, that it was faster than saving a record in the database (finishing transaction related with save). So, looking below on this scheme actions (1) and (2), were faster, then (3).
Don’t you think it isn’t intuitive? It is logical, when you know that already, but it is not intuitive. At least, not for me.
We have three possible solutions:
after_commit- We will be pretty sure that the record is in the database, but our model will still take care of sending emails.
deliver_later(wait: 10.seconds)- It is not ideal, because what if saving a record in the database will take longer?
- extract completely sending email from model - This is not responsibility of the model, to send any emails. I think we should create some kind of process (we can call it service or workflow) to do a sequence of steps when a user wants to invite a friend to our app.
I think the last solution is the best. We should remove all external actions, not related to data persistency out from the model.
Which solution you will choose and why? Or maybe you see another solution? Let me know in the comments below. If you like this article share with your friends and I will be so grateful for all your feedback.