When building out email functionality in the different APIs for my various side projects, I find myself carrying out a lot of repetitive actions. Figure out the templating language for the language/framework I'm working in (Blade, Twig for PHP, EJS for Node, Jinja for Python), figure out how the framework implements emails and if the project consisted of multiple services that needed to send emails, perform setup for each service. Also, maybe this is just a personal peeve but I've never liked looking through a repo for an API and seeing templates. For example, going through a Laravel API repo and seeing Blade templates or going through a Flask API and seeing Jinja templates mixed in with the blueprints and resource logic. I've always felt that APIs should only deal with logic and not with anything UI or presentation facing. It would be fine if it was a full stack, monolithic app in which the framework was making use of the templates to serve the UIs to the user but for APIs I do not like that paradigm.

Cyclone - Introduction

As a nerdy engineer, able to build tools to solve problems, I I set out to build a solution to this. A service to which I could delegate any and everything relating to emails in my APIs. The respective APIs simply make requests to this service indicating what email should be sent, a list of variables to be embedded in the email and a list of recipients. To build this, I opted to make use of FastAPI, (a python framework with type hints), along with PostgreSQL. This year I've made it a goal to write more Python based backends. I used to write a lot of Flask several years back but in the last few years its been majorly PHP and Typescript.

The service has 4 tables which are

• applications
• credentials
• emails
• dispatches

The applications table contains the different applications whose emails are managed by the service. It also stores important fields like api_key and layout. The api_key field represents the unique api key for each application which is sent along as part of the Authorization header in the format Bearer {api_key} when sending emails. The layout field represents the general structure and body for emails of the application. Its the template that'll be inherited by the child emails of the application (More on this later).

The credentials table stores the mail credentials for the different applications and is a simple 1-1 relationship with the applications table. I typically make use of mailgun to send emails in my different side projects. The emails table contains the different, unique emails sent by the different applications for example VERIFY_EMAIL, UPDATE_TELEGRAM. And finally, the dispatches table, this I would liken to a sort of logs table for each email. It basically just records the emails sent along with any potential errors or issues if something went wrong so I can easily see it.

Challenges

An issue I faced with building this was how to model/implement the inheritance with templates for each application. Think about it, 99% of the time if not more, the email templates for an application are all going to look the same in terms of layout, structure and body. What is likely to differ is the text content for each respective email but the overall layout would be the same. As a result of this, to avoid code repetition, the way to go would be to define a template layout for all of the application's templates and then for each email, inherit the template layout. With templates, this is known as template inheritance and is a common concept in templating languages/engines. If any changes needed to be made in the general structure of the templates, simply make the change in the layout and it would be reflected across all the templates that inherit the layout. Now, what I just explained would be easily implementable if the templates were on the filesystem (in the application as they usually are) but with the service, the layouts and templates are in the database. Essentially, the layout for each email template which it needs to inherit is stored under the layout field of its parent application. The initial idea I had was to write the layout along with the email template to the file system when it was to be sent but the problem I had with this approach was that I would end up having two different sources of truths for the templates, the database and the file system. It would mean that when updating the email templates, I needed to update it both in the database and if it existed on the filesystem, there too. That would not work. Also, doing that for every single template would lead to a cluttered filesystem which I also did not like. After a lot of thinking, I realized that I was approaching the problem from the wrong angle. I was looking at it from the perspective of having the child templates inheriting from the layouts which would be entirely fine if I had all the templates on the file system as usual. But with this my use case, a better way to approach it would be from the angle of embedding the child templates into the layouts. I don't know if there's a word for the opposite of inheritance but basically that. With this, I could write only the layouts for each application to the filesystem and when a particular email was to be sent, embed its template into the layout and have it sent. This is much more manageable, a lot less files getting written to the filesystem allowing me have my database as the primary source of truth.

Conclusion

As ever, it was quite the experience building this. The API is live at cyclone.olamileke.dev while its code is here> if anyone wants to have a look. I got to learn some new things about SQLAlchemy which I've used before but which I was able to explore in an in-depth manner. I also learnt some new tips and tricks relating to Jinja owing to the non-conventional way the templates had to work here. Over time I'll be integrating all of my side projects to make use of Cyclone, for now, only Yeet has been integrated. Building out the API was just the first step. Right now, I can only perform operations by making requests to the API. The next step is to build out the frontend which will be written in Vue. The frontend will also especially be important because for the longest time, one of the pain points I've had with emails is the fact that its hard to test them independently. Services like Mailtrap do not adequately simulate the quirks that email clients have with interpreting HTML/CSS. Several times years back, I used mailtrap and had the emails looking all nice and good, only to actually send to an actual email client and have it looking all weird and broken. As a result, testing out emails has been a trial and error process, I build out the template, add it to the application, send it, look at it in the email client, make adjustments, re-send again, do this over and over until I'm satisfied with the UI. I want to be able to build out the templates, and test them out in email clients independently of the application. With the built out frontend, I can easily add in functionality that would enable me to send these templates to myself, view them in an email client, make all the necessary adjustments before integrating the template with the application.