Open sourcing our server

March 01, 2024
manav@ente.io

We take the final step in our open source journey. Our clients had always been open source. Today, we are also releasing the source code for our servers.

Ente is now fully open source.

We’ve consolidated all our code into a single repository – just clone ente-io/ente on GitHub, and you will have at your disposal a state of the art, end-to-end encrypted, full stack (mobile/web/desktop clients, the server, and a CLI to boot) alternative to Google Photos and Apple Photos.

Us at work!

This is the same code that we are using for our own Cloud offering, so it is battle tested in production. And it’s not a code dump – this is the live, breathing, continuously updated repository where we go about our work everyday.

We feel many of you will find this useful - maybe you are an inquisitive future-founder wondering about how to go about implementing your end-to-end encrypted something, or maybe you’re a self hosting connoisseur who gets happy at the prospect of tinkering with this, or maybe you’re a Go/Flutter/TypeScript engineer who wants to look at an example of these technologies talking amongst each other in a real product at scale. To all of you, we’re happy to be of use.

But our real aim behind open sourcing our servers is to offer yet another reason for Ente customers to feel proud that by choosing Ente, they’ve made the right choice.


The mechanics of the announcement are just that - Ente’s fully open source, and all the code’s at github.com/ente-io/ente. If you’re a customer or a potential one, that’s probably all you need to know.

For the rest of the post, I’d like to give a more philosophical perspective on why we’re doing this, and also give the story behind why we’re doing this now.

Why

The fact that our server was not already open source was just an accidental quirk of how the product had evolved. In our minds, it was always meant to be open source, and even if it was not already, we knew that getting there was an inevitability.

Why did we feel so sure? Posterity, trust, and gratitude.

Posterity

With our end-to-end encryption, and open source (and externally audited) client code, customers don’t need to trust us. It is nice that they do, but they don’t have to – the mathematics of cryptography ensures the privacy of their data, not trust. Somewhat counterintuitively, this point caused us to deprioritise open sourcing our server code. We knew that the secret sauce was all on the client, the servers were just dumb conduits for the encrypted blob storage. Open sourcing the server, while good to do, would not technically add anything more - the product already was trustlessly secure.

However, open sourcing the server adds a non-technical dimension. With the server being also open, all the pieces our customers use get out there in public. No matter what happens to Ente, their fork of Ente’s code remains untouched, and they can rest secure. Of course, we don’t want anything bad to happen to Ente, but think of this as insurance that one buys for one’s family lest something bad happen to oneself. For us, as cheeky as it sounds, all of our customers are like an extended family, and keeping our code open is our insurance for posterity.

Trust

While I just mentioned that our customers don’t need to trust us, we like it when they do. Generally, life is too short to deal with people or companies one doesn’t like, and trust is the bedrock of affection. Trust also builds psychological safety, which is perhaps as important for a customer as the technical safety of their data.

When we open sourced our Figma files, many of our customers reached out to us and thanked us for our general openness. To summarise their feedback, “you didn’t have to do it – no body else has ever open-sourced their live Figma files, it is not a thing – but the fact that you did it anyway, and voluntarily, makes us trust you more”. There were some secondary benefits too (e.g. customers commenting on early prototypes of future features), but for us the biggest win was gaining trust. And we hope open sourcing our servers adds to it.

Gratitude

Finally, gratitude. There are years of 24/7/365 work that we all have put into building Ente, and we’re even a bit proud of what we’ve achieved so far, but we realize that none of this was possible without the existing open source technologies on whose shoulders we stand. Without the various research papers around security and reliability we’ve read and implemented. Without the many mathematicians scribbling away at their dusty whiteboards, alone and unacknowledged by the greater world, who’ve given us the cryptography we need and the computers to run it.

Fully open sourcing Ente is our small contribution to this stream. And the biggest thank you note we could give.

Why now?

Well, if we felt so strongly about open sourcing, why not do it yesterday? The reason is more mundane than you’d expect - we’re swamped!

We’re a small team. Intentionally. It is not just that we believe small teams can do big things, we believe only small teams can do big things.

At the same time, we have a huge number of feature requests that come pouring in. As more and more people use our products, especially photos, we get more and more feedback that we have to incorporate in our products. It is exhilarating, and it is not an excuse - we simply have to get better at prioritising - but what it meant is that open sourcing our server just kept getting pushed back on our priority log, since it was more in the good to have rather than in the essential category of things.

Okay, fine. But what changed now?

To explain, let me rewind a bit. Last January, we decided to block out some time for open sourcing our server.

There wasn’t much to do: just going through the code, making sure there were no embedded secrets, move some things to the config files, improving README instructions - you know the type of minor cleanup I’m talking about. This just took a day or two, but when going through this process it dawned on us that, as excited as we were to release it, we should perhaps hold on to our horses and get an external audit first. If the clients were getting audited, the server should too before it joins them on the public stage.

So we got one. But audits are asynchronous, and they take time. We reached out to Fallible, but them being good meant that they were also in demand, and the earliest window of commitment they gave us was a month down the line. The audit itself took another month.

We passed the audit, and fixes for the audit issues were also done (the issues was more embarrassing than critical, e.g. using math/random instead of crypto/random to generate the email verification OTPs. You can read the full report here), but by this time mentally we’d moved on to other tasks.

Fallible also recommended we squash the git history before open sourcing lest there be some accidental tokens in prior commits, which, as trivial as it is to do, added yet another mental barrier / "step" we had to complete instead of just going to GitHub and hitting the change visibility button.

So things just remained how they were. Until recently.

A new feature we’ve been working on is adding support for Passkeys (it’s coming along nicely, expect an announcement on that front soon!). Since passkeys are an alternative second-factor authentication flow, it needed server side changes too.

Some of you with more experience in traditional client-server architectures might be surprised, but we rarely need to make changes to our server’s code. For end-to-end encrypted products, the situation is sort of inverted from traditional setups - the work and the magic is all on the client, and servers have limited, and mostly fixed, APIs.

Passkeys however needed new APIs, so one of us got working on the server side code. Slowly others got pulled in, discussing the changes, and when reviewing them. All this sort of brought back the pending task around open sourcing this code back into our collective mindspace.

And this time, we've made it. Now the time has come, and Ente’s open sourcing is complete.

Go clone!

To the stars!