Tailscale is a startup that develops a zero config VPN based on WireGuard. It's an easy and useful product. Tailscale is a small team, but they employ some seriously smart engineers.
They have a peculiar data architecture – they used to run their entire database as a single JSON file. Now, they're running it with SQLite.
That's not the only company engineering against the grain. Expensify builds expense management systems for small businesses. Their database? "Bedrock – a simple, modular, WAN-replicated, Blockchain-based data foundation for global scale applications".
Both solutions are novel and sound interesting to work on. But optimization is fragile – and this is optimization at its finest. I have no doubt the engineers at Tailscale could pull it off for now. But what happens when someone else needs to maintain bespoke stacks?
Engineering with the grain is boring and undifferentiated. You won't have an advantage running AWS RDS off the shelf. You'll have just as much complexity and just as many features as everyone else. Engineering against the grain means that you can sometimes simplify a problem by orders of magnitude. The downside is that you need to sustain that advantage. There are diseconomies of scale at companies like Google with bespoke stacks. Open source moves too fast (and that's why companies like Uber, Lyft, Airbnb, Google, and Facebook open source significant projects).
Tailscale doesn't need all the features of a large database, and doesn't want cloud vendor lock-in, so they opted for a custom solution that meets their minimal requirements. New hires will join, requirements will change, and maybe they will need another migration, or maybe they won't. What other interesting problems could the team have worked on instead? How many resumes or customers do they get from content marketing with a controversial blog post like this?
It's always hard to estimate the present value of engineering against the grain.