I spent almost 5 years at Amazon… and learned so much there. I really do appreciate that experience. The tooling available for developers was quite good, and really focused on providing benefits to the developers and then getting out of the way. Everything existed as an optional component. If you had a better way to do something you could. Though there are many things I miss from Amazon, like their orchestrated build/deploy Pipelines project, the one thing I miss most is KeyMaster.
KeyMaster was an internal set of tools that predates their KMS project. It provides the ability to (securely, out-of-process) encrypt and decrypt payloads for services running from within Amazon. That by itself sounds minor… but for KeyMaster to do this correctly, it had to make use of many other security tools within the Amazon ecosystem. It had the ability to ensure the application was even allowed to encrypt and/or decrypt the data; with an ownership model to validate who the owner was. It needed a mechanism to get the keys securely from the backend. Everything is completely audited so any bad actors in the system (or misbehaving code) could be quickly identified manually or via data analysis. Keys were (pre) cached to reduce latency issues. It had a mechanism available to rotate the keys, though admittedly fairly clunky when I left. And, of course, it was fast and scaled to Amazon levels. To do this all this, KeyMaster relied on many tools within the Amazon’s security tool-chain. My memory ascribed to KeyMaster was really the evolution of security from within Amazon with separate tooling able to be linked together by the KeyMaster project. Yet using it was always optional; you could rely on KMS instead if you wanted, other other tooling.
As a computer programmer… what I describe with KeyMaster is the best possible outcome from years of building up tools. What we don’t see is the projects created for the security tool-chain that were quickly (or unfortunately delayed) decommissioned because they didn’t work, or worse, ended up reducing security regardless of good intentions. The results shown above is an evolution of these systems and requirements… where the final product excels far better than anyone could have designed 10 years ago. Each project of the security tool-chain has a set of independent requirements. Each component stands up on its own. But, as they say, the sum is greater than the parts.
You can create a KeyMaster today, outside of Amazon, as an individual computer programmer. The parts that make up KeyMaster all exist. Nothing is unique to Amazon. There are proprietary solutions that are similar, though many of them are really doing too much at once. They are not components that can be used as needed, rather frameworks that exist only to engulf the consumer in sales channel lock-down and ecosystem reliance. This is the failure of today’s commercial software.
There is a need for an open-source encryption service today. One built from replaceable components; many of these already exist in the open-source world. I’m going to try my hand and putting this together. It’d be great to get some help.