For the last decade or so, the shift to cloud has been extremely profitable for companies. Shifting capital expenditures (capex) to operating expenditures (opex) has allowed companies to focus on their core business and not about purchasing, upgrading, and maintaining hardware and datacenters. But now that nearly half of infrastructure spend is cloud spend, we've reached an inflection point. Cloud spend is not immune to waste — and in some ways, costs can balloon in new and unexpected ways.
There are a number of startups like Vantage and Kubecost that were created to solve this problem. Most of these companies start with an observability approach: Where are costs coming from? Are there potentially unused or idle resources? Some tried to structure their pricing around taking a percentage of cost-savings, but that doesn't really work in enterprise sales.
The shift to cloud has also changed the equation for services. Companies like Splunk became prohibitively expensive in the cloud world where egress costs are high. Even companies like Datadog can quickly become expensive as more metrics and logs flow out of the system.
Architecture changes like moving analysis closer to the edge or data can work. Serverless offerings and better autoscaling can also help.
Average CPU and RAM utilization is low, even for best-of-breed engineering companies at scale (like Google).
We're talking 10-60%
Intelligent autoscaling, scale to 0 without cold starts, and high abstract layers like lambda and fargate bring down infrastructure costs
— Matt Rickard (@mattrickard) February 9, 2021
Corey Quinn has built a large following for his analysis of AWS costs and services. He bills himself as the "first cloud economist".
As the cloud tax increases and software margins contract, I think we'll see companies become even more conscious about their cloud costs. However, there's so much room for optimization and software companies that use cloud are growing faster than ever – so maybe in the long run it doesn't matter that much.