Over the past year the team at Firefly has been researching and analyzing Terraform provider adoption and usage, to help build the tooling to support the most popular technologies and stacks in use today to support everything-as-code.

By reviewing the data and charting different trends, we discovered some pretty compelling changes our ecosystem is undergoing. In this post, we’d like to take a look at some of the statistics and metrics around Terraform provider adoption and speculate about current and future technology trends based on this data.

For some context on why this matters, in the recent 2023 State of IaC Report, it was found that 90% of cloud users are using Infrastructure as Code (IaC). An open source project managed by HashiCorp, Terraform has taken the world by storm since its inception as one of the most popular tools to build infrastructure as code, and as a result, its marketplace is becoming an integral part of many technology strategies. This can be witnessed in the growing number of providers available in the Terraform Registry with hundreds (even thousands if we count the smaller, community-created ones) more providers being added every single year.

For those who aren’t familiar, Terraform is a popular open-source tool that enables users to define and manage their infrastructure in a declarative way. Terraform providers are plugins that allow Terraform to interact with various cloud providers, APIs, services, and other systems that Terraform doesn’t natively support out of the box.

Each provider is responsible for communicating with the target system’s API, creating, updating, and deleting resources, and keeping track of their state. Terraform providers are maintained by the community and by the provider companies themselves many times, and new providers are continually being added to support more services and platforms.

Terraform Providers Benchmarking

While we often perceive the DevOps ecosystem to be for emerging companies and technologies, being more than a decade in practice, this is really no longer the case at all. Any technology company looking to remain relevant in a cloud native world is now focusing on providing the services and tools required in a DevOps-driven world.

Even long-standing technology companies and incumbents understand the importance of supporting modern cloud native stacks, and you can see this through the growing number of providers available from major industry players from Splunk, to Palo Alto, Dell, Cisco, and Juniper and ZScaler.

Terraform Providers are divided into three primary categories:

  • Official: Providers created and supported by the HashiCorp team.
  • Partner: Providers created and supported by official HashiCorp technology partners.
  • Community: Providers created and maintained by community-based individual contributors.  (A popular example is Anton Babenko’s AWS provider repository).

Taking a look at the growth across the different categories, and even specific areas in each category — whether monitoring and observability or DevOps and DevSecOps tools, can tell us a lot about what’s happening in the current DevOps and cloud native landscape. Below we’ll unpack some of the interesting data we have found.

Insights from Terraform Provider Adoption

Let’s take a look at popular areas in modern software delivery — DevOps, Identity and PaaS, CDN, and even open source and containers, as common tooling that is widely adopted to facilitate cloud operations.

We’ll explore tools like Artifactory by JFrog, Heroku (still one of the most popular PaaS solutions), CloudFlare, KubeCTL as a representative of the open source ecosystem, and Okta, which has become nearly the de facto standard for identity management in scale-ups and enterprises today.

The numbers are simply astounding. Take a look at the snapshot of growth for these tools from June 2022 to June 2023.

  • The JFrog Artifactory provider grew nearly 800% in a year — from 900K downloads, to 14 million downloads.
  • The Okta provider grew more than 350% from 4 million downloads to 17.4 million downloads.
  • The Heroku provider grew nearly 500% from 1.2 million downloads to 6.5 million downloads.
  • The CloudFlare provider simply exploded growing from 16 million downloads to 46 million downloads.
  • Last but not least, KubeCTL provider, a community-contributed and maintained provider has grown a whopping 400% as well from 6.5 million downloads to 32.7 million downloads.

What we can learn from these numbers is that the move to codification in SaaS applications and platforms (aka everything-as-code), at this point, now outpaces codification of common public clouds that have also grown, but at a much less staggering rate. With the move to managing our clouds as code largely complete, the benefits that engineering teams have derived from greater agility and automation, as well as better security and cost guardrails, governance and policy management, have demonstrated that this does not need to be confined to our CloudOps alone.

Engineering organizations are starting to understand that by managing EVERYTHING as code you can derive the same benefits for all of your platforms and mission-critical applications, and manage them in a unified way, in a single place — such as your code repository or your cloud asset inventory management.  As cost considerations become a critical piece in cloud native engineering, not just because of the current climate but as an ongoing business driver, having a unified inventory of everything makes it possible to then manage cloud at scale, much more efficiently.

This means that with everything-as-code and inventoried resources, you can apply the same kind of alerts you would for system failures to system anomalies such as drift detection between your code and cloud SaaS applications, apply unified policy and alert when this is breached, and this isn’t just limited to a single SaaS platform, cloud or solution.  This is now available cross-cloud, platform, OS, and tech stack, as it is agnostic to the underlying environment with third-party tooling.

What This Means for Us

Engineering organizations that are already widely using IaC to manage their cloud resources, are now downloading Terraform providers in the millions to enable the rest of their tooling and services to catch up with their clouds best practices. The key to efficient DevOps management is in the second half — the Ops. With your tooling and services being managed in a fragmented way, it’s impossible to get a grasp on it all, manage costs, and apply policy and security controls required today. Which adds a lot of friction to cloud operations.

DevOps has brought us a long way in the world of automation, and our CI/CD pipelines have enabled us to streamline software delivery from code to cloud. Our services and tooling should be able to apply the same governance standards of shipping and operating code at scale, and this can now happen with codified resources.

Looking ahead, we will likely only see growth in this trend, as the need to manage all things as code will take us to places where automation and AI will be de facto for all parts of the stack, as they will now be machine-readable. This will enable the humans in the process to focus on higher-order problems than just managing and maintaining disparate software stacks and unlock our next phase of growth for our development and operations.

‍