Messy cloud scenarios

Does this sound familiar?

Imagine a typical engineering scenario, perhaps one you can relate to: you are a global-scale company, with 50 different engineering teams all building products that are then deployed and hosted on the cloud.  Each team has their own unique snowflake way of delivering to production. Now let’s add more complexity. Your company has recently acquired a company where the primary practice for cloud deployments has been “ClickOps” whose standard operating practice is using the cloud console to provision all cloud resources – manually, and one at a time. 

Ok, maybe that’s a little far-fetched, perhaps they’ve progressed enough to copy infrastructure-as-code for repeatability, but the code that is  being copied is not compliant with organizational and IT policies, or is using some really bad or obsolete practices. 

Nothing is compatible or compliant. And this becomes a growing mess of complexity to manage in the long-term.

‍

But keep calm - because this blog post will share what you can do. You’ll learn some quick ways to clean up your cloud, improve compliance, and at the same time simplify your management and maintenance efforts of your cloud operations.

This is a common scenario with large companies that have undergone a lift-and-shift to the cloud. They took a messy infrastructure, pushed that legacy architecture into the cloud and claimed victory. Now they are learning about the difficulties in maintaining a reliable, cost effective cloud infrastructure without the tools and processes that can deal with the ephemeral nature of the cloud.

What we often hear is that innovators that grow by acquisition rarely consult their infrastructure team during the M&A. They don’t get a vote in the due diligence process, but they ultimately inherit a mess that they must untangle and align. Like any big change, it involves people, processes, and technology. ClickOps, while the path of least resistance and the easiest to do, is difficult, if not impossible, to maintain in the long run.

Another common scenario is when a quick prototype is created using only the console, and now there’s no time to do the work to convert it into infrastructure-as-code. Repeat. Repeat. Now you have a mess of bespoke, fragile infrastructure to maintain.

‍

In fact, this blog came about after hearing some real prospects describing their pain: 

  • “We have 50 teams using the Cloud and 55 ways of doing it. It is sort of chaotic and we are here to get it under control.”
  • “I built out this thing, but I built it using the console’. ‘How do I convert it to Terraform?”
  • “I understand the concept behind modules and reuse but where do I start first: the module or the code?”

There is light at the end of the tunnel! 

‍

Three steps to clean up your cloud

‍

1. Use a scanner to identify all of your cloud resources 

Following a simple scan, 80% of Firefly customers find resources they didn’t even know they had. Firefly will show you what’s out there, along with its status. Scanning your cloud enables you to:

  • Find and assess the status of the actual cloud infrastructure you’ve acquired or inherited (not just what’s been “Terraformed” or documented)
  • Identify resources not yet managed by IaC like Terraform, Pulumi, CloudFormation, etc., including multi-cloud, K8s, and SaaS app configurations like Okta, DataDog, etc. 
  • Map resource relationships automatically to find critical points of focus and understand visually modules and their dependencies.

‍

2. Automate the creation and use of IaC

A good practice is to stop manually creating Terraform and Pulumi resources. Tools like Firefly can do this for you. Firefly can automatically:

  • Create IaC, including Terraform modules and dependencies, from your actual cloud configurations
  • Align existing cloud configurations with existing modules, and
  • Create new infrastructure using existing modules 
  • Deliver new code and edits to existing infrastructure code via a pull request, allowing you to deploy new Infrastructure and changes using the same DevOps processes and tools you already use for application code. Now your infrastructure is also governed with version control, pipeline approvals, and all of the other benefits of DevOps.

‍

3. Apply Policies-as-Code within your automated processes 

Firefly Insights provide visibility into violations of common best practices for reliability, cost and potentially dangerous misconfigurations. In a study of actual proof-of-concept projects, prospects found $186k in savings, (averaging $56k in cloud waste alone) in their first 2 weeks of using Firefly.

  • Find things like forgotten, unused or underutilized assets.  You may also find ghost assets from environments that were torn down, with assets left behind.
  • Use out-of-the-box OPA (Open Policy Agent) policies or create your own using Firefly’s AI-driven custom policy generator, a popular tool to ensure bespoke asset tagging.

‍

Cloud Management in the Long Term

As organizations grow and evolve, change is inevitable, whether through new practices, technologies, or even new stacks and incompatible stacks inherited by different means. This often impacts cloud operations first, but luckily this is manageable.

With some easily applicable good practices of continuous scanning, automatic codification, and insights into bad practices, you can get a handle on your CloudOps rapidly, without investing hours of manual effort.  

Infrastructure as Code was just the first step in the automation revolution. Once codified, we can leverage automation to ensure systems are consistent, cost effective, and secure through continuous IT governance built for DevOps native workflows.

Learn more about managing configuration drift and other "Day 2" challenges through the Firefly knowledge center, IaCademy.

‍