DevOps Still Has Room for Testing

You wouldn’t think that Boise, Idaho would be a hotbed of technology.  However, with the headquarters of HP Printers, a large Micron Technology facility, a number of technology oriented financial institutions, and even a few dot-coms, it combines a high quality of life with a bent toward technology jobs.

Boise held its first DevOps Days conference in October, and almost two hundred practitioners showed up to interact and learn more about how they can work with devops concepts.  I had the privilege of being invited to speak, and over the course of two days, interacted with many of those practitioners.

I learned that even in Boise there are a large number of IT professionals that are highly interested in bringing devops concepts into their organizations.  They work for technology companies, banks, government, and even startups.  They trended young and casual, as you might expect in a city with two major universities.

There were technical speakers from the likes of Docker, Chef, and JFrog, who explained how to use their respective tools within a devops environment.  Some of these presentations were in the weeds, focusing on what new features in containers and artifact factories brought to bear on creating better containerized software.  In places, they acknowledged that they contradicted each other, especially in places where their respective tools offered significant features.

Speakers from JPL and Sonatype discussed how to set up working environments for continuous integration and continuous delivery.  The focus was on getting the right tools and workflow in place, and trusting that the software pipeline will fill up at some point

DevOps Still About the Plumbing

One impression that stood out was that virtually all of the practitioners were still in the very early stages of a devops transformation, seeking the right way to configure their devops infrastructure.  Specifically, they were still looking for a continuous integration/continuous deployment (CI/CD) solution that actually worked for them and delivered on the promise of devops.  That involves specific tools, and how those tools work together in designing the software process workflow.

Testing remains a big dark hole in devops.  The speaker from JPL, Dan Isla, was asked how testing was accomplished.  He gave a weak laugh, and said that some of the developers did unit testing, but their focus was on getting code deployed quickly, and if necessarily failing quickly.

To most of us, failing in production is simply failing.  And in a mission-critical environment such as JPL and NASA, failure often means the loss of millions of dollars and years of time.  In other organizations, it could mean the loss of business and revenue.  Yet many devops professionals seem to lack the appreciation of the need for testing, or don’t believe it’s critical in a devops shop because they can fix and redeploy quickly.  That approach needs to change.

There seems to be a lot of opportunity for solid testing in the process, as long as that testing can be done in a CI/CD environment.  There is the future challenge for testing.  There has to be a major role in ensuring that devops delivers truly working software that can withstand actual production use.  It’s simply not there yet, and testers have to play a key role in getting it there.  Today, practitioners are largely aren’t thinking about how to test either the infrastructure or the software that uses that infrastructure.

It is largely a new language and workflow for testers, and the learning curve may be steep for some.  Some testers will be left behind by the new approaches to development and delivery, and may not fully appreciate the role of the devops workflow.

And to be fair, most organizations are not yet in the position to create containerized software.  There is a lot of discussion surrounding the use of Docker and similar tools, but few seem to be actually doing it.  It’s not clear that containers are necessarily the best way to build software into a known production environment, and testers need to weigh in on that, too.

Most of the devops vendors are offering very technical solutions to very specific environments.  But software development and delivery is a very unique process; every organization has its quirks.  My impression was that practitioners are confused on the approach they should be using, and how to integrate a devops process into their existing workflow.

This is an excellent time for testers to assert a role in devops.  That role might be as test consultants, providing testing standards and objectives to teams, or it might be setting up test automation environments as part of the workflow.  Most devops thought leaders admit that devops is a cultural rather than technical challenge, and testing can be made a part of that cultural shift.

Advertisement