Nailing CI/CD in a Telco Digital Transformation
Updated: Feb 5, 2020
As we start a new decade, we look over our shoulder to look at how DevOps has evolved and we can surely see that a lot has changed in the past 10 years. From code integration (merging), testing, quality control, build, release and deploy; it's become all about automation. The fact of the matter is that the levels of maturity and capabilities in the field of DevOps using CI/CD tools have surpassed the confidence levels of Telco operators to be able to use them efficiently - largely stemming from a legacy thinking that big changes equal big impact. But it's not just the Telco operators who are reluctant to take that leap of faith but more often than not, so are the software vendors and system integrators too.
There are two flavors of CD. Continuous Delivery & Continuous Deployment. Continuous Delivery contrasts with Continuous Deployment, a similar approach in which software is also produced in short cycles but through automated deployments rather than manual ones - in other words, it goes the one crucial step further to actually deploy the software to your various environments rather than just to produce it.
At PiA with our DNext product; we have achieved the latter. With one of our major European Telco customers, we are deploying our BSS stack to pre-production at times 100's of times a day. This is achievable also due to our architecture where everything is containerized and everything is a microservice. Our OKD/Kubernetes with Jenkins setup will deploy code-commits from Git into pre-production after automated code verification, build, unit testing - all in a matter of minutes. When things fail the system will automatically bring containers down and keep the last working version running and automatically notify the developer. The idea here is when you do fail, make sure you fail fast!
The next step, of course, is to achieve fully automated full Continuous Verification (CV) where UAT is automated too so that we know that things won't fail when things do go into production (automatically). This doesn't need to be that far off from a technical stand-point but it will require an even bigger leap of faith to achieve.
Getting your CI/CD pipelines in place is no easy feat. It requires a lot of effort to set up, automation rules, processes to get in place, adhered and committed to by all teams. At PiA this has taken us the better part of 2 years with a lot of painful trial and error. It is crucial to do this from the getgo and have all initiatives implemented with CI/CD in mind from the very beginning. Keep things simple and keep things small. A proverbial Titanic will never be agile. There are some real and very tangible benefits of CI/CD - among others:
Improved Productivity and Efficiency - We let the coders and developers concentrate on what they do best - shielding them from the knowledge of what the others are doing and worrying about release compatibility. Release managers are largely gone, testing is largely automated.
Reliability and Quality - At PiA we have automated a set of more than 2000 rules applied to the code before building and code is rejected back to the developer when norms, standards, and policies are not followed ensuring a tightened quality. Environments are reliable because our trio of OpenShift/Kubernetes/Jenkins brings faulty components down and working ones up.
CI/CD is not in itself to do with Agile. Agile is more about a planning and commitment model whereas CI/CD is a way how to ship code from the developer to a working environment. We are still planning in sprints but our releases are now decoupled from the sprint planning. This provides us with the flexibility to execute changes at a much faster rate. CI/CD is not the end of the story, so keep watching this space. There is more to come. Suffice to say; at PiA we think we have probably nailed it.