This is the story of a forward-thinking CIO of corporate IT named Bruno Delas. He had a vision to create a new kind of startup team inside his IT department. He wanted to identify and overcome the limitations that keep traditional organizations from being able to develop applications at a rapid pace the way startups can. Could his organization do scrum, DevOps, or lean and be successful?
Was it a matter of organizational change, as Clayton M. Christensen suggested in Innovator’s Dilemma? Would the team need to be autonomous in order to shed heavy organizational structures and limitations? He knew he needed some help.
One day, Delas met a CTO named Fabrice Bernhard. Bernhard was from a small, agile web and mobile development firm.
Their organizations couldn’t be more different. Société Générale, Delas’ firm, is a 150-year-old multinational banking and finance company with more than 175.000 employees. Bernhard’s firm, Theodo, was less than 10 years old and employed barely more than 100 people.
Together, they set a lofty goal: Build and deploy new apps in less than 2 months.
They agreed on some limitations, including:
- No consulting, just build apps.
- Focus on lead time (less than 2 months).
- Use clearly defined standards shared with everyone.
- Hold a weekly tactical meeting for continuous improvement.
- Start with an all-Theodo staff, and progressively integrate SocGen (the shortened, and obviously cooler, name for Société Générale) developers.
The digital revolution scares big organizations
In the last 20 years, a new generation of companies has been growing. FAST. Bernhard asked his audience to consider the growth of Amazon versus Walmart. Amazon took off and grew massively in a short period of time, while Walmart’s sustained, plodding growth continued. Amazon has quickly raced ahead of Walmart, and this is what big, traditional organizations are afraid of.
This has not yet come to pass in finance… but Bernhard (and many others) are certain it’s coming. And banks are scared.
Why? Bernhard explained: “Because on the internet, the fast eat the slow.” If you’re fast, you get faster and the slow fall further behind. Time-to-market is vital.
The story of fast IT at Société Générale
As Bernhard got more involved with SocGen, he uncovered one interesting challenge: The financial institution’s Dev and Ops teams were quite literally separated–by 28 kilometers–in offices on opposite sides of Paris. And this wasn’t the only obstacle. When they looked at roadblocks, they also found that this new team would need to:
- get Dev and Ops in the same room (both literally and physically)
- find a product owner to lead the team
- have decent internet access
- get sufficiently powerful development machines
- introduce a new stack based on NodeJS and Angular
- make internal apps reachable on the internet
Fortunately, a good InfoSec executive was on board who could help move things through. Bernhard emphasized how important high-level sponsorship is to these kinds of projects. The sponsor needs to be high enough in the organization that he or she is over both Dev and Ops employee groups.
The team agreed that there would be no compromises. They embraced the startup culture by:
- Decorating their desks and floor
- Holding weekly standup meetings
- Having informal brown bag lunches together
- Making sure their engineering staff had the cool, useful, powerful computers they needed (and wanted)
- Making work fun
Security is priority one
The biggest issue for the project (and the business) is security. Banking security requirements are very strict and presented a huge challenge to their startup methodology. For instance, connecting any part of the SocGen infrastructure to the internet was frowned upon. However, Bernhard’s team was surprised by how supportive SocGen’s architects were. They were excited about the promise of Node.js and mobile development.
Once Node.js was ready to go, however, the team encountered resistance. Ops wasn’t involved early enough in the process, and when they were ready to deploy, Ops responded that Dev had to install everything themselves–in user space.
This is where Ansible (and Ansible product manager Justin Nemmers) comes in.
Ansible saves the day (in user space)
IT professionals spend a lot of time dealing with complex systems and tasks. Ansible makes this managing this complexity simple. By modelling processes through an agentless architecture, with access control to set permissions and tasks, anyone in the DevOps spectrum can use it safely.
The team automated the build and user space creation and prepared to deploy by getting on a train and travelling across Paris to take their code to the Ops building. This was an interesting initial deployment strategy, and clearly not the best way to build and deliver applications sustainably. (Imagine how continuous integration and deployment might look in this scenario!)
There were 2 key requirements that had to be met before the team could deploy in a more digitally connected way:
- Separation of concern – Dev and Ops must be in completely separate (but identical) environments
- Traceability – When something stupid happened, they needed to be able to quickly identify who did it
To satisfy the separation of concern requirement, they used Ansible to create separate server and application roles.
|Ops roles||Dev roles|
|Things that require root access. Most are written by Devs, validated by Ops.||App-related roles, stored in the same repo as the app. They used a pull request to contribute to server-related roles.|
DevOps is not necessarily about tools–but tools can help. Like beer. And Ansible Tower.
Ansible Tower by Red Hat helped the team resolve the traceability issue. It gave them a friendlier interface for Ansible (which is natively used on the command line) and let them see system information and role behavior in real time. But there was more.
Ansible helped the team carefully separate Dev and Ops behaviors to satisfy the strict banking requirements. They put Ansible Tower in the Ops network, and Jenkins in the Dev network. Production deployments can only be done from the Ops network. But by giving Jenkins access to the Ansible Tower API, Devs can trigger deployments… Ops still has full control and can stop the pipeline if needed.
How did SocGen’s rebel agile team do? 11 of their last 14 projects were in production within their 2-month window. The first few projects they completed together took longer–on average, almost 18 months. But as the team solved more problems and refined their playbooks, they got more efficient and were able to complete work well within the 2-month goal.
3 things to take away
- Digital revolution is about organizational transformation.
- One key measure is innovation lead time.
- Simple tools, like Ansible and Ansible Tower, can help transform IT organizations.