Over the past year, I’ve had the opportunity to attend a number of conferences, ranging from the most establishment events you can think of all the way to highly technical yet informal meetups. The dress at the events ranged from suits and ties to t-shirts and shorts.
During the year, I’ve talked with an enormous range of IT people. Some work at vendors. Some at service providers. I’ve spoken to many analysts, associated from the very largest firms down to one-person shops. And I’ve talked to well over a hundred IT users, working in large enterprise organizations as well as Web-native startups.
And the universal feeling is that IT is changing in a huge way. And that change is not just in terms of the technology used in IT – though there is incredible change going on there. But also in in the very role of IT.
In short, person after person, no matter their role or their employer, echoed the same message: IT is undergoing a profound transformation.
I agree. And here’s my four tenets for the future of the IT industry.
1. Software is eating the world
The four tenets start off with no controversy: Marc Andreessen’s 2011 Wall Street Journal editorial pointed out that software is transforming entire industry segments, including video entertainment, music, but also old-line industries like transportation, retail, and even defense. His message was that software is now a central component of every industry’s offerings, and every company needed to focus on software as a core competency. And everyone jumped on it.
The phrase “software is eating the world” has become a standard opening slide in many vendor’s decks. More recently, the line “Don’t get Ubered” has joined it, warning audience members that their business could be disrupted by a Silicon Valley startup that is reconfiguring an industry value chain and being fueled by endless venture capital.
In fact, software is eating the world has become such a trope that it’s trotted out as an easy shorthand to represent that change is going on – but the endless repetition attenuates the underlying truth: The shift to digitization is affecting every industry, to the disadvantage of incumbents.
[Related: How to prepare for the IT department of the future]
Part of this is the fact that products or processes that used to require significant investment or manual effort are being replaced by digitized alternatives. More tellingly, small companies, unencumbered by practices required by analog approaches fossilized into inefficient industry traditions, can rethink the customer value chain and redesign it as a more efficient offering.
The key challenge for incumbents is their existing infrastructure and practices, typically developed as a key part of their differentiation and competitive advantage, are now shackles constricting their ability to respond to the software revolution.
For example, Walmart once leveraged its ability to build and stock giant stores to gain industry competitive advantage. Today, however, it finds expensive real estate strewn across the nation out of step with consumers increasingly shifting to online buying. Even if Walmart wants to move aggressively toward e-commerce, it’s stuck with costly retail infrastructure. More subtle, but worse, is that much of its executive talent is experience in tasks associated with managing large numbers of physical outlets and supply chains attuned to distributing pallets of goods to them. In other words, it has a skills issue as well as a misaligned value chain.
What we will see over the next decade is akin to the revolution wrought by the emergence of assembly line mass production. Any company that wanted to serve a mass market had to figure out how to automate production; those that couldn’t, or couldn’t raise enough capital to fund the effort dwindled away.
What is different today is that this software-centric reality will affect every industry and every company. Most are unprepared, and many will fail to make the leap to being a software company.
2. Open source is eating the technology industry
Within the overall wrenching restructuring of the economy, it’s easy to overlook the massive change going in the technology industry itself. The shift to a software-centric world should redound to the immense benefit of the big technology firms, but they’re in trouble, too. Instead of reaping the gain of everyone else being forced to focus on technology, big vendors are themselves suffering from the ongoing shift to software.
The evidence is everywhere: IBM has suffered 15 straight quarters of reduced revenues. Dell and EMC are set for a merger (consolidation is a classic response to an industry in financial trouble). BMC and Informatica have gone through private equity restructurings (another classic response to poor prospects).
So why is what should be a field of dreams for technology companies turning into a nightmare? The answer is two little words: open source. Simply put, open source is commoditizing the vendor offerings, and vendors are suffering from cost structures misaligned with the new realities of the industry.
Not only does this directly affect software products, but it also affects hardware (remember, software is eating the world, and that extends to hardware as well). The new mantra is software-defined anything, and by placing open source into commodity hardware, previously-unassailable market positions are now showing reduced growth and even profit drops.
Open source is proving to be more than a low-cost replacement, however. Technology innovation has shifted decisively to open source, as I wrote about in my most recent blog post. It’s hard to think of an emerging technology area that isn’t based on open source. Obviously, the software-defined hardware trends reflects this. We see open source in other emerging areas as well: distributed NoSQL databases; machine learning, containers and container orchestration.
The open source trend means more than incumbent vendor financial pain. It means that the ongoing industrial restructuring I described in the previous section of this piece will be achieved via the use of open source, which brings us to my third tenet.
3. No enterprise is Netflix … or ever will be
Netflix is an awesome company. Given how early and effectively it dismantled the video rental company Blockbuster, we should perhaps speak of the “Netflixization of an industry” rather than give that epithet to Uber. And it hasn’t rested on its laurels, either. It no sooner became top dog in the rental industry than it shifted to video streaming, where it appears about ready to upend the existing studio/carrier ecosystem.
At its heart, Netflix is a technology company. And it’s awesome there as well. Early in its streaming journey, Netflix concluded that operating data centers is not its forte; moreover, given its uncertain growth and erratic viewer patterns, trying to manage infrastructure would be an intractable headache. So it turned to Amazon Web Services, and used it as the basis for its widely-used streaming service (so widely used, in fact, that Netflix streaming accounts for 37 percent of all North American evening Internet traffic).
Instead of managing infrastructure, Netflix focused on building out its streaming application capability. And it built far more than just application functionality. Netflix created supporting functionality appropriate to deploying and monitoring vast numbers of AWS resources in an environment that often experiences failure. It built orchestration tooling to allow application components to continuously join and leave the resource pool. It built complementary tools to perform destructive testing to better ensure resiliency.
In other words, Netflix designed an incredibly sophisticated software environment tuned for its needs, and invested heavily to implement it. And it did all this with open source, much of which it created and then released to the larger world to allow others to use Netflix tools.
The company is often held up as an example to be emulated. One analyst firm told me they have an unending stream of enterprise IT clients querying it to find out how they can operate and execute like Netflix.
Here’s the thing, though. Netflix is sui generis, and has capabilities far beyond the average or even top-notch enterprise IT organization. Netflix focuses on its IT capabilities as a core competence and invests as much as is needed, facilitated by its software-like margins. It also attracts some of the best talent in the industry, drawn by the challenge of working on the Netflix technology, and a set of HR principles unique unto Netflix. Oh, and it pays really well, and offers options in its high-flying stock.
None of this sounds like a typical enterprise IT organization to me, and I doubt it does to you, either. Despite the best intentions, the average enterprise IT organization doesn’t have the money to attract high-quality talent. It’s unlikely to be capable of designing its own application environment. Even trying to leverage the Netflix tools is likely to be beyond its capabilities, because the tools require top talent to adapt and operate them for a company’s own environment.
Consequently, exhorting enterprise IT organizations to “be like Netflix” is like encouraging me to run like Usain Bolt. It’s not going to happen. Whatever solution arrives to help enterprise IT organizations meet their future, it’s not going to occur by aping Netflix. Convenient, this leads directly to my fourth tenet on the future of IT.
4. Helping enterprises becoming software companies via open source is the next decade’s IT challenge … and opportunity
There’s an almost perfect storm engulfing enterprise IT organizations. Their charter is rapidly moving from internal process automation to creating new business offerings to help their companies remain competitive. The stalwarts of technology vendors are unlikely to be capable of helping them achieve this, because their own prospects are being buffeted by the innovation and economics of open source. Meanwhile, emulating the best open source-using organizations like Netflix seems beyond their grasp. So where will their solutions come from?
[Related: For a future-ready IT organization, think multithreaded, not bimodal]
Almost certainly, new open source offerings will arrive that combine important components into preconfigured stacks that can be implemented as-is, thereby reducing the competence required to use them. I know that not all open source communities have covered themselves in glory on this issue, but the pressure – and opportunity – to make open source more consumable will provide powerful incentives for creation of these offerings.
Second, one can expect a new breed of vendor to spring up to offer help to enterprises with innovative open source projects. This new breed is likely to combine consultancy and training and many will also offer managed services to offload operational burden from users.
Third, we are likely to see interesting consortia spring up to provide best practices and reference architectures, with participation from both vendors and end users. Both have an interest in making open source more consumable, and working together can ensure both provider and user can contribute and benefit from collaboration.
One can envisage tremendous disarray in the vendor space as incumbents try to retool themselves to meet the emerging needs of enterprise customers. There will be a flood of new entrants who will all be honing their “product/market fit” in the throes of a rapidly changing industry landscape. Old and new will be jousting to serve as the primary interface to customers because the entity that “owns the customer” always obtains the lion’s share of the revenue. One can anticipate a lengthy period of vicious natural selection as the new ecosystem participants go up against existing inhabitants in a battle for critical resources: customer budgets.
Likewise, enterprise IT organizations will go through extended disruption as they attempt to develop skills more suitable to their new role as the technology factory of their companies. The organizational strife implicit in Gartner’s bi-modal IT will play out in full force as IT groups try to build new capabilities and circumscribe investment in legacy systems. The rapid technology innovation we will witness over the next decade will make this process even more difficult, as it will be challenging, to say the least, to determine exactly how to make a plan in such a dynamic environment.
Making such an enormous change across the entire technology landscape will be a protracted effort. Everyone will be walking on constantly shifting sands as the siroccos of innovation and disruption blow through the industry. But on the other side of the chaos lies a new kind of industrial organization, one powered by open source software and delivering its value via digital technology.
Join the CIO New Zealand group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.