Exploding data needs and the future for the data centre

The data centre industry isn't just changing because of advances in technology any more, advances in technology are forcing change upon it.

By 2030, we're expected to have 25bn Internet of Things devices connected (https://explodingtopics.com/blog/iot-stats) and an AGR of over 37% in artificial intelligence (https://www.forbes.com/advisor/business/ai-statistics/).

That means the world's data needs – already stretched to breaking point in many areas – are going to trend exponentially upwards.

And it's happening not just because of the natural evolution of innovation we see in tech, it's how the industry is organised. As Ashley Martin, Data Centre Solutions Manager at data services provider CommScope, says; "We're able to access a lot of new technologies because big names like AWS and Microsoft make them freely available, so businesses will have the opportunity to adopt them faster and data centres have to cope with that growth."

But even though we have to deploy more of the physical elements of data storage and transport like cabling, we still need to ensure manageability. Innovating more compact fibre cables or reducing connector footprints is all well and good, but they still need to be accessed and manipulated physically by technicians.

Even then, the cooling systems of many data centres are doing all they can with the physics of air cooling, so any savings we make shrinking components might be cancelled out by the cost or space of liquid cooling.

A new paradigm

That means – unlike in areas like software or transistors – advancements can't just come from size alone. Even though a deluge of data is coming and new applications like AI will connect in all new ways, the basic idea of connectivity being required is pretty fixed for the time being. "You'll still have people racking and stacking equipment, you'll still have devices that need to be plugged in and managed, Martin says.

So for now, we have to keep pace with improvements in the software and applications developments and what this does to the connection and transmission frameworks. More importantly still, the data centre industry needs to continually upskill about how such new protocols and applications fit together, educating network designers, installers and technicians on the floor.

In a perfect world we might be able to wipe the slate clean and rebuild data centres from the ground up to prepare for the AI and IoT age. But as Martin says – referring back to the mainframe era where a change in computing infrastructure often meant rewiring a whole workplace – we can't throw everything out and start again.

We live in a world of economics rather than just technology, so data centres have to start from where they are now, which means they need to service the technologies of yesterday that we still use even while keeping an eye on the future. "We need to offer customers what they need, but have flexibility around added capacity," Martin says.

A cloud of clouds

The future also means data centres will increasingly need to connect with each other as efficiently as they do their own internal environments. Data Centre Interconnect (DCI) will become an increasingly common concern.

With standard network speeds going from 100Gb to 400Gb, the industry now looking at 800Gb, 1.6Tb and beyond and switch capacity doubling every 6-12 months, the challenge is (once more) around fitting physical capacity in the same space, which makes troubleshooting and field testing harder.

"The ports don't change, but the amount of bandwidth you need to pump through them is rising faster all the time, so from an infrastructure point of view it becomes a headache – sometimes you can't just take the old lead, plug it into the new device and have everything work," Martin says.

But another factor he says might help transform DCI is machine learning, ironically one of the reasons our data needs are exploding. While we've always had Data Centre Infrastructure Management (DCIM), he says it has its limits.

'Systems that bring intelligence to connectivity' is how he describes new methods where AI might have an impact. "[AI] can not only tell you what port is connected, it can tell you where and join the dots to give you a full snapshot of the link."

He also thinks AI might also help in the pre-planning stage, mapping out the most efficient paths and connections automatically, enabling a lot more of what he calls 'network visibility'.

Data Centres are facing a bold new future. All that's left is to see whether providers are up to the task.