Ambitious AI roadmaps are challenging existing infrastructure
Flexential recently surveyed IT leaders and found that not many cloud or on-prem data centers were originally designed to handle high-density AI workloads or latency-sensitive AI applications.As the gen AI trend drives significant investments in this computing-intensive technology, IT leaders need to rapidly adjust and enhance existing infrastructure to accommodate the constantly evolving AI requirements. Read on to discover more key findings from the Flexential 2024 State of AI Infrastructure Report.
IT leaders are optimistic but face significant roadblocks, including:
- Challenges networking and scaling data centers
- A skills gap related to specialized infrastructure
- Increased data privacy and security risks
- Growing concerns about IT infrastructure sustainability
- A disconnect between C-suite leaders and the challenges on the ground
Key findings
- 59% of respondents from organizations with AI roadmaps said increasing IT infrastructure investments was an element of that roadmap.
- 53% of respondents reported skills gaps or staffing shortages related to the management of specialized computing infrastructure
- 42% of respondents said their organizations have pulled an AI workload back from public cloud due to data privacy and security concerns
- 51% of respondents are addressing performance issues by using third-party colocation data centers to process data closer to the edge
- 45% of respondents from organizations with AI roadmaps said failing to meet the goals in their roadmap would affect their ability to innovate
- 94% of respondents would pay more for data centers or third-party cloud vendors to use clean or renewable energy and/or buy credits to offset their carbon footprints
IT leaders are excited about their AI roadmaps, but the ability to execute remains in question
In the current AI boom, largely driven by next-gen AI solutions, the importance of machine learning cannot be underestimated. Widely used in sectors like finance and healthcare, a large majority of organizations (99%) report having a well-documented AI strategy, highlighting the integral role of ML in early technology adoption.
Is infrastructure a top priority when executing AI roadmaps?
The importance of bolstering data center infrastructure is recognized within these AI strategies, with the majority (59%) identifying augmenting infrastructure investments as a key component of strategic planning. Following closely, the focus on reinforcing cybersecurity measures for AI applications was marked as a critical investment by 54% of the respondents. This prioritization reflects the growing awareness of the data security challenges that accompany the deployment of sophisticated AI technologies.
IT leaders are under increased scrutiny
AI investments are a top-down initiative, with over half of respondents saying the C-suite is the driving force behind AI adoption. However, while IT leaders are enthusiastic, they aren’t confident about execution, with over a third flagging their organizations as “nascent” or “emerging,” indicating there is some catchup to do.
{{statBigOne}}
While more than half of respondents (53%) are extremely confident in their organizations’ ability to execute AI roadmaps, a significant percentage (46%) express doubt, with a lack of confidence more prevalent among smaller organizations with under $500m in revenue.
What happens if AI roadmaps aren’t achieved?
Organizations face significant consequences if they fail to achieve the goals outlined in their AI roadmaps. Nearly half (45%) said they will struggle to innovate or must delay plans for new products or services, while 42% said they would lose market share to competitors.
Skills gaps, performance issues, data privacy, and security concerns pose barriers
Skills gaps make it difficult to meet a pressing need for high-density infrastructure, while performance issues tied to networking and data center scaling difficulties hold back AI initiatives’ time to revenue. At the same time, AI’s unique data privacy and security challenges are a concern.
Skills gaps hinder AI progress
The progress of AI is being impeded by major skills gaps at enterprise organizations, especially around high-density computing infrastructure. While building AI infrastructure is the top priority for organizations, many struggle to find staff with the necessary skills to support it, with management of specialized computing infrastructure a significant concern.
{{statOne}}
{{statTwo}}
Given these challenges, organizations are likely to need to rely on AI-enabled private cloud services, technical consultants, or other external resources to achieve their ambitious goals in a timely manner.
Networking challenges impact AI performance
82% of respondents have encountered some kind of performance issue with their AI workloads in the past 12 months, reducing the efficiency of AI applications and increasing time to revenue—a major concern given the mounting pressure on C-suite leaders. This was further exacerbated by senior leaders' lack of awareness of the issues on the ground, with almost a quarter (22%) reporting no skills or staffing gaps over the past 12 months.
Data privacy and security concerns are top of mind
Nearly all respondents (95%) believe their organizations’ increased investment in AI has also increased their vulnerability to cyber threats, with 40% saying their cybersecurity teams don’t understand how to protect AI applications and workloads. This represents a major skills gap organizations will need to fill with third-party expertise and infrastructure, especially as investments in AI continue to grow.
Organizations are leveraging colocation to meet AI infrastructure challenges, but there’s room for optimization
Moving compute and data storage to the edge of the network reduces latency. Moreover, colocation data centers provide additional efficiencies by allowing the deployment of GPU hardware next to CPU stacks and key data lakes, which further enhances performance. Improved efficiency leads to cost reductions, which is crucial for AI workloads.
Top considerations for AI workload deployment:
- Security. Securing sensitive data processed in AI applications through robust security measures that extend throughout an entire IT architecture helps ensure compliance with ever-evolving privacy rules.
- Cost-efficiency. Strategic decision-making about where to deploy AI workloads is vital for optimizing cost efficiency. Additionally, greater C-suite involvement in driving AI initiatives at large organizations may be driving an increased focus on infrastructure costs.
- Application performance. Performance issues tied to networking and data center scaling difficulties hold back AI initiatives’ time to revenue. Colocation data centers help shift compute and data storage to the edge of the network, inherently minimizing latency.
{{statBigTwo}}
Interconnection-led infrastructure is becoming vital for optimization
Organizations are not yet deploying their heaviest AI workloads at the edge, which could limit their performance benefits. To start, turning public connections into private ones and moving GPUs and other AI infrastructure into colocation data centers closer to the edge of the network would improve security and performance.
Improving data center sustainability is a top priority
The push to decrease the carbon footprint of data centers is not a new concept. However, with AI infrastructure projected to consume as much as 134 terawatt-hours of power per year by 2027, as the New York Times reports, the conversation has become more pressing. Surprisingly, it's the C-suite or board that is identified as the main driving force behind this pressure, outweighing even customers in terms of pressuring IT sustainability.
IT leaders are personally invested in improving IT infrastructure sustainability
While seemingly satisfied with their organizations’ current efforts, nearly all respondents (93%) feel at least somewhat personally invested in improving sustainability in IT, and a majority (53%) believe the issue doesn’t get enough attention. Most interestingly, nearly all respondents (94%) are willing to pay a premium for better sustainability outcomes from third-party data centers or cloud vendors.
{{statThree}}
{{statFour}}
Conclusion
As AI accelerates the pace and scope of organizational change and innovation, IT leaders must reimagine how they architect infrastructure to achieve ambitious business goals.
- Sourcing high-density compute capacity that can scale with AI workloads while evolving to integrate new tools and technologies.
- Securing the sensitive data processed in AI applications through robust security measures that extend throughout their entire IT architecture
- Leveraging software-led interconnection to support real-time collaboration across IT ecosystems and large, distributed workloads.
- Deploying liquid-cooling technologies and other strategies to address sustainability without sacrificing performance.
Click below to download this resource!
