GenAI has taken the world by storm, with organisations big and small eager to pilot use cases for automation and productivity boosts. Tech giants like Google, AWS, and Microsoft are offering cloud-based GenAI tools, but the demand is straining current infrastructure capabilities needed for training and deploying large language models (LLMs) like ChatGPT and Bard.
Understanding the Demand for Chips
The microchip manufacturing process is intricate, involving hundreds of steps and spanning up to four months from design to mass production. The significant expense and lengthy manufacturing process for semiconductor plants have led to global demand surpassing supply. This imbalance affects technology companies, automakers, and other chip users, causing production slowdowns.
Supply chain disruptions, raw material shortages (such as rare earth metals), and geopolitical situations have also had a fair role to play in chip shortages. For example, restrictions by the US on China’s largest chip manufacturer, SMIC, made it harder for them to sell to several organisations with American ties. This triggered a ripple effect, prompting tech vendors to start hoarding hardware, and worsening supply challenges.
As AI advances and organisations start exploring GenAI, specialised AI chips are becoming the need of the hour to meet their immense computing demands. AI chips can include graphics processing units (GPUs), application-specific integrated circuits (ASICs), and field-programmable gate arrays (FPGAs). These specialised AI accelerators can be tens or even thousands of times faster and more efficient than CPUs when it comes to AI workloads.
The surge in GenAI adoption across industries has heightened the demand for improved chip packaging, as advanced AI algorithms require more powerful and specialised hardware. Effective packaging solutions must manage heat and power consumption for optimal performance. TSMC, one of the world’s largest chipmakers, announced a shortage in advanced chip packaging capacity at the end of 2023, that is expected to persist through 2024.
The scarcity of essential hardware, limited manufacturing capacity, and AI packaging shortages have impacted tech providers. Microsoft acknowledged the AI chip crunch as a potential risk factor in their 2023 annual report, emphasising the need to expand data centre locations and server capacity to meet customer demands, particularly for AI services. The chip squeeze has highlighted the dependency of tech giants on semiconductor suppliers. To address this, companies like Amazon and Apple are investing heavily in internal chip design and production, to reduce dependence on large players such as Nvidia – the current leader in AI chip sales.
How are Chipmakers Responding?
NVIDIA, one of the largest manufacturers of GPUs, has been forced to pivot its strategy in response to this shortage. The company has shifted focus towards developing chips specifically designed to handle complex AI workloads, such as the A100 and V100 GPUs. These AI accelerators feature specialised hardware like tensor cores optimised for AI computations, high memory bandwidth, and native support for AI software frameworks.
While this move positions NVIDIA at the forefront of the AI hardware race, experts say that it comes at a significant cost. By reallocating resources towards AI-specific GPUs, the company’s ability to meet the demand for consumer-grade GPUs has been severely impacted. This strategic shift has worsened the ongoing GPU shortage, further straining the market dynamics surrounding GPU availability and demand.
Others like Intel, a stalwart in traditional CPUs, are expanding into AI, edge computing, and autonomous systems. A significant competitor to Intel in high-performance computing, AMD acquired Xilinx to offer integrated solutions combining high-performance central processing units (CPUs) and programmable logic devices.
Global Resolve Key to Address Shortages
Governments worldwide are boosting chip capacity to tackle the semiconductor crisis and fortify supply chains. Initiatives like the CHIPS for America Act and the European Chips Act aim to bolster domestic semiconductor production through investments and incentives. Leading manufacturers like TSMC and Samsung are also expanding production capacities, reflecting a global consensus on self-reliance and supply chain diversification. Asian governments are similarly investing in semiconductor manufacturing to address shortages and enhance their global market presence.
Japan is providing generous government subsidies and incentives to attract major foreign chipmakers such as TSMC, Samsung, and Micron to invest and build advanced semiconductor plants in the country. Subsidies have helped to bring greenfield investments in Japan’s chip sector in recent years. TSMC alone is investing over USD 20 billion to build two cutting-edge plants in Kumamoto by 2027. The government has earmarked around USD 13 billion just in this fiscal year to support the semiconductor industry.
Moreover, Japan’s collaboration with the US and the establishment of Rapidus, a memory chip firm, backed by major corporations, further show its ambitions to revitalise its semiconductor industry. Japan is also looking into advancements in semiconductor materials like silicon carbide (SiC) and gallium nitride (GaN) – crucial for powering electric vehicles, renewable energy systems, and 5G technology.
South Korea. While Taiwan holds the lead in semiconductor manufacturing volume, South Korea dominates the memory chip sector, largely due to Samsung. The country is also spending USD 470 billion over the next 23 years to build the world’s largest semiconductor “mega cluster” covering 21,000 hectares in Gyeonggi Province near Seoul. The ambitious project, a partnership with Samsung and SK Hynix, will centralise and boost self-sufficiency in chip materials and components to 50% by 2030. The mega cluster is South Korea’s bold plan to cement its position as a global semiconductor leader and reduce dependence on the US amidst growing geopolitical tensions.
Vietnam. Vietnam is actively positioning itself to become a major player in the global semiconductor supply chain amid the push to diversify away from China. The Southeast Asian nation is offering tax incentives, investing in training tens of thousands of semiconductor engineers, and encouraging major chip firms like Samsung, Nvidia, and Amkor to set up production facilities and design centres. However, Vietnam faces challenges such as a limited pool of skilled labour, outdated energy infrastructure leading to power shortages in key manufacturing hubs, and competition from other regional players like Taiwan and Singapore that are also vying for semiconductor investments.
The Potential of SLMs in Addressing Infrastructure Challenges
Small language models (SLMs) offer reduced computational requirements compared to larger models, potentially alleviating strain on semiconductor supply chains by deploying on smaller, specialised hardware.
Innovative SLMs like Google’s Gemini Nano and Mistral AI’s Mixtral 8x7B enhance efficiency, running on modest hardware, unlike their larger counterparts. Gemini Nano is integrated into Bard and available on Pixel 8 smartphones, while Mixtral 8x7B supports multiple languages and suits tasks like classification and customer support.
The shift towards smaller AI models can be pivotal to the AI landscape, democratising AI and ensuring accessibility and sustainability. While they may not be able to handle complex tasks as well as LLMs yet, the ability of SLMs to balance model size, compute power, and ethical considerations will shape the future of AI development.
2024 has started cautiously for organisations, with many choosing to continue with tech projects that have already initiated, while waiting for clearer market conditions before starting newer transformation projects. This means that tech providers must continue to refine their market messaging and enhance their service/product offerings to strengthen their market presence in the latter part of the year. Ecosystm analysts present five key considerations for tech providers as they navigate evolving market and customer trends, this year.
Navigating Market Dynamics
Continuing Economic Uncertainties. Organisations will focus on ongoing projects and consider expanding initiatives in the latter part of the year. This means that tech providers should maintain visibility and trust with existing clients. They also need to help their customers meet multiple KPIs.
Popularity of Generative AI. For organisations, this will be the time to go beyond the novelty factor and assess practical business outcomes, allied costs, and change management. Tech providers need to include ROI discussions for short-term and mid-term perspectives as organisations move beyond pilots.
Infrastructure Market Disruption. Tech leaders will keep an eye out for advancements and disruptions in the market (likely to originate from the semiconductor sector). The disruptions might require tech vendors to re-assess the infrastructure partner ecosystem.
Need for New Tech Skills. Tech leaders will evaluate Generative AI’s impact on AIOps and IT Architecture; invest in upskilling for talent retention. Tech providers must prioritise creating user-friendly experiences to make technology accessible to business users. Training and partner enablement will also need a higher focus.
Increased Focus on Governance. Tech leaders will consult tech vendors on how to implement safeguards for data usage, sharing, and cybersecurity. This opens up opportunities in offering governance-related services.
5 Key Considerations for Tech Vendors
#1 Get Ready for the Year of the AI Startup
While many AI companies have been around for years, this will be the year that many of them make a significant play into enterprises in Asia Pacific. This comes at a time when many organisations are attempting to reduce tech debt and simplify their tech architecture.
For these AI startups to succeed, they will need to create watertight business cases, and do a lot of the hard work in pre-integrating their solutions with the larger platforms to reduce the time to value and simplify the systems integration work.
To respond to these emerging threats, existing tech providers will need to not only accelerate their own use of AI in their platforms, but also ramp up the education and promotion of these capabilities.
#2 Lead With Data, Not AI Capabilities
Organisations recognise the need for AI to enhance their workforce, improve customer experience, and automate processes. However, the initial challenge lies in improving data quality, as trust in early AI models hinges on high-quality training data for long-term success.
Tech vendors that can help with data source discovery, metadata analysis, and seamless data pipeline creation will emerge as trusted AI partners. Transformation tools that automate deduplication and quality assurance tasks empower data scientists to focus on high-value work. Automation models like Segment Anything enhance unstructured data labeling, particularly for images. Finally synthetic data will gain importance as quality sources become scarce.
Tech vendors will be tempted to capitalise on the Generative AI hype but for sake of positive early experiences, they should begin with data quality.
#3 Prepare Thoroughly for AI-driven Business Demand
Besides pureplay AI opportunities, AI will drive a renewed and increased interest in data and data management. Tech and service providers can capitalise on this by understanding the larger picture around their clients’ data maturity and governance. Initial conversations around AI can be door openers to bigger, transformational engagements.
Tech vendors should avoid the pitfall of downplaying AI risks. Instead, they should make all efforts to own and drive the conversation with their clients. They need to be forthcoming about their in-house responsible AI guidelines and understand what is happening in AI legislation world-wide (hint: a lot!)
Tech providers must establish strong client partnerships for AI initiatives to succeed. They must address risk and benefit equally to reap the benefits of larger AI-driven transformation engagements.
#4 Converge Network & Security Capabilities
Networking and security vendors will need to develop converged offerings as these two technologies increasingly overlap in the hybrid working era. Organisations are now entering a new phase of maturity as they evolve their remote working policies and invest in tools to regain control. They will require simplified management, increased visibility, and to provide a consistent user experience, wherever employees are located.
There has already been a widespread adoption of SD-WAN and now organisations are starting to explore next generation SSE technologies. Procuring these capabilities from a single provider will help to remove complexity from networks as the number of endpoints continue to grow.
Tech providers should take a land and expand approach, getting a foothold with SASE modules that offer rapid ROI. They should focus on SWG and ZTNA deals with an eye to expanding in CASB and FWaaS, as customers gain experience.
#5 Double Down on Your Partner Ecosystem
The IT services market, particularly in Asia Pacific, is poised for significant growth. Factors, including the imperative to cut IT operational costs, the growing complexity of cloud migrations and transformations, change management for Generative AI capabilities, and rising security and data governance needs, will drive increased spending on IT services.
Tech services providers – consultants, SIs, managed services providers, and VARs – will help drive organisations’ tech spend and strategy. This is a good time to review partners, evaluating whether they can take the business forward, or whether there is a need to expand or change the partner mix.
Partner reviews should start with an evaluation of processes and incentives to ensure they foster desired behaviour from customers and partners. Tech vendors should develop a 21st century partner program to improve chances of success.
2024 will be another crucial year for tech leaders – through the continuing economic uncertainties, they will have to embrace transformative technologies and keep an eye on market disruptors such as infrastructure providers and AI startups. Ecosystm analysts outline the key considerations for leaders shaping their organisations’ tech landscape in 2024.
Navigating Market Dynamics
Continuing Economic Uncertainties. Organisations will focus on ongoing projects and consider expanding initiatives in the latter part of the year.
Popularity of Generative AI. This will be the time to go beyond the novelty factor and assess practical business outcomes, allied costs, and change management.
Infrastructure Market Disruption. Keeping an eye out for advancements and disruptions in the market (likely to originate from the semiconductor sector) will define vendor conversations.
Need for New Tech Skills. Generative AI will influence multiple tech roles, including AIOps and IT Architecture. Retaining talent will depend on upskilling and reskilling.
Increased Focus on Governance. Tech vendors are guide tech leaders on how to implement safeguards for data usage, sharing, and cybersecurity.
5 Key Considerations for Tech Leaders
#1 Accelerate and Adapt: Streamline IT with a DevOps Culture
Over the next 12-18 months, advancements in AI, machine learning, automation, and cloud-native technologies will be vital in leveraging scalability and efficiency. Modernisation is imperative to boost responsiveness, efficiency, and competitiveness in today’s dynamic business landscape.
The continued pace of disruption demands that organisations modernise their applications portfolios with agility and purpose. Legacy systems constrained by technical debt drag down velocity, impairing the ability to deliver new innovative offerings and experiences customers have grown to expect.
Prioritising modernisation initiatives that align with key value drivers is critical. Technology leaders should empower development teams to move beyond outdated constraints and swiftly deploy enhanced applications, microservices, and platforms.
#2 Empowering Tomorrow: Spring Clean Your Tech Legacy for New Leaders
Modernising legacy systems is a strategic and inter-generational shift that goes beyond simple technical upgrades. It requires transformation through the process of decomposing and replatforming systems – developed by previous generations – into contemporary services and signifies a fundamental realignment of your business with the evolving digital landscape of the 21st century.
The essence of this modernisation effort is multifaceted. It not only facilitates the integration of advanced technologies but also significantly enhances business agility and drives innovation. It is an approach that prepares your organisation for impending skill gaps, particularly as the older workforce begins to retire over the next decade. Additionally, it provides a valuable opportunity to thoroughly document, reevaluate, and improve business processes. This ensures that operations are not only efficient but also aligned with current market demands, contemporary regulatory standards, and the changing expectations of customers.
#3 Employee Retention: Consider the Strategic Role of Skills Acquisition
The agile, resilient organisation needs to be able to respond at pace to any threat or opportunity it faces. Some of this ability to respond will be related to technology platforms and architectures, but it will be the skills of employees that will dictate the pace of reform. While employee attrition rates will continue to decline in 2024 – but it will be driven by skills acquisition, not location of work.
Organisations who offer ongoing staff training – recognising that their business needs new skills to become a 21st century organisation – are the ones who will see increasing rates of employee retention and happier employees. They will also be the ones who offer better customer experiences, driven by motivated employees who are committed to their personal success, knowing that the organisation values their performance and achievements.
#4 Next-Gen IT Operations: Explore Gen AI for Incident Avoidance and Predictive Analysis
The integration of Generative AI in IT Operations signifies a transformative shift from the automation of basic tasks, to advanced functions like incident avoidance and predictive analysis. Initially automating routine tasks, Generative AI has evolved to proactively avoiding incidents by analysing historical data and current metrics. This shift from proactive to reactive management will be crucial for maintaining uninterrupted business operations and enhancing application reliability.
Predictive analysis provides insight into system performance and user interaction patterns, empowering IT teams to optimise applications pre-emptively, enhancing efficiency and user experience. This also helps organisations meet sustainability goals through accurate capacity planning and resource allocation, also ensuring effective scaling of business applications to meet demands.
#5 Expanding Possibilities: Incorporate AI Startups into Your Portfolio
While many of the AI startups have been around for over five years, this will be the year they come into your consciousness and emerge as legitimate solutions providers to your organisation. And it comes at a difficult time for you!
Most tech leaders are looking to reduce technical debt – looking to consolidate their suppliers and simplify their tech architecture. Considering AI startups will mean a shift back to more rather than fewer tech suppliers; a different sourcing strategy; more focus on integration and ongoing management of the solutions; and a more complex tech architecture.
To meet business requirements will mean that business cases will need to be watertight – often the value will need to be delivered before a contract has been signed.
