ÀÖÓ㣨Leyu£©ÌåÓý¹ÙÍø

Generative artificial intelligence (Gen AI) is revolutionising the business landscape, building on years of progress in data and AI adoption. Its potential to drive competitive advantage and fuel growth is undeniable. However, capitalising on its benefits requires organisations to fully understand and mitigate its unique risks, particularly in managing data and evaluating organisational readiness.

Using it safely, though, requires they understand not only the risks and quality of organisational data specific to implementing Gen AI—which leaders in the ÀÖÓ㣨Leyu£©ÌåÓý¹ÙÍø AI Quarterly Pulse Survey Q1 2025 highlighted as the biggest challenge—but also how to manage that data. To deploy Gen AI safely and effectively, businesses must address risks in four key areas.

Unlike traditional AI, where development and deployment were largely limited to specialist teams, Gen AI reaches across functions and business units. This widespread use raises the risk of employees misinterpreting or over-relying on Gen AI outputs. Without proper understanding, teams may trust the results as infallible, particularly in decision-critical contexts. This could lead to financial or reputational damage to the organisation. 

Managing data security and data quality is a critical challenge when using Gen AI. While it is straightforward for organisations to develop policies that prevent the use of confidential or personally identifiable information (PII) data by a Gen AI model, technical enforcement of these rules is far more complex. The primary reason is the proliferation of consumer solutions with multi-modality capabilities, increasing the risk of employees inadvertently exposing confidential data to third-party providers.

Furthermore, the popular adoption of Retrieval Augmented Generation (RAG) architectures could create vulnerabilities if the data sources are not adequately secured. Mismanagement of these aspects not only opens the door to regulatory breaches; it also risks unintentional data exposure, both internally and externally.

To utilise Gen AI, many organisations must expand their technology stack, whether on-premises or in the cloud. This rapid scaling introduces operational risks, including integration gaps between new tools and existing systems as well as increased technological footprint complexity. Besides data disclosure risks, it is important to pay special attention to the risks associated with integrating third-party tools and ensuring API security. 

Gen AI models—all of which operate probabilistically rather than deterministically—introduce another layer of complexity. These models are pre-trained for a specific purpose, and determining whether a model is fit for purpose demands careful analysis.

A rigorous benchmarking process is essential. Businesses must evaluate each model’s intended application, limitations and safeguards to ensure compatibility with their operational requirements and ethical standards. This process not only mitigates risk but also ensures the technology is used responsibly and effectively. 

Despite these risks, avoiding Gen AI altogether is not the solution. Technology offers unparalleled opportunities to boost efficiencies and innovation, but its rapid developments also bring evolving threats. How can organisations new to Gen AI approach its deployment wisely? 

Most organisations already have processes in place for managing technology risks. The challenge lies in tailoring these frameworks to accommodate Gen AI. For limited-scale deployment, a modest expansion of their technology risk management approach may suffice. However, broader Gen AI adoption might require establishing dedicated AI-specific steering committees to address strategy and risks specific to AI’s usage in the organisation.

Clear ethical guidelines should govern the use Gen AI, including prohibited use cases outside the appetite of the organisation and pre-defined risk categories. This guidance provides clarity for business functions pursuing innovation and helps risk and audit functions establish control expectations. Transparency and trust are foundational as AI’s role proliferates. This involves understanding regulatory and compliance obligations, uplifting governance processes, bringing together cross-functional stakeholders and assigning responsibility for mitigating risks.

Organisations can introduce Gen AI incrementally by applying governance proportional to the risk level in line with the stage of the innovation idea. For prototypes in low-risk scenarios (e.g., minimal financial investment or data sensitivity), oversight can be lighter. As prototypes scale toward deployment, more comprehensive assessments, including cybersecurity evaluations and risk analyses, should be conducted to reinforce defences.

Deploying Gen AI should not be radically different from implementing standard software tools. Much like other technologies, it carries risks that businesses must carefully evaluate and mitigate. The upcoming document for AI system impact assessment offers useful guidance on how to evaluate the potential impact of AI on the organisation and its stakeholders. 

Furthermore, organisations must decide the degree of human oversight required in Gen AI use cases. provides a useful structure by categorising oversight into three levels: human-in-the-loop, human-out-of-the-loop and human over-the-loop. Determining which to use is a matter of balance—outcomes with a major impact could see more involved human oversight even though faster straight-through decision-making is not possible. Which option to choose should be made by cross-functional teams that assess risks and recommend controls. 

Looking ahead, the emergence of Agentic AI has the potential to transform operations even further. Agentic AI, when embedded in businesses, has the ability to mature beyond content generation to include reasoning and decision-making. This demands heightened governance to manage its influence on business processes including ensuring resilience in multi-agent environments and equipping organisations to investigate and respond to incidents effectively. 

As with today’s Gen AI, the key to success lies in a consistent, risk-based approach to deployment combined with robust cybersecurity. By balancing innovation with caution, organisations can harness Gen AI’s potential while minimising exposure to its risks.