Pitfalls of AI in Procurement Processes: Key Highlights

In the ever-changing world of procurement, artificial intelligence (AI) is creating a buzz by promising more efficiency, smarter decision-making and smoother processes. However, as with any ground breaking technology, there are potential pitfalls that businesses need to be aware of before fully integrating AI into their procurement strategies.

Here’s are our key takeaways from our recent webinar– Pitfalls of AI in Procurement Processes.  

Understanding Large Language Models and Predictive Text 

A significant portion of AI in procurement relies on large language models (LLMs) such as ChatGPT, Gemini or similar tools. These systems are trained on massive datasets to predict outcomes and generate responses. A simple example is predictive text: when you start typing a phrase, the system suggests likely next words based on patterns it has learned. While this is helpful, it also highlights the need to understand that AI works based on probabilities —it generates answers based on past data, which isn't always perfect or applicable to your specific scenario. 

Beware of Bias in AI 

One of the most pressing concerns with AI in procurement is the potential for bias. There have been real-world examples where AI trained on historical data, reinforced existing biases. For instance, Amazon’s AI system showed a preference for male candidates in hiring, while a financial AI tool favoured men over women in credit scoring. In procurement, if your historical data shows a bias toward a particular supplier, the AI could reflect that in its recommendations. Being aware of these biases and addressing them with data cleansing and proper model oversight is crucial. 

The Context and Memory Challenge 

Another limitation of AI in procurement is its struggle with context retention. In long, complex interactions, AI can lose track of previous information, leading to repetitive or irrelevant answers. While improvements are being made to increase AI’s memory capabilities, it’s important to understand this limitation when using AI systems for detailed procurement tasks. 

Training Costs and Data Ownership Concerns 

Implementing AI systems comes with significant costs, not just in terms of hardware but also in training and maintaining these models. Moreover, businesses must be mindful of data ownership. When you provide your data to train an AI model, it is important to establish clear terms regarding who owns the intellectual property generated by the AI. 

AI Success Stories in Procurement 

Despite these challenges, AI has found several successful applications in procurement. For example, contract management tools can help extract and organise information from standardised templates, saving valuable time. AI tools are also being used to flag maverick spending—unauthorised small purchases—helping companies maintain tighter control over budgets. 

Questions we addressed during the webinar -  

1. What benefits can be achieved by integrating AI during the contract execution period, and how can we ensure the best outcome? 

2. Can AI help with analysing data from supplier performance questionnaires?  

Conclusion: Balancing Caution with Opportunity 

AI offers enormous potential for procurement teams, from enhancing contract management to streamlining day-to-day operations. However, it is essential to approach AI with a balanced perspective, keeping an eye on biases, training costs and data management concerns. By understanding both the benefits and risks, organisations can harness AI effectively, leading to more strategic decision-making and operational efficiencies.