There is no shortage of AI froth that fails to live up to its promise when it comes to real-world, production deployments. For Yaron Haviv, CTO of AI company Iguazio, acquired by McKinsey & Company in 2023, turning AI projects into real business impact is a daily reality – and he recognises that at the coalface of delivering business value, it’s gritty.

“People go read some buzzy articles and go ‘oh, it's so easy to go and implement that’. Sometimes we see the vendors in the space trying to get higher adoption and saying ‘Now it's really easy to do all these things.’”

Haviv – who has extensive experience operationalising and de-risking machine learning and GenAI applications at scale – is more than a little cutting on some of the sound and fury surrounding the generative AI space. He’s also a fan of hard-won knowledge and sitting down with The Stack, is happy to share it. 

As soon as Haviv and his team at Iguazio begin asking questions around how risk is governed, whether observability is built into the application and what happens when LLM updates break the application, it becomes clear that achieving tangible results requires a well thought out data strategy.

“If you don't build the right architecture, the model doesn't provide good results. You can start with preparing the data properly, putting in guard rails and direct prompts. All those issues need to be addressed,” he adds.

Strategic approach

With only one in ten GenAI projects making it from experimentation to production, preparation is central to unlocking the full benefits of AI and GenAI tools. Haviv’s clients tend to be multinational in scale and are typically working on multiple use cases, meaning the focus is firmly on building the right infrastructure that will serve multiple goals as their plans mature.

The first task is to build a platform for AI and GenAI that will serve the various aspects of the lifecycle and then implement the initial use case and continue forward with more use cases. “It's not really a one off. Building the right technologies for data pipelines and data storage, building the automation for how to move from development to production and CI/CD is what we do.”

Many Iguazio and McKinsey clients operate in the customer service sector, with Haviv finding that automating how people engage with chatbots or call centres can have powerful benefits for customer service businesses. “You can shrink the call duration by at least 50%,” he says, based on his recent experience.

Not only do these improvements translate to the operation of a more cost-effective call centre but executed well, measurably higher levels of customer satisfaction also accompany these AI-backed changes. 

The journey is far from over when the production stage is reached. Clients usually have new use cases in mind, or even want to increase the sophistication of their current solutions, he adds. 

“They’re often trying to build an MVP. So they will, for example, introduce a documentation bot or something for internal use that answers questions about their content,” says the CTO of Iguazio. “They'll have multiple iterations on the same project. Maybe the next revisions will index more documents or allow you to ask more sophisticated questions.”

Mature data landscape

There’s no question that data is really the foundation for any successful AI, machine learning or GenAI application. Essential questions around how to store, process and access data must be fully answered upfront, with the real-world applications of the project needing to remain at the forefront at all times. 

In many GenAI use cases that Haviv has seen, he believes they could be deployed using MongoDB. “It's actually very convenient that I don't need to install MySQL and then install the new bus, or whatever vector database. 

“A lot of focus today is on vector databases, but in the real world, you'll have all the different models of access to a given database. I don't need to work with five different types of databases, each one with its authentication. 

“MongoDB has the enterprise maturity for serving different data types, and has just extended it to vector data, authentication and versioning and so on - all those fundamentals that you need to get production ready, without patching. And you get to choose from an a la carte menu of data access patterns.”

MongoDB is able to meet the most important factors that are non-negotiable for Haviv in the deployments he works on. As he sums up: “I trust MongoDB because they have security, enterprise functionality and reliability.”

Delivered in partnership with MongoDB

The link has been copied!