Navigating AI’s Proof-of-Concept Plateau: A Call for ROI Focus
The rapid ascent of artificial intelligence has led to a flurry of experimentation across various industries. However, many companies are experiencing “proof-of-concept fatigue,” where initial trials fail to translate into tangible business value. Ivan Zhang, co-founder of Cohere, a leading enterprise large language model (LLM) company, addressed this growing frustration during a recent Web Summit, urging potential customers to maintain their faith in AI while emphasizing the critical need to focus on return on investment (ROI).
The Proof-of-Concept Pitfall
Zhang highlighted the disillusionment among enterprises that have invested heavily in AI pilots without seeing a corresponding payoff. He acknowledged that many Cohere clients, despite building initial applications, have struggled to move them into production due to issues ranging from cost and governance to data security and privacy. This sentiment reflects a broader trend where the promise of AI often clashes with the practical realities of implementation.
He pointed out the issues of expense, regulatory compliance, data protection and privacy protocols, which Cohere hopes to solve with its new workspace platform offering, North.
The ROI Imperative
In an interview, Zhang emphasized that the next phase of AI adoption must be driven by demonstrable ROI. Companies need to see a clear financial justification for their AI investments, ensuring that the benefits outweigh the costs. He warned that some AI systems are so expensive to operate that they negate any potential cost savings from automating tasks.
"Sometimes the systems they end up building, the cost of the model itself is more expensive than the humans that are actually running it," he said.
The essential question of is there an actual improvement with AI implementations has to be addressed to overcome the burned bridges of AI companies taking on projects that never pan out.
AI Augmentation vs. Productivity
Zhang also noted instances where companies have attempted to augment existing workforces with AI but failed to see any improvement in productivity. In some cases, employees simply reduced their workload without increasing output, effectively negating the benefits of AI. This highlights the importance of carefully considering how AI is integrated into existing workflows and ensuring that it leads to genuine efficiency gains. Careful analysis of current processes versus AI’s ability to improve is vital.
Overcoming Early Setbacks
Zhang anticipates that AI startups will now be tasked with winning back companies “burned” by projects that didn’t pan out. “The next phase of go-to-market for this technology is, ‘where is the ROI?’” He believes that AI companies will need to rebuild trust by demonstrating the tangible value of their solutions and focusing on delivering measurable results. This involves clear communication, transparent pricing, and realistic expectations.
Echoes from the Research Community
Zhang’s observations are supported by research from organizations like the National Bureau of Economic Research, which found "no significant impact on earnings or recorded hours in any occupation" after surveying 7,000 workplaces using AI chatbots. Similarly, a Boston Consulting Group study revealed that only a quarter of executives surveyed have seen significant value from AI, suggesting that companies often spread their investments too thinly across multiple pilots. Focusing on a few key areas and proving value is more effective than a widespread but shallow approach.
Prioritizing Business Problems over Flashy Solutions
Zhang’s advice to companies considering LLMs is to focus on solving specific business problems rather than building elaborate solutions without clear use cases. He cautioned against getting “lost in building something and searching for a problem,” emphasizing the importance of aligning AI investments with strategic business goals. A clear understanding of the pain points and how AI can alleviate them is crucial for success.
AI as a Tool in the Toolbox
Zhang arguedthat AI should be viewed as just one tool in the toolbox for solving business problems and creating value for customers. He cautioned against overhyping the technology’s potential to solve all the world’s problems, emphasizing that it is most effective when used strategically and in conjunction with other solutions. Integration with existing systems and processes is key to maximizing its effectiveness.
The Hallucination Challenge
While AI has made significant strides, challenges remain, particularly in the area of “hallucinations,” where LLMs generate false or fabricated information. Despite progress in this area, LLM hallucination rates have remained stubbornly high, with even the latest models from leading companies producing errors. This issue underscores the importance of transparency and providing users with insights into how AI models arrive at their conclusions. Proper validation and verification of AI-generated content are essential.
A human-in-the-loop approach can help to catch and correct errors.
The co-founder acknowledged to numerous professionals that hallucination remains a problem in generative AI. He stated that the company has tried to help by being transparent, including showing users “the raw thinking” of its LLMs, and what tools its systems use, along with how and citations to derived answers.
The Competitive Landscape
Cohere faces stiff competition from better-funded rivals in the AI space. However, Zhang believes that bigger is not always better when it comes to building cost-effective and energy-efficient AI models. He argued that a model is “only as good as the data and systems it can access,” emphasizing the importance of building solutions that can be run completely within customers’ environments. Zhang touted Cohere’s “intense growth” and said the “relatively nascent” nature of the space leaves plenty of room for the company to expand. This suggests a focus on niche markets and specialized solutions.
Revenue Growth and Challenges
Cohere’s growth has been a recent topic of focus for tech media. Cohere reached $100 million USD ($138 million CAD) in annualized revenue this month after more than doubling its sales since the start of 2025, and CEO Aidan Gomez recently told Bloomberg the company was “not far away” from profitability. But The Information has reported this is still $350-million USD behind what Cohere told investors in 2023 it expected to be making annually by now. Revenue targets and stiff competition are not the only challenges Cohere must contend with. Maintaining strong growth and achieving profitability is a constant challenge in the competitive AI market.
Copyright Infringement Lawsuit
The AI startup also has what one expert called a potentially “precedent-setting” copyright-infringement lawsuit from major media companies on its plate. A group of media organizations including the Toronto Star, Condé Nast, and Vox have alleged Cohere scraped media content without consent and used it to train AI models, accessed content in real time without permission, and generated infringing outputs. Cohere is just one of many AI startups facing similar lawsuits. Cohere has denied these claims, arguing that the suing publishers had gone out of their way to “manufacture” a case and disputed the notion that any practical copyright infringement had occurred. Legal challenges related to copyright and data usage are a growing concern for AI companies.
Zhang declined to offer much comment on the matter, pointing BetaKit to a blog post detailing Cohere’s thinking. “We’re confident in that,” he said.
A Deeper Dive into AI Implementation Challenges
Many businesses initially dive into AI initiatives with considerable enthusiasm, believing that AI will quickly revolutionize their operations and create previously unheard-of efficiencies. But many find themselves facing substantial challenges that they did not anticipate. These difficulties can take various forms, from technical complexity to organizational resistance. Understanding these challenges is essential for businesses hoping to successfully implement AI and get a positive return on their investments.
Technical Complexity and Data Requirements
One of the first hurdles businesses frequently encounter is the technical complexity of AI systems. AI models, particularly those based on deep learning, are computationally demanding and require specialized knowledge to create, train, and deploy. Data is also required. The training data’s quality and quantity have a substantial impact on the performance of AI models. Collecting and preparing hugedatasets may be a time-consuming and resource-intensive process. AI projects may be hampered by a lack of high-quality, labeled data, which results in inaccurate or prejudiced models. Organizations should invest in data quality and data governance to ensure successful AI implementations.
Furthermore, the selection of the right AI platform and tools is critical for managing this complexity.
Furthermore, guaranteeing the interoperability of AI systems with existing IT infrastructure introduces further complexity. Different AI platforms and frameworks may not be compatible with legacy systems, necessitating substantial changes to existing workflows and architectures. Integrating AI into complicated organizational environments often necessitates considerable experience and a strong grasp of both AI technologies and the underlying commercial operations. Using APIs and standard data formats can help to improve interoperability.
Organizational and Cultural Barriers
Besides technical obstacles, organizations may encounter substantial organizational and cultural hurdles to AI uptake. One prevalent issue is workers’ reluctance to embrace AI-driven changes. Employees may be concerned about job displacement as well as the need to learn new talents and adapt to new working methods. Resistance from workers may hamper AI initiatives and impede the realization of anticipated advantages. Open communication, training programs, and demonstrating the benefits of AI to employees can help to overcome this resistance.
Leadership buy-in and support are also essential for driving cultural change.
Furthermore, AI deployment necessitates considerable collaboration between departments and teams. Data scientists, IT professionals, business analysts, and subject matter experts must collaborate to define problems, create AI solutions, and deploy them into production. Silos and a lack of communication may stifle cooperation and impede the effective integration of AI into commercial operations. Overcoming these organizational and cultural obstacles necessitates strong leadership, effective communication, and a dedication to change management. Establishing cross-functional teams and promoting a culture of collaboration can significantly improve AI adoption.
Ethical and Governance Concerns
As AI becomes more widespread, ethical and governance issues become increasingly important. AI systems have the ability to perpetuate prejudices, make unfair judgments, and infringe on people’s privacy. Organizations must address these concerns by developing robust ethical guidelines and governance procedures for AI design, development, and deployment. Transparency, accountability, and fairness are key principles for responsible AI. Regular audits and impact assessments can help to identify and mitigate potential ethical risks.
Data privacy is an important issue to consider. Data privacy rules must be followed while building AI systems, along with safeguards to protect sensitive information from unwanted access or abuse. Organizations must obtain user consent for data collection and usage, as well as provide transparency about how AI models are making choices. Furthermore, organizations should have mechanisms in place for monitoring and auditing AI systems to discover and mitigate any ethical risks or unwelcome consequences. Implementing robust data security measures and complying with regulations like GDPR and CCPA are crucial for maintaining user trust and avoiding legal penalties.
Measuring and Demonstrating ROI
Ultimately, the success of any AI project depends on its capacity to produce a quantifiable return on investment (ROI). However, determining the ROI of AI projects may be difficult, particularly when benefits are intangible or long-term. Organizations must establish clear goals and indicators for their AI initiatives, as well as track progress and measure results regularly. This necessitates a thorough grasp of the business value AI is expected to deliver as well as the resources necessary to attain that value. Defining key performance indicators (KPIs) and using appropriate metrics to track progress are essential for demonstrating ROI.
Furthermore, communicating the benefits of AI to stakeholders is critical for gaining support and establishing confidence in AI investments. This may entail presenting use cases, showcasing early triumphs, and quantifying the impact of AI on essential business indicators. To successfully quantify and show the ROI of AI, businesses must create a defined framework for measuring performance and clearly express the value proposition to stakeholders. Regularly reporting on the progress and impact of AI initiatives can help to maintain stakeholder engagement and secure continued investment.
The Future of AI Adoption: A Balanced Perspective
Ivan Zhang’s insights highlight the importance of a balanced approach to AI adoption, one that acknowledges the technology’s potential while remaining grounded in practical realities. As AI continues to evolve, companies will need to focus on building solutions that deliver tangible ROI, address ethical concerns, and integrate seamlessly into existing workflows. By prioritizing business problems over flashy solutions and viewing AI as a tool in the toolbox, organizations can unlock the true potential of AI and drive meaningful business outcomes. This requires a strategic, data-driven approach that focuses on delivering real business value.
Organizations should invest in building internal AI capabilities and fostering a culture of innovation to fully realize the potential of AI. Continuous learning and adaptation are essential for staying ahead in the rapidly evolving field of artificial intelligence.