May 14, 2024

Digital Journal A recent survey found that the biggest challenge facing software developers in 2024 is incorporating AI By Photo courtesy of Ariel Weinberger Opinions expressed by Digital Journal contributors are their own. Artificial intelligence represents a significant leap forward in technology, and its use cases for growing businesses are vast. However, many companies struggle to actually integrate AI into their products. There are many reasons for this lost opportunity. One is that IT teams trying to incorporate artificial intelligence into their work don’t have the necessary tooling. Although there’s been a surge in AI adoption over the past year, there’s been little to no improvement in tooling for production-level AI use. As a result, it can be quite difficult to turn cutting-edge AI research into practical business use cases. That’s where Pezzo steps in. Pezzo is an open-source LLMOps (large language model operations) platform that helps businesses implement artificial intelligence into their products. It allows anyone in the company to use and collaborate on AI features without having to hire scarce and expensive technical experts. In doing so, businesses can enrich their products with AI and keep innovation in the hands of those who know their products best. Learn more about the common challenges businesses face with AI integration, how Pezzo’s open-source platform can fill the gaps, and how CEO and founder, Ariel Weinberger, is democratizing AI for developers and product managers. AI’s integration problem A recent survey found that the biggest challenge facing software developers in 2024 is incorporating AI. Many developers named generative AI’s lack of creativity, innovation, and security as their top concerns , along with its frequent errors and bugs. For example, while LLMs like OpenAI’s GPT (powering ChatGPT) are quite capable, it’s impossible to control their knowledge base , which is composed of data from all over the internet. This can lead to inaccurate responses, also known as hallucinations. These mistakes pose a significant risk to businesses, and without the ability to observe the AI, hallucinations are very difficult to catch. What’s more, AI tools often alienate those who know their business best: non-technical stakeholders such as product managers. And in an era where tech is made increasingly accessible so professionals can use it without advanced knowledge, this is a missed opportunity. Finally, there is the ever-present problem of security, especially in emerging tech like AI. Without the ability for business owners to keep a tight grip on their applications, they expose themselves to the liability of costly fines and serious damage to their brand reputation. To overcome these and other challenges, businesses often fight for expensive talent to make sure they have the required resources to use AI in their organizations. But even then, artificial language and machine learning experts usually work independently from product teams — who, again, are the ones who usually understand the customer best. AI integration, then, becomes a siloed and expensive process where businesses aren’t able to deliver maximum value to their end users. How Pezzo streamlines AI integration into product development Pezzo aims to solve these challenges by being an open-source, user-friendly platform that businesses can use to more effectively manage their AI operations. It’s a toolkit designed to overcome AI integration and development issues by streamlining prompt management and increasing observability. Pezzo’s prompt management feature lets users easily create and validate AI prompts in plain English. For example, a sales manager might input a prompt like “Generate a monthly sales report for region X using parameters Y and Z.” Once validated, these prompts are deployed into APIs that can be used directly by developers, the same way they consume any other API. The process of AI ideation, fine-tuning, and quality control can be done by anyone in the business, without requiring any coding skills. According to founder and CEO, Ariel Weinberger , “This approach makes AI accessible to developers, like any other integration — just one API call away.” Pezzo’s observability features further empower businesses to manage AI operations. As Ariel puts it, “When we ship services to production, we need to have observability features in place to troubleshoot errors, ping us with notifications, and identify the precise piece of code that caused the problem.” However, AI models don’t have this observability built in. But with Pezzo, users can monitor AI requests and get a granular overview of details like cost per user, per organization, per AI model, and per prompt — down to the day, week, or year. “AI can also hallucinate sometimes,” Ariel says, referring to AI’s tendency to produce bizarre responses that don’t fit the prompt. “With Pezzo, you can view historical errors, basically time travel, and change the prompt to see what the output would have been. If you like that, you can push a new version instantly. And guess what? You don’t need to write a single line of code.” This introduces observability to AI integration in a way that significantly reduces technical errors, bias, inappropriate responses, and unexpected costs. The key to successful AI integration: Open source For Ariel Weinberger, the key to Pezzo’s success is that it’s entirely open source. He has over ten years of experience leading teams in developer tools and open source, but he quickly learned that the AI space was dominated by a different group: scientists and researchers. “These people don’t ship things to production,” he says. “They work in very sandbox environments, in lab conditions.” And while those conditions are great for experimenting, they are vastly different from production environments, where users may have malicious intentions and applications face inputs they never expected. “Production environments serving real users simply work differently,” says Ariel. Open-source software like Pezzo democratizes AI tool development by letting more people contribute to and improve it. Developers can modify the software according to real needs, experiment with the code, and share those changes with other users — making Pezzo’s software better and more relevant for everyone. “I don’t regret building Pezzo as an open-source project from day one,” says Ariel. Pezzo’s open-source version has been downloaded over 500,000 times. According to Ariel, there are more benefits to Pezzo being open-source than not: “People often ask me why I made Pezzo’s source code entirely open and free. The value I get from having access to early adopters of AI — helping them in the trenches as they solve unfamiliar problems — is an incredible opportunity, and I do not regret it whatsoever.” Developers at the helm of AI While large-scale innovations like AI often come from painstaking research, the traditional developer community has a real knack for open-sourcing and improving these technologies for businesses to use. Pezzo’s open-source AI integration platform makes this innovation handoff easier, faster, and more affordable so that product managers can focus on refining their products with AI and developers can handle the nitty-gritty details that bring these products to life. Building and maintaining open-source software is challenging, as Ariel explains: “Open-sourcing a project makes it significantly harder to build. People criticize your code, hackers look for vulnerabilities in your software, and you have to document everything you work on to make it accessible to other contributors.” On the flip side, these same challenges help open-source software like Pezzo better earn the trust of the developer community. To Ariel Weinberger, it’s all about democratizing AI integration through prompt management and robust observability features, which starts with open-source principles at the core.