AI-Powered Assistant Creation for a Global Finance Data Provider

• Project: PoC of an AI-powered assistant that interacts with clients searching for information of interest.
• Duration: 5 weeks
• Technologies: Python, ChatGPT 4.0, Azure DevOps

Barchart is a global financial technology leader providing market data and services to the financial, media, and commodity industries.

The Client’s Request

Manual information search does not contribute to a good user experience. Having acknowledged this plain truth, the client decided to automate their interaction with customers and implement the capabilities of GenAI into their workflows. The idea was to create an AI-powered research assistant providing the requested information in a clear and well-structured form.

Check out how we delivered an AI Call Analytics Solution for Insurance Company

Being fully aware that many software development projects fail and many comprehensive solutions turn out to be not as cost-effective as expected, the customer decided to test the waters in a smart way. They chose in favor of a POC creation, covering only one aspect — all data aggregated from grain elevators.

To create an assistant within strict timelines, Barchart needed a reliable software development partner, proficient in AI implementation and data management. Velvetech suited in all respects: our strong technical background and rich portfolio convinced the customer to involve our engineers into this ambitious initiative.

The Process

The finance AI assistant is built on the basis of ChatGPT-4.0, which interacts with users through a live chat. It works as follows: when a customer enters a query, the bot sends the request to API and returns the result in a formatted, well-structured, and digestible form.

Our team elaborated the major functionality, which included backend development and API integration, having ensured correct data transfer from data sources and real-time processing.

Streaming Implementation

The customer emphasized the critical role of user experience in this project. That’s why it was agreed to incorporate the streaming feature into the assistant to accelerate request handling. In other words, when a user submits an inquiry in the dialog box, the cloud-hosted backend sends it to the LLM, which processes it and transfers the response back to the backend for real-time delivery to the user.

This feature makes interactions with the bot more seamless and dynamic, significantly reducing response times. As a result, users enjoy a more fluid experience, which ultimately boosts overall satisfaction.

Function Calling

Only several weeks after OpenAI released function calling tool usage, we implemented it in this project. In general, this ability empowers programs to call or launch particular functions (with pre-written pieces of code) for concrete task fulfillment.

“Tool usage and function calling are some of the most important and promising GenAI patterns. They enable AI models to interact with external systems, APIs, or tools to extend their functionality beyond text generation. With these capabilities, models can retrieve real-time data, execute code, automate workflows, or control applications.”

In our case, the language model acknowledges which data it needs to request and which functions need to be called during the text generation. With the pre-written code, the model understands which toolset is in its possession, operates it as needed, and might call one function from another.

For example, we have a function written with a regular code and named “call the client list.” Our model is backed by links to this code, and there’s no need to generate it repeatedly. Therefore, the LM returns the client list very quickly.

ON-DEMAND WEBINAR

GenAI for Business

Watch our webinar to uncover how to integrate GenAI for improved productivity and decisions.

WATCH NOW

The Challenge

As a whole, the project ran smoothly and without serious technical or communicative obstacles. The only complexity our engineering team faced was the amount of data our client operates with.

Even though the POC covered only one aspect related to grain elevators, huge data volumes turned out to be a real challenge for streaming implementation, especially in terms of the backend part. However, our engineers successfully handled the task, and the model generates data smoothly in real time, which is highly appreciated by the client.

The Outcome

The project has now been completed, and the customer is currently in the testing phase for the assistant. Overall, the POC was delivered on time and within the agreed budget.

During the development, we held regular sync-ups with the customer to maintain full transparency on project progress. Our team also proactively suggested product enhancements and demonstrated flexibility in adapting to the client’s evolving requirements.

What’s Next

The team successfully delivered the POC, marking a significant step towards the final product. However, it’s just the foundation, and functionality still needs to be expanded before the solution is ready for full deployment.

For instance, the current version of the AI-powered finance assistant can only interact with one user at a time, and there is no database integration, meaning that communication history cannot be saved or referenced for future interactions.

Looking ahead, the customer plans to get back the product development and transform the POC into a fully-functional AI-powered assistant. This will involve enabling multi-threading for simultaneous user interactions, integrating a robust database to store conversation histories, and implementing advanced analytics to gain insights into customer behavior and preferences.

These enhancements will significantly elevate the assistant’s functionality, making it a more comprehensive tool for streamlining communication and improving user experience.