Leora: AI-Powered Voice Sales Assistant

To improve the accuracy of responses, we created a prioritization strategy, according to which Leobit’s service descriptions were given the highest importance, followed by case studies and blog posts. That way, when the LLM queries Cognitive Search, it gets the most reliable, relevant content first.

The web scraping part of our system uses Selenium, a popular web automation tool. Selenium is usually used by automation testers to simulate user actions in a web browser. We adapted Selenium to scrape web pages by automatically browsing the Leobit website and extracting the needed information. This data is then saved into a spreadsheet for further use.

We built the front end using modern React patterns to manage state and handle asynchronous communication with Azure services, including speech recognition, text generation, and voice output. React’s modular design allowed us to prototype quickly, test features in isolation, and update components independently as the system evolved. We wrote the codebase in TypeScript to reduce errors and improve long-term maintainability. For a faster and smoother development experience, we used Vite with Rollup, which also helped speed up the release process.

We explored a few options and decided to go with a pre-made 3D character model, which we customized to reflect Leobit’s branding. We also used publicly available animations, like hand gestures and head movements, to make the avatar feel more natural. We used the Three.js engine to display the animated avatar in a browser. The result is a speaking, animated character that gestures and moves in sync with its voice.

While some parts of the system are already multilingual (e.g., Leobit’s LLM can understand and respond in many languages), the current voice interface works only in English. That means users can interact with the system by speaking or hearing responses only in English, even though the model itself could handle other languages if given written input. In future updates, we plan to add multilingual support to the speech recognition and text-to-speech components. This would allow users to speak and receive responses in different languages, making the entire experience fully multilingual.

Since the webpage is not public yet and is accessible only within the company, it is secured behind Azure Active Directory (Azure AD) using Enterprise ID authentication. This means users must be authorized through Azure AD to access the site, so that only company employees can access the content.

We gather content from several sources, not just our public website, but also our CRM records, internal documents, case studies, and presentations. After collecting these materials, we run them through a cleanup pipeline: we remove duplicates, filter out irrelevant items, convert the files to a consistent format, and organize everything into a clear structure. Initially, we handled some of this work manually, but we have since automated most tasks with custom tools that run as Azure Functions. These functions now process new data once a day, keeping the knowledge base current without extra effort from the team.