Leobit
Claude-Powered AI Support Agent in Microsoft Teams
Custom development for for ACT Bermuda, a Caribbean IT services company, that reduces the support team’s workload by 3x
ABOUT the project
- Client:
- ACT Bermuda
- Location:
-
Bermuda
- Company Size:
- 50+ Employees
- Industry:
-
IT Services and Consulting
- Solution:
- Custom AI-Powered Software
Leobit helped ACT Bermuda build an AI support agent powered by Claude that automates first-line support in Microsoft Teams. The customer provides a customized Claude desktop deployment to its end clients, and the agent answers users’ questions. It handles routine requests using the customer’s knowledge base and routes the rest to support specialists. Our team validated the concept in a four-week proof of concept with a small group of the customer’s clients. After that, we moved the agent to Claude and rolled it out to the customer’s end clients.
The PoC showed that answer quality depends on how well the agent works with the customer’s own documentation. When we moved the agent to Claude, we kept the Microsoft 365 Agents SDK, the Teams integration, and the Azure retrieval layer and changed the model layer. The customer took the validated PoC to production without a rebuild.
Customer
ACT Bermuda operates primarily in Bermuda, the Cayman Islands, and the Bahamas, providing technology services to government entities and enterprise organizations across the region, including a customized Claude desktop deployment for end clients.
Business Challenge
As demand grew, the customer’s support team faced a growing volume of client inquiries, which put pressure on response times and service quality. The customer decided to build an AI support agent that could handle routine client queries on its own.
Why Leobit
The customer chose Leobit for its track record in AI development and engaged us as a technology partner for a wider AI transformation initiative that includes several solutions. For this project, Leobit’s experience building Leora, our AI-driven sales assistant, proved especially relevant.
Project
in detail
The agent started as a proof of concept on Azure, built with the Microsoft 365 Agents SDK and Azure OpenAI. After the customer validated the PoC, our team moved the agent’s model layer to Claude and took the solution to production.
Our team configured the development environment, repository, CI/CD pipelines, Azure Resource Group, and the project foundation required for the AI agent. Even at the PoC stage, this groundwork was necessary for the solution to work as expected.
Our team built the agent with the Microsoft 365 Agents SDK, connected it to the customer’s curated knowledge base, and implemented tenant-aware security and data isolation to protect sensitive information. The team configured and refined prompts, semantic search, response behavior, and uncertainty handling to improve the quality of the agent’s answers.
Our specialists configured a query-escalation workflow to prioritize user requests more efficiently and integrated the agent with Microsoft Teams. The team then tested the key workflows and ran pilot testing in real-world conditions with a small group of the customer’s clients. This helped us validate the PoC within four weeks.
Our team replaced Azure OpenAI with Claude as the agent’s model and adapted the prompts, retrieval, uncertainty handling, and escalation logic for Claude. For end-client answers, the team runs Claude with adaptive thinking at the extra-high effort level, so the model reasons more thoroughly before it replies. The customer already provides Claude to its end clients, so the support agent now runs on the same model family as the customer’s end-client product.
Leobit engineers prepared the production environment on Azure and rolled the agent out to the customer’s end clients in Microsoft Teams.
AI Agent development
Our team built the agent with the Microsoft 365 Agents SDK, connected it to the customer’s curated knowledge base, and implemented tenant-aware security and data isolation to protect sensitive information. The team configured and refined prompts, semantic search, response behavior, and uncertainty handling to improve the quality of the agent’s answers.
Information Retrieval
Our team organized a curated knowledge base with the customer’s documentation, system runbook, and additional marketing materials. The agent uses retrieval-augmented generation (RAG): Azure AI Search with a custom RAG configuration finds the relevant fragments in the approved datasets, and Claude writes the answer from these fragments instead of relying on general knowledge. As a result, users receive answers within seconds, based on relevant and consistent company data.
Query Escalation and Notifications
Leobit implemented the query escalation workflow as part of the agent built with the Microsoft 365 Agents SDK. When the knowledge base does not cover a question or the user asks for a specialist, Claude Opus 5 calls the escalation tool and returns a structured assessment with the request’s relevance, urgency, priority, and category. The agent matches the category with the customer’s specialist roster and adds a conversation summary with client details from the Teams context. The specialist receives the request by email through Azure Communication Services and as an Adaptive Card in Microsoft Teams, sent as a proactive bot message. Every escalation is logged in Azure Application Insights.
Project
Phases
Technology Solutions
- Built the support agent with the Microsoft 365 Agents SDK and moved its model layer to Claude for natural language answers, request assessment, and conversation summaries.
- Configured retrieval over the customer’s knowledge base with Azure AI Search and custom RAG configurations.
- Deployed the solution on scalable Azure infrastructure: Azure App Service, Azure Blob Storage, Azure Key Vault, and supporting cloud resources.
- Set up email and Microsoft Teams notifications for support specialists through Azure Communication Services.
- Implemented tenant-aware security and data isolation to protect sensitive information.
Value Delivered
- Automated routine client queries with an agent that reduced the support team’s workload by 3x.
- Automated first-line support for the customer’s end clients in Microsoft Teams, a familiar interface for them.
- Provided support specialists with prioritized requests, conversation summaries, and client context.
- Put the customer’s end-client product and its support agent on the same model family, Claude.
- Provided a flexible foundation that lets the agent become more accurate and cover a wider range of support requests over time.
