Leobit
AI-Based Digital Dermoscopy Application
Development of a Compex HealthTech Solution Detecting Skin Changes
ABOUT
the project
Client:
Location:
Germany
Company Size:
Industry:
Solution:
Leobit helped a European skin imaging system company enhance its existing application for skin change detection and develop a new kiosk app that would work with the company’s hardware. Our skilled developers performed refactoring and made architectural code changes to ensure the code works simultaneously in two applications without causing any disruptions. The newly developed application works paired with a special dermoscopy case – designed to close the gap between conventional handheld dermatoscopes and stationary mole mapping systems.
We were thrilled to contribute to this innovative healthcare project, empowering our client to maximize the full potential of their portable dermoscope device.
CUSTOMER
Our client is a renowned global brand specializing in medical skin imaging systems. They provide comprehensive hardware and software solutions designed to detect skin changes and identify skin cancer at the earliest stages. Their hardware portfolio includes automated total body mapping devices and portable dermoscopy devices that transform smartphones into digital dermoscopes. The company’s AI-powered medical imaging software, validated in numerous clinical studies, also offers AI Scoring for pre-assessing melanocytic and non-melanocytic skin lesions.
BUSINESSCHALLENGE
The client wanted to revamp the business logic and architecture of an existing solution, aiming for more flexibility and easier maintenance of its modules. Concurrently, they were grappling with a distortion issue in photos taken using hardware, which posed a challenge for AI skin change detection algorithms to analyze properly.
During the development journey, the product owners opted to pivot the concept. They wanted to create a new kiosk application using the existing code components. Through a specific connection protocol, this application would communicate with a specialized hardware case (transforming a smartphone into a digital dermoscope).
WhyLeobit
The customer chose Leobit because of our expertise in healthtech and high Clutch ratings. They were drawn to our reviews, extensive experience in embedded software development, and the innovative ideas proposed by our tech experts to address their technical challenges. Having established trust in our capabilities, the client subsequently engaged Leobit in developing an additional application using our UI/UX design services as well.
Project
in detail
Leobit refactored and redeveloped several modules, altering the architectural logic of an existing application, and developed a new kiosk app with a primary focus on camera functionality. Our approach to coding ensured code compatibility for use in both mobile applications.
distortion elimination
When a lens is placed over the phone camera, it produces distortion (similar to a fish-eye effect), posing a challenge for proper analysis by AI skin change detection algorithms. This presented challenges for both user experience, as users saw distorted images, and the final logic (distorted images triggered AI algorithms as suspicious). Leobit experts found a solution by dynamically rewriting the camera preview. We developed a plugin powered by OpenCV, running the image in real time through an algorithm that corrects this distortion. As a result, users now see stabilized images that AI algorithms can properly analyze.
Hardware SDK Development
Previously, the lens case communicated with the application via Bluetooth and had limited functionality, such as switching on/off the lens light and adjusting brightness. Leobit addressed this limitation by developing a specialized Hardware SDK that utilizes the D2XX library for communication with the FTDI case.
The application seamlessly integrates with a mobile device housed in a specialized case, connecting through USB. A Hardware SDK enables deep integration with the case through byte commands. This integration facilitates retrieving data from the case and allows controlling the lens connection, toggling its power, and changing the lens light type between polarized and non-polarized modes. Additionally, the SDK monitors lens connection and disconnection events, ensuring high interactivity and control over the system's physical components.
AI-powered skin change detection
The skin change detection application leverages a deep learning algorithm, previously validated in numerous clinical studies. Leobit's role was to prepare information for a company’s deep-learning algorithm to be able to analyze it properly. It involved utilizing a Tensorflow model capable of processing photos presented as bytes. Before analysis, the photos undergo optimization to align with the input format required by the deep learning model, a process facilitated in part by OpenCV.
The Tensorflow model yields two types of data: the photo's type (whether it is valid and suitable for effective processing by deep learning) and whether human intervention is needed for analysis. The second type is an artificial intelligence assessment expressed as a fractional number ranging from 0 to 1. A higher score indicates an elevated risk of the patient having a malignant skin formation.
Furthermore, we implemented an SDK plugin that allows the client to use another pre-trained ML model or enhance the existing one. We’ve also fine-tuned a process, so users can now specify a mole's location on the body, facilitating more precise AI analysis.
Data synchronization
Data synchronization plays a crucial role as the application supports offline mode. Upon reconnecting to the internet, it necessitates synchronizing all data, including patient information and photos, with the central system. For certain clients with extensive patient databases and numerous photos, the synchronization process could take up to 10 minutes.
To enhance this process, Leobit implemented improvements and modifications to the synchronization logic using Dart isolates and employing SQL Lite queries. This approach involved dividing the synchronization into three distinct blocks that could synchronize concurrently. The outcome was a remarkable acceleration in synchronization speed, making it 3x faster. Additionally, users can continue using the application seamlessly, even during the synchronization process.
QR code provisioning and authentication
To ensure the seamless deployment of the application on new devices, we implemented a QR code provisioning feature. Scanning the QR code on a new phone automatically initiates the installation of the application. It activates the kiosk mode, significantly simplifying the setup and deployment process at a large scale.
We've incorporated QR code authentication into the app alongside the conventional email and password method to simplify the login procedure. Users can choose the "Login via QR" option on their web page profile. Afterward, they can access the "Login via QR Code" page in the mobile app, where they scan the code using the camera. If the code is valid, the system notifies the user of the successful action and proceeds to the next steps. In case of any issues, the system offers user-friendly prompts to guide them through the login process.
The Journey
Behind Our Success
Technology Solutions
- Altering the architectural logic, refactoring, and redeveloping several modules of the existing application and developing a new kiosk application from scratch.
- Real-time photo rendering to eliminate distortion when using a lens thanks to using Darf FFi and OpenCV library.
- Preparing photos for ML analysis utilizing a Tensorflow model and OpenCV library.
- Using Dart isolates and employing SQL Lite queries to streamline data synchronization.
- Developing a specialized Hardware SDK that utilizes the D2XX library for communication with the FTDI case.
- Using F-Droid to enable automatic application updates without any user interaction.
Valuedelivered
- Seamless bidirectional communication between hardware and software.
- Blazing fast app installation and login thanks to using QR code provisioning and authentification.
- 3x faster data synchronization in the background.
- User-friendly UI/UX design for mobile and web version of the software.
- Enhanced camera capabilities.