Today, smart technology can bring equal improvements to all the stakeholders engaged in regular B2C deals. Innovation fits seamlessly into our daily processes: from business to family traditions. And sometimes, it can even solve work-family conflicts.
Surprised? Let me tell you about one of our Computer Vision use cases and how we have helped save a holiday and our customer’s business idea.
Once upon a time, before Christmas
You are probably familiar with a classic pre-holiday situation: you are overwhelmed by strategic deadlines at the end of the year while your household asks you to choose Christmas decorations to impress your neighbours and friends.
You need to make time to finish important business tasks, order festive lights, buy presents, and by Christmas Eve, you feel like a warmed-over omelet. And retailers serving a massive flow of Santa’s elves also do.
Modern challenges require modern solutions; that’s true. When we get acquainted with Reegeel, they definitely have one powered by Computer Vision. Just imagine: you upload a photo of your house into the system and receive a ready-made lighting plan, including the type and number of lamps and light strings required. Sounds great.
However, our customers could not achieve accurate system outcomes and deliver a valuable service to their clients for a long time. What was the case? Let’s find out.
The challenge
As we have said before, Reegeel has an idea to make outdoor lighting easy for their clients to plan. This way they wanted to optimize design process by reducing efforts of the team and estimated time for each design project.
They developed a service for its users to automatically generate outdoor lighting plans for buildings based on Computer Vision: not only for festive illumination but for all kinds of outdoor lighting. Thanks to it, their users can upload a photo of their house to the system, get different lighting options, choose the one they like, then pay and wait for their order.
When Reegeel came to us, they had already worked with two software development teams but were still not satisfied with the accuracy of the results of the algorithms. They asked us to analyze the existing solution and develop suggestions for improvement.
The solution
We started with analyzing the current solution to understand why the algorithm produces inaccurate results. We had several hypotheses, and the main was the problem with the quality of the initial data used to create the dataset and train the model.
One of the main tasks the model performed was image segmentation. The dataset was created based on images of buildings freely available on the Internet and was labeled into 11 classes. Classes included walls, roofs, plants, driveways, and so on.
Semantic segmentation
According to the documentation provided, the accuracy according to the IOU score (the standard metric used to evaluate data segmentation) ranged from 50% to 82%, depending on the class. Far from the best result.
We analyzed approximately 5% of randomly selected photos from the dataset provided by the customer. During the analysis, we identified the following problems with the data used to train the model:
- Low photo resolution (less than 1024 px on the smaller side);
- Lack of unified labeling rules (for example, somewhere the houses in the background are marked, somewhere not). Is there discrimination in Real Estate?
- Syntax errors (for example, the wall is marked as a road);
- Photos were taken mainly in the USA (architecture there has its specifics);
- No snow was seen in the dataset! All photos were taken primarily in summer but what is Christmas without a snowman?
- Low quality of the mask.
Obviously, the result of segmentation can be significantly improved by better preparation of the training data. To do this, our team has developed a plan of actions required to create a high-quality dataset and train the model once again:
- Determine the target group of images that will be as similar as possible to client images and have a diverse context – time of year, angle, type of architecture. Collect them in a test dataset in the amount of about 100 pieces;
- Based on the test dataset, determine the main data characteristics on which the training dataset should be developed;
- Develop data labeling rules and determine the quality of detail (The quality of detail affects the labeling speed and the final values of the IOU metric. That is, for values of 80%, you can mark up faster and less efficiently; for values closer to 90%, the markup quality should be higher);
- Form and label a dataset to train the model one more time.
In our experience, to obtain more than 80% IOU accuracy of results for all 11 classes, we need from 3500 to 5000 well-labeled images. For ensuring the highest quality of entire solution output, we prepared an automatic script for validating the accuracy and tested it on a reference dataset of 100 qualitatively labeled images.
The system output
Now, the client can use the script to prevent poor performance of the solution. Our Solution Architects also analyzed the current code and, together with the team, drew up a detailed roadmap for the further development of the project to obtain maximum results at minimum cost.
The successfully completed Discovery Phase has already brought value, reduced risks and became the great foundation for the future development of the client’s project.
Your business deserves innovation
Whether you sell outdoor lighting or run a complex production of sewing equipment, you can adopt modern technology for your business. Bringing innovation to your daily business processes helps to stay focused on your main goals, optimize resources, and stay competitive in the market.
In case you are afraid of making an unprofitable investment, we can help you to validate your specific business case. Contact us to get professional consulting services from our experts, or visit Computer Vision page to learn more.
