QA service

Git: https://git.picalike.corpex-kunden.de/incubator/ai-similarity-model

URL: http://report-db01.picalike.corpex-kunden.de:31415/docs

Landing page: http://report-db01.picalike.corpex-kunden.de:31415

The goal of this simple service is to provide a frontend so that people can vote on which (OSA) attributes are, in fact, visible in the presented images. This is serve as training data for our similarity model. Please check the git for more information on the README. This is a subproject inside ai-similarity-model, the source code is in ai-similarity-model/qa_service.

Remarks

Currently, otto_de_crawler and about_you_de_crawler data are imported into the service
The mapping team has QAed attributes that are generally visible on the images, guided by use-cases (e.g. which attributes are likely relevant for consumers interested in shoes, for instance)<HTML></ol></HTML>

Operations

It should be noted that importing a shop will block the service, so this should be done only outside regular working hours. For smaller shops, a “maintenance window” could be announced, but for larger shops it definitely tooks too long.

Doing QA

1. Basic QA

Head to the /qa endpoint. Select one group then one attribute. Then, select the images where the attribute is clearly visible and click on send answers. You'll receive more images from this endpoint as long as there are images to be QA-ed.

2. Second QA

Head to the /second_qa endpoint. Select one group then one attribute. The visibility status is marked with an emoji (checkmark for visible, red X for not visible). If you find obvious mistakes, select them and click on delete selected (be cautious). Only those will be deleted and will be presented again for QA. Then, select bad images (e.g. placeholder images) that should be hidden (not back to QA and not returned as labels) and click on hide selected. All images, including those unmarked, will be set as second-QA-done and will not be shown again anywhere.

If you find mistakes on the second QA

If they are QA mistakes, select the images and click on the delete selected button. These images will return to the QA queue. This button does nothing to the unselected images.

If they are bad images (e.g. placeholder images), select them and click on hide selected button. This will mark these images as hidden and all the other images in the page as OK and second-QA-ed.

Mapping Team

The mapping team created a Bug Tracker with open issues: https://docs.google.com/spreadsheets/d/1PupOe6tQ7ZdAA_1Vp_dNY6IE0p2-Fcb6Bzc63aAmJew/edit#gid=0