Picalike Dokuwiki Archive

repo: https://git.picalike.corpex-kunden.de/incubator/visual-mapping

An extension of the mapping service functionalities that relies on visual data to enhance product information mapping.

There are two data sources right now:

• BigNet features from the v5/feeds (stored on dev02);
• ColorNet features;

For now the service is minimal and just enriches OSA attributes with colors.

As of now, attributes are imported using two scripts:

• scripts/sqlite2postgres.py to import sqlite DBs (BigNet attributes from v5/feeds on dev02)
• scripts/postgres2postgres.py to import data from the v5_features table (postgres sim database on report-db01)

• schedule_shops: takes a list of shop_ids, a list of attribute groups and batch size as input. It's a fire and forget endpoint that returns immediately.
• map_shop: takes a shop_id, a list of attribute groups and batch size as input. Returns a JSON with total, modified and error counts.

Any visual inference is orders of magnitude slower than simple keyword search (unless we use GPUs). Therefore, features must be extracted before the mapping happens. Still, some features are likely missing and this mapping step might be very slow. For this reason it was chosen to put this service in a separate repository outside of mapping-service.

This choice, however, brings a major downside: as new shops are schedules for mapping, all the attributes added by the visual-mapping will be erased until the visual-mapping runs again for that shop. There needs to be a way to integrate both services and keep them in sync. This is not implemented.

Check repository README.md