dev02: The new home of the v5

The v5_extractor plus database lives on this host. The base folder is

~/v5

• v5_extractor_container to refresh shops images with an enrichment
• v5_extractor_db to query features

In the ~/v5 folder, there is a requirements.txt that contains all packages that should be available in the default Python 3 environment.

• pypg.py a wrapper to execute SQL commands non-interactively (vaccum, create xyz)
• refresh_prelive_db.sh to refresh the database after each crawl

This is a recurring task every Monday

The cronjobs send a summary mail that can be browsed with mutt. Use 'd' for delete and 'q' for quit (as shown in the menu bar).

This allows an overview of the last updates. The output of the 'prelive refresh' should be analyzed more thoroughly to ensure that there were no problems with the latest data.

A simple script (~v5/scripts/speed_monitor.sh) runs once an hour to perform some queries. If the runtime is above a threshold, an alert is send to the slack channel.

A netcup ping monitoring script is run by the cronjob every 15 minutes (scripts/netcup_ping.py) and alerts in the slack channel if the ping takes too long.

Keywords: dev02 v5 features psql ping alert