Hi all,
There is a new release of the Overpass API https://dev.overpass-api.de/releases/osm-3s_v0.7.56.6.tar.gz that adheres to these rules.
Unfortunately, getting network communication right is hard. In that version, there is a race condition. For that reason, there is another new release 0.7.56.7. You can run that release and keep the database, in particular if an update based on 0.7.56.4 or 0.7.56.6 has ended with gzip: stdin: unexpected end of file Reading XML file ...Parse error at line 23105: unclosed token This means that apply_osc_to_db.sh has tried to use an XML file before fetch_osc.sh has completed to download it. It is harmless, in this case the automatic emergency stop has applied early enough. Please do the following (with EXEC_BASE_DIR, DB_DIR and DIFF_DIR appropriately replaced): # Install new bash scripts over the exisiting broken ones $ pushd "$EXEC_BASE_DIR" $ cp ~/osm-3s_v0.7.56.7/bin/fetch_osc.sh bin/ $ cp ~/osm-3s_v0.7.56.7/bin/apply_osc_to_db.sh bin/ # Kill the frozen existing processes and remove the lock file $ ps -ef | grep apply roland 19676 1 0 Aug13 ? 00:00:08 bash bin/apply_osc_to_db.sh /opt/osm3s/diff/ auto --meta=attic $ ps -ef | grep 19676 roland 19676 1 0 Aug13 ? 00:00:08 bash bin/apply_osc_to_db.sh /opt/osm3s/diff/ auto --meta=attic roland 20652 4841 0 04:09 pts/0 00:00:00 grep --color=auto 19676 roland 21629 19676 0 Aug13 ? 00:00:00 ./update_from_dir --osc-dir=/tmp/osm-3s_update_VUkjcE --version=2020-08-13T20\:33\:03Z --keep-attic --flush-size=0 $ kill 19676 21629 $ rm /ssd/osm3s/db/osm_base_shadow.lock $ ps -ef | grep fetch roland 23675 1 0 Aug13 ? 00:00:06 bash bin/fetch_osc.sh 4146694 https://planet.openstreetmap.org/replication/minute/ /opt/osm3s/diff/ roland 27830 4841 0 04:11 pts/0 00:00:00 grep --color=auto fetch $ kill 23675 # Restart the diff processes $ nohup bin/fetch_osc.sh $(cat /ssd/osm3s/db/replicate_id) 'https://planet.openstreetmap.org/replication/minute/' "$DIFF_DIR" >>fetch_osc.out & $ cat "$DB_DIR/replicate_id" 4150150 # Must exist and contain a meaningful diff id, this is a new lock file $ nohup bin/apply_osc_to_db.sh /opt/osm3s/diff/ auto --meta=attic >>apply_osc_to_db.out & $ sleep 30 $ tail apply_osc_to_db.out /opt/osm3s/bin /opt/osm3s Reading XML file ... # Should show some messages after the "read unclosed token" error # The first ones are the output of pushd to $DB_DIR/bin and the progress line "Reading XML file ..." and so on I'm sorry for the inconvenience and hope that this has fixed all race conditions. Not yet adapted is "fetch_osc_and_apply.sh". If someone uses this then please mail me. I will then watch with you whether an improved version of that script is stable. Best regards, Roland _______________________________________________ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk