Hi again, I think I may have figured this out by experimentation (tinkering). The following command succeeds:
lstmtraining --stop_training \ --continue_from /data/output/meme0.512_97818.checkpoint \ --traineddata /data/starter/eng/eng.traineddata \ --model_output /data/output/mem.traineddata where meme0.512_97818.checkpoint is the most recent checkpoint other than the one that (I presume) is currently being modified by the main lstmtraining process. So am I right in guessing that while lstmtraining is running, the meme_checkpoint file is not only susceptible to being changed, but is missing some kind of "completion" that is required to use the stop_training subcommand to generate the model? Thanks, Adam On 06/01/2020 13:40, Adam Funk wrote: > > Thanks! Unfortunately I'm now getting the dreaded "Failed to read > continue from:" error. > > I have used docker exec to start another terminal in the running > container, and am trying to use the following command: > > lstmtraining \ > --stop_training \ > --continue_from /data/output/meme_checkpoint \ > --traineddata /data/starter/eng/eng.traineddata \ > --model_output /data/output/mem.traineddata > > The file I'm using in --continue_from always has a fresh timestamp, but > the other checkpoints (with numbers in the filenames) in the same > directory are quite old. > > I'd be grateful for any suggestions. > > > > > On 23/12/2019 17:10, Shree Devi Kumar wrote: >> You can create traineddata with the --stop-training while lstmtraining >> continues to run. >> >> If you are using tesstrain makefile then it has a target called >> traineddata which will generate traineddata file for each intermediate >> checkpoint. >> >> You can stop and start training but I have a feeling that training >> runs longer in that case. >> >> On Mon, Dec 23, 2019, 17:33 Adam Funk <[email protected] >> <mailto:[email protected]>> wrote: >> >> Hi, >> >> I have an lstmtraining job running without --max_iterations; it's >> been >> going for a couple of weeks now (in a docker container in screen on a >> server that I ssh into). >> >> Can I safely use ctrl-C to stop it, use lstmtraining --stop_training >> (with appropriate settings for --continue_from --traineddata and >> --model_output) to create a trained model that I can drop into >> tesseraact somewhere else, and then restart the lstmtraining job? >> >> I don't want to lose anything that's been produced so far. >> >> Thanks, >> Adam >> >> -- You received this message because you are subscribed to the >> Google >> Groups "tesseract-ocr" group. >> To unsubscribe from this group and stop receiving emails from it, >> send an email to [email protected] >> <mailto:tesseract-ocr%[email protected]>. >> To view this discussion on the web visit >> >> https://groups.google.com/d/msgid/tesseract-ocr/8b53685f-6368-6239-726c-3b2e5305fe64%40sheffield.ac.uk. >> >> >> -- >> You received this message because you are subscribed to the Google >> Groups "tesseract-ocr" group. >> To unsubscribe from this group and stop receiving emails from it, send >> an email to [email protected] >> <mailto:[email protected]>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/tesseract-ocr/CAG2NduWwka7RJ1FG7dGdx_WjTnhVq3h%3DfNDrfC7OhYAY56t_yw%40mail.gmail.com >> <https://groups.google.com/d/msgid/tesseract-ocr/CAG2NduWwka7RJ1FG7dGdx_WjTnhVq3h%3DfNDrfC7OhYAY56t_yw%40mail.gmail.com?utm_medium=email&utm_source=footer>. >> > -- You received this message because you are subscribed to the Google Groups "tesseract-ocr" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/tesseract-ocr/131f6776-74a8-7afc-5a5b-43e7f20d4e93%40sheffield.ac.uk.
