dr0ptp4kt added a comment.
For me the first 300 GB of the file went really, really fast. But `axel` was
dropping connections, similar to when I had downloaded the large 1 TB file. So
this download took about 5 hours. I'm pretty sure it could be done in 1-3
hours, though, if everything were working well.
Now, I encountered an error, and this was reproducible with two separate
downloads. @bking does a test on the file yield the same corrupted block
detected warning for you by any chance if you download the the zst? What about
if you do it with your already existing copy?
/mnt/x $ time unzstd --output-dir-flat /mnt/y/ wikidata.jnl.zst
wikidata.jnl.zst : 649266 MB... wikidata.jnl.zst : Decoding error
(36) : Corrupted block detected
real 124m59.115s
user 17m44.647s
sys 7m24.509s
/mnt/x $ ls -l wikidata.jnl.zst
-rwxrwxrwx 1 adam adam 342189138219 Oct 3 02:32 wikidata.jnl.zst
I've kicked off a `sha1sum`, but this will take a while to run.
/mnt/x$ time sha1sum wikidata.jnl.zst
TASK DETAIL
https://phabricator.wikimedia.org/T347605
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: dr0ptp4kt
Cc: Addshore, dr0ptp4kt, Aklapper, bking, Danny_Benjafield_WMDE, Astuthiodit_1,
AWesterinen, BTullis, karapayneWMDE, Invadibot, maantietaja, ItamarWMDE,
Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE,
GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper,
Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984,
Manybubbles, Mbch331
_______________________________________________
Wikidata-bugs mailing list -- [email protected]
To unsubscribe send an email to [email protected]
