bennofs added a comment.
Now the same with zstd:
$ time zstdcat -v -d wikidata-20190506-all.json.bz2 | cat > /dev/null
real 3m48.657s
user 0m3.792s
sys 0m58.768s
here's the sizes:
35G wikidata-20190506-all.json.bz2
39G wikidata-20190506-all.json.zstd-def
so zstd is about the same size compressed but is about 10 times faster
decompressing.
TASK DETAIL
https://phabricator.wikimedia.org/T222985
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bennofs
Cc: ArielGlenn, Liuxinyu970226, bennofs, darthmon_wmde, alaa_wmde, Nandana,
Lahi, Gq86, GoranSMilovanovic, Lunewa, QZanden, LawExplorer, _jensen,
rosalieper, gnosygnu, Wikidata-bugs, aude, Mbch331
_______________________________________________
Wikidata-bugs mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
