dr0ptp4kt added a comment.
Addressing @Addshore's comment in T344905#9210122 <https://phabricator.wikimedia.org/T344905#9210122>... > I think the amount of time taken to decompress the JNL file should also be taken into consideration on varying hardware if compression is being considered. Here's what I saw for performance: /mnt/x $ time unzstd --output-dir-flat /mnt/y/ wikidata.jnl.zst wikidata.jnl.zst : 1265888788480 bytes real 219m10.733s user 29m51.350s sys 12m53.425s This was on an i7-8700 CPU @ 3.20GHz. When I checked this with `top` it seemed to be using about 0.8-1.6 processor, but hovering around 1 processor, at any given time. `unzstd` doesn't support multiple processor decompress from what I see. TASK DETAIL https://phabricator.wikimedia.org/T347605 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dr0ptp4kt Cc: Addshore, dr0ptp4kt, Aklapper, bking, Danny_Benjafield_WMDE, Astuthiodit_1, AWesterinen, BTullis, karapayneWMDE, Invadibot, maantietaja, ItamarWMDE, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331
_______________________________________________ Wikidata-bugs mailing list -- [email protected] To unsubscribe send an email to [email protected]
