NealMcB added a comment.
The recommended download format continues to be JSON as discussed at https://www.wikidata.org/wiki/Wikidata:Database_download Since this was reported in 2015, the smallest version of the "latest-all" database has grown more than tenfold from 5.4 GB to 64 GB in size, making the usage challenges far greater. From https://dumps.wikimedia.org/wikidatawiki/entities/: latest-all.json.bz2 31-Mar-2021 17:03 64697800080 Others are running across the issues, motivating the duplicate issue T278204 <https://phabricator.wikimedia.org/T278204> which was recently merged. They note that > dumps are currently in fact already produced by multiple shards and then combined into one file and > There are already no guarantees on the order of documents in dumps making it seem yet more reasonable to provide them as multiple files not a single file. What would it take to resolve this issue? How can we help? TASK DETAIL https://phabricator.wikimedia.org/T115223 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: NealMcB Cc: Addshore, Mitar, abian, JanZerebecki, Hydriz, hoo, Halfak, NealMcB, Aklapper, Invadibot, maantietaja, Akuckartz, Nandana, Lahi, Gq86, GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, Svick, Mbch331, jeremyb
_______________________________________________ Wikidata-bugs mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
