fgiunchedi added a comment.
Thank you for following up, I think the culprit is the fact that the S3 <https://phabricator.wikimedia.org/S3> compat API stores chunks of big files in a separate container (suffixed with `+segments`). See also the audit below I ran logged into swift as `wdqs:flink`: # swift list | xargs -n1 swift stat | grep -e Container -e Objects -e Bytes Container: rdf-streaming-updater-codfw Objects: 125 Bytes: 336259981 Container: rdf-streaming-updater-codfw+segments Objects: 3752575 Bytes: 19043034820255 Container: rdf-streaming-updater-codfw-T314835 Objects: 0 Bytes: 0 Container: rdf-streaming-updater-eqiad Objects: 3079 Bytes: 61140716537 Container: rdf-streaming-updater-eqiad+segments Objects: 13832 Bytes: 55454475470 Container: rdf-streaming-updater-staging Objects: 1423 Bytes: 5878764535 Container: thanos-swift Objects: 48 Bytes: 25856176 Container: updater Objects: 2921 Bytes: 86172646251 Container: updater+segments Objects: 552 Bytes: 2856364302 Container: updater-zbyszko Objects: 31 Bytes: 1622577 Container: updater-zbyszko-v2 Objects: 36 Bytes: 120047231 TASK DETAIL https://phabricator.wikimedia.org/T316031 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bking, fgiunchedi Cc: Aklapper, fgiunchedi, dr0ptp4kt, LSobanski, dcausse, Ottomata, elukey, gmodena, MatthewVernon, EBernhardson, bking, Raineydaz, Astuthiodit_1, AWesterinen, BTullis, karapayneWMDE, joanna_borun, Invadibot, Lalamarie69, MPhamWMF, Devnull, maantietaja, Muchiri124, CBogen, ItamarWMDE, Akuckartz, Di3sel1975, Chambersjay, RhinosF1, Legado_Shulgin, ReaperDawn, Nandana, Namenlos314, Conradrock, Davinaclare77, Techguru.pc, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Hfbn0, QZanden, EBjune, merbst, LawExplorer, Zppix, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, Wong128hk, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, faidon, Mbch331, Jay8g
_______________________________________________ Wikidata-bugs mailing list -- [email protected] To unsubscribe send an email to [email protected]
