[Wikidata-bugs] [Maniphest] [Commented On] T171607: Main snak and reference snaks do not include hash in JSON output
Lucas_Werkmeister_WMDE added a comment. Just to clarify: the current situation in dumps and EntityData isn’t “no hashes”, it’s “qualifier hashes but not main snak hashes or reference hashes”. And we will lose the ability to make this distinction as soon as we switch to wikibase/data-model-serialization 3.0.0 (see pull request), and if we go for “no hashes” then (which is fine IMO), it will be a breaking change.TASK DETAILhttps://phabricator.wikimedia.org/T171607EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: thiemowmde, Lucas_Werkmeister_WMDECc: Lydia_Pintscher, WMDE-leszek, thiemowmde, gerritbot, Addshore, Aklapper, daniel, Lucas_Werkmeister_WMDE, PokestarFan, GoranSMilovanovic, QZanden, Izno, Wikidata-bugs, aude, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T171607: Main snak and reference snaks do not include hash in JSON output
Lydia_Pintscher added a comment. Do we need them in the json dumps and Special:EntityData? If the answer is "unclear" then let's not do it until someone asks for it.TASK DETAILhttps://phabricator.wikimedia.org/T171607EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Lydia_PintscherCc: Lydia_Pintscher, WMDE-leszek, thiemowmde, gerritbot, Addshore, Aklapper, daniel, Lucas_Werkmeister_WMDE, PokestarFan, GoranSMilovanovic, QZanden, Izno, Wikidata-bugs, aude, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T171607: Main snak and reference snaks do not include hash in JSON output
gerritbot added a comment. Change 373859 abandoned by Lucas Werkmeister (WMDE): Emit all snak hashes in API responses Reason: Already done in Ia143563ed1. (Merging this into current master produces an empty diff. That’s also why the tests succeeded – Leszek went through the trouble of fixing them already :) ) https://gerrit.wikimedia.org/r/373859TASK DETAILhttps://phabricator.wikimedia.org/T171607EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: gerritbotCc: gerritbot, Addshore, Aklapper, daniel, Lucas_Werkmeister_WMDE, PokestarFan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Lewizho99, Maathavan, Izno, Wikidata-bugs, aude, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T171607: Main snak and reference snaks do not include hash in JSON output
Lucas_Werkmeister_WMDE added a comment. It might not even need to be configurable. (Daniel: “I’m okay with always including the hashes in API output, and output of Special:EntityData.”) Perhaps it should be configurable after all, because otherwise the tests are going to become ugly.TASK DETAILhttps://phabricator.wikimedia.org/T171607EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Lucas_Werkmeister_WMDECc: gerritbot, Addshore, Aklapper, daniel, Lucas_Werkmeister_WMDE, PokestarFan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Lewizho99, Maathavan, Izno, Wikidata-bugs, aude, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T171607: Main snak and reference snaks do not include hash in JSON output
gerritbot added a comment.Herald added a subscriber: jeblad. Change 373859 had a related patch set uploaded (by Lucas Werkmeister (WMDE); owner: Lucas Werkmeister (WMDE)): [mediawiki/extensions/Wikibase@master] Emit all snak hashes in API responses https://gerrit.wikimedia.org/r/373859TASK DETAILhttps://phabricator.wikimedia.org/T171607EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: gerritbotCc: jeblad, gerritbot, Addshore, Aklapper, daniel, Lucas_Werkmeister_WMDE, PokestarFan, GoranSMilovanovic, QZanden, Izno, Wikidata-bugs, aude, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T171607: Main snak and reference snaks do not include hash in JSON output
Lucas_Werkmeister_WMDE added a comment. I have no idea how to implement this nicely for API modules. There should be an optional flag parameter to include hashes in the output of wbsetclaim and other modules, but the configuration of the serializers is buried in WikibaseRepo::getApiHelperFactory, which is called way before the API module is even instantiated, let alone has had a chance to parse the API parameters. So it seems to me that we can either change a lot of scaffolding to inject serializer factories instead of serializers, and only instantiate a serializer factory at the last possible moment, or we add a way to change the serializer’s behavior after it has been instantiated, which is risky because it might be shared and also tricky because we need to propagate that change through the tree of serializers.TASK DETAILhttps://phabricator.wikimedia.org/T171607EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Lucas_Werkmeister_WMDECc: Addshore, Aklapper, daniel, Lucas_Werkmeister_WMDE, PokestarFan, GoranSMilovanovic, QZanden, Izno, Wikidata-bugs, aude, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T171607: Main snak and reference snaks do not include hash in JSON output
daniel added a comment. In T171607#3478253, @Addshore wrote: tldr, the only reason the options exist was to maintain the output format exactly as it was before refactoring. Yay for bug-by-bug compatibility :)TASK DETAILhttps://phabricator.wikimedia.org/T171607EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: danielCc: Addshore, Aklapper, daniel, Lucas_Werkmeister_WMDE, PokestarFan, GoranSMilovanovic, QZanden, Izno, Wikidata-bugs, aude, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T171607: Main snak and reference snaks do not include hash in JSON output
daniel added a comment. In T171607#3478233, @Lucas_Werkmeister_WMDE wrote: As far as I can tell, we’ll need hashes in the output of wbeditentity if we want to have those hashes in the DOM (T171725). Or can the _javascript_ view calculate the hashes? No, the frontend code should not do its own hash calculations. That's going to fail horribly. The has is not based on the JSON representation. When does the frontend code need to build a DOM fragment from JSON? After an edit completes, I assume. II still think it would save us a ton of trouble to simply ask the backend to do that, instead of doing it in JS... oh well. If we need that, hashes should be optional in the output of all API modules (and off per default).TASK DETAILhttps://phabricator.wikimedia.org/T171607EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: danielCc: Addshore, Aklapper, daniel, Lucas_Werkmeister_WMDE, PokestarFan, GoranSMilovanovic, QZanden, Izno, Wikidata-bugs, aude, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T171607: Main snak and reference snaks do not include hash in JSON output
Addshore added a comment. So, this was part of the LARGE refactoring of the serialization stuff / removing lib serializers for those that remember that mess (it spans about 30 patches or something like that). The options were introduced to keep the output in various areas of wikibase exactly the same through the refactoring / removing of the lib serializers. We said that if we wanted to change something then we would discuss it further down the line, and it looks like we may now be at that point. tldr, the only reason the options exist was to maintain the output format exactly as it was before refactoring.TASK DETAILhttps://phabricator.wikimedia.org/T171607EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: AddshoreCc: Addshore, Aklapper, daniel, Lucas_Werkmeister_WMDE, PokestarFan, GoranSMilovanovic, QZanden, Izno, Wikidata-bugs, aude, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T171607: Main snak and reference snaks do not include hash in JSON output
daniel added a comment. The following places currently construct statement serializers: Scribunto_LuaWikibaseLibrary::newEntityAccessor() WikibaseRepo::getApiHelperFactory() SpecialEntityData::newDefaultRequestHandler() DumpJson::execute() All of them, set use the OPTION_SERIALIZE_MAIN_SNAKS_WITHOUT_HASH and OPTION_SERIALIZE_REFERENCE_SNAKS_WITHOUT_HASH flags. None of them sets the OPTION_SERIALIZE_QUALIFIER_SNAKS_WITHOUT_HASH. I cannot remember any good reason for this. I suspect that the flags were simply copied around. Perhaps the OPTION_SERIALIZE_QUALIFIER_SNAKS_WITHOUT_HASH mode was added later.TASK DETAILhttps://phabricator.wikimedia.org/T171607EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: danielCc: Addshore, Aklapper, daniel, Lucas_Werkmeister_WMDE, PokestarFan, GoranSMilovanovic, QZanden, Izno, Wikidata-bugs, aude, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T171607: Main snak and reference snaks do not include hash in JSON output
daniel added a comment. I guess this was just caused by confusion during refactoring. I'm looking into it now. A good point to start the investigation is SerializerFactory::newStatementSerializer().TASK DETAILhttps://phabricator.wikimedia.org/T171607EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: danielCc: Addshore, Aklapper, daniel, Lucas_Werkmeister_WMDE, PokestarFan, GoranSMilovanovic, QZanden, Izno, Wikidata-bugs, aude, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
