[Wikidata-bugs] [Maniphest] [Commented On] T117032: Create configuration for specifying units conversions
gerritbot added a comment. Change 319401 abandoned by Smalyshev: Tools for creating unit conversion config Reason: we can do it without backporting https://gerrit.wikimedia.org/r/319401TASK DETAILhttps://phabricator.wikimedia.org/T117032EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Smalyshev, gerritbotCc: Stashbot, Esc3300, Laddo, Lydia_Pintscher, TomT0m, gerritbot, Smalyshev, Aklapper, daniel, aude, mschwarzer, Avner, Lewizho99, Maathavan, debt, Gehel, D3r1ck01, Jonas, FloNight, Xmlizer, Izno, jkroll, Wikidata-bugs, Jdouglas, Deskana, Manybubbles, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T117032: Create configuration for specifying units conversions
gerritbot added a comment. Change 298407 merged by jenkins-bot: Tools for creating unit conversion config https://gerrit.wikimedia.org/r/298407TASK DETAILhttps://phabricator.wikimedia.org/T117032EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Smalyshev, gerritbotCc: Stashbot, Esc3300, Laddo, Lydia_Pintscher, TomT0m, gerritbot, Smalyshev, Aklapper, daniel, aude, mschwarzer, Avner, Lewizho99, Maathavan, debt, Gehel, D3r1ck01, Jonas, FloNight, Xmlizer, Izno, jkroll, Wikidata-bugs, Jdouglas, Deskana, Manybubbles, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T117032: Create configuration for specifying units conversions
Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2016-10-06T21:53:06Z]Synchronized wmf-config/Wikibase-production.php: SWAT: [[gerrit:311206|Add config for units on Wikidata (T117032)]] PART II (duration: 00m 50s)TASK DETAILhttps://phabricator.wikimedia.org/T117032EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: StashbotCc: Stashbot, Esc3300, Laddo, Lydia_Pintscher, TomT0m, gerritbot, Smalyshev, Aklapper, daniel, aude, mschwarzer, Avner, Lewizho99, Maathavan, debt, Gehel, D3r1ck01, Jonas, FloNight, Xmlizer, Izno, jkroll, Wikidata-bugs, Jdouglas, Deskana, Manybubbles, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T117032: Create configuration for specifying units conversions
Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2016-10-06T21:51:28Z]Synchronized wmf-config/unitConversionConfig.json: SWAT: [[gerrit:311206|Add config for units on Wikidata (T117032)]] PART I (duration: 00m 48s)TASK DETAILhttps://phabricator.wikimedia.org/T117032EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: StashbotCc: Stashbot, Esc3300, Laddo, Lydia_Pintscher, TomT0m, gerritbot, Smalyshev, Aklapper, daniel, aude, mschwarzer, Avner, Lewizho99, Maathavan, debt, Gehel, D3r1ck01, Jonas, FloNight, Xmlizer, Izno, jkroll, Wikidata-bugs, Jdouglas, Deskana, Manybubbles, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T117032: Create configuration for specifying units conversions
Smalyshev added a comment. If there's no conversion to SI units or no SI units, we shouldn't use P2370. We should use P2442 instead. SI is well-defined system and we should only use SI properties to signify SI units. But we do plan to support other units, just a bit later after we see how the system fares with SI ones. For dates, I'm not sure we should use these - dates are entirely different can of worms which usually don't play by the rules the other measure play. So we'll approach it on case-by-case basis.TASK DETAILhttps://phabricator.wikimedia.org/T117032EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: Esc3300, Laddo, Wesalius, Lydia_Pintscher, TomT0m, gerritbot, Smalyshev, Aklapper, daniel, aude, mschwarzer, Avner, Lewizho99, Maathavan, debt, Gehel, D3r1ck01, Jonas, FloNight, Xmlizer, Izno, jkroll, Wikidata-bugs, Jdouglas, Deskana, Manybubbles, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T117032: Create configuration for specifying units conversions
daniel added a comment. @Smalyshev @Esc3300: When extracting conversion factors from statements on Wikidata, we should only use factory that are marked +/- 0. +/-0 indicates that the factory applies per definition, which is what we want for conversion. For things like calendar year, month, or historical units, the conversion factor should be given with some uncertainty. Thinking about it, we could even uncertain conversion factory, but we would need to carry the uncertainty through the conversion mechanism, applying it during conversion. Possible, but I don't think we need that right now, and it would complicate things quite a bit.TASK DETAILhttps://phabricator.wikimedia.org/T117032EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: danielCc: Esc3300, Laddo, Wesalius, Lydia_Pintscher, TomT0m, gerritbot, Smalyshev, Aklapper, daniel, aude, mschwarzer, Avner, Lewizho99, Maathavan, debt, Gehel, D3r1ck01, Jonas, FloNight, Xmlizer, Izno, jkroll, Wikidata-bugs, Jdouglas, Deskana, Manybubbles, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T117032: Create configuration for specifying units conversions
Esc3300 added a comment. Initially https://www.wikidata.org/wiki/Property:P2370 (conversion to SI units) wasn't used for units that don't have a reliable conversion to these (e.g. calendar year, month, historic units). What is the plan for these? Should we leave P2370 or use something else? TASK DETAILhttps://phabricator.wikimedia.org/T117032EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Esc3300Cc: Esc3300, Laddo, Wesalius, Lydia_Pintscher, TomT0m, gerritbot, Smalyshev, Aklapper, daniel, aude, mschwarzer, Avner, Lewizho99, Maathavan, debt, Gehel, D3r1ck01, Jonas, FloNight, Xmlizer, Izno, jkroll, Wikidata-bugs, Jdouglas, Deskana, Manybubbles, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T117032: Create configuration for specifying units conversions
gerritbot added a comment. Change 311206 had a related patch set uploaded (by Smalyshev): Add config for units on Wikidata https://gerrit.wikimedia.org/r/311206TASK DETAILhttps://phabricator.wikimedia.org/T117032EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: gerritbotCc: Lydia_Pintscher, TomT0m, gerritbot, Smalyshev, Aklapper, daniel, aude, mschwarzer, Avner, debt, Gehel, D3r1ck01, Jonas, FloNight, Xmlizer, Izno, jkroll, Wikidata-bugs, Jdouglas, Deskana, Manybubbles, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T117032: Create configuration for specifying units conversions
Smalyshev added a comment. More mismatches: DIFF: [congius(Q3646719)->cubic metre]: GNU 0.0034806123936, ours 0.00323 DIFF: [sextarius(Q14333713)->cubic metre]: GNU 0.0005801020656, ours 0.000546 DIFF: [choinix(Q15794456)->cubic metre]: GNU 0.00108, ours 0.001087 DIFF: [amphora(Q2844434)->cubic metre]: GNU 0.0278448991488, ours 0.02586 DIFF: [chous(Q1076762)->cubic metre]: GNU 0, ours 0.00312 <-- here GNU doesn't seem to think it's volume measure DIFF: [modius(Q669909)->cubic metre]: GNU 0.0092816330496, ours 0.008736 DIFF: [heredium(Q3785200)->square metre]: GNU 5046.6816, ours 5036 DIFF: [jugerum(Q251545)->square metre]: GNU 2523.3408, ours 2518 Most of these units seem to be rather exotic, and used only in cross-definitions (not sure yet how to filter those). The question is - are these differences important? Which data is better?TASK DETAILhttps://phabricator.wikimedia.org/T117032EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: Lydia_Pintscher, TomT0m, gerritbot, Smalyshev, Aklapper, daniel, aude, mschwarzer, Avner, debt, Gehel, D3r1ck01, Jonas, FloNight, Xmlizer, Izno, jkroll, Wikidata-bugs, Jdouglas, Deskana, Manybubbles, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T117032: Create configuration for specifying units conversions
Smalyshev added a comment. So I checked our data against GNU units data and found these mismatches, among used units: DIFF: [stadion(Q1645966)->metre]: GNU 189.738, ours 186 DIFF: [plethron(Q2099374)->metre]: GNU 31.623, ours 29.55 DIFF: [scruple(Q1573593)->kilogram]: GNU 0.0012959782, ours 0.001244 DIFF: [pood(Q923539)->kilogram]: GNU 16.3806872, ours 16.3804964 DIFF: [grain(Q693944)->kilogram]: GNU 6.479891E-5, ours 6.22E-5 DIFF: [pace(Q691543)->metre]: GNU 0.762, ours 1.48 DIFF: [pous(Q7235735)->metre]: GNU 0.31623, ours 0.308 But this not counting for square/cubic/etc. yet.TASK DETAILhttps://phabricator.wikimedia.org/T117032EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: Lydia_Pintscher, TomT0m, gerritbot, Smalyshev, Aklapper, daniel, aude, mschwarzer, Avner, debt, Gehel, D3r1ck01, Jonas, FloNight, Xmlizer, Izno, jkroll, Wikidata-bugs, Jdouglas, Deskana, Manybubbles, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T117032: Create configuration for specifying units conversions
Smalyshev added a comment. We need then a solution to introduce new unit conversions into dataset. Currently the only way it full reload, but in theory we can add new conversions incrementally. Not change though, change is more complex.TASK DETAILhttps://phabricator.wikimedia.org/T117032EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: Lydia_Pintscher, TomT0m, gerritbot, Smalyshev, Aklapper, daniel, aude, mschwarzer, MelodyKramer, Avner, debt, Gehel, D3r1ck01, Jonas, FloNight, Xmlizer, Izno, jkroll, Wikidata-bugs, Jdouglas, Deskana, Manybubbles, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T117032: Create configuration for specifying units conversions
Lydia_Pintscher added a comment. I'd prefer starting small and then extending on-demand.TASK DETAILhttps://phabricator.wikimedia.org/T117032EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Lydia_PintscherCc: Lydia_Pintscher, TomT0m, gerritbot, Smalyshev, Aklapper, daniel, aude, mschwarzer, MelodyKramer, Avner, debt, Gehel, D3r1ck01, Jonas, FloNight, Xmlizer, Izno, jkroll, Wikidata-bugs, Jdouglas, Deskana, Manybubbles, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T117032: Create configuration for specifying units conversions
daniel added a comment. To use GNU units, we'll at least need a mapping between item IDs and GNU unit symbols. Anyway @Smalyshev do you think it makes sense to start small, with a manually written config of maybe 100 units, or do you think we should try to cover as much as possible right away, so we don't have to reload the data multiple times?TASK DETAILhttps://phabricator.wikimedia.org/T117032EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: danielCc: TomT0m, gerritbot, Smalyshev, Aklapper, daniel, aude, mschwarzer, MelodyKramer, Avner, debt, Gehel, D3r1ck01, Jonas, FloNight, Xmlizer, Izno, jkroll, Wikidata-bugs, Jdouglas, Deskana, Manybubbles, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T117032: Create configuration for specifying units conversions
Smalyshev added a comment. BTW, if we really wanted to use GNU units, I think it won't be hard to create UnitStorage driver that uses it, or generate JSON config from it, but I am not sure it's better than having one generated from Wikidata.TASK DETAILhttps://phabricator.wikimedia.org/T117032EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: TomT0m, gerritbot, Smalyshev, Aklapper, daniel, aude, mschwarzer, MelodyKramer, Avner, debt, Gehel, D3r1ck01, Jonas, FloNight, Xmlizer, Izno, jkroll, Wikidata-bugs, Jdouglas, Deskana, Manybubbles, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T117032: Create configuration for specifying units conversions
Smalyshev added a comment. True, these are the issues. This is why I propose the two-stage scheme, where units config file is generated from Wikidata statements, but this should be done very rarely once initial version is created. Mostly updates will be only new unit additions, and changing existing unit should require good explanation. Yes, this means we need to reload whole data set (or at least quantity statements, which is not much better) when we change unit conversion tables - excepting the case where new conversions/units are added (in this case, only statements using these units need to be reloaded and existing statements do not need to be changed). Also, this means the conversion config would not really be synchronized with Wikidata data as such. I don't think it is a very big issue. Until we have per-property permissions, I don't think direct usage of Wikidata properties is a good thing.TASK DETAILhttps://phabricator.wikimedia.org/T117032EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: TomT0m, gerritbot, Smalyshev, Aklapper, daniel, aude, mschwarzer, MelodyKramer, Avner, debt, Gehel, D3r1ck01, Jonas, FloNight, Xmlizer, Izno, jkroll, Wikidata-bugs, Jdouglas, Deskana, Manybubbles, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T117032: Create configuration for specifying units conversions
daniel added a comment. In T117032#2609531, @TomT0m wrote: Maybe instead of this configuration we can make the mechanism more flexible by just specifying such a property on the database. The set of unit supported by the export could then be extended with no privilege at all. The problem is that it could then also be changed and deleted with no privileges at all. What should happen when a statement specifying a conversion factor is edited? Shall we re-import the entire RDF dump into the query service? Changes that have big impact should not be too easy... We'd have to implement per-property permissions, or per-statement protection, before we could allow this. For now, a configuration file is the best solution, I think.TASK DETAILhttps://phabricator.wikimedia.org/T117032EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: danielCc: TomT0m, gerritbot, Smalyshev, Aklapper, daniel, aude, mschwarzer, MelodyKramer, Avner, debt, Gehel, D3r1ck01, Jonas, FloNight, Xmlizer, Izno, jkroll, Wikidata-bugs, Jdouglas, Deskana, Manybubbles, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T117032: Create configuration for specifying units conversions
Smalyshev added a comment. @TomT0m yes, we plan to use "conversion to SI unit".TASK DETAILhttps://phabricator.wikimedia.org/T117032EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: TomT0m, gerritbot, Smalyshev, Aklapper, daniel, aude, mschwarzer, MelodyKramer, Avner, debt, Gehel, D3r1ck01, Jonas, FloNight, Xmlizer, Izno, jkroll, Wikidata-bugs, Jdouglas, Deskana, Manybubbles, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T117032: Create configuration for specifying units conversions
TomT0m added a comment. Hi, maybe I come after the battle, but if you're not aware of this, Wikidata has now a "conversion to SI unit" that could come valuable for this as it stores the value of the configuration file. Maybe instead of this configuration we can make the mechanism more flexible by just specifying such a property on the database. The set of unit supported by the export could then be extended with no privilege at all.TASK DETAILhttps://phabricator.wikimedia.org/T117032EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: TomT0mCc: TomT0m, gerritbot, Smalyshev, Aklapper, daniel, aude, mschwarzer, MelodyKramer, Avner, debt, Gehel, D3r1ck01, Jonas, FloNight, Xmlizer, Izno, jkroll, Wikidata-bugs, Jdouglas, Deskana, Manybubbles, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T117032: Create configuration for specifying units conversions
gerritbot added a comment. Change 298407 had a related patch set uploaded (by Smalyshev): Tools for creating unit conversion config https://gerrit.wikimedia.org/r/298407TASK DETAILhttps://phabricator.wikimedia.org/T117032EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: gerritbotCc: gerritbot, Smalyshev, Aklapper, daniel, aude, Avner, debt, Gehel, D3r1ck01, Jonas, FloNight, Xmlizer, Izno, jkroll, Wikidata-bugs, Jdouglas, Deskana, Manybubbles, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T117032: Create configuration for specifying units conversions
Smalyshev added a comment. One more concern - to actually do the calculations, we may need arbitrary precision arithmetics... And we need a fast one if we want to do it inside dump. We can maybe require bcmath/gmp to calculate normalized values. But some of those will be big, like: https://www.wikidata.org/wiki/Q180892 - I'm not even sure if RDF tools can deal with such numbers. Or if you try to display it in infobox, it won't blow up the page. TASK DETAIL https://phabricator.wikimedia.org/T117032 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Smalyshev, Aklapper, daniel, aude, debt, Gehel, D3r1ck01, FloNight, Izno, jkroll, Wikidata-bugs, Jdouglas, Deskana, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T117032: Create configuration for specifying units conversions
Smalyshev added a comment. Makes sense, thanks @daniel. TASK DETAIL https://phabricator.wikimedia.org/T117032 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Smalyshev, Aklapper, daniel, aude, debt, jkroll, Wikidata-bugs, Jdouglas, Deskana, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T117032: Create configuration for specifying units conversions
Smalyshev added a comment. Note that we have https://www.wikidata.org/wiki/Property:P2370 now, so I wonder how the two should be interplaying. TASK DETAIL https://phabricator.wikimedia.org/T117032 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Smalyshev, Aklapper, daniel, aude, jkroll, Wikidata-bugs, Jdouglas, Deskana, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T117032: Create configuration for specifying units conversions
daniel added a comment. In addition to the multiplier, we need at least an optional offset (for °F to °C conversion) and perhaps also an exponent (for reciprocal conversions like miles/gallon to liter/100km). Both can come from GNU Units. TASK DETAIL https://phabricator.wikimedia.org/T117032 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: daniel Cc: Smalyshev, Aklapper, daniel, aude, jkroll, Wikidata-bugs, Jdouglas, Deskana, Manybubbles ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs