Hi guys. Today's installment in things I learned today that you will want to know if I get abducted by aliens is all about translation statistics.
Bug 903532 arises from the fact that verify-pofile-stats-daily has been removed from the crontab (by whom I know not) and that I wasn't aware that bug 886067 wasn't done. To fix bug 903532 in the short term we can reinstate verify-pofile-stats-daily and perhaps even run a tweaked version of it with a bigger window to fix the translation statistics that have been out of whack for more than a week (it's current window is 7 days). I (or you, if the aliens have their way) will have a LOSA, er WebOp(s) add it back tomorrow. I might be nice to know why it was removed so it's not removed again. (We'll need it even after bug 886067 is done because of some unnamed database wind that I'm not totally clear on). As for how to actually fix bug 886067, here's the conversation jtv and I had about that: 16:09 < jtv:#launchpad-dev> benji: that would only apply if the bug it refers to had been fully fixed. 16:09 < jtv:#launchpad-dev> Instead, we now have a partial implementation. 16:10 < jtv:#launchpad-dev> What the bug specified (with a bit of excessive detail that I would argue with, which I'll skip over for now) is “do the statistics updates in a separate job, and have each job update not just a single POFile but its entire sharing set.” 16:10 < jtv:#launchpad-dev> The latter part is not implemented AFAICS. 16:11 < jtv:#launchpad-dev> Which leaves us without the biggest thing the statistics script did for us: patch up statistics for POFiles that were affected by changes in shared translations. 16:11 < benji:#launchpad-dev> jtv: gotcha, that leads me to the second half; I can't say that I completely understand the details of how the shared translations need to be updated when an individual POFile is changed; will you outline that for me? 16:12 < jtv:#launchpad-dev> It's not so much that shared translations need to be updated, but that they _are_ updated and the statistics need to reflect that. 16:12 < jtv:#launchpad-dev> A template can share translations with a bunch of other templates. 16:13 < jtv:#launchpad-dev> Therefore, a POFile can share translations with other POFiles: those whose templates share with its template, translated to the same language. 16:13 < jtv:#launchpad-dev> Say POFile A shares with POFile B. 16:13 < jtv:#launchpad-dev> They are both completely untranslated. 16:14 < jtv:#launchpad-dev> Going through the UI for POFile A, I translate POTMsgSet x. 16:14 < jtv:#launchpad-dev> But x is also shared with B's template. 16:14 < jtv:#launchpad-dev> We always updated statistics for A at this point, 16:14 < jtv:#launchpad-dev> but B obviously also needs its statistics updated because it is no longer completely untranslated. 16:15 < jtv:#launchpad-dev> There was no code to do that immediately; too complex and too costly. 16:15 < jtv:#launchpad-dev> Instead, we relied on the statistics updater that was already in place. 16:16 < jtv:#launchpad-dev> And so, in the current design, the pofilestatsjob needs to look up a full set of sharing POFiles and update all their statistics. 16:16 < jtv:#launchpad-dev> In my opinion the current design is not that great for template imports, where we'd basically have to look up the sharing set twice (and once in the critical path). But that's another topic. 16:17 < jtv:#launchpad-dev> benji: still with me? 16:18 < benji:#launchpad-dev> jtv: I have a glimmer of understanding of the words you are saying. Do you know of a place in the code which we already "full set of sharing POFiles"? 16:18 < jtv:#launchpad-dev> You're hitting a sore spot there. 16:18 < jtv:#launchpad-dev> No, we don't have that. 16:18 < jtv:#launchpad-dev> What we have is code to look up a full set of sharing templates. 16:19 < jtv:#launchpad-dev> So you'd have to go over the full set of sharing templates (POTemplateSharingSubset IIRC), and for each, look up their POFile for the language you're interested in. 16:20 < jtv:#launchpad-dev> Also, the template import code needs to do this for all languages that any of the sharing templates are translated to. Which is why I would have preferred for POFileStatsJob to refer to a POTemplate and an optional language, rather than a POFile. 16:20 < jtv:#launchpad-dev> A POFile is basically a tuple of a POTemplate and a language. But who's to say there's always a POFile? 16:23 * benji considers buying a very large whiteboard. 16:26 < benji:#launchpad-dev> jtv: so... it looks like I need to make the job use POTemplateSharingSubset.getSharingPOTemplates(POFile_this_job_is_about.template) to get a set of templates and then call updateStatistics on each of those templates 16:27 * benji crosses his fingers. 16:28 < jtv:#launchpad-dev> benji: you'll be happy to hear that that is almost entirely correct. The only small missing detail is in what you call updateStatistics on: 16:28 < jtv:#launchpad-dev> for each of those sharing templates you need to find the POFile (if any) that translates it to POFile_this_job_is_about.language. And if there is one, call updateStatistics on that POFile. 16:28 < benji:#launchpad-dev> oh! yeah; I need to go from the templates to... /some/ set of POFiles, right? 16:29 < jtv:#launchpad-dev> benji: yes, and tragically there's no direct support for that. Not that it'll be _very_ hard, but still. 16:32 < benji:#launchpad-dev> jtv: thanks much -- Benji York -- Mailing list: https://launchpad.net/~yellow Post to : [email protected] Unsubscribe : https://launchpad.net/~yellow More help : https://help.launchpad.net/ListHelp
