[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 Nemo changed: What|Removed |Added Blocks||71815 Depends on||52661 --- Comment #47 from Nemo --- (In reply to Nemo from comment #46) > I understand this has some value in itself for PDF export, see bug 34919 > comment 17. And more were filed, like bug 71815. Should they all depend from this? Does this really depend on bug 43547? Maybe this should just be converted to a tracking bug so that we're free to add dependencies without hairsplitting. -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 Nemo changed: What|Removed |Added Keywords||i18n CC||federicol...@tiscali.it, ||smole...@eunet.rs --- Comment #46 from Nemo --- (In reply to Liangent from comment #0) > Phase 1: Capsule conversion syntax (-{}- markups) into non-editable blocks > to avoid breakage. (In reply to Gabriel Wicke from comment #2) > It would be good to look into implementing phase 1 (recognize and protect > language conversion content). I understand this has some value in itself for PDF export, see bug 34919 comment 17. -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 C. Scott Ananian changed: What|Removed |Added Blocks||34919 -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 --- Comment #45 from C. Scott Ananian --- I've written up some notes about nested conversion blocks and other discoveries at https://www.mediawiki.org/wiki/Parsoid/MediaWiki_DOM_spec/Language_conversion_blocks#Notes -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 Gerrit Notification Bot changed: What|Removed |Added Status|NEW |PATCH_TO_REVIEW -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 --- Comment #44 from Gerrit Notification Bot --- Change 140235 had a related patch set uploaded by Cscott: WIP: parse language converter markup. https://gerrit.wikimedia.org/r/140235 -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 --- Comment #43 from Liangent --- They're mostly used in template. See also [[zh:Template:DISPLAYTITLE]] and [[zh:Module:Template:地区用词]]. -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 --- Comment #42 from C. Scott Ananian --- @Liangent: can you describe the use cases for "the other kind" of nested markup? That is, -{ }- inside -{ }-? Our proposed DOM tree (https://www.mediawiki.org/wiki/Parsoid/MediaWiki_DOM_spec/Language_conversion_blocks) can handle: -{ foo -{ bar }- bat }- but not foo-{zh-cn:blog -{ nested }-; zh-hk:WEBJOURNAL; zh-tw:WEBLOG;}- quux etc. How are nested -{ }- markups of this sort actually used? -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 C. Scott Ananian changed: What|Removed |Added Assignee|gwi...@wikimedia.org|canan...@wikimedia.org -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 --- Comment #41 from Gerrit Notification Bot --- Change 50767 abandoned by GWicke: Revert "(bug 41716) Add variant config to siprop=general" Reason: This ship has sadly sailed. Too late to clean it up I guess. Sigh. https://gerrit.wikimedia.org/r/50767 -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 --- Comment #40 from Liangent --- (In reply to comment #39) > (In reply to comment #38) > > (In reply to comment #37) > > > David, Roan, Scott, Subbu and me met in the office to discuss this. Short > > > summary of the plans for the next steps: > > > > > > 1) Find nesting issues and see if we can fix them up with a bot. Also > > > investigate use cases for markup in variant conversion rules. > > > > Why do we want to get rid of nested -{}- markups? It's useful in some cases. > > > > See [[模块:Template:地区用词]] which has a wrapper at [[Template:地区用词3]] (proposed > > replacement for [[Template:地区用词]]). > > > > Try to expand a [[Template:地区用词3]] call and see its result: > > > > {{地区用词3|zh-cn=cn|zh-tw=tw}} > > Liangent: Gwicke did not fully explain the nesting issue we were talking > about. > > What we had in mind was use in attributes. Ex:-{zh-cn= style='color:red';zh-tw=. We are > proposing > using a bot to fix this to: style='-{zh-cn=color:red;zh-tw=color:green}-'>foo. The rewritten form > has the property that all HTML snippets have a well-formed DOM representation > whereas the original does not. Oh that's what we've discussed before - and that's fine. -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 --- Comment #39 from ssas...@wikimedia.org --- (In reply to comment #38) > (In reply to comment #37) > > David, Roan, Scott, Subbu and me met in the office to discuss this. Short > > summary of the plans for the next steps: > > > > 1) Find nesting issues and see if we can fix them up with a bot. Also > > investigate use cases for markup in variant conversion rules. > > Why do we want to get rid of nested -{}- markups? It's useful in some cases. > > See [[模块:Template:地区用词]] which has a wrapper at [[Template:地区用词3]] (proposed > replacement for [[Template:地区用词]]). > > Try to expand a [[Template:地区用词3]] call and see its result: > > {{地区用词3|zh-cn=cn|zh-tw=tw}} Liangent: Gwicke did not fully explain the nesting issue we were talking about. What we had in mind was use in attributes. Ex:-{zh-cn=. We are proposing using a bot to fix this to: foo. The rewritten form has the property that all HTML snippets have a well-formed DOM representation whereas the original does not. -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 --- Comment #38 from Liangent --- (In reply to comment #37) > David, Roan, Scott, Subbu and me met in the office to discuss this. Short > summary of the plans for the next steps: > > 1) Find nesting issues and see if we can fix them up with a bot. Also > investigate use cases for markup in variant conversion rules. Why do we want to get rid of nested -{}- markups? It's useful in some cases. See [[模块:Template:地区用词]] which has a wrapper at [[Template:地区用词3]] (proposed replacement for [[Template:地区用词]]). Try to expand a [[Template:地区用词3]] call and see its result: {{地区用词3|zh-cn=cn|zh-tw=tw}} -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 --- Comment #37 from Gabriel Wicke --- David, Roan, Scott, Subbu and me met in the office to discuss this. Short summary of the plans for the next steps: 1) Find nesting issues and see if we can fix them up with a bot. Also investigate use cases for markup in variant conversion rules. 2) Parse all -{ }- syntax and represent it in the DOM. Exact spec TBD in https://www.mediawiki.org/wiki/Parsoid/MediaWiki_DOM_spec#Language_conversion_blocks. Render the default variant according to the fallback chain for output-producing rules. 3) Enable editing of inline (once-only) rules in the VE. Most rule table modifications seem to be templated and will not be applied, so are not directly relevant. Rules that only modify the table but produce no output directly in page content can be represented as mw:Placeholder and will simply be preserved. This will make the VE usable for typical editors on variant-enabled wikis without requiring the variant conversion overhaul to be done first. For the longer-term strategy, we (mostly) agreed on: 1) Add the capability to associate an ordered list of glossaries with a page. These can either be stored in a separate namespace, or something like Special:Glossary. They should be revision-controlled and machine-readable for processing and UI purposes (JSON). 2) Add the capability to add page-specific rules that override glossary rules. Only glossaries and global rules associated with the top-level page itself are considered. This makes the set of conversion rules independent of dynamic template expansions. 3) Apply the combined rule set to the entire page including templated content. Rationales: * Simple mental model * efficient to implement * consistent conversion of passed-in content, even if it is massaged further during transclusion expansion * content in templates (labels, also real content in some infoboxes) themselves can still be protected or converted differently with local inline rules, as is done right now The details on how this can be implemented depend on whether we reach our goal of implementing multi-part revision storage that we can use for metadata by the next quarter. PS @David: Conversion rules should be passed into a pure function that converts each template expansion. Nothing at all should leak- otherwise our function would no longer be pure, and we could no longer efficiently update template expansions independently. -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 David Chan changed: What|Removed |Added CC||da...@sheetmusic.org.uk --- Comment #36 from David Chan --- I think we should be *extremely* restrictive about where language rules can leak. This is because they lead to several problems: (1) Rule changes make it hard to give a faithful real-time view of *any* plaintext. (2) Rule changes can cause unexpected errors in distant text. (3) Few people can proofread both zh-Hans and zh-Hant. Therefore, almost anyone who makes an edit will be unable to proofread at least one of the variants it might affect. On the other hand, leaking currently allows pages to import rules. I think we should preserve this facility but make it more separate. 1. In general, there should be no leakage: rules should be page-global, and should not leak into or out of templates. This means template *arguments* should be subject to the rules of the page in which they are written, but text generated by a template should not. 2. As an exception to the "no leakage" rule, there should be a new type of template called a Glossary, whose only purpose is to leak rules into the calling page. That way, language rules are completely separate and independent of any other template behaviour. These Glossaries should be referenced at the top of the page only. 3. The page which defines a template is free to use rules and Glossaries too. But they will only affect the text generated by the template itself -- they won't leak into any text defined in the calling page. This includes the arguments passed into the template, because they're written in the calling page. As you can see, this is just cscott's "Global" proposal, but with the additional restriction that the templates that leak rules cannot have any other functionality. -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 Liangent changed: What|Removed |Added Blocks||53784 -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 --- Comment #35 from C. Scott Ananian --- Can you improve your RFC to specify more precisely the scoping you anticipate for 'global' rules? In particular, it seems that a global rule defined in a page *does* affect the content of templates included in the page (a sort of leak). What happens to when a template defines a global rule? Does it get added to the inherited global rules from the parent page, and then applies to any subtemplates? FWIW, my Category proposal does state that page-scope templates do not leak -- neither into templates nor up to enclosing context. I'm not 100% sure that's desirable, but that's how it currently reads.) -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 --- Comment #34 from Gabriel Wicke --- (In reply to comment #33) > Scott's '"Category" Proposal' seems not leaking? The category variant in Scott's RFC is close to what I have been advocating for a while. He does not rule out leaking of rules out of templates, but mentions the problems associated with doing so. So it might or might not be leaking. See https://www.mediawiki.org/wiki/Requests_for_comment/Page_and_category_based_language_variant_conversion for a more detailed write-up of my proposal. -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 --- Comment #33 from Liangent --- (In reply to comment #32) > (In reply to comment #31) > > However this plan is only doable after Parsoid become the default parser, > > and > > all migration process must be done at exactly the same time as Parsoid > > becoming > > so... to keep everything working. Because (1) PHP parser doesn't understand > > your schema (2) Parsoid doesn't understand PHP parser's -{A| }- markups. > > No, this is not depending on Parsoid becoming the default parser. It does > depend on efficient access to global conversion rules at parse time, which is > true for both approaches. > > The main difference is that I favor direct (and mostly automatic) migration > of > rules to versioned page metadata for efficient access and gadget / UI-based > editing. The processing model is also designed to be efficient with > independent > transclusion expansions as done in Parsoid. > > Scott prefers to store rules in page and template content instead, and lets > rules leak out of templates. Scott's '"Category" Proposal' seems not leaking? -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 --- Comment #32 from Gabriel Wicke --- (In reply to comment #31) > However this plan is only doable after Parsoid become the default parser, and > all migration process must be done at exactly the same time as Parsoid > becoming > so... to keep everything working. Because (1) PHP parser doesn't understand > your schema (2) Parsoid doesn't understand PHP parser's -{A| }- markups. No, this is not depending on Parsoid becoming the default parser. It does depend on efficient access to global conversion rules at parse time, which is true for both approaches. The main difference is that I favor direct (and mostly automatic) migration of rules to versioned page metadata for efficient access and gadget / UI-based editing. The processing model is also designed to be efficient with independent transclusion expansions as done in Parsoid. Scott prefers to store rules in page and template content instead, and lets rules leak out of templates. -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 --- Comment #31 from Liangent --- (In reply to comment #30) > A rough outline of my proposal as developed during Wikimania with Liangent is > at > http://www.mediawiki.org/wiki/Parsoid/ > MediaWiki_DOM_spec#Language_conversion_blocks. > Scott would like to add additional syntax, while I am proposing a two-phase > approach that 1) aims at supporting visual editing of existing content and 2) > builds the infrastructure for clean language variant conversion based on > page-global and category-global rules and then migrates dynamic rule table > modifications out of articles and templates. However this plan is only doable after Parsoid become the default parser, and all migration process must be done at exactly the same time as Parsoid becoming so... to keep everything working. Because (1) PHP parser doesn't understand your schema (2) Parsoid doesn't understand PHP parser's -{A| }- markups. -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 --- Comment #30 from Gabriel Wicke --- A rough outline of my proposal as developed during Wikimania with Liangent is at http://www.mediawiki.org/wiki/Parsoid/MediaWiki_DOM_spec#Language_conversion_blocks. Scott would like to add additional syntax, while I am proposing a two-phase approach that 1) aims at supporting visual editing of existing content and 2) builds the infrastructure for clean language variant conversion based on page-global and category-global rules and then migrates dynamic rule table modifications out of articles and templates. -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 --- Comment #29 from C. Scott Ananian --- See also http://www.mediawiki.org/wiki/Requests_for_comment/Scoped_language_converter gwicke has an alternate proposal, which I'm sure he'll link here at some point. As I understand it, we will parse the language converter markup, and then we will have a post-processing step which will actually apply the rules and markup to convert the text into the desired variant. As discussed (to some extent) in bug 15161, ideally visual editor would present the text in the user's preferred variant and then we would leverage the selser mechanism to ensure a change in variant applies only to the edited portion of the text. Again, ideally DOM blocks would be annotated with the variant they were written in. -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 --- Comment #28 from C. Scott Ananian --- @Gwicke wrt comment 26 -- because it would fix the bugs documented in bug 52661. (In particular, is in sad shape right now.) -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 --- Comment #27 from Liangent --- (In reply to comment #26) > (In reply to comment #24) > > See also bug 52661 -- the language converter should be integrated better > > with > > the preprocessor, in both PHP and Parsoid. > > The language converter is actually a post-processor rather than a > preprocessor. > Why should that change? Their point is to have the preprocess understand those markups, to avoid interpreting them as something else. -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 --- Comment #26 from Gabriel Wicke --- (In reply to comment #24) > See also bug 52661 -- the language converter should be integrated better with > the preprocessor, in both PHP and Parsoid. The language converter is actually a post-processor rather than a preprocessor. Why should that change? -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 --- Comment #25 from Gabriel Wicke --- More info: https://www.mediawiki.org/wiki/Writing_systems/Syntax -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 C. Scott Ananian changed: What|Removed |Added CC||canan...@wikimedia.org --- Comment #24 from C. Scott Ananian --- See also bug 52661 -- the language converter should be integrated better with the preprocessor, in both PHP and Parsoid. -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 John Mark Vandenberg changed: What|Removed |Added Blocks||51792 -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 --- Comment #23 from Gabriel Wicke --- Yes, this is the meta bug that depends on several other bugs (see the "Depends on" field). Once we have a good overview of the issues we should probably get together to discuss possible solutions. Will you be at Wikimania? -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 --- Comment #22 from Liangent --- (In reply to comment #21) > Changed the title back to "Support language variant conversion in Parsoid" as > this is not just about the syntax. There're too many things, far more from what I mentioned in comment 0 and I may be going to add some separate bugs from time to time... or do you want to use this one as some "meta" bug? -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 --- Comment #21 from Gabriel Wicke --- Changed the title back to "Support language variant conversion in Parsoid" as this is not just about the syntax. -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 Gabriel Wicke changed: What|Removed |Added Summary|Support language variant|Support language variant |conversion syntax in|conversion in Parsoid |Parsoid | -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 Gabriel Wicke changed: What|Removed |Added Assignee|mtrac...@member.fsf.org |gwi...@wikimedia.org -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 Gabriel Wicke changed: What|Removed |Added Depends on||51587 -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 Liangent changed: What|Removed |Added CC||kel...@kiwix.org --- Comment #20 from Liangent --- *** Bug 51325 has been marked as a duplicate of this bug. *** -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 Gabriel Wicke changed: What|Removed |Added Status|ASSIGNED|NEW --- Comment #19 from Gabriel Wicke --- (In reply to comment #18) > (In reply to comment #16) > > About global state of dictionaries: the table affected by -{H| }- is used > > for > > link & categorylink resolution too. We may want to keep this behavior. > > One thing more about -{H| }-: the current behavior is that it only affects > text > after it and this behavior is sometimes deliberately used. We may want to > keep > it. For us mutable global state is very hard to support in any sane way. Having page-global dictionary definitions or self-contained manual conversions is fine, but changing global state in the middle of the page (even from a dynamically changing template) conflicts with a lot of optimizations and is hard to represent in a UI. -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 --- Comment #18 from Liangent --- (In reply to comment #16) > About global state of dictionaries: the table affected by -{H| }- is used for > link & categorylink resolution too. We may want to keep this behavior. One thing more about -{H| }-: the current behavior is that it only affects text after it and this behavior is sometimes deliberately used. We may want to keep it. -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 Andre Klapper changed: What|Removed |Added Component|JS/General |General --- Comment #17 from Andre Klapper --- [Parsoid component reorg by merging JS/General and General. See bug 50685 for more information. Filter bugmail on this comment. parsoidreorg20130704] -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 --- Comment #16 from Liangent --- About global state of dictionaries: the table affected by -{H| }- is used for link & categorylink resolution too. We may want to keep this behavior. -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 Bug 41716 depends on bug 46658, which changed state. Bug 46658 Summary: Tpl-style encapsulation for <*include*> and lang-variant conversions https://bugzilla.wikimedia.org/show_bug.cgi?id=46658 What|Removed |Added Status|NEW |RESOLVED Resolution|--- |INVALID -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 James Forrester changed: What|Removed |Added Blocks||47913 -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 James Forrester changed: What|Removed |Added Blocks||47411 -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 --- Comment #15 from Liangent --- (In reply to comment #14) > The channel is public anyway, but pasting them on the wiki certainly makes it > easier to search for names. It might be a good idea for you to get an IRC > hostmask cloak, so that the IP does not show up in IRC logs. I already have one, but I often see this happening: [09:23] --> spectie has joined this channel (~fran@***). [09:23] <-- spectie has left this server (Changing host). [09:23] --> spectie has joined this channel (~fran@unaffiliated/spectie). I guess it happpens when the user /msg nickserv identify xxx after they joins the channel, and the sequence is usually decided by their IRC client. -- You are receiving this mail because: You are on the CC list for the bug. You are watching all bug changes. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 --- Comment #14 from Gabriel Wicke --- The channel is public anyway, but pasting them on the wiki certainly makes it easier to search for names. It might be a good idea for you to get an IRC hostmask cloak, so that the IP does not show up in IRC logs. -- You are receiving this mail because: You are on the CC list for the bug. You are watching all bug changes. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 --- Comment #13 from Liangent --- (In reply to comment #12) > Some more info: > http://www.mediawiki.org/wiki/Parsoid/Language_conversion Maybe you want to avoid pasting IPs in those join-messages onto the wiki next time. :) -- You are receiving this mail because: You are on the CC list for the bug. You are watching all bug changes. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 --- Comment #12 from Gabriel Wicke --- Some more info: http://www.mediawiki.org/wiki/Parsoid/Language_conversion -- You are receiving this mail because: You are on the CC list for the bug. You are watching all bug changes. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 Liangent changed: What|Removed |Added Depends on||43547 -- You are receiving this mail because: You are on the CC list for the bug. You are watching all bug changes. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 --- Comment #11 from Liangent --- (In reply to comment #9) > A few notes from IRC: Let me explain more: The main entry point should be convertTo(), with convert() as a shortcut to use the "preferred" (= automatically guessed from request) variant. It accepts an almost-parsed HTML document (string) with -{}- markups embedded. convertTo() is just a loader. It calls recursiveConvert* afterwards, which parse -{}- syntax, and break text into pieces based on -{}- markups. These pieces are fed into autoConvert(). autoConvert() extracts text snippets which actually need conversion (with HTML tags, blocks etc. excluded, but include "title" attribs in HTML tags again...), then send these snippets to translate(). translate() transforms text finally using strtr()-like mechanism. -- You are receiving this mail because: You are on the CC list for the bug. You are watching all bug changes. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 Gabriel Wicke changed: What|Removed |Added Depends on||46658 -- You are receiving this mail because: You are on the CC list for the bug. You are watching all bug changes. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 --- Comment #10 from Gabriel Wicke --- DOM spec being developed at http://www.mediawiki.org/wiki/Parsoid/MediaWiki_DOM_spec#Language_conversion_blocks. -- You are receiving this mail because: You are on the CC list for the bug. You are watching all bug changes. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 --- Comment #9 from Gabriel Wicke --- A few notes from IRC: [09:59] marktraceur: I browsed the LanguageConverter source a bit [09:59] there is an autoConvert method that simply converts all text based on a dictionary lookup [10:00] it only excludes markup and script/code blocks [10:00] the default search language for Chinese seems to be zh-hans (simplified) [10:01] am not sure when the special conversion syntax is used in practice [10:03] --> tewwy has joined this channel (~tychay@wikimedia/Tychay). [10:03] conversion is restricted to those blocks when using convert() and convertTo() [10:03] plus special conversion for link targets and headings [10:04] the conversion itself is performed using autoConvert (the dictionary-based method) [10:06] * cscott is reading backlog [10:07] yeah, i mentioned getting minority-language buy-in in the meeting yesterday, thinking specifically of how hard it's been to get i18n feedback [10:07] --> HaeB has joined this channel (~quassel@wikipedia/HochaufeinemBaum). [10:08] this languageconverter thing is changing simplified chinese to traditional, and vice-versa? ie, mainland-to-taiwan and back? [10:13] cscott: there are four variants for Chinese I think [10:13] Serbian and some other languages have variants too [10:14] marktraceur: so my reading is that normally convert() is used, which only converts marked-up blocks (-{ }-) [10:15] except for search, which uses autoconvert directly [10:16] the conversion is also lossy, but less ambiguous when converting from traditional to simplified for example [10:16] now the question is how we should represent all this in the DOM [10:20] *** edsanders|away is now known as edsanders. [10:20] on one hand it would be nice to abstract the issue, but with the conversion being lossy that does not seem to be possible without preserving the original (potentially mixed-variant) text -- You are receiving this mail because: You are on the CC list for the bug. You are watching all bug changes. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 Liangent changed: What|Removed |Added Depends on||19044 -- You are receiving this mail because: You are on the CC list for the bug. You are watching all bug changes. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 --- Comment #8 from Mark Holmquist --- We really need to know about how this is *supposed* to go, and we need English documentation for it if our team is going to work on it. The current offerings are all in other languages I think. -- You are receiving this mail because: You are on the CC list for the bug. You are watching all bug changes. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 --- Comment #7 from Liangent --- (In reply to comment #4) > Next question: Should a construct like {{variantopen}}令{{variantclose}} work > (assume it expands to -{令}-)? If not, would it be difficult to phase that > construct out as deprecated and go forward with Parsoid not supporting it? > > Thanks for your help, we'd love to get Parsoid working with zh-wikis. That construct works in the PHP converter. -- You are receiving this mail because: You are on the CC list for the bug. You are watching all bug changes. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 --- Comment #6 from Liangent --- (In reply to comment #3) > Liangent, can you please link us to documentation about how this works? > Initial > searches have been less than fruitful. Do you means how it's done in the PHP parsing process, or what is expected to be done (specification of related syntax)? -- You are receiving this mail because: You are on the CC list for the bug. You are watching all bug changes. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 Gabriel Wicke changed: What|Removed |Added CC||gwi...@wikimedia.org --- Comment #5 from Gabriel Wicke --- Some documentation: * https://www.mediawiki.org/wiki/Language_converter * https://meta.wikimedia.org/wiki/Automatic_conversion_between_simplified_and_traditional_Chinese -- You are receiving this mail because: You are on the CC list for the bug. You are watching all bug changes. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 --- Comment #4 from Mark Holmquist --- Next question: Should a construct like {{variantopen}}令{{variantclose}} work (assume it expands to -{令}-)? If not, would it be difficult to phase that construct out as deprecated and go forward with Parsoid not supporting it? Thanks for your help, we'd love to get Parsoid working with zh-wikis. -- You are receiving this mail because: You are on the CC list for the bug. You are watching all bug changes. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 Mark Holmquist changed: What|Removed |Added CC||mtrac...@member.fsf.org --- Comment #3 from Mark Holmquist --- Liangent, can you please link us to documentation about how this works? Initial searches have been less than fruitful. -- You are receiving this mail because: You are on the CC list for the bug. You are watching all bug changes. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 Gabriel Wicke changed: What|Removed |Added Status|NEW |ASSIGNED Assignee|gwi...@wikimedia.org|mtrac...@member.fsf.org --- Comment #2 from Gabriel Wicke --- It would be good to look into implementing phase 1 (recognize and protect language conversion content). -- You are receiving this mail because: You are on the CC list for the bug. You are watching all bug changes. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 Gabriel Wicke changed: What|Removed |Added Component|CPP/General |JS/General -- You are receiving this mail because: You are on the CC list for the bug. You are watching all bug changes. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 Gabriel Wicke changed: What|Removed |Added Blocks||43332 -- You are receiving this mail because: You are on the CC list for the bug. You are watching all bug changes. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 41716] Support language variant conversion in Parsoid
https://bugzilla.wikimedia.org/show_bug.cgi?id=41716 Gabriel Wicke changed: What|Removed |Added Priority|Unprioritized |Low Component|General |CPP/General --- Comment #1 from Gabriel Wicke 2012-11-02 22:46:59 UTC --- Oh, that will be fun ;) -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l