Jira (PUP-8024) Delay loading module translations until 'translate' is used for that module
Title: Message Title
Maggie Dreyer updated an issue
Puppet / PUP-8024
Delay loading module translations until 'translate' is used for that module
Change By:
Maggie Dreyer
Currently, we attempt to initialize translations machinery for all modules as soon as they are loaded, despite the fact in many contexts we do not need to display any translations (e.g. {{module list}}). We do this even for modules that do not have any translation files or marked strings.We also attempt to cache translations after they're loaded, but not associated with any existing cache lifetimes. So the only way to cause a translation to be reloaded is to restart the process. Instead, we We should look into only initializing gettext for modules when we first need add a function to call the {{ translate() module.rb }} or {{_()}} functions in that loads a module code. This would spread out expensive initialization and avoid 's translations, then call it altogether in contexts where it is unnecessary before we think we need module translations to be displayed (e . g. before compilation, or when running a puppet application). *Notes on decoupling module naming from locales/config.yaml project name*From comment below:{quote}As part of this, we should remove the requirement that the module have metadata (relies on forge_name) and remove the coupling of metadata naming and locales/config.yaml's project_name.{quote}Benchmarks in puppet have shown a non-trivial performance degradation if there is a mismatch between the module's metadata name, and the locales/config.yaml's project_name:With matching names:{code}mkdir -p /var/folders/0_/6qt793gx6w706cq1q9kc0w78gq/T/many_modules20171009-2426-1l58rtz user system totalrealRun 13.70 0.46 4.17 ( 4.171436)Run 22.84 0.13 2.97 ( 2.974983)Run 32.89 0.12 3.01 ( 3.016188)Run 42.91 0.13 3.04 ( 3.036231)Run 52.77 0.15 2.92 ( 2.932649)Run 62.80 0.12 2.92 ( 2.910116)Run 73.24 0.11 3.35 ( 3.359197)Run 82.76 0.15 2.91 ( 2.909707)Run 93.26 0.15 3.41 ( 3.414169)Run 10 3.69 0.14 3.83 ( 3.835340)> total:30.86 1.66 32.53 ( 32.560017)> avg: 3.086000 0.166000 3.253000 ( 3.256002){code}with mismatched names:{code}mkdir -p /var/folders/0_/6qt793gx6w706cq1q9kc0w78gq/T/many_modules20171009-2350-11p565v user system totalrealRun 13.68 0.46 4.15 ( 4.157170)Run 23.30 0.25 3.55 ( 3.552481)Run 33.18 0.25 3.43 ( 3.430290)Run 43.40 0.26 3.66 ( 3.662123)Run 53.69 0.31 4.00 ( 4.010465)Run 63.60 0.26 3.86 ( 3.866366)Run 73.32 0.25 3.57 ( 3.571592)Run 83.61 0.32 3.93 ( 3.943484)Run 93.56 0.31 3.87 ( 3.866563)Run 10 3.99 0.26 4.25 ( 4.247610)> total:35.33 2.93 38.27 ( 38.308143)> avg: 3.533000 0.293000 3.827000 ( 3.830814){code}Note ~ 6 second cumula
Jira (PUP-8024) Delay loading module translations until 'translate' is used for that module
Title: Message Title
Michael Smith updated an issue
Puppet / PUP-8024
Delay loading module translations until 'translate' is used for that module
Change By:
Michael Smith
Currently, we attempt to initialize translations machinery for all modules as soon as they are loaded, despite the fact in many contexts we do not need to display any translations (e.g. {{module list}}). We do this even for modules that do not have any translation files or marked strings.We also attempt to cache translations after they're loaded, but not associated with any existing cache lifetimes. So the only way to cause a translation to be reloaded is to restart the process.Instead, we should look into only initializing gettext for modules when we first need to call the {{translate()}} or {{_()}} functions in module code. This would spread out expensive initialization and avoid it altogether in contexts where it is unnecessary.*Notes on decoupling module naming from locales/config.yaml project name*From comment below:{quote}As part of this, we should remove the requirement that the module have metadata (relies on forge_name) and remove the coupling of metadata naming and locales/config.yaml's project_name.{quote}Benchmarks in puppet have shown a non-trivial performance degradation if there is a mismatch between the module's metadata name, and the locales/config.yaml's project_name:With matching names:{code}mkdir -p /var/folders/0_/6qt793gx6w706cq1q9kc0w78gq/T/many_modules20171009-2426-1l58rtz user system totalrealRun 13.70 0.46 4.17 ( 4.171436)Run 22.84 0.13 2.97 ( 2.974983)Run 32.89 0.12 3.01 ( 3.016188)Run 42.91 0.13 3.04 ( 3.036231)Run 52.77 0.15 2.92 ( 2.932649)Run 62.80 0.12 2.92 ( 2.910116)Run 73.24 0.11 3.35 ( 3.359197)Run 82.76 0.15 2.91 ( 2.909707)Run 93.26 0.15 3.41 ( 3.414169)Run 10 3.69 0.14 3.83 ( 3.835340)> total:30.86 1.66 32.53 ( 32.560017)> avg: 3.086000 0.166000 3.253000 ( 3.256002){code}with mismatched names:{code}mkdir -p /var/folders/0_/6qt793gx6w706cq1q9kc0w78gq/T/many_modules20171009-2350-11p565v user system totalrealRun 13.68 0.46 4.15 ( 4.157170)Run 23.30 0.25 3.55 ( 3.552481)Run 33.18 0.25 3.43 ( 3.430290)Run 43.40 0.26 3.66 ( 3.662123)Run 53.69 0.31 4.00 ( 4.010465)Run 63.60 0.26 3.86 ( 3.866366)Run 73.32 0.25 3.57 ( 3.571592)Run 83.61 0.32 3.93 ( 3.943484)Run 93.56 0.31 3.87 ( 3.866563)Run 10 3.99 0.26 4.25 ( 4.247610)> total:35.33 2.93 38.27 ( 38.308143)> avg: 3.533000 0.293000 3.827000 ( 3.830814){code}Note ~ 6 second cumulative runtime increase Note that a further implication of how we're loading modules is that if modules don't have metadata, they [will not have a forge_name|https://github.com/puppetlabs/puppet/blob/5.3.1/
Jira (PUP-8024) Delay loading module translations until 'translate' is used for that module
Title: Message Title Michael Smith updated an issue Puppet / PUP-8024 Delay loading module translations until 'translate' is used for that module Change By: Michael Smith Labels: i18n Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-8024) Delay loading module translations until 'translate' is used for that module
Title: Message Title Craig Gomes updated an issue Puppet / PUP-8024 Delay loading module translations until 'translate' is used for that module Change By: Craig Gomes Fix Version/s: PUP 5.3.3 Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-8024) Delay loading module translations until 'translate' is used for that module
Title: Message Title
Moses Mendoza updated an issue
Puppet / PUP-8024
Delay loading module translations until 'translate' is used for that module
Change By:
Moses Mendoza
Currently, we attempt to initialize translations machinery for all modules as soon as they are loaded, despite the fact in many contexts we do not need to display any translations (e.g. {{module list}}). We do this even for modules that do not have any translation files or marked strings.We also attempt to cache translations after they're loaded, but not associated with any existing cache lifetimes. So the only way to cause a translation to be reloaded is to restart the process.Instead, we should look into only initializing gettext for modules when we first need to call the {{translate()}} or {{_()}} functions in module code. This would spread out expensive initialization and avoid it altogether in contexts where it is unnecessary. *Notes on decoupling module naming from locales/config.yaml project name*From comment below:{quote}As part of this, we should remove the requirement that the module have metadata (relies on forge_name) and remove the coupling of metadata naming and locales/config.yaml's project_name.{quote}Benchmarks in puppet have shown a non-trivial performance degradation if there is a mismatch between the module's metadata name, and the locales/config.yaml's project_name:With matching names:{code}mkdir -p /var/folders/0_/6qt793gx6w706cq1q9kc0w78gq/T/many_modules20171009-2426-1l58rtz user system totalrealRun 13.70 0.46 4.17 ( 4.171436)Run 22.84 0.13 2.97 ( 2.974983)Run 32.89 0.12 3.01 ( 3.016188)Run 42.91 0.13 3.04 ( 3.036231)Run 52.77 0.15 2.92 ( 2.932649)Run 62.80 0.12 2.92 ( 2.910116)Run 73.24 0.11 3.35 ( 3.359197)Run 82.76 0.15 2.91 ( 2.909707)Run 93.26 0.15 3.41 ( 3.414169)Run 10 3.69 0.14 3.83 ( 3.835340)> total:30.86 1.66 32.53 ( 32.560017)> avg: 3.086000 0.166000 3.253000 ( 3.256002){code}with mismatched names:{code}mkdir -p /var/folders/0_/6qt793gx6w706cq1q9kc0w78gq/T/many_modules20171009-2350-11p565v user system totalrealRun 13.68 0.46 4.15 ( 4.157170)Run 23.30 0.25 3.55 ( 3.552481)Run 33.18 0.25 3.43 ( 3.430290)Run 43.40 0.26 3.66 ( 3.662123)Run 53.69 0.31 4.00 ( 4.010465)Run 63.60 0.26 3.86 ( 3.866366)Run 73.32 0.25 3.57 ( 3.571592)Run 83.61 0.32 3.93 ( 3.943484)Run 93.56 0.31 3.87 ( 3.866563)Run 10 3.99 0.26 4.25 ( 4.247610)> total:35.33 2.93 38.27 ( 38.308143)> avg: 3.533000 0.293000 3.827000 ( 3.830814){code}Note ~ 6 second cumulative runtime increase
Jira (PUP-8024) Delay loading module translations until 'translate' is used for that module
Title: Message Title Michael Smith commented on PUP-8024 Re: Delay loading module translations until 'translate' is used for that module Most likely you get the original english string rather than a translation. I'm actually not sure, there's a chance the i18n registry never forgets translations so they just add on to it and it's not an issue (unless you have two different modules with different translations for the same string). We should test this out, but it may be a non-issue. Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-8024) Delay loading module translations until 'translate' is used for that module
Title: Message Title Henrik Lindberg commented on PUP-8024 Re: Delay loading module translations until 'translate' is used for that module Michael Smith What can happen if translations are wrong/off - can that crash because of parameter mismatch, or is the worse case that you get garbage messages/messages with information missing ? Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-8024) Delay loading module translations until 'translate' is used for that module
Title: Message Title Maggie Dreyer updated an issue Puppet / PUP-8024 Delay loading module translations until 'translate' is used for that module Change By: Maggie Dreyer Fix Version/s: PUP 5.3.3 Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-8024) Delay loading module translations until 'translate' is used for that module
Title: Message Title Michael Smith commented on PUP-8024 Re: Delay loading module translations until 'translate' is used for that module Another implication of this work: I think translations for modules will bleed across environments. Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-8024) Delay loading module translations until 'translate' is used for that module
Title: Message Title Henrik Lindberg commented on PUP-8024 Re: Delay loading module translations until 'translate' is used for that module Michael Smith For 4.x loaders there is an instantiator per thing that can be loaded. For all 3.x things it is the Autoloader that is configured to load from places, and then the ruby code in the file that autoloader finds is supposed to just do the right thing... Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-8024) Delay loading module translations until 'translate' is used for that module
Title: Message Title Michael Smith commented on PUP-8024 Re: Delay loading module translations until 'translate' is used for that module Unfortunately PUP-8026 - and the implication that Ruby code should be able to use translations - makes when to load module translations a little more difficult. We need to lazily load in code either when we hit a translate function, or call another Ruby function or type or load custom facts from modules. Each of those implies much smaller performance hits than the current implementation, but it's a fairly broad surface area of triggers to instrument. Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-8024) Delay loading module translations until 'translate' is used for that module
Title: Message Title
Michael Smith updated an issue
Puppet / PUP-8024
Delay loading module translations until 'translate' is used for that module
Change By:
Michael Smith
Currently, we attempt to initialize translations machinery for all modules as soon as they are loaded, despite the fact in many contexts we do not need to display any translations (e.g. {{module list}}). We do this even for modules that do not have any translation files or marked strings. We also attempt to cache translations after they're loaded, but not associated with any existing cache lifetimes. So the only way to cause a translation to be reloaded is to restart the process. Instead, we should look into only initializing gettext for modules when we first need to call the {{translate()}} or {{_()}} functions in module code. This would spread out expensive initialization and avoid it altogether in contexts where it is unnecessary.
Add Comment
This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe)
--
You received this message because you are subscribed to the Google Groups "Puppet Bugs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com.
To post to this group, send email to puppet-bugs@googlegroups.com.
Visit this group at https://groups.google.com/group/puppet-bugs.
For more options, visit https://groups.google.com/d/optout.
Jira (PUP-8024) Delay loading module translations until 'translate' is used for that module
Title: Message Title Michael Smith commented on PUP-8024 Re: Delay loading module translations until 'translate' is used for that module As part of this, we should remove the requirement that the module have metadata (relies on forge_name) and remove the coupling of metadata naming and locales/config.yaml's project_name. Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-8024) Delay loading module translations until 'translate' is used for that module
Title: Message Title Michael Smith updated an issue Puppet / PUP-8024 Delay loading module translations until 'translate' is used for that module Change By: Michael Smith Priority: Normal Major Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-8024) Delay loading module translations until 'translate' is used for that module
Title: Message Title Michael Smith commented on PUP-8024 Re: Delay loading module translations until 'translate' is used for that module Serving files or file metadata is another example of when we do this unnecessarily. That can be pretty expensive. Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-8024) Delay loading module translations until 'translate' is used for that module
Title: Message Title Maggie Dreyer created an issue Puppet / PUP-8024 Delay loading module translations until 'translate' is used for that module Issue Type: Improvement Assignee: Unassigned Created: 2017/10/04 1:04 PM Priority: Normal Reporter: Maggie Dreyer Currently, we attempt to initialize translations machinery for all modules as soon as they are loaded, despite the fact in many contexts we do not need to display any translations (e.g. module list). We do this even for modules that do not have any translation files or marked strings. Instead, we should look into only initializing gettext for modules when we first need to call the translate() or _() functions in module code. This would spread out expensive initialization and avoid it altogether in contexts where it is unnecessary. Add Comment
