Re: [Repoze-dev] Separate 'compile' and 'execute' steps in Chameleon [was: Re: repoze.bfg trunk "C-free"]
Malthe Borch wrote: > 2009/5/4 Carlos de la Guardia : >> If Malthe or anyone else has a good idea about how to do this and will give >> me some pointers, I'm willing to use my limited abilities to make this work. >> Otherwise, I might try to bring in old zpt. > > It can be done, but it requires the introduction of a new compilation > mode, which will essentially produce a source-file which will "fit > all"; currently, the compiler produces one source-file per > keyword-argument signature. That's obviously not something you can > reasonably determine at compile-time. > > The way to go about this is probably to have the render-method do > something like this: > > def render(**kwargs): >kwargs['econtext'].update(kwargs) >return render_template(econtext=kwargs['econtext']) > > Such that all variables will be pulled from the dynamic variable > scope; this should work, but otoh, there's always some trickery with > that 'econtext'. This might also slowdown the lookup of the argument values. Especially whatever regular "context" is often passed in as an argument. But it may not be dramatic. In general I'd like the idea of this. I'd assume that there's gonna be some nice subtle problems with it, but nothing the excellent test coverage shouldn't find. > Chameleon has a mode of operation triggered by the CHAMELEON_EAGER > flag; this basically parses all templates at initialization; I think > we can hijack this mode to force the above-mentioned "fallback" > compilation. The render-logic would then be something like this: > > 1) Do we have ``ast`` and ``compiler`` available? > 2) If not, is a fallback source-file available? If we'd get this, I'd like to expose the "compile" step in an additional way. It should be possible to do this as a distutils build_templates command in the setup.py. That'd make it similar in concept to building extensions (or for example gettext .mo files). It's something you can do as part of your build before releasing a package. The underlying compiled templates format is just a pickle protocol 2 dump and thus doesn't vary on any platform or Python build settings. So putting those files into a source distribution should be fine in general. Hanno ___ Repoze-dev mailing list Repoze-dev@lists.repoze.org http://lists.repoze.org/listinfo/repoze-dev
Re: [Repoze-dev] Separate 'compile' and 'execute' steps in Chameleon [was: Re: repoze.bfg trunk "C-free"]
2009/5/4 Carlos de la Guardia : > If Malthe or anyone else has a good idea about how to do this and will give > me some pointers, I'm willing to use my limited abilities to make this work. > Otherwise, I might try to bring in old zpt. It can be done, but it requires the introduction of a new compilation mode, which will essentially produce a source-file which will "fit all"; currently, the compiler produces one source-file per keyword-argument signature. That's obviously not something you can reasonably determine at compile-time. The way to go about this is probably to have the render-method do something like this: def render(**kwargs): kwargs['econtext'].update(kwargs) return render_template(econtext=kwargs['econtext']) Such that all variables will be pulled from the dynamic variable scope; this should work, but otoh, there's always some trickery with that 'econtext'. Chameleon has a mode of operation triggered by the CHAMELEON_EAGER flag; this basically parses all templates at initialization; I think we can hijack this mode to force the above-mentioned "fallback" compilation. The render-logic would then be something like this: 1) Do we have ``ast`` and ``compiler`` available? 2) If not, is a fallback source-file available? \malthe ___ Repoze-dev mailing list Repoze-dev@lists.repoze.org http://lists.repoze.org/listinfo/repoze-dev
Re: [Repoze-dev] Separate 'compile' and 'execute' steps in Chameleon [was: Re: repoze.bfg trunk "C-free"]
I tried a separate compile step yesterday, but chameleon uses marshal for recreating the code objects from the .cache files. It also uses compiler and ast to generate the compiled templates that are stored in a registry once the app is running. Compiler is OK until it tries to use parser, which is not included because it can generate code objects. In fact, like Chris says, any attempt to produce code objects by other means (like new.code()) will fail with a restricted error. If Malthe or anyone else has a good idea about how to do this and will give me some pointers, I'm willing to use my limited abilities to make this work. Otherwise, I might try to bring in old zpt. Carlos de la Guardia On Sun, May 3, 2009 at 8:21 PM, Chris McDonough wrote: > On 5/3/09 8:51 PM, Sidnei da Silva wrote: > >> On Sat, May 2, 2009 at 1:53 PM, Carlos de la Guardia >> wrote: >> >>> Chameleon.* will not work because it needs to generate code objects and >>> that's forbidden in GAE (no marshal.py, parser.py or direct calls to >>> new.code() or similar). >>> >> >> I would like to bring up an idea that Paul Everitt had: >> >> - What if chameleon supported separate 'compile' and 'execute' steps? >> >> 1. Chameleon basically compile templates to Python source code. >> 2. It is possible to save the generated source code to .py files >> (CHAMELEON_SOURCE or something in the environment) >> 3. Therefore, it should be possible to copy the .py files to GAE and >> have them be executed without recompiling from the .pt file (of course >> with some changes to chameleon) >> > > IIRC, the "cache" files in Chameleon aren't just .py files, they hold a > marshalled code object plus other info in the pickle that Chameleon uses to > resolve macros (I think). I suspect it's not falling-off-a-log easy as a > result. > > - C > ___ Repoze-dev mailing list Repoze-dev@lists.repoze.org http://lists.repoze.org/listinfo/repoze-dev
Re: [Repoze-dev] Separate 'compile' and 'execute' steps in Chameleon [was: Re: repoze.bfg trunk "C-free"]
On 5/3/09 8:51 PM, Sidnei da Silva wrote: > On Sat, May 2, 2009 at 1:53 PM, Carlos de la Guardia > wrote: >> Chameleon.* will not work because it needs to generate code objects and >> that's forbidden in GAE (no marshal.py, parser.py or direct calls to >> new.code() or similar). > > I would like to bring up an idea that Paul Everitt had: > > - What if chameleon supported separate 'compile' and 'execute' steps? > > 1. Chameleon basically compile templates to Python source code. > 2. It is possible to save the generated source code to .py files > (CHAMELEON_SOURCE or something in the environment) > 3. Therefore, it should be possible to copy the .py files to GAE and > have them be executed without recompiling from the .pt file (of course > with some changes to chameleon) IIRC, the "cache" files in Chameleon aren't just .py files, they hold a marshalled code object plus other info in the pickle that Chameleon uses to resolve macros (I think). I suspect it's not falling-off-a-log easy as a result. - C ___ Repoze-dev mailing list Repoze-dev@lists.repoze.org http://lists.repoze.org/listinfo/repoze-dev
