Re: Pluggable template retrival/caching logic (Was: Template Loader Implementation that loads from a database (DataSource))
Friday, June 9, 2017, 12:19:02 AM, Daniel Dekany wrote: [snip] > DefaultTemplateResolver happens to support them. If > Configuration.templateLoader is set to `new MyTemplateResolver()`, and I meant "If Configuration.templateResolver is set to `new MyTemplateResolver()`" of course... -- Thanks, Daniel Dekany
Re: Pluggable template retrival/caching logic (Was: Template Loader Implementation that loads from a database (DataSource))
Now this mechanism is working in FM3. At the end I did this: http://freemarker.org/builds/fm3/api/freemarker-core/org/apache/freemarker/core/templateresolver/TemplateResolver.html Explanation: If you implement a custom TemplateResolver, then you are required to declare which of the Configuration settings you support, out of those that are related to template loading/caching. Then the user just sets those setting on the Configuration-level, pretty much like in FM2 (except that in FM3 you set settings in a Configuration.Builder object, not directly in the Configuration). DefaultTemplateResolver is not treated specially, it works just like any custom TemplateResolver; you are allowed to set Configuration.templateLoader, Configuration.templateUpdateDellayMilliseconds, etc. because DefaultTemplateResolver happens to support them. If Configuration.templateLoader is set to `new MyTemplateResolver()`, and MyTemplateResolver only cares about templateUpdateDellayMilliseconds, but can't use the TemplateLoader abstraction of FM, then the user can set templateUpdateDellayMilliseconds as usual, but can't set Configuration.templateLoader (they get an exception that tells they that MyTemplateResolver doesn't support the templateLoader setting). The author of a custom TemplateResolver admittedly has to face some extra complexity here, but I believe it's not a big deal in practice. Also, very few developers will ever implement a custom TemplateResolver. This thing is optimized for the users who just configure FreeMarker. Any insights? Like as a reality check, how well would this work with Moqui? Thursday, February 16, 2017, 6:33:01 PM, David E Jones wrote: > On Thu, 2017-02-16 at 15:56 +0100, Daniel Dekany wrote: >> Thursday, February 16, 2017, 6:17:00 AM, David E Jones wrote: >> >> > >> > This is cleaner, more obvious what's going on underneath, but since >> > the DefaultTemplateResolver will be the most commonly used you >> > could just leave the current setting methods as they are and just >> > document that if you set a different TemplateResolver they will be >> > ignored. >> >> Or they should just throw exception if the template resolver set is >> not the default one. It can be quite annoying when someone sets the >> templateUpdateDelay for example and it has silently no effect. > > Good idea, I was thinking more of calls before setting a TemplateResolver. > >> Also, it's maybe quite speculative, but perhaps some custom >> TemplateResolver have some of the standard settings, like for example >> the templateUpdateDelayMilliseconds. Then it's somewhat confusing that >> cfg.setTemplateUpdateDelayMilliseconds doesn't work, while >> myResolver.setTemplateUpdateDelayMilliseconds does. So I guess if we >> keep those setters in the Configuration, then they should forward to >> the templateResolver, if it implements the interface that contains the >> setter. > > It isn't too cumbersome to have extra methods to implement, but > would be nice to have those methods with a default implementation > that throws the appropriate Exception. In Java 8 you can do this > with default methods on the interface, but in Java 7- this could > simply be an abstract class to have the same effect. > > -David -- Thanks, Daniel Dekany
Re: Pluggable template retrival/caching logic (Was: Template Loader Implementation that loads from a database (DataSource))
Note that there's quick draft of TemplateResolver committed in the "3" branch: https://github.com/apache/incubator-freemarker/blob/3/src/main/java/org/apache/freemarker/core/templateresolver/TemplateResolver.java It's not integrated with the other parts of FM (it's assumed everywhere that we have a DefaultTemplateResolver, not a generic TemplateResolver), nor does it deal with the configuration aspects discussed today. But it shows an idea for the main functionality. Tuesday, February 7, 2017, 11:59:08 AM, Daniel Dekany wrote: > Tuesday, February 7, 2017, 7:40:18 AM, David E Jones wrote: > >> On Tue, 2017-01-24 at 23:47 +0100, Daniel Dekany wrote: >>> >>> Should the TemplateResolver-s (the work name for the new interface) >>> still get the template name already resolved to absolute and >>> normalized? So when cfg.getTemplate is called, then it invokes >>> cfg.templateNameFormat.normalizeAbsoluteName(String) to normalize the >>> path, and only then it calls the TemplateResolver. Also #import and >>> such currently calls templateNameFormat.toAbsoluteName(String, >>> String), and that would happen earlier than TemplateResolver is >>> involved. >>> >>> And how would it look... perhaps like this: >>> >>> public interface TemplateResolver { >>> >>> Template getTemplate(String templatePath, Local lookupLocale, Object >>> customLookupCondition) >>> throws IOException; >>> >>> } >>> >>> You may notice that it misses the `encoding` and `parsed` parameter of >>> cfg.getTemplate. That's because I belive that in FM3 we should not >>> allow specifying those as getTemplate parameters anymore, but that >>> will be another thread when we get there. >> >> IMO it should pass through the exact text from the include/etc in >> the template. It might be worth constraining to a valid URI syntax >> but other than that a TemplateResolver would be much more flexible if no >> normalization/etc is done. > > In FM2 there's a class like this: > > public abstract class TemplateNameFormat { > ... > abstract String toAbsoluteName(String baseName, String > targetName) throws MalformedTemplateNameException; > abstract String normalizeAbsoluteName(String name) throws > MalformedTemplateNameException; > } > > My idea was that if we allow users to provide a custom implementation, > then if the standard rules don't fit your custom TemplateResolver, you > just create a custom TemplateNameFormat too. But now that I think > about it more, pushing the TemplateNameFormat functionality on the > TemplateResolver implementation does indeed look like a better > approach, because then it can't happen that a configuration > accidentally contains a TemplateResolver with a TemplateNameFormat > that doesn't work well with it. But then the above two methods should > be part of the TemplateResolver interface (because, for example, when > you #import something, FM has to find out if the same FTL was already > imported in the current Environment, so it needs to normalize the > #import argument without loading anything). WDYT? > >> Passing in the Locale rather than trying to call it multiple times >> with different locale extensions is a good idea too as different >> underlying content or file stores may have their own approach for >> this. > > That's even a problem with the TemplateLoader interface proposed to > FM3. Like in the recent database template loader thread, the author > wanted to use a separate column to indicate the locale of the template > (and the same could happen with custom lookup conditions too), so you > don't want to encode that information into the template name in the > case of that particular TemplateLoader. It's something we may try to > address in a second iteration of the FM3 TemplateLoader interface. > >> For example a JCR back end has quite a bit for meta data and >> content variations. >> >> In general more parameters and less built into the path/name is a good thing >> IMO. > > BTW, I still want to see if in FM3 the standard TemplateResolver (the > one that's basically FM2's TemplateCache, relies on TemplateLoader and > TemplateLookupStrategy and TemplateNameFormat) could cover more use > cases without becoming overly intricate. Also, even if you decide to > use a custom TemplateResolver, perhaps you still want to call some of > the stock TemplateLoader-s from it (while you replace the caching > logic, etc., or load from non-TemplateLoader sources too in case of > more sophisticated storages). I suspect you have experience with these > things, so it would be good if you can look at the FM3 TemplateLoader > thread. > >> -David > -- Thanks, Daniel Dekany
Re: Pluggable template retrival/caching logic (Was: Template Loader Implementation that loads from a database (DataSource))
Thursday, February 16, 2017, 5:33:01 PM, David E Jones wrote: > On Thu, 2017-02-16 at 15:56 +0100, Daniel Dekany wrote: >> Thursday, February 16, 2017, 6:17:00 AM, David E Jones wrote: >> >> > >> > This is cleaner, more obvious what's going on underneath, but since >> > the DefaultTemplateResolver will be the most commonly used you >> > could just leave the current setting methods as they are and just >> > document that if you set a different TemplateResolver they will be >> > ignored. >> >> Or they should just throw exception if the template resolver set is >> not the default one. It can be quite annoying when someone sets the >> templateUpdateDelay for example and it has silently no effect. > > Good idea, I was thinking more of calls before setting a TemplateResolver. I would clarify there that it's the build() method that throws the exception. Because we don't want to constraint the order in which you set the settings. Like maybe you set the templateLoader on the builder, and only then set the TemplateLoader to MyTemplateResolver that doesn't support a templateLoader. >> Also, it's maybe quite speculative, but perhaps some custom >> TemplateResolver have some of the standard settings, like for example >> the templateUpdateDelayMilliseconds. Then it's somewhat confusing that >> cfg.setTemplateUpdateDelayMilliseconds doesn't work, while >> myResolver.setTemplateUpdateDelayMilliseconds does. So I guess if we >> keep those setters in the Configuration, then they should forward to >> the templateResolver, if it implements the interface that contains the >> setter. > > It isn't too cumbersome to have extra methods to implement, but > would be nice to have those methods with a default implementation > that throws the appropriate Exception. In Java 8 you can do this > with default methods on the interface, but in Java 7- this could > simply be an abstract class to have the same effect. Yes, that looks to be the simplest solution. (Even if not the cleanest... that would be introducing interfaces like HasTemplateLoaderSetting, and so you can detect this capability without "probing" the setter methods. Certainly an overkill anyway.) > -David > > -- Thanks, Daniel Dekany
Re: Pluggable template retrival/caching logic (Was: Template Loader Implementation that loads from a database (DataSource))
On Thu, 2017-02-16 at 15:56 +0100, Daniel Dekany wrote: > Thursday, February 16, 2017, 6:17:00 AM, David E Jones wrote: > > > > > This is cleaner, more obvious what's going on underneath, but since > > the DefaultTemplateResolver will be the most commonly used you > > could just leave the current setting methods as they are and just > > document that if you set a different TemplateResolver they will be > > ignored. > > Or they should just throw exception if the template resolver set is > not the default one. It can be quite annoying when someone sets the > templateUpdateDelay for example and it has silently no effect. Good idea, I was thinking more of calls before setting a TemplateResolver. > Also, it's maybe quite speculative, but perhaps some custom > TemplateResolver have some of the standard settings, like for example > the templateUpdateDelayMilliseconds. Then it's somewhat confusing that > cfg.setTemplateUpdateDelayMilliseconds doesn't work, while > myResolver.setTemplateUpdateDelayMilliseconds does. So I guess if we > keep those setters in the Configuration, then they should forward to > the templateResolver, if it implements the interface that contains the > setter. It isn't too cumbersome to have extra methods to implement, but would be nice to have those methods with a default implementation that throws the appropriate Exception. In Java 8 you can do this with default methods on the interface, but in Java 7- this could simply be an abstract class to have the same effect. -David
Re: Pluggable template retrival/caching logic (Was: Template Loader Implementation that loads from a database (DataSource))
Thursday, February 16, 2017, 6:17:00 AM, David E Jones wrote: > > This is cleaner, more obvious what's going on underneath, but since > the DefaultTemplateResolver will be the most commonly used you > could just leave the current setting methods as they are and just > document that if you set a different TemplateResolver they will be > ignored. Or they should just throw exception if the template resolver set is not the default one. It can be quite annoying when someone sets the templateUpdateDelay for example and it has silently no effect. Also, it's maybe quite speculative, but perhaps some custom TemplateResolver have some of the standard settings, like for example the templateUpdateDelayMilliseconds. Then it's somewhat confusing that cfg.setTemplateUpdateDelayMilliseconds doesn't work, while myResolver.setTemplateUpdateDelayMilliseconds does. So I guess if we keep those setters in the Configuration, then they should forward to the templateResolver, if it implements the interface that contains the setter. > -David > > > On Wed, 2017-02-15 at 14:56 +0100, Daniel Dekany wrote: >> An interesting consequence of introducing the TemplateResolver concept >> is that it the earlier top-level configuration settings, like >> templateLoader, templateLookupStrategy, templateNameFormat, >> cacheStorage, and templateUpdateDelayMillisecond should now be the >> settings (JavaBean properties) of DefaultTemplateResolver, not of >> Configuration. In theory at least. It's the same things as with >> ObjectWrapper, where multiple implementations are possible, and each >> has different settings. >> >> This has some inconvenient consequences for the majority who uses the >> DefaultTemplateResolver. Earlier, you have written: >> >> cfg.setTemplateLoader(new FooTemplateLoader()) >> cfg.setTemplateUpdateDelayMilliseconds(10_000); >> >> but now you had to write either: >> >> DefaultTemplateResolver dtr = new DefaultTemplateResolver(); >> dtr.setTemplateLoader(new FooTemplateLoader()); >> dtr.setTemplateUpdateDelayMilliseconds(10_000); >> cfg.setTemplateResolver(dtr); >> >> or alternatively: >> >> DefaultTemplateResolver dtr = (DefaultTemplateResolver) >> dtr.getTemplateResolver(); >> dtr.setTemplateLoader(new FooTemplateLoader()); >> dtr.setTemplateUpdateDelayMilliseconds(10_000); >> >> Also configuring with j.u.Properties becomes different, because you >> can't have this anymore: >> >> templateLoader = com.example.FooTemplateLoader >> templateUpdateDelay = 10s >> >> I guess instead it should be something like: >> >> templateResolver.templateLoader = com.example.FooTemplateLoader >> templateResolver.templateUpdateDelay = 10s >> >> which is a syntax that we don't yet support (dotted setting names). >> This syntax we do support though (in FM2): >> >> templateResolver=DefaultTemplateResolver( \ >> templateLoader = com.example.FooTemplateLoader, \ >> templateUpdateDelayMilliseconds = 10_000 \ >> ) >> >> but it has the problem that you can't, for example, set the >> templateUpdateDelay without also specifying the whole templateResolver >> with all of its settings. So that's where the dotted notation is >> better. OTOH for some other things the above syntax is better, like >> for specifying the templateConfigurations (see >> http://freemarker.org/docs/pgui_config_templateconfigurations.html). >> So I think we should have both, and since we must ensure that >> templateResolver=com.example.MyTemplateResolver is "executed" earlier >> than templateResolver.mySetting=foo anyway, if mySetting is specified >> on both places (i.e., you have >> templateResolver=com.example.MyTemplateResolver(mySetting="foo") >> there), then obviously templateResolver.mySetting wins. >> >> And if we support the dotted notation in j.u.Properties, that should >> work for the objectWrapper setting too, naturally. >> >> >> > -- Thanks, Daniel Dekany
Re: Pluggable template retrival/caching logic (Was: Template Loader Implementation that loads from a database (DataSource))
This is cleaner, more obvious what's going on underneath, but since the DefaultTemplateResolver will be the most commonly used you could just leave the current setting methods as they are and just document that if you set a different TemplateResolver they will be ignored. -David On Wed, 2017-02-15 at 14:56 +0100, Daniel Dekany wrote: > An interesting consequence of introducing the TemplateResolver concept > is that it the earlier top-level configuration settings, like > templateLoader, templateLookupStrategy, templateNameFormat, > cacheStorage, and templateUpdateDelayMillisecond should now be the > settings (JavaBean properties) of DefaultTemplateResolver, not of > Configuration. In theory at least. It's the same things as with > ObjectWrapper, where multiple implementations are possible, and each > has different settings. > > This has some inconvenient consequences for the majority who uses the > DefaultTemplateResolver. Earlier, you have written: > > cfg.setTemplateLoader(new FooTemplateLoader()) > cfg.setTemplateUpdateDelayMilliseconds(10_000); > > but now you had to write either: > > DefaultTemplateResolver dtr = new DefaultTemplateResolver(); > dtr.setTemplateLoader(new FooTemplateLoader()); > dtr.setTemplateUpdateDelayMilliseconds(10_000); > cfg.setTemplateResolver(dtr); > > or alternatively: > > DefaultTemplateResolver dtr = (DefaultTemplateResolver) > dtr.getTemplateResolver(); > dtr.setTemplateLoader(new FooTemplateLoader()); > dtr.setTemplateUpdateDelayMilliseconds(10_000); > > Also configuring with j.u.Properties becomes different, because you > can't have this anymore: > > templateLoader = com.example.FooTemplateLoader > templateUpdateDelay = 10s > > I guess instead it should be something like: > > templateResolver.templateLoader = com.example.FooTemplateLoader > templateResolver.templateUpdateDelay = 10s > > which is a syntax that we don't yet support (dotted setting names). > This syntax we do support though (in FM2): > > templateResolver=DefaultTemplateResolver( \ > templateLoader = com.example.FooTemplateLoader, \ > templateUpdateDelayMilliseconds = 10_000 \ > ) > > but it has the problem that you can't, for example, set the > templateUpdateDelay without also specifying the whole templateResolver > with all of its settings. So that's where the dotted notation is > better. OTOH for some other things the above syntax is better, like > for specifying the templateConfigurations (see > http://freemarker.org/docs/pgui_config_templateconfigurations.html). > So I think we should have both, and since we must ensure that > templateResolver=com.example.MyTemplateResolver is "executed" earlier > than templateResolver.mySetting=foo anyway, if mySetting is specified > on both places (i.e., you have > templateResolver=com.example.MyTemplateResolver(mySetting="foo") > there), then obviously templateResolver.mySetting wins. > > And if we support the dotted notation in j.u.Properties, that should > work for the objectWrapper setting too, naturally. > > >
Re: Pluggable template retrival/caching logic (Was: Template Loader Implementation that loads from a database (DataSource))
An interesting consequence of introducing the TemplateResolver concept is that it the earlier top-level configuration settings, like templateLoader, templateLookupStrategy, templateNameFormat, cacheStorage, and templateUpdateDelayMillisecond should now be the settings (JavaBean properties) of DefaultTemplateResolver, not of Configuration. In theory at least. It's the same things as with ObjectWrapper, where multiple implementations are possible, and each has different settings. This has some inconvenient consequences for the majority who uses the DefaultTemplateResolver. Earlier, you have written: cfg.setTemplateLoader(new FooTemplateLoader()) cfg.setTemplateUpdateDelayMilliseconds(10_000); but now you had to write either: DefaultTemplateResolver dtr = new DefaultTemplateResolver(); dtr.setTemplateLoader(new FooTemplateLoader()); dtr.setTemplateUpdateDelayMilliseconds(10_000); cfg.setTemplateResolver(dtr); or alternatively: DefaultTemplateResolver dtr = (DefaultTemplateResolver) dtr.getTemplateResolver(); dtr.setTemplateLoader(new FooTemplateLoader()); dtr.setTemplateUpdateDelayMilliseconds(10_000); Also configuring with j.u.Properties becomes different, because you can't have this anymore: templateLoader = com.example.FooTemplateLoader templateUpdateDelay = 10s I guess instead it should be something like: templateResolver.templateLoader = com.example.FooTemplateLoader templateResolver.templateUpdateDelay = 10s which is a syntax that we don't yet support (dotted setting names). This syntax we do support though (in FM2): templateResolver=DefaultTemplateResolver( \ templateLoader = com.example.FooTemplateLoader, \ templateUpdateDelayMilliseconds = 10_000 \ ) but it has the problem that you can't, for example, set the templateUpdateDelay without also specifying the whole templateResolver with all of its settings. So that's where the dotted notation is better. OTOH for some other things the above syntax is better, like for specifying the templateConfigurations (see http://freemarker.org/docs/pgui_config_templateconfigurations.html). So I think we should have both, and since we must ensure that templateResolver=com.example.MyTemplateResolver is "executed" earlier than templateResolver.mySetting=foo anyway, if mySetting is specified on both places (i.e., you have templateResolver=com.example.MyTemplateResolver(mySetting="foo") there), then obviously templateResolver.mySetting wins. And if we support the dotted notation in j.u.Properties, that should work for the objectWrapper setting too, naturally. Tuesday, February 7, 2017, 11:59:08 AM, Daniel Dekany wrote: > Tuesday, February 7, 2017, 7:40:18 AM, David E Jones wrote: > >> On Tue, 2017-01-24 at 23:47 +0100, Daniel Dekany wrote: >>> >>> Should the TemplateResolver-s (the work name for the new interface) >>> still get the template name already resolved to absolute and >>> normalized? So when cfg.getTemplate is called, then it invokes >>> cfg.templateNameFormat.normalizeAbsoluteName(String) to normalize the >>> path, and only then it calls the TemplateResolver. Also #import and >>> such currently calls templateNameFormat.toAbsoluteName(String, >>> String), and that would happen earlier than TemplateResolver is >>> involved. >>> >>> And how would it look... perhaps like this: >>> >>> public interface TemplateResolver { >>> >>> Template getTemplate(String templatePath, Local lookupLocale, Object >>> customLookupCondition) >>> throws IOException; >>> >>> } >>> >>> You may notice that it misses the `encoding` and `parsed` parameter of >>> cfg.getTemplate. That's because I belive that in FM3 we should not >>> allow specifying those as getTemplate parameters anymore, but that >>> will be another thread when we get there. >> >> IMO it should pass through the exact text from the include/etc in >> the template. It might be worth constraining to a valid URI syntax >> but other than that a TemplateResolver would be much more flexible if no >> normalization/etc is done. > > In FM2 there's a class like this: > > public abstract class TemplateNameFormat { > ... > abstract String toAbsoluteName(String baseName, String > targetName) throws MalformedTemplateNameException; > abstract String normalizeAbsoluteName(String name) throws > MalformedTemplateNameException; > } > > My idea was that if we allow users to provide a custom implementation, > then if the standard rules don't fit your custom TemplateResolver, you > just create a custom TemplateNameFormat too. But now that I think > about it more, pushing the TemplateNameFormat functionality on the > TemplateResolver implementation does indeed look like a better > approach, because then it can't happen that a configuration > accidentally contains a TemplateResolver with a TemplateNameFormat > that doesn't work well with it. But then the above two methods should > be part of the TemplateResolver interface (because, for
Re: Template Loader Implementation that loads from a database (DataSource)
I have only had the chance to look it on my phone, so have not read the thread thoroughly. I will go through it on my desktop later today. --Roberto Sent from Yahoo Mail on Android On Tue, Feb 7, 2017 at 8:14 AM, Daniel Dekanywrote: In case you have missed it in the other thread, note that it seems that there won't be a new alternative TemplateLoader interface in FM2, so this will have to be solved with the existing TemplateLoader there. -- Thanks, Daniel Dekany Wednesday, January 25, 2017, 4:17:10 AM, Roberto Benitez wrote: >>Also, I hope that freemarker-datasource-loader can be simplified over >>time, so it doesn't employ this many interfaces/classes. > Currently, the only essential classes are: 1)JdbcMetaData --A > central place to store DB table/column names. 2) TemplateSourceDao > --to perform the JDBC/CRUD work 3) DataSourceTemplateLoader -- the > loader implementation. 4) (Maybe)?) JdbcTemplateSource -- template > data record including the template content itself. This should be > helpful to a client such that no additional work should be necessary > (in most general cases) to maintain the templates > (name/content)--the DAO and JdbcTemplateSource will do the work. > > I am experimenting with the other classes/interfaces. In > particular, TemplateName and TemplateSource (which extends > TemplateName). The idea is to describe template metadata that is > more abstract that is more abstract than File, such that it could > describe a File or some database record--with the relevant data such > as name, locale. BUT, as you noted, necessitates pre-fetching data. > > > > >>My main problem with FreeMarker's TemplateLoader interface is that it >>assumes that you execute existence check, last modification query, and >>oading the actual content as 3 separate steps. While that fits how >>you deal with plain files (and I guess that's what it was modelled >>after), it doesn't fit how you do things with a database. In a >>straightforward implementation we would have 3 SQL round trips per new >>template loaded, and 2 for each up-to-date checks later. In a smarter >>implementation, and as you did it too, you can reduce the number of >>SQL operations be prefetching data into the templateSource object >>during the existence check (i.e., in findTemplateSource). As >>closeTemplateSource will be called pretty soon anyway, working from a >>such "cache" is certainly fine (because closeTemplateSource will >>invalidate it). But I don't think that you should prefetch the >>template content there, because then that will be an unnecessarily >>prefetched for template cache up-to-date checks, which is the dominant >>TemplateLoader usage in many applications. So it's sill 2 SQL-s per >>new template loaded, and 1 per template cache up-to-date checks. >>Certainly not a problem in most apps, but it's still bugs me. In >>FreeMarker 3 I definitely want to fix this be radically changing the >>TemplateLoader interface, and it's also possible that I will backport >>that solution into FreeMarker 2 (if people here will agree), though >>there it will co-exists with the traditional TemplateLoader, probably >>under the name TemplateLoader2 or something, so it's kind of messy. > > Would the changes to the TemplateLoader be that drastic? Sometimes > it is feasible to deprecate old/non-conformant methods (perhaps > provide defaults that rely on the new methods), which can later be removed > completely. > > --Roberto > > > >
Re: Template Loader Implementation that loads from a database (DataSource)
In case you have missed it in the other thread, note that it seems that there won't be a new alternative TemplateLoader interface in FM2, so this will have to be solved with the existing TemplateLoader there. -- Thanks, Daniel Dekany Wednesday, January 25, 2017, 4:17:10 AM, Roberto Benitez wrote: >>Also, I hope that freemarker-datasource-loader can be simplified over >>time, so it doesn't employ this many interfaces/classes. > Currently, the only essential classes are: 1)JdbcMetaData --A > central place to store DB table/column names. 2) TemplateSourceDao > --to perform the JDBC/CRUD work 3) DataSourceTemplateLoader -- the > loader implementation. 4) (Maybe)?) JdbcTemplateSource -- template > data record including the template content itself. This should be > helpful to a client such that no additional work should be necessary > (in most general cases) to maintain the templates > (name/content)--the DAO and JdbcTemplateSource will do the work. > > I am experimenting with the other classes/interfaces. In > particular, TemplateName and TemplateSource (which extends > TemplateName). The idea is to describe template metadata that is > more abstract that is more abstract than File, such that it could > describe a File or some database record--with the relevant data such > as name, locale. BUT, as you noted, necessitates pre-fetching data. > > > > >>My main problem with FreeMarker's TemplateLoader interface is that it >>assumes that you execute existence check, last modification query, and >>oading the actual content as 3 separate steps. While that fits how >>you deal with plain files (and I guess that's what it was modelled >>after), it doesn't fit how you do things with a database. In a >>straightforward implementation we would have 3 SQL round trips per new >>template loaded, and 2 for each up-to-date checks later. In a smarter >>implementation, and as you did it too, you can reduce the number of >>SQL operations be prefetching data into the templateSource object >>during the existence check (i.e., in findTemplateSource). As >>closeTemplateSource will be called pretty soon anyway, working from a >>such "cache" is certainly fine (because closeTemplateSource will >>invalidate it). But I don't think that you should prefetch the >>template content there, because then that will be an unnecessarily >>prefetched for template cache up-to-date checks, which is the dominant >>TemplateLoader usage in many applications. So it's sill 2 SQL-s per >>new template loaded, and 1 per template cache up-to-date checks. >>Certainly not a problem in most apps, but it's still bugs me. In >>FreeMarker 3 I definitely want to fix this be radically changing the >>TemplateLoader interface, and it's also possible that I will backport >>that solution into FreeMarker 2 (if people here will agree), though >>there it will co-exists with the traditional TemplateLoader, probably >>under the name TemplateLoader2 or something, so it's kind of messy. > > Would the changes to the TemplateLoader be that drastic? Sometimes > it is feasible to deprecate old/non-conformant methods (perhaps > provide defaults that rely on the new methods), which can later be removed > completely. > > --Roberto > > > >
Re: Pluggable template retrival/caching logic (Was: Template Loader Implementation that loads from a database (DataSource))
Tuesday, February 7, 2017, 7:40:18 AM, David E Jones wrote: > On Tue, 2017-01-24 at 23:47 +0100, Daniel Dekany wrote: >> >> Should the TemplateResolver-s (the work name for the new interface) >> still get the template name already resolved to absolute and >> normalized? So when cfg.getTemplate is called, then it invokes >> cfg.templateNameFormat.normalizeAbsoluteName(String) to normalize the >> path, and only then it calls the TemplateResolver. Also #import and >> such currently calls templateNameFormat.toAbsoluteName(String, >> String), and that would happen earlier than TemplateResolver is >> involved. >> >> And how would it look... perhaps like this: >> >> public interface TemplateResolver { >> >> Template getTemplate(String templatePath, Local lookupLocale, Object >> customLookupCondition) >> throws IOException; >> >> } >> >> You may notice that it misses the `encoding` and `parsed` parameter of >> cfg.getTemplate. That's because I belive that in FM3 we should not >> allow specifying those as getTemplate parameters anymore, but that >> will be another thread when we get there. > > IMO it should pass through the exact text from the include/etc in > the template. It might be worth constraining to a valid URI syntax > but other than that a TemplateResolver would be much more flexible if no > normalization/etc is done. In FM2 there's a class like this: public abstract class TemplateNameFormat { ... abstract String toAbsoluteName(String baseName, String targetName) throws MalformedTemplateNameException; abstract String normalizeAbsoluteName(String name) throws MalformedTemplateNameException; } My idea was that if we allow users to provide a custom implementation, then if the standard rules don't fit your custom TemplateResolver, you just create a custom TemplateNameFormat too. But now that I think about it more, pushing the TemplateNameFormat functionality on the TemplateResolver implementation does indeed look like a better approach, because then it can't happen that a configuration accidentally contains a TemplateResolver with a TemplateNameFormat that doesn't work well with it. But then the above two methods should be part of the TemplateResolver interface (because, for example, when you #import something, FM has to find out if the same FTL was already imported in the current Environment, so it needs to normalize the #import argument without loading anything). WDYT? > Passing in the Locale rather than trying to call it multiple times > with different locale extensions is a good idea too as different > underlying content or file stores may have their own approach for > this. That's even a problem with the TemplateLoader interface proposed to FM3. Like in the recent database template loader thread, the author wanted to use a separate column to indicate the locale of the template (and the same could happen with custom lookup conditions too), so you don't want to encode that information into the template name in the case of that particular TemplateLoader. It's something we may try to address in a second iteration of the FM3 TemplateLoader interface. > For example a JCR back end has quite a bit for meta data and > content variations. > > In general more parameters and less built into the path/name is a good thing > IMO. BTW, I still want to see if in FM3 the standard TemplateResolver (the one that's basically FM2's TemplateCache, relies on TemplateLoader and TemplateLookupStrategy and TemplateNameFormat) could cover more use cases without becoming overly intricate. Also, even if you decide to use a custom TemplateResolver, perhaps you still want to call some of the stock TemplateLoader-s from it (while you replace the caching logic, etc., or load from non-TemplateLoader sources too in case of more sophisticated storages). I suspect you have experience with these things, so it would be good if you can look at the FM3 TemplateLoader thread. > -David -- Thanks, Daniel Dekany
Re: Template Loader Implementation that loads from a database (DataSource)
Wednesday, January 25, 2017, 4:17:10 AM, Roberto Benitez wrote: >>Also, I hope that freemarker-datasource-loader can be simplified over >>time, so it doesn't employ this many interfaces/classes. > > Currently, the only essential classes are: > 1) JdbcMetaData --A central place to store DB table/column names. > 2) TemplateSourceDao --to perform the JDBC/CRUD work I think we shouldn't have a TemplateSourceDao inside FreeMarker. The only concern of FreeMarker is reading the templates, so only the R in CRUD. You might ask, but why not solve other operations as well, it can be convenient for some users. It's a trap FreeMarker has fell a few times in the past, and my conclusion is that it's in general not wise to address things that are outside our scope. It's a maintenance problem for one (all those little "utilities" we are dragging with ourselves, being responsible for them). Also, often we aren't even in the position to give the best solution, as the best solution depends on the application. Like here, it's likely that the user uses JPA, and has a hundred other tables that he already manipulates through that, has his company standards for it, like even some JPA hooks. I think everyone should just use his own project conventions for this, not a DAO provided by a template engine. Also we probably don't want to add yet another thing to our public API (and to the jar...), which the developers will look at, and contemplate, if they should use that, or their usual persistence solution. (We don't do anything special in there really, like we don't invalidate the related template cache entry when a row is deleted.) Components that stretch their scope too much are better be in separate jars than inside FreeMarker's core jar. > 3) DataSourceTemplateLoader -- the loader implementation. BTW, DatabaseTemplateLoader or JDBCTemplateLoader perhaps would be a more telling name. > 4) (Maybe)?) JdbcTemplateSource -- template data record including > the template content itself. This should be helpful to a client > such that no additional work should be necessary (in most general > cases) to maintain the templates (name/content)--the DAO and > JdbcTemplateSource will do the work. > > I am experimenting with the other classes/interfaces. In > particular, TemplateName and TemplateSource (which extends > TemplateName). The idea is to describe template metadata that is > more abstract that is more abstract than File, such that it could > describe a File or some database record--with the relevant data such > as name, locale. BUT, as you noted, necessitates pre-fetching data. > > > > >>My main problem with FreeMarker's TemplateLoader interface is that it >>assumes that you execute existence check, last modification query, and >>oading the actual content as 3 separate steps. While that fits how >>you deal with plain files (and I guess that's what it was modelled >>after), it doesn't fit how you do things with a database. In a >>straightforward implementation we would have 3 SQL round trips per new >>template loaded, and 2 for each up-to-date checks later. In a smarter >>implementation, and as you did it too, you can reduce the number of >>SQL operations be prefetching data into the templateSource object >>during the existence check (i.e., in findTemplateSource). As >>closeTemplateSource will be called pretty soon anyway, working from a >>such "cache" is certainly fine (because closeTemplateSource will >>invalidate it). But I don't think that you should prefetch the >>template content there, because then that will be an unnecessarily >>prefetched for template cache up-to-date checks, which is the dominant >>TemplateLoader usage in many applications. So it's sill 2 SQL-s per >>new template loaded, and 1 per template cache up-to-date checks. >>Certainly not a problem in most apps, but it's still bugs me. In >>FreeMarker 3 I definitely want to fix this be radically changing the >>TemplateLoader interface, and it's also possible that I will backport >>that solution into FreeMarker 2 (if people here will agree), though >>there it will co-exists with the traditional TemplateLoader, probably >>under the name TemplateLoader2 or something, so it's kind of messy. > > > Would the changes to the TemplateLoader be that drastic? It's a completely new interface. > Sometimes it is feasible to deprecate old/non-conformant methods > (perhaps provide defaults that rely on the new methods), which can > later be removed completely. > --Roberto -- Thanks, Daniel Dekany
Re: Template Loader Implementation that loads from a database (DataSource)
>Also, I hope that freemarker-datasource-loader can be simplified over >time, so it doesn't employ this many interfaces/classes. Currently, the only essential classes are: 1)JdbcMetaData --A central place to store DB table/column names. 2) TemplateSourceDao --to perform the JDBC/CRUD work 3) DataSourceTemplateLoader -- the loader implementation. 4) (Maybe)?) JdbcTemplateSource -- template data record including the template content itself. This should be helpful to a client such that no additional work should be necessary (in most general cases) to maintain the templates (name/content)--the DAO and JdbcTemplateSource will do the work. I am experimenting with the other classes/interfaces. In particular, TemplateName and TemplateSource (which extends TemplateName). The idea is to describe template metadata that is more abstract that is more abstract than File, such that it could describe a File or some database record--with the relevant data such as name, locale. BUT, as you noted, necessitates pre-fetching data. >My main problem with FreeMarker's TemplateLoader interface is that it >assumes that you execute existence check, last modification query, and >oading the actual content as 3 separate steps. While that fits how >you deal with plain files (and I guess that's what it was modelled >after), it doesn't fit how you do things with a database. In a >straightforward implementation we would have 3 SQL round trips per new >template loaded, and 2 for each up-to-date checks later. In a smarter >implementation, and as you did it too, you can reduce the number of >SQL operations be prefetching data into the templateSource object >during the existence check (i.e., in findTemplateSource). As >closeTemplateSource will be called pretty soon anyway, working from a >such "cache" is certainly fine (because closeTemplateSource will >invalidate it). But I don't think that you should prefetch the >template content there, because then that will be an unnecessarily >prefetched for template cache up-to-date checks, which is the dominant >TemplateLoader usage in many applications. So it's sill 2 SQL-s per >new template loaded, and 1 per template cache up-to-date checks. >Certainly not a problem in most apps, but it's still bugs me. In >FreeMarker 3 I definitely want to fix this be radically changing the >TemplateLoader interface, and it's also possible that I will backport >that solution into FreeMarker 2 (if people here will agree), though >there it will co-exists with the traditional TemplateLoader, probably >under the name TemplateLoader2 or something, so it's kind of messy. Would the changes to the TemplateLoader be that drastic? Sometimes it is feasible to deprecate old/non-conformant methods (perhaps provide defaults that rely on the new methods), which can later be removed completely. --Roberto
Re: Pluggable template retrival/caching logic (Was: Template Loader Implementation that loads from a database (DataSource))
Tuesday, January 24, 2017, 9:27:37 PM, David E Jones wrote: > On Jan 24 2017, at 10:25 am, Daniel Dekanywrote: > >> Tuesday, January 24, 2017, 6:48:16 PM, David E Jones wrote: > >> > >> > but all other Configuration.getTemplate() methods call this method > >> > >> > so it is an easy way to override all Template loading (assuming you >> are doing your own caching and so on, which is part of what I wanted >> in this case, ie use a managed cache external to FreeMarker that >> other things also use instead of the cache internal to FreeMarker). > >> > >> That's another thing I definitely want to solve in FM3 (I'm not sure > if I want to go into that in FM2 though), but by providing a setting > like cfg.templateResolver (TemplateResolver interface), which has some > minimalistic "give me a template like this somehow" method, and > default implementation that delegates to TemplateCache. > > > > Being able to implement an interface and register it with the Configuration > would be great, much better than hacking around in the implementation of > FreeMarker while still being able to override the Template loading behavior > completely (including caching, location resolution, etc). So yes, this sounds > great. Should the TemplateResolver-s (the work name for the new interface) still get the template name already resolved to absolute and normalized? So when cfg.getTemplate is called, then it invokes cfg.templateNameFormat.normalizeAbsoluteName(String) to normalize the path, and only then it calls the TemplateResolver. Also #import and such currently calls templateNameFormat.toAbsoluteName(String, String), and that would happen earlier than TemplateResolver is involved. And how would it look... perhaps like this: public interface TemplateResolver { Template getTemplate(String templatePath, Local lookupLocale, Object customLookupCondition) throws IOException; } You may notice that it misses the `encoding` and `parsed` parameter of cfg.getTemplate. That's because I belive that in FM3 we should not allow specifying those as getTemplate parameters anymore, but that will be another thread when we get there. > -David > > > > > -- Thanks, Daniel Dekany
Re: Template Loader Implementation that loads from a database (DataSource)
On Jan 24 2017, at 10:25 am, Daniel Dekanywrote: > Tuesday, January 24, 2017, 6:48:16 PM, David E Jones wrote: > > > but all other Configuration.getTemplate() methods call this method > > > so it is an easy way to override all Template loading (assuming you > are doing your own caching and so on, which is part of what I wanted > in this case, ie use a managed cache external to FreeMarker that > other things also use instead of the cache internal to FreeMarker). > > That's another thing I definitely want to solve in FM3 (I'm not sure if I want to go into that in FM2 though), but by providing a setting like cfg.templateResolver (TemplateResolver interface), which has some minimalistic "give me a template like this somehow" method, and default implementation that delegates to TemplateCache. Being able to implement an interface and register it with the Configuration would be great, much better than hacking around in the implementation of FreeMarker while still being able to override the Template loading behavior completely (including caching, location resolution, etc). So yes, this sounds great. -David
Re: Template Loader Implementation that loads from a database (DataSource)
Tuesday, January 24, 2017, 6:48:16 PM, David E Jones wrote: > A simpler TemplateLoader would be helpful. > > > > One approach I've used to simplify it is to override all loading by creating > an object that extends the freemarker.template.Configuration and override this > method: > > > > Template getTemplate(final String name, Locale locale, Object > customLookupCondition, String encoding, > boolean parseAsFTL, boolean ignoreMissing) > > > > The method to override has changed a little over time and I don't think it is > meant to be used this way, It's not designed for that for sure. (When it comes to widely used libraries like FreeMarker, I'm in camp who wants every class to only have methods that are either final or abstract, and maybe protected. That is, something is either designed to be overridden, or can't be overridden.) > but all other Configuration.getTemplate() methods call this method > so it is an easy way to override all Template loading (assuming you > are doing your own caching and so on, which is part of what I wanted > in this case, ie use a managed cache external to FreeMarker that > other things also use instead of the cache internal to FreeMarker). That's another thing I definitely want to solve in FM3 (I'm not sure if I want to go into that in FM2 though), but by providing a setting like cfg.templateResolver (TemplateResolver interface), which has some minimalistic "give me a template like this somehow" method, and default implementation that delegates to TemplateCache. > Hopefully this is helpful feedback, ie the need for and one approach I've > found to a simple override for all Template loading. > > > > -David > > > > > On Jan 24 2017, at 4:28 am, Daniel Dekanywrote: > >> Tuesday, January 24, 2017, 6:27:38 AM, Roberto Benitez wrote: > >> > >> > I am working on a TemplateLoader implementation that loads from a >> database. From a previous e-mail with Daniel Dekany, he mentioned >> that this is something you still need, and may be interested in integrating. >> >> I have the source code on github: >> rbenitez22/freemarker-datasource-loader >> >> >> | | >> rbenitez22/freemarker-datasource-loader >> | | > >> > >> Your link above didn't go through in one piece somehow, so again: > https://github.com/rbenitez22/freemarker-datasource-loader > (To clone: > https://github.com/rbenitez22/freemarker-datasource-loader.git) > >> > >> > This is something I am still tweaking as I learn more about the >> inner workings of the frame work. Let me know if you have further >> thoughts on what could be modified to make it integrate better. > >> > >> The TemplateLoader interface have some tricky aspects, so reading the > JavaDoc carefully is important. (It took a while for me to figure all > details out too, but then I have JavaDoc-ed them.) Especially look > into how a templateSource works (the return value of findTemplate). In > your case I think it should just be the things that you are using in > the SQL `where`: name and locale. And note that it must implement > `equals` (and `hash`) properly. > >> > >> Also, I hope that freemarker-datasource-loader can be simplified over > time, so it doesn't employ this many interfaces/classes. > >> > >> My main problem with FreeMarker's TemplateLoader interface is that it > assumes that you execute existence check, last modification query, and > loading the actual content as 3 separate steps. While that fits how > you deal with plain files (and I guess that's what it was modelled > after), it doesn't fit how you do things with a database. In a > straightforward implementation we would have 3 SQL round trips per new > template loaded, and 2 for each up-to-date checks later. In a smarter > implementation, and as you did it too, you can reduce the number of > SQL operations be prefetching data into the templateSource object > during the existence check (i.e., in findTemplateSource). As > closeTemplateSource will be called pretty soon anyway, working from a > such "cache" is certainly fine (because closeTemplateSource will > invalidate it). But I don't think that you should prefetch the > template content there, because then that will be an unnecessarily > prefetched for template cache up-to-date checks, which is the dominant > TemplateLoader usage in many applications. So it's sill 2 SQL-s per > new template loaded, and 1 per template cache up-to-date checks. > Certainly not a problem in most apps, but it's still bugs me. In > FreeMarker 3 I definitely want to fix this be radically changing the > TemplateLoader interface, and it's also possible that I will backport > that solution into FreeMarker 2 (if people here will agree), though > there it will co-exists with the traditional TemplateLoader, probably > under the name TemplateLoader2 or something, so it's kind of messy. > >> > >> > \--Roberto > >> > >> \-- > Thanks, > Daniel Dekany > --
Re: Template Loader Implementation that loads from a database (DataSource)
A simpler TemplateLoader would be helpful. One approach I've used to simplify it is to override all loading by creating an object that extends the freemarker.template.Configuration and override this method: Template getTemplate(final String name, Locale locale, Object customLookupCondition, String encoding, boolean parseAsFTL, boolean ignoreMissing) The method to override has changed a little over time and I don't think it is meant to be used this way, but all other Configuration.getTemplate() methods call this method so it is an easy way to override all Template loading (assuming you are doing your own caching and so on, which is part of what I wanted in this case, ie use a managed cache external to FreeMarker that other things also use instead of the cache internal to FreeMarker). Hopefully this is helpful feedback, ie the need for and one approach I've found to a simple override for all Template loading. -David On Jan 24 2017, at 4:28 am, Daniel Dekanywrote: > Tuesday, January 24, 2017, 6:27:38 AM, Roberto Benitez wrote: > > > I am working on a TemplateLoader implementation that loads from a > database. From a previous e-mail with Daniel Dekany, he mentioned > that this is something you still need, and may be interested in integrating. > I have the source code on github: > rbenitez22/freemarker-datasource-loader > > > | | > rbenitez22/freemarker-datasource-loader > | | > > Your link above didn't go through in one piece somehow, so again: https://github.com/rbenitez22/freemarker-datasource-loader (To clone: https://github.com/rbenitez22/freemarker-datasource-loader.git) > > > This is something I am still tweaking as I learn more about the > inner workings of the frame work. Let me know if you have further > thoughts on what could be modified to make it integrate better. > > The TemplateLoader interface have some tricky aspects, so reading the JavaDoc carefully is important. (It took a while for me to figure all details out too, but then I have JavaDoc-ed them.) Especially look into how a templateSource works (the return value of findTemplate). In your case I think it should just be the things that you are using in the SQL `where`: name and locale. And note that it must implement `equals` (and `hash`) properly. > > Also, I hope that freemarker-datasource-loader can be simplified over time, so it doesn't employ this many interfaces/classes. > > My main problem with FreeMarker's TemplateLoader interface is that it assumes that you execute existence check, last modification query, and loading the actual content as 3 separate steps. While that fits how you deal with plain files (and I guess that's what it was modelled after), it doesn't fit how you do things with a database. In a straightforward implementation we would have 3 SQL round trips per new template loaded, and 2 for each up-to-date checks later. In a smarter implementation, and as you did it too, you can reduce the number of SQL operations be prefetching data into the templateSource object during the existence check (i.e., in findTemplateSource). As closeTemplateSource will be called pretty soon anyway, working from a such "cache" is certainly fine (because closeTemplateSource will invalidate it). But I don't think that you should prefetch the template content there, because then that will be an unnecessarily prefetched for template cache up-to-date checks, which is the dominant TemplateLoader usage in many applications. So it's sill 2 SQL-s per new template loaded, and 1 per template cache up-to-date checks. Certainly not a problem in most apps, but it's still bugs me. In FreeMarker 3 I definitely want to fix this be radically changing the TemplateLoader interface, and it's also possible that I will backport that solution into FreeMarker 2 (if people here will agree), though there it will co-exists with the traditional TemplateLoader, probably under the name TemplateLoader2 or something, so it's kind of messy. > > > \--Roberto > > \-- Thanks, Daniel Dekany
Re: Template Loader Implementation that loads from a database (DataSource)
Tuesday, January 24, 2017, 6:27:38 AM, Roberto Benitez wrote: > I am working on a TemplateLoader implementation that loads from a > database. From a previous e-mail with Daniel Dekany, he mentioned > that this is something you still need, and may be interested in integrating. > I have the source code on github: > rbenitez22/freemarker-datasource-loader > > > || > rbenitez22/freemarker-datasource-loader > | | Your link above didn't go through in one piece somehow, so again: https://github.com/rbenitez22/freemarker-datasource-loader (To clone: https://github.com/rbenitez22/freemarker-datasource-loader.git) > This is something I am still tweaking as I learn more about the > inner workings of the frame work. Let me know if you have further > thoughts on what could be modified to make it integrate better. The TemplateLoader interface have some tricky aspects, so reading the JavaDoc carefully is important. (It took a while for me to figure all details out too, but then I have JavaDoc-ed them.) Especially look into how a templateSource works (the return value of findTemplate). In your case I think it should just be the things that you are using in the SQL `where`: name and locale. And note that it must implement `equals` (and `hash`) properly. Also, I hope that freemarker-datasource-loader can be simplified over time, so it doesn't employ this many interfaces/classes. My main problem with FreeMarker's TemplateLoader interface is that it assumes that you execute existence check, last modification query, and loading the actual content as 3 separate steps. While that fits how you deal with plain files (and I guess that's what it was modelled after), it doesn't fit how you do things with a database. In a straightforward implementation we would have 3 SQL round trips per new template loaded, and 2 for each up-to-date checks later. In a smarter implementation, and as you did it too, you can reduce the number of SQL operations be prefetching data into the templateSource object during the existence check (i.e., in findTemplateSource). As closeTemplateSource will be called pretty soon anyway, working from a such "cache" is certainly fine (because closeTemplateSource will invalidate it). But I don't think that you should prefetch the template content there, because then that will be an unnecessarily prefetched for template cache up-to-date checks, which is the dominant TemplateLoader usage in many applications. So it's sill 2 SQL-s per new template loaded, and 1 per template cache up-to-date checks. Certainly not a problem in most apps, but it's still bugs me. In FreeMarker 3 I definitely want to fix this be radically changing the TemplateLoader interface, and it's also possible that I will backport that solution into FreeMarker 2 (if people here will agree), though there it will co-exists with the traditional TemplateLoader, probably under the name TemplateLoader2 or something, so it's kind of messy. > --Roberto -- Thanks, Daniel Dekany
Template Loader Implementation that loads from a database (DataSource)
I am working on a TemplateLoader implementation that loads from a database. From a previous e-mail with Daniel Dekany, he mentioned that this is something you still need, and may be interested in integrating. I have the source code on github: rbenitez22/freemarker-datasource-loader || rbenitez22/freemarker-datasource-loader | | This is something I am still tweaking as I learn more about the inner workings of the frame work. Let me know if you have further thoughts on what could be modified to make it integrate better. --Roberto