Re: 3.3.1 to 4.0 migration

[Helge Weissig]

No worries! Thanks again for your help earlier.

I have submitted two patches at 
https://issues.apache.org/jira/browse/TORQUE-337 
 and 
https://issues.apache.org/jira/browse/TORQUE-338 
. The first adds the ability 
to easily override the logging configuration for the Generator and the second 
fixes the implementation of the undocumented (hint! hint! ;) ) configuration 
setting runOnlyOnSourceChange, which already provides for a way to skip the 
re-generation of sources when the schema files have not changed.

Additionally, I submitted https://issues.apache.org/jira/browse/TORQUE-339 
, which provides a patch for 
the generator’s configuration documentation.

hth,
h.

> On Jul 7, 2015, at 6:28 PM, Thomas Fox  wrote:
> 
> Hi Helge
> 
>> Helge Weissig wrote:
> 
>> Just a quick follow-up to report on progress on the template conversions. 
>> I managed to get the single mergepoint override working by changing the 
>> order of outlets and mergepoints. 
>> As for the bean classes, I was able to just use the baseBean.vm template and 
>> it’s outlet directly from the distribution.
>> 
>> Let me know if you are still interested in the two patches I mentioned below.
> 
> Sorry not to answer sooner. There is not much time nowadays for me and Torque.
> I'm glad you got it finally working. Are there still open questions?
> Yes, I'm still interested in the two patches. Please create jira issues.
> 
>Thanks,
> 
>  Thomas
> 
>> cheers,
>> h.
> 
> 
>> On Jul 2, 2015, at 5:04 PM, Helge Weissig  wrote:
>> 
>> Thanks Thomas! Some comments and follow-up questions below:
>> 
>>> On Jul 1, 2015, at 6:03 PM, Thomas Fox  wrote:
>>> 
>>> Helge Weissig wrote:
>>> 
 I forgot about the HTML filter this group uses… pardon the crappy 
 formatting. 
 Here is a text-only version:
 
 Our code base is heavily invested in some of the functionality removed 
 from torque 4.0 
 and I was wondering if any one had some advice on a migration path:
 
1. We make extensive use of village records obtained via 
   BasePeer.doSelect(Criteria), for example. 
   Those results could probably be considered a view of the data
   and I am wondering if that would be the correct/recommended/best 
 approach.
>>> 
>>> If you are using this to read data only, have a look at the RecordMapper 
>>> functionality. A record mapper maps DB Columns to an object. You can use 
>>> different record mappers for the same table (e.g. it can be passed in 
>>> through SomePeer.doSelect(Criteria, RecordMapper), so have different 
>>> object representations. For a generic village-record-like representation, 
>>> see org.apache.torque.om.mapper.ObjectListMapper (personally I do not like 
>>> this representation, put perhaps it serves your needs). Views (now 
>>> supported !) are also a more db-centric option. 
>> 
>> It looks like RecordMapper may ultimately be the best way to go although I 
>> still like the idea of using views because it would simplify the assembly of 
>> criteria in our code and allow for some generalization as well.
>> 
>> 
2. We have added our own caching implementation 
   (configurable via the schema definition) to the Object.vm 
   and Peer.vm templates. I have read through the new generator 
 documentation 
   and the customization part of the OR Mapping Reference 
   
 (https://db.apache.org/torque/torque-4.0/documentation/orm-reference/customizing-generation.html)
   but I don’t seem to be able to find the relevant information 
   to put it all together. Basically, I would like to override one or 
 more templates.
   I think I know how to specify that in an outlet, but do I also need 
 the control configuration
   in the conf directory? The maven plugin (we use maven) has 
 configuration parameters 
   for overrideConfigDir and overrideConfigPackage… do I need to set 
 those?
>>> 
>>> Have you checked the code-gen tutorial 
>>> (http://db.apache.org/torque/torque-4.0/documentation/tutorial/orm/index.html)
>>>  for a basic understanding ?
>>> Also, there is an example in the torque test project 
>>> (https://svn.apache.org/repos/asf/db/torque/torque4/trunk/torque-test/src/main/torque-gen)
>>>  which basically does what you want, i.e. overrides the Peer template 
>>> (adding @SuppressWarnings annotations). The torque generation config which 
>>> invokes this can be found in 
>>> https://svn.apache.org/repos/asf/db/torque/torque4/trunk/torque-test/pom.xml
>>>  lines 183ff.
>>> To be more specific:
>>> Yes, you need at least an empty control configuration.
>>> You need to set either overrideConfigDir or overrideConfigPackage, 
>>> depending whether the additional templates can be found in the file sytsem 
>>> or in the classpath.
>>

Re: 3.3.1 to 4.0 migration

[Thomas Fox]

Hi Helge

> Helge Weissig wrote:

> Just a quick follow-up to report on progress on the template conversions. 
> I managed to get the single mergepoint override working by changing the order 
> of outlets and mergepoints. 
> As for the bean classes, I was able to just use the baseBean.vm template and 
> it’s outlet directly from the distribution.
> 
> Let me know if you are still interested in the two patches I mentioned below.

Sorry not to answer sooner. There is not much time nowadays for me and Torque.
I'm glad you got it finally working. Are there still open questions?
Yes, I'm still interested in the two patches. Please create jira issues.

Thanks,

  Thomas

> cheers,
> h.


> On Jul 2, 2015, at 5:04 PM, Helge Weissig  wrote:
> 
> Thanks Thomas! Some comments and follow-up questions below:
> 
>> On Jul 1, 2015, at 6:03 PM, Thomas Fox  wrote:
>> 
>> Helge Weissig wrote:
>> 
>>> I forgot about the HTML filter this group uses… pardon the crappy 
>>> formatting. 
>>> Here is a text-only version:
>>> 
>>> Our code base is heavily invested in some of the functionality removed from 
>>> torque 4.0 
>>> and I was wondering if any one had some advice on a migration path:
>>> 
>>> 1. We make extensive use of village records obtained via 
>>>BasePeer.doSelect(Criteria), for example. 
>>>Those results could probably be considered a view of the data
>>>and I am wondering if that would be the correct/recommended/best 
>>> approach.
>> 
>> If you are using this to read data only, have a look at the RecordMapper 
>> functionality. A record mapper maps DB Columns to an object. You can use 
>> different record mappers for the same table (e.g. it can be passed in 
>> through SomePeer.doSelect(Criteria, RecordMapper), so have different 
>> object representations. For a generic village-record-like representation, 
>> see org.apache.torque.om.mapper.ObjectListMapper (personally I do not like 
>> this representation, put perhaps it serves your needs). Views (now supported 
>> !) are also a more db-centric option. 
> 
> It looks like RecordMapper may ultimately be the best way to go although I 
> still like the idea of using views because it would simplify the assembly of 
> criteria in our code and allow for some generalization as well.
> 
> 
>>> 2. We have added our own caching implementation 
>>>(configurable via the schema definition) to the Object.vm 
>>>and Peer.vm templates. I have read through the new generator 
>>> documentation 
>>>and the customization part of the OR Mapping Reference 
>>>
>>> (https://db.apache.org/torque/torque-4.0/documentation/orm-reference/customizing-generation.html)
>>>but I don’t seem to be able to find the relevant information 
>>>to put it all together. Basically, I would like to override one or 
>>> more templates.
>>>I think I know how to specify that in an outlet, but do I also need 
>>> the control configuration
>>>in the conf directory? The maven plugin (we use maven) has 
>>> configuration parameters 
>>>for overrideConfigDir and overrideConfigPackage… do I need to set 
>>> those?
>> 
>> Have you checked the code-gen tutorial 
>> (http://db.apache.org/torque/torque-4.0/documentation/tutorial/orm/index.html)
>>  for a basic understanding ?
>> Also, there is an example in the torque test project 
>> (https://svn.apache.org/repos/asf/db/torque/torque4/trunk/torque-test/src/main/torque-gen)
>>  which basically does what you want, i.e. overrides the Peer template 
>> (adding @SuppressWarnings annotations). The torque generation config which 
>> invokes this can be found in 
>> https://svn.apache.org/repos/asf/db/torque/torque4/trunk/torque-test/pom.xml 
>> lines 183ff.
>> To be more specific:
>> Yes, you need at least an empty control configuration.
>> You need to set either overrideConfigDir or overrideConfigPackage, depending 
>> whether the additional templates can be found in the file sytsem or in the 
>> classpath.
>> Please ask again if you cannot get it working. Although more complicated 
>> :-(, the new generator is much more extensible than the old one. In the 
>> current trunk, you can even use groovy templates :-)
> 
> OK, I am getting somewhere, my outlets and templates are now found, alas I am 
> having trouble achieving the desired results. For one, I am not able to get 
> the example at the bottom of 
> http://db.apache.org/torque/torque-4.0/documentation/codegen-reference/configuration.html
>  
> 
>  
>   
> >
>  to work. I get an error about a missing outlet tag. I think if I could 
> actually override just a specific mergepoint, that would get me 90% of the 
> way. However, we also modifi

Re: 3.3.1 to 4.0 migration

[Helge Weissig]

Just a quick follow-up to report on progress on the template conversions. I 
managed to get the single mergepoint override working by changing the order of 
outlets and mergepoints. As for the bean classes, I was able to just use the 
baseBean.vm template and it’s outlet directly from the distribution.

Let me know if you are still interested in the two patches I mentioned below.

cheers,
h.


> On Jul 2, 2015, at 5:04 PM, Helge Weissig  wrote:
> 
> Thanks Thomas! Some comments and follow-up questions below:
> 
>> On Jul 1, 2015, at 6:03 PM, Thomas Fox  wrote:
>> 
>> Helge Weissig wrote:
>> 
>>> I forgot about the HTML filter this group uses… pardon the crappy 
>>> formatting. 
>>> Here is a text-only version:
>>> 
>>> Our code base is heavily invested in some of the functionality removed from 
>>> torque 4.0 
>>> and I was wondering if any one had some advice on a migration path:
>>> 
>>> 1. We make extensive use of village records obtained via 
>>>BasePeer.doSelect(Criteria), for example. 
>>>Those results could probably be considered a view of the data
>>>and I am wondering if that would be the correct/recommended/best 
>>> approach.
>> 
>> If you are using this to read data only, have a look at the RecordMapper 
>> functionality. A record mapper maps DB Columns to an object. You can use 
>> different record mappers for the same table (e.g. it can be passed in 
>> through SomePeer.doSelect(Criteria, RecordMapper), so have different 
>> object representations. For a generic village-record-like representation, 
>> see org.apache.torque.om.mapper.ObjectListMapper (personally I do not like 
>> this representation, put perhaps it serves your needs). Views (now supported 
>> !) are also a more db-centric option. 
> 
> It looks like RecordMapper may ultimately be the best way to go although I 
> still like the idea of using views because it would simplify the assembly of 
> criteria in our code and allow for some generalization as well.
> 
> 
>>> 2. We have added our own caching implementation 
>>>(configurable via the schema definition) to the Object.vm 
>>>and Peer.vm templates. I have read through the new generator 
>>> documentation 
>>>and the customization part of the OR Mapping Reference 
>>>
>>> (https://db.apache.org/torque/torque-4.0/documentation/orm-reference/customizing-generation.html)
>>>but I don’t seem to be able to find the relevant information 
>>>to put it all together. Basically, I would like to override one or 
>>> more templates.
>>>I think I know how to specify that in an outlet, but do I also need 
>>> the control configuration
>>>in the conf directory? The maven plugin (we use maven) has 
>>> configuration parameters 
>>>for overrideConfigDir and overrideConfigPackage… do I need to set 
>>> those?
>> 
>> Have you checked the code-gen tutorial 
>> (http://db.apache.org/torque/torque-4.0/documentation/tutorial/orm/index.html)
>>  for a basic understanding ?
>> Also, there is an example in the torque test project 
>> (https://svn.apache.org/repos/asf/db/torque/torque4/trunk/torque-test/src/main/torque-gen)
>>  which basically does what you want, i.e. overrides the Peer template 
>> (adding @SuppressWarnings annotations). The torque generation config which 
>> invokes this can be found in 
>> https://svn.apache.org/repos/asf/db/torque/torque4/trunk/torque-test/pom.xml 
>> lines 183ff.
>> To be more specific:
>> Yes, you need at least an empty control configuration.
>> You need to set either overrideConfigDir or overrideConfigPackage, depending 
>> whether the additional templates can be found in the file sytsem or in the 
>> classpath.
>> Please ask again if you cannot get it working. Although more complicated 
>> :-(, the new generator is much more extensible than the old one. In the 
>> current trunk, you can even use groovy templates :-)
> 
> OK, I am getting somewhere, my outlets and templates are now found, alas I am 
> having trouble achieving the desired results. For one, I am not able to get 
> the example at the bottom of 
> http://db.apache.org/torque/torque-4.0/documentation/codegen-reference/configuration.html
>  
> 
>  
>   
> >
>  to work. I get an error about a missing outlet tag. I think if I could 
> actually override just a specific mergepoint, that would get me 90% of the 
> way. However, we also modified the Bean.vm template to allow us to continue 
> using fields like “modified” or “new” in our DB schema. Those attributes and 
> their corresponding getters and setters are now in the template 
> bean/base/baseBean.vm which is referenced from the bean.xml outlet 
> definition. If I simply bring over

Re: 3.3.1 to 4.0 migration

[Helge Weissig]

Thanks Thomas! Some comments and follow-up questions below:

> On Jul 1, 2015, at 6:03 PM, Thomas Fox  wrote:
> 
> Helge Weissig wrote:
> 
>> I forgot about the HTML filter this group uses… pardon the crappy 
>> formatting. 
>> Here is a text-only version:
>> 
>> Our code base is heavily invested in some of the functionality removed from 
>> torque 4.0 
>> and I was wondering if any one had some advice on a migration path:
>> 
>>  1. We make extensive use of village records obtained via 
>> BasePeer.doSelect(Criteria), for example. 
>> Those results could probably be considered a view of the data
>> and I am wondering if that would be the correct/recommended/best 
>> approach.
> 
> If you are using this to read data only, have a look at the RecordMapper 
> functionality. A record mapper maps DB Columns to an object. You can use 
> different record mappers for the same table (e.g. it can be passed in through 
> SomePeer.doSelect(Criteria, RecordMapper), so have different object 
> representations. For a generic village-record-like representation, see 
> org.apache.torque.om.mapper.ObjectListMapper (personally I do not like this 
> representation, put perhaps it serves your needs). Views (now supported !) 
> are also a more db-centric option. 

It looks like RecordMapper may ultimately be the best way to go although I 
still like the idea of using views because it would simplify the assembly of 
criteria in our code and allow for some generalization as well.


>>  2. We have added our own caching implementation 
>> (configurable via the schema definition) to the Object.vm 
>> and Peer.vm templates. I have read through the new generator 
>> documentation 
>> and the customization part of the OR Mapping Reference 
>> 
>> (https://db.apache.org/torque/torque-4.0/documentation/orm-reference/customizing-generation.html)
>> but I don’t seem to be able to find the relevant information 
>> to put it all together. Basically, I would like to override one or 
>> more templates.
>> I think I know how to specify that in an outlet, but do I also need 
>> the control configuration
>> in the conf directory? The maven plugin (we use maven) has 
>> configuration parameters 
>> for overrideConfigDir and overrideConfigPackage… do I need to set 
>> those?
> 
> Have you checked the code-gen tutorial 
> (http://db.apache.org/torque/torque-4.0/documentation/tutorial/orm/index.html)
>  for a basic understanding ?
> Also, there is an example in the torque test project 
> (https://svn.apache.org/repos/asf/db/torque/torque4/trunk/torque-test/src/main/torque-gen)
>  which basically does what you want, i.e. overrides the Peer template (adding 
> @SuppressWarnings annotations). The torque generation config which invokes 
> this can be found in 
> https://svn.apache.org/repos/asf/db/torque/torque4/trunk/torque-test/pom.xml 
> lines 183ff.
> To be more specific:
> Yes, you need at least an empty control configuration.
> You need to set either overrideConfigDir or overrideConfigPackage, depending 
> whether the additional templates can be found in the file sytsem or in the 
> classpath.
> Please ask again if you cannot get it working. Although more complicated :-(, 
> the new generator is much more extensible than the old one. In the current 
> trunk, you can even use groovy templates :-)

OK, I am getting somewhere, my outlets and templates are now found, alas I am 
having trouble achieving the desired results. For one, I am not able to get the 
example at the bottom of 
http://db.apache.org/torque/torque-4.0/documentation/codegen-reference/configuration.html
 

 to work. I get an error about a missing outlet tag. I think if I could 
actually override just a specific mergepoint, that would get me 90% of the way. 
However, we also modified the Bean.vm template to allow us to continue using 
fields like “modified” or “new” in our DB schema. Those attributes and their 
corresponding getters and setters are now in the template bean/base/baseBean.vm 
which is referenced from the bean.xml outlet definition. If I simply bring over 
that definition, the generator seems to expect to also find all other templates 
contained in bean.xml to be in my code base. If I remove all mergepoints and 
other outlets and simply keep the empty outlet definition for 
bean/base/baseBean.vm, my template is used with a lot of warning messages about 
missing mergepoint definitions and in the end, the generated beans contain only 
the above mentioned attributes and their getters and setters and nothing else. 
Putting it more generally, how can I override parts of a template that also 
contains a lot of mergepoints?

>>  3. This is more of a maven question, maybe, alas I have the feeling
>> it isn’t actually possible to implement through it, but is there a 
>> way
>> 

Re: 3.3.1 to 4.0 migration

[Thomas Fox]

Helge Weissig wrote:

> I forgot about the HTML filter this group uses… pardon the crappy formatting. 
> Here is a text-only version:
>
> Our code base is heavily invested in some of the functionality removed from 
> torque 4.0 
> and I was wondering if any one had some advice on a migration path:
>
>   1. We make extensive use of village records obtained via 
>  BasePeer.doSelect(Criteria), for example. 
>  Those results could probably be considered a view of the data
>  and I am wondering if that would be the correct/recommended/best 
> approach.

If you are using this to read data only, have a look at the RecordMapper 
functionality. A record mapper maps DB Columns to an object. You can use 
different record mappers for the same table (e.g. it can be passed in through 
SomePeer.doSelect(Criteria, RecordMapper), so have different object 
representations. For a generic village-record-like representation, see 
org.apache.torque.om.mapper.ObjectListMapper (personally I do not like this 
representation, put perhaps it serves your needs). Views (now supported !) are 
also a more db-centric option. 

>   2. We have added our own caching implementation 
>  (configurable via the schema definition) to the Object.vm 
>  and Peer.vm templates. I have read through the new generator 
> documentation 
>  and the customization part of the OR Mapping Reference 
>  
> (https://db.apache.org/torque/torque-4.0/documentation/orm-reference/customizing-generation.html)
>  but I don’t seem to be able to find the relevant information 
>  to put it all together. Basically, I would like to override one or 
> more templates.
>  I think I know how to specify that in an outlet, but do I also need 
> the control configuration
>  in the conf directory? The maven plugin (we use maven) has 
> configuration parameters 
>  for overrideConfigDir and overrideConfigPackage… do I need to set 
> those?

Have you checked the code-gen tutorial 
(http://db.apache.org/torque/torque-4.0/documentation/tutorial/orm/index.html) 
for a basic understanding ?
Also, there is an example in the torque test project 
(https://svn.apache.org/repos/asf/db/torque/torque4/trunk/torque-test/src/main/torque-gen)
 which basically does what you want, i.e. overrides the Peer template (adding 
@SuppressWarnings annotations). The torque generation config which invokes this 
can be found in 
https://svn.apache.org/repos/asf/db/torque/torque4/trunk/torque-test/pom.xml 
lines 183ff.
To be more specific:
Yes, you need at least an empty control configuration.
You need to set either overrideConfigDir or overrideConfigPackage, depending 
whether the additional templates can be found in the file sytsem or in the 
classpath.
Please ask again if you cannot get it working. Although more complicated :-(, 
the new generator is much more extensible than the old one. In the current 
trunk, you can even use groovy templates :-)

>   3. This is more of a maven question, maybe, alas I have the feeling
>  it isn’t actually possible to implement through it, but is there a 
> way
>  to skip the code generation steps if the schema sources have not 
> changed?

Sorry, this is not any more possible. The problem is to reliably find out 
whether the file has changed (e.g. some OS do not change the change date of a 
file when it is copied). I have seen more than one situation where schema 
changes did not get picked up because of this option, so my personal 
recommendation is not to rely on such a mechanism. Also, mvn generate is 
typically not executed often while developing, so long generation times should 
not slow down development too much.
However, if you still want to use it, you can patch the Mojo class in the 
Torque maven plugin, this should not be too difficult (please share if you 
choose to do it).

>   4. I know how to configure logging for the generator, but the switch 
> between loglevel
>  WARN and INFO is quite severe. Is there a way to log at INFO level 
> but only to a file, 
>  not to the console? We use log4j throughout our project.

Yes, check the log4j configuration documentation. Basically you need to 
override the log4j configuration file and use two appenders, one file appender 
and one console appender, with different log levels. The log4j configuration 
needs to be in the classpath /org/apache/torque/generator/log4j.properties and 
it must have a higher classpath priority than the torque-generator.jar (I am 
currently not sure how to achieve this using maven, the last resort is patching 
the generator)

Good luck!

   Thomas

-
To unsubscribe, e-mail: torque-user-unsubscr...@db.apache.org
For additional commands, e-mail: torque-user-h...@db.apache.org

Re: 3.3.1 to 4.0 migration

[Helge Weissig]

I forgot about the HTML filter this group uses… pardon the crappy formatting. 
Here is a text-only version:

Our code base is heavily invested in some of the functionality removed from 
torque 4.0 and I was wondering if any one had some advice on a migration path:

1. We make extensive use of village records obtained via 
BasePeer.doSelect(Criteria), for example. Those results could probably be 
considered a view of the data and I am wondering if that would be the 
correct/recommended/best approach.

2. We have added our own caching implementation (configurable via the 
schema definition) to the Object.vm and Peer.vm templates. I have read through 
the new generator documentation and the customization part of the OR Mapping 
Reference 
(https://db.apache.org/torque/torque-4.0/documentation/orm-reference/customizing-generation.html)
 but I don’t seem to be able to find the relevant information to put it all 
together. Basically, I would like to override one or more templates. I think I 
know how to specify that in an outlet, but do I also need the control 
configuration in the conf directory? The maven plugin (we use maven) has 
configuration parameters for overrideConfigDir and overrideConfigPackage… do I 
need to set those?

3. This is more of a maven question, maybe, alas I have the feeling it 
isn’t actually possible to implement through it, but is there a way to skip the 
code generation steps if the schema sources have not changed?

4. I know how to configure logging for the generator, but the switch 
between loglevel WARN and INFO is quite severe. Is there a way to log at INFO 
level but only to a file, not to the console? We use log4j throughout our 
project.

Thanks very much in advance for any advice and pointers!

cheers,
h.

> On Jul 1, 2015, at 9:51 AM, Helge Weissig  wrote:
> 
> Our code base is heavily invested in some of the functionality removed from 
> torque 4.0 and I was wondering if any one had some advice on a migration path:
> 
> We make extensive use of village records obtained via 
> BasePeer.doSelect(Criteria), for example. Those results could probably be 
> considered a view of the data and I am wondering if that would be the 
> correct/recommended/best approach.
> We have added our own caching implementation (configurable via the schema 
> definition) to the Object.vm and Peer.vm templates. I have read through the 
> new generator documentation and the customization part of the OR Mapping 
> Reference 
> (https://db.apache.org/torque/torque-4.0/documentation/orm-reference/customizing-generation.html
>  
> )
>  but I don’t seem to be able to find the relevant information to put it all 
> together. Basically, I would like to override one or more templates. I think 
> I know how to specify that in an outlet, but do I also need the control 
> configuration in the conf directory? The maven plugin (we use maven) has 
> configuration parameters for overrideConfigDir and overrideConfigPackage… do 
> I need to set those?
> This is more of a maven question, maybe, alas I have the feeling it isn’t 
> actually possible to implement through it, but is there a way to skip the 
> code generation steps if the schema sources have not changed?
> I know how to configure logging for the generator, but the switch between 
> loglevel WARN and INFO is quite severe. Is there a way to log at INFO level 
> but only to a file, not to the console? We use log4j throughout our project.
> 
> Thanks very much in advance for any advice and pointers!
> 
> cheers,
> h.
> 
> Helge Weissig, Ph.D.
> Director, Bioinformatics & IT
> ActivX Biosciences, Inc.
> www.activx.com 
> www.kinativ.com 


-
To unsubscribe, e-mail: torque-user-unsubscr...@db.apache.org
For additional commands, e-mail: torque-user-h...@db.apache.org