Am Montag, 3. September 2012 23:30:25 UTC+2 schrieb Lukas Eder: Some background info: I needed a more powerful code generator now, so my 2nd gen code generator reuses most of the old API and code. This isn't a goal, it's pure necessity.
But since the new code generator is more flexible (and needs less lines of code) than the old one, it will help you when you start with refactoring the old API because less things will break / it will be easier to fix the broken parts. > Just as a teaser, here is the configuration of the new code generator. > I'm > > not a big fan of XML-based configurations, I prefer DSLs but the API is > > public, so writing an XML front-end for the config is just an issue of > > someone wanting to scratch this itch. > > A DSL contribution is nice, too. If it works well, I can add the > XML-parts, which will be required to be able to run code generation > through Maven, for instance. That's just like what we have today, > where the XML configuration is loaded by GenerationTool or by the > Maven plugin, and passed on to the DefaultGenerator. > I'm now finished with the prototype. You can find the code here: https://bitbucket.org/digulla/jooq-codegen2 I used Mercurial because that was easier for me to set up. Let me know if you need a Git repo. Overview of the features: * Most types can be generated (table definition, interfaces, records and pojos) * It's easy to write tests for the code generator. All output is written via an OutputWriter. Test override that with a MockOutputWriter that collects all output in a map (path -> content) * 70% test coverage * Code generators are written in Xtend * Each output type gets their own generator to make it easy to extend the generated code * Configuration uses a DSL * There is a MockDatabase to help writing test cases (no need to actually create a database). It uses DSL to quickly define schemas, tables and columns. * A pluggable strategy to create a serialVersionUID. I also created a new strategy which removed comments and white space before calculating the UID. I also made sure the UID is never negative to make Cut&paste easier. I suggest to have a look at the existing test cases to get an idea how everything works together. There are quite a few TODOs left because I only implemented those features which I needed to create the code for my application. But near the end, I created a new generator in about 4 hours. A comment about XML based configs. I need a good way to define configs in Java so I can create configs for unit tests. When creating custom code generators, it's very important to be able to run the generator and check the output against a "known good" baseline. Otherwise, it's too hard to see what effects a change has. So I end up with this setup: 1. A project which extends jOOQ-codegen2 and which contains the custom generators 2. There is a "mini model" project that defines a minimal database that demonstrates all the features of the custom generators. It uses a MockDatabase. 3. The real model with the complete DDL. This one uses an XML-based config and a Maven plugin to generate the code. In the past, it wasn't really possible to extend the code generator, so there was no need to be able to define a config with 2-3 lines of code. Test cases for #2 look like this: @Test public void testSchema() throws Exception { check( "Blu.java" ); } @Test public void testFactory() throws Exception { check( "BluFactory.java" ); } @Test public void testKeys() throws Exception { check( "Keys.java" ); } @Test public void testTables() throws Exception { check( "Tables.java" ); } @Test public void testClientInfo() throws Exception { check( "tables/adm/ClientInfo.java" ); } @Test public void testGlobalPrivGrant() throws Exception { check( "tables/adm/GlobalPrivGrant.java" ); } ... hundreds of tests; one per generated file ... private void check( String file ) throws IOException { JavaGenerator2 generator = runCodeGen(); String path = generator.config().getPackageName().replace( ".", "/" ) + "/" + file; File expected = new File( "src/main/java", path ); File actual = new File( generator.config().getRootFolder(), path ); TestUtils.compareFiles( expected, actual, Encoding.UTF_8.charset() ); } This means I define one test case for each generated file. When I run JUnit, I get failures for each output file that did change. That way, I can quickly see whether my changes have the desired/expected impact on the code. And when I run the tests in an IDE, I can use the result comparison to see the expected and actual output side by side. This approach makes it very safe to add features to the code generator. After checking the output, I either copy the updated code into the source tree or run the tests with the System property "overwriteSource=true" to let the tests write into src/main/java/. > .renameBooleanColumn( "BLU.ADM_LANGUAGE.IS_DEFAULT", "default" ) > > That shouldn't be necessary. The boolean type information is available > on the ColumnDefinition. If you prefer isXXX() instead of getXXX(), > this should be placed elsewhere. In your case, it should be placed > into the BluNamingStrategy implementation, not the API > You're right. I'll add that to the list of TODOs. > .dialect( SQLDialect.H2 ) > > I'd prefer if the org.jooq.util.Database implementation is still > referenced rather than a SQLDialect, for two reasons [...] > Actually, this information is used just in a single place: log.info(" dialect: {}", config.getDialect()); Deleted. > > .schema( "PUBLIC", "BLU" ) > > Did you plan to keep multi-schema support? > I didn't plan to drop anything because I don't know what's safe to drop. > Also, I wonder whether object renames should still be possible in two > places, i.e. naming strategy AND the overall schema reference. The > current generator strategy is a strategy for giving names to Java > objects (it would affect what you call the GeneratorFactory) > Yes, this is very confusing, especially since it leads to getInputName() and getOutputName() in a lot of places and I'm always wondering which one to use... A new "database strategy" that would contain database object renames > (such as physical schema renames), and also the include() / exclude() > / includeAll() methods, and maybe the type mapping. > Sounds good to me. Why is this necessary at all? Shouldn't we use the original names and then allow to override those at runtime? What renames to we really need? In my limited view, we only need to allow to set a different schema to allow people to run jOOQ against one database server with two schemas that have the same tables but different names. > > .factory( new BluGeneratorFactory( generator.config() ) ) > > I'm guessing that this will replace what is called DefaultGenerator > today, reading the meta data and generating Java classes? > No, that's an instance factory. In the current version of the prototype, I have already replaced that with DI based on Guice (com.google.inject). This makes it easy to define a custom code generator for POJOs. It's not needed at runtime, only to generate the code. > Anyway, we should coordinate your goals. Your own personal goals seem > to diverge a little from the general jOOQ 3.0 roadmap. > > Here's the jOOQ 3.0 roadmap: > 1. There should be a clean API for jooq-meta. It should be completely > independent of the code generator. > 2. The 1) jooq-meta API can be implemented in dialect-independent > ways, reading meta-information from non-JDBC data sources, such as > XML. This can be used to read non-jOOQ XML formats, such as > Hibernate's .hbm.xml > 3. The jooq-meta API should have a "database strategy", which models > today's include/exclude features, database type mappings, database > object renames > I agree with the include/exclude and type mappings but isn't renaming part of the naming strategy? Do we really want to strategies? I'm asking because the Java naming strategy will contain a lot of code to determine what to rename. A lot of that would have to replicated in the database naming strategy when there are two instances since you can only extend a single type in Java. One thing that would be great is custom attributes. I would like to add something like "this table has columns for a time range" to the TableDefinition so I can quickly look that up in custom generators. > 4. The output produced by 3) "database strategy" is fed into a > "JavaGeneratorFactory" or into a "Generator". Today's DefaultGenerator > could be a sample implementation thereof > I like the idea of creating a database model and then run a (bunch of) postprocessors on that before its fed into the code generator. > 6. Today's "GeneratorStrategy", or what you call the "Naming Strategy" > could be called "JavaNamingStrategy", used to map database objects to > Java objects. It is specific to 4) "JavaGeneratorFactory". Today's > "Naming Strategy" is too low-level, more explicit features would be > better. > I already improved that a bit. There is now a method getType() which returns a JavaType instance. This encodes the type information (package + type name, generic parameters, isArray). For the code generator, it has methods like getPath() which returns the path this type would have. > For me, the most important features to add are 2) (new data sources), > 5) (new target formats) and maybe 6) (improved Java naming > strategies). For you, 6) seems to be the most important improvement, > as well as the fact that 4) can be re-implemented in your own way, > using Xtend. > That sounds about right. > If we can agree on the roadmap, I can try to devise an API for 1), 3), > 4) and 6), and post it on this user group within the next 1-2 weeks. > You could then again challenge this API with your requirements > Well, the prototype works for me, so you can now have fun with it. I'll improve it over the next few weeks as I add features which we need for our application. In the process, I'll try hard to meet your goals, so the final result can be included in jOOQ 3.0. Regards, A. Digulla
