I used log4jdbc and didn’t see anything strange. I also updated to the latest Oracle driver with no change. There doesn’t appear to be any debug activity on FacetMethodsBuilder either.
Although I think this may be a local resource constraint. I've found that after adding the schema to the metamodel definition in @PersistenceCapable, it only takes about 5 seconds to perform the column validation. The new query contains a 'where' clause with the schema name, so the number of records is very small. Using the schema name is acceptable for my use case. Thanks for your help Dan! Jeremy D. Branham Technology Architect - Sprint -----Original Message----- From: Dan Haywood [mailto:[email protected]] Sent: Tuesday, January 12, 2016 7:13 PM To: users Subject: Re: Very slow column validation Are you sure that this is DN that is causing the issue? You've said "in memory metamodel validation", which makes think you've observed something specifically? Isis itself does also do validation, and also builds its metamodel dynamically... but it tends to build most of it fairly eagerly. You could turn on log4j logging of FacetsMethodsBuilder, down at debug level, just to see if Isis is busy or not. On 13 January 2016 at 01:01, Branham, Jeremy [HR] < [email protected]> wrote: > The delay does not go away when validation is turned off. > It seems the in memory metamodel validation occurs when the object is > accessed, regardless of the property. > > I haven’t tried a 3rd party driver yet, but I will give that a shot, > and monitor if a separate query is occurring for each row. > > > > Jeremy D. Branham > Technology Architect - Sprint > O: +1 (972) 405-2970 | M: +1 (817) 791-1627 | **DotNet > [email protected] #gettingbettereveryday > > > > -----Original Message----- > From: Dan Haywood [mailto:[email protected]] > Sent: Tuesday, January 12, 2016 6:54 PM > To: users > Subject: Re: Very slow column validation > > On 13 January 2016 at 00:35, Branham, Jeremy [HR] < > [email protected]> wrote: > > > Thanks Dan - > > Yes, it is the JDO/DataNucleus column validation against the RDMS > > [Oracle in this case] > > > > I've disabled the validation in the properties file like this. > > > > isis.persistor.datanucleus.impl.datanucleus.schema.autoCreateAll=fal > > se > > isis.persistor.datanucleus.impl.datanucleus.schema.validateTables=fa > > ls > > e > > > > isis.persistor.datanucleus.impl.datanucleus.schema.validateConstrain > > ts > > =false > > > > > OK, so I'm clear... does the pause go away when you disable the validation? > > per > http://www.datanucleus.org/products/accessplatform_4_1/jdo/schema.html > > what do you observe with: > > datanucleus.schema.validateTables > vs > > datanucleus.schema.validateColumns > > vs > > datanucleus.schema.validateConstraints > > (each with the "isis.persistor.datanucleus.impl" prefix, of course, > when listed in our properties file). > > > > > > I'm using @PersistenceCapable on the domain objects and implementing > > the AppManifest interface [DomainAppDomManifest]. > > I used the Simple archetype to create a project, but I am using an > > existing DB. > > > > > ok, good. > > > > > > > > The following query is generated when the new object is accessed - > > > > SELECT NULL TABLE_CAT, OWNER TABLE_SCHEM, TABLE_NAME, COLUMN_NAME, > > NULL DATA_TYPE, DATA_TYPE TYPE_NAME, > > DECODE(DATA_TYPE,'NUMBER',DATA_PRECISION,DATA_LENGTH) COLUMN_SIZE, 0 > > BUFFER_LENGTH, DATA_SCALE DECIMAL_DIGITS, 10 NUM_PREC_RADIX, > > DECODE(NULLABLE,'Y',1,0) NULLABLE, NULL REMARKS, NULL COLUMN_DEF, 0 > > SQL_DATA_TYPE, 0 SQL_DATETIME_SUB, DATA_LENGTH CHAR_OCTET_LENGTH, > > COLUMN_ID ORDINAL_POSITION, DECODE(NULLABLE,'Y','YES','NO') > > IS_NULLABLE FROM ALL_TAB_COLUMNS ORDER BY TABLE_SCHEM, TABLE_NAME, > > ORDINAL_POSITION > > > > This returns 61446 records. > > > > I think this is the source of the delay, as each of these records > > are interated in memory. > > > > > that doesn't seem like a huge amount... I imagine this takes only a > second or two to return when you run it from a standalone query tool? > > The only way I could see this query eating up 9 minutes would be if, > as JDO/DN iterates over it, there ends up being making a network call > for each row... I doubt that happens, but perhaps you could monitor to > see. (I don't know if Oracle provides such a tool; worst comes to > worse you could use the log4jdbc driver that we have commented out in > persistor.properties. > > I did also notice that in the DN docs there is a remark about > "validateColumns" being buggy for some JDBC drivers. Are there any > third-party JDBC drivers that you could try out? > > -- Dan > > > > > > > > > Jeremy D. Branham > > Technology Architect - Sprint > > > > > > -----Original Message----- > > From: Dan Haywood [mailto:[email protected]] > > Sent: Tuesday, January 12, 2016 6:15 PM > > To: users > > Subject: Re: Very slow column validation > > > > Hi Jeremy, > > > > Is this JDO/Datanucleus' column validation we are talking about > > here, where it is verifying its in-memory metamodel of the > > persistent entities against the RDBMS? Or is it some other sort of > > query occuring > in your code? > > > > I'm guessing the former. > > > > Just to narrow this down, if you disable column validation in the > > persistor_datanucleus.properties file, then does the delay disappear? > > > > ~~~ > > Speculating, I wonder if your app is being bootstrapped without JDO > > knowing about all the persistent entities, and then only later when > > you hit this second entity does a whole bunch more of the metamodel > > become "apparent" to JDO, hence a pause. In earlier versions of > > Isis we didn't necessarily do a good job of telling JDO/DN about all > > the entities, though the AppSpec stuff has, I think, sorted out that > > issue because we now do a classpath scan for > > all @PersistenceCapables. Are you using an AppSpec, or still relying on > > isis.properties? > > > > Whatever your answers to the above, 9 minutes does sound like a > > ridiculously long amount of time, so we've obviously got to try to > > figure this one out for you. > > > > Thx > > Dan > > > > > > > > > > > > > > On 12 January 2016 at 23:39, Branham, Jeremy [HR] < > > [email protected]> wrote: > > > > > I'm experiencing some latency, which I think is occurring due to a > > > large number of tables in my DB. > > > The first domain object that is loaded on the home page loads very > quick. > > > But the second loads very slow. > > > > > > I notice there is a difference in the column validation query - > > > The first query has a table name in the 'where' clause. > > > The second query contains 2 objects/tables in the debugging > > > statement, and there is no 'where' clause in the SQL. > > > > > > Hangs at this line while debugging - > > > Loading column info for table(s) "OBJECT_A, OBJECT_B" in > > > Catalog "", Schema "" > > > > > > It will eventually return, but it takes about 9 minutes. > > > I'd like to speed this up, maybe by forcing the column validation > > > to use the table name every time, or some other way. > > > > > > Any thoughts? > > > > > > > > > > > > Jeremy D. Branham > > > Technology Architect - Sprint > > > > > > > > > ________________________________ > > > Learn more on how to switch to Sprint and save 50% on most > > > Verizon, AT&T or T-Mobile rates. See > > > sprint.com/50off<http://sprint.com/50off> > > > for details. > > > > > > ________________________________ > > > > > > This e-mail may contain Sprint proprietary information intended > > > for the sole use of the recipient(s). Any use by others is prohibited. > > > If you are not the intended recipient, please contact the sender > > > and delete all copies of the message. > > > > > > > ________________________________ > > Learn more on how to switch to Sprint and save 50% on most Verizon, > > AT&T or T-Mobile rates. See > > sprint.com/50off<http://sprint.com/50off> > > for details. > > > > ________________________________ > > > > This e-mail may contain Sprint proprietary information intended for > > the sole use of the recipient(s). Any use by others is prohibited. > > If you are not the intended recipient, please contact the sender and > > delete all copies of the message. > > > > ________________________________ > Learn more on how to switch to Sprint and save 50% on most Verizon, > AT&T or T-Mobile rates. See sprint.com/50off<http://sprint.com/50off> > for details. > > ________________________________ > > This e-mail may contain Sprint proprietary information intended for > the sole use of the recipient(s). Any use by others is prohibited. If > you are not the intended recipient, please contact the sender and > delete all copies of the message. > ________________________________ Learn more on how to switch to Sprint and save 50% on most Verizon, AT&T or T-Mobile rates. See sprint.com/50off<http://sprint.com/50off> for details. ________________________________ This e-mail may contain Sprint proprietary information intended for the sole use of the recipient(s). Any use by others is prohibited. If you are not the intended recipient, please contact the sender and delete all copies of the message.
