The delay does not go away when validation is turned off. It seems the in memory metamodel validation occurs when the object is accessed, regardless of the property.
I haven’t tried a 3rd party driver yet, but I will give that a shot, and monitor if a separate query is occurring for each row. Jeremy D. Branham Technology Architect - Sprint O: +1 (972) 405-2970 | M: +1 (817) 791-1627 | **DotNet [email protected] #gettingbettereveryday -----Original Message----- From: Dan Haywood [mailto:[email protected]] Sent: Tuesday, January 12, 2016 6:54 PM To: users Subject: Re: Very slow column validation On 13 January 2016 at 00:35, Branham, Jeremy [HR] < [email protected]> wrote: > Thanks Dan - > Yes, it is the JDO/DataNucleus column validation against the RDMS > [Oracle in this case] > > I've disabled the validation in the properties file like this. > > isis.persistor.datanucleus.impl.datanucleus.schema.autoCreateAll=false > isis.persistor.datanucleus.impl.datanucleus.schema.validateTables=fals > e > > isis.persistor.datanucleus.impl.datanucleus.schema.validateConstraints > =false > > OK, so I'm clear... does the pause go away when you disable the validation? per http://www.datanucleus.org/products/accessplatform_4_1/jdo/schema.html what do you observe with: datanucleus.schema.validateTables vs datanucleus.schema.validateColumns vs datanucleus.schema.validateConstraints (each with the "isis.persistor.datanucleus.impl" prefix, of course, when listed in our properties file). > I'm using @PersistenceCapable on the domain objects and implementing > the AppManifest interface [DomainAppDomManifest]. > I used the Simple archetype to create a project, but I am using an > existing DB. > > ok, good. > > The following query is generated when the new object is accessed - > > SELECT NULL TABLE_CAT, OWNER TABLE_SCHEM, TABLE_NAME, COLUMN_NAME, > NULL DATA_TYPE, DATA_TYPE TYPE_NAME, > DECODE(DATA_TYPE,'NUMBER',DATA_PRECISION,DATA_LENGTH) COLUMN_SIZE, 0 > BUFFER_LENGTH, DATA_SCALE DECIMAL_DIGITS, 10 NUM_PREC_RADIX, > DECODE(NULLABLE,'Y',1,0) NULLABLE, NULL REMARKS, NULL COLUMN_DEF, 0 > SQL_DATA_TYPE, 0 SQL_DATETIME_SUB, DATA_LENGTH CHAR_OCTET_LENGTH, > COLUMN_ID ORDINAL_POSITION, DECODE(NULLABLE,'Y','YES','NO') > IS_NULLABLE FROM ALL_TAB_COLUMNS ORDER BY TABLE_SCHEM, TABLE_NAME, > ORDINAL_POSITION > > This returns 61446 records. > > I think this is the source of the delay, as each of these records are > interated in memory. > > that doesn't seem like a huge amount... I imagine this takes only a second or two to return when you run it from a standalone query tool? The only way I could see this query eating up 9 minutes would be if, as JDO/DN iterates over it, there ends up being making a network call for each row... I doubt that happens, but perhaps you could monitor to see. (I don't know if Oracle provides such a tool; worst comes to worse you could use the log4jdbc driver that we have commented out in persistor.properties. I did also notice that in the DN docs there is a remark about "validateColumns" being buggy for some JDBC drivers. Are there any third-party JDBC drivers that you could try out? -- Dan > > Jeremy D. Branham > Technology Architect - Sprint > > > -----Original Message----- > From: Dan Haywood [mailto:[email protected]] > Sent: Tuesday, January 12, 2016 6:15 PM > To: users > Subject: Re: Very slow column validation > > Hi Jeremy, > > Is this JDO/Datanucleus' column validation we are talking about here, > where it is verifying its in-memory metamodel of the persistent > entities against the RDBMS? Or is it some other sort of query occuring in > your code? > > I'm guessing the former. > > Just to narrow this down, if you disable column validation in the > persistor_datanucleus.properties file, then does the delay disappear? > > ~~~ > Speculating, I wonder if your app is being bootstrapped without JDO > knowing about all the persistent entities, and then only later when > you hit this second entity does a whole bunch more of the metamodel > become "apparent" to JDO, hence a pause. In earlier versions of Isis > we didn't necessarily do a good job of telling JDO/DN about all the > entities, though the AppSpec stuff has, I think, sorted out that issue > because we now do a classpath scan for > all @PersistenceCapables. Are you using an AppSpec, or still relying on > isis.properties? > > Whatever your answers to the above, 9 minutes does sound like a > ridiculously long amount of time, so we've obviously got to try to > figure this one out for you. > > Thx > Dan > > > > > > > On 12 January 2016 at 23:39, Branham, Jeremy [HR] < > [email protected]> wrote: > > > I'm experiencing some latency, which I think is occurring due to a > > large number of tables in my DB. > > The first domain object that is loaded on the home page loads very quick. > > But the second loads very slow. > > > > I notice there is a difference in the column validation query - The > > first query has a table name in the 'where' clause. > > The second query contains 2 objects/tables in the debugging > > statement, and there is no 'where' clause in the SQL. > > > > Hangs at this line while debugging - > > Loading column info for table(s) "OBJECT_A, OBJECT_B" in Catalog > > "", Schema "" > > > > It will eventually return, but it takes about 9 minutes. > > I'd like to speed this up, maybe by forcing the column validation to > > use the table name every time, or some other way. > > > > Any thoughts? > > > > > > > > Jeremy D. Branham > > Technology Architect - Sprint > > > > > > ________________________________ > > Learn more on how to switch to Sprint and save 50% on most Verizon, > > AT&T or T-Mobile rates. See > > sprint.com/50off<http://sprint.com/50off> > > for details. > > > > ________________________________ > > > > This e-mail may contain Sprint proprietary information intended for > > the sole use of the recipient(s). Any use by others is prohibited. > > If you are not the intended recipient, please contact the sender and > > delete all copies of the message. > > > > ________________________________ > Learn more on how to switch to Sprint and save 50% on most Verizon, > AT&T or T-Mobile rates. See sprint.com/50off<http://sprint.com/50off> > for details. > > ________________________________ > > This e-mail may contain Sprint proprietary information intended for > the sole use of the recipient(s). Any use by others is prohibited. If > you are not the intended recipient, please contact the sender and > delete all copies of the message. > ________________________________ Learn more on how to switch to Sprint and save 50% on most Verizon, AT&T or T-Mobile rates. See sprint.com/50off<http://sprint.com/50off> for details. ________________________________ This e-mail may contain Sprint proprietary information intended for the sole use of the recipient(s). Any use by others is prohibited. If you are not the intended recipient, please contact the sender and delete all copies of the message.
