Philip and I chatted this morning on IRC://irc.freenode.net/#derby
about DERBY-213.
* We ran our resultset.java test and updated the master.
We decided to add a test for fetching past the end of a
scrollable cursor too.
* We talked about the protocol specs from the 10,000 foot level.
They are available at http://www.opengroup.org/dbiop/
* C043 - Vol. 1: Distributed Relational Database
Architecture
Describes overall protocol flow.
* C044 Vol. 2: Formatted Data Object Content Architecture
Describes format of data types for exchange between
server and client
* C045 Vol. 3: Distributed Data Management Architecture
Describes the commands, objects, and parameters used
in the protocol described in Volume 1.
* We looked at QRYCLSIMP. in Vol 3
Client can send one of three values that affect how the
server behaves when fetching past
the end of a FORWARD_ONLY ResultSet.
01 - Server has to close
02 - Server should not close
00 - Server decides.
Scrollable cursors are not affected.
* We are changing the server to default to 02 and will clean
up some errors in DRDAStatement at the same time.
We decided that qryclsimp should only be in DRDAResultSet,
not DRDAStatement as it is only sent for
OPNQRY, not EXCSQLSTT.
* We think the client sends 00 (Server choice) and JCC sends
01 (Server has to close) but still need to verify,
so we may only be able to fix this with network client for
right now.
Attached are the transcripts from the chat. Next meeting is Thursday,
June 2, 9am PST .
If you made it this far, I am looking for input on summaries for IRC chats..
* Is this too much / too little detail.
* Are the transcripts at all helpful? I think I might drop them if
they are not useful.
* Would a weekly summary be better?
Thanks
Kathey
kmarsden good morning Philip
Bogomips_ Morning
kmarsden how are your tests?
Bogomips_ Much more promising
Bogomips_ Total of 594 tests, 92 of which fail. For what looks like legitimate
reasons.
Bogomips_ This is for derbyAll
kmarsden What fails?
Bogomips_ Many of the dml tests that were running before as well as the netmat
tests.
Bogomips_ However this was before I new of removing the trace, so you might be
able to cut down on those failures as well.
kmarsden did you see my note about not using ij.exceptionTrace=true
kmarsden ok
Bogomips_ :-)
kmarsden What fails in derbynetmats
kmarsden ?
Bogomips_
derbyall/derbynetclientmats/derbynetclientmats.fail:derbynet/netxaPositive.sql
Bogomips_
derbyall/derbynetclientmats/derbynetmats.fail:derbynet/NSinSameJVM.java
Bogomips_
derbyall/derbynetclientmats/derbynetmats.fail:derbynet/badConnection.java
Bogomips_ derbyall/derbynetclientmats/derbynetmats.fail:derbynet/callable.java
Bogomips_
derbyall/derbynetclientmats/derbynetmats.fail:derbynet/checkSecMgr.java
Bogomips_
derbyall/derbynetclientmats/derbynetmats.fail:derbynet/csPrepStmt.java
Bogomips_
derbyall/derbynetclientmats/derbynetmats.fail:derbynet/dblook_test_net.java
Bogomips_
derbyall/derbynetclientmats/derbynetmats.fail:derbynet/dataSourcePermissions_net.java
Bogomips_ among others.
kmarsden do you know why yet?
kmarsden oh this is probably because you do not have db2jcc.jar and
db2jcc_license_c.jar
kmarsden in your classpath
Bogomips_ Some of them complain about a lack of master file. Another has a
security exception.
kmarsden hmmm
kmarsden derbynetclientmats is ok?
Bogomips_ There were a lot of netclient problems, but I'm looking at the tests
prior to removing your the trace.
Bogomips_ Shall I try removing the trace for just the singular resultset test?
kmarsden The trace I am quite sure does not affect the network server tests. I
run those just about every day
kmarsden It had been a while since I had run derbyAll so that is why I hit the
nist thing.
kmarsden But yes, let's work on resultset.java and then you can look at your
test failures later today.
kmarsden and ping me if you have questions.
kmarsden Does that sound ok?
Bogomips_ Sure, sounds good.
kmarsden ok. first we run resultset.java for embedded
Bogomips_ k
kmarsden oh I have an outdated copy. Is the one you mailed me the latest.
Bogomips_ ?
Bogomips_ This is from the second email I sent you?
kmarsden there was a mail delay. I have the one from Jira but I will look for
your mail now
Bogomips_ Ok, should be titled "Derby-213 files May 31, 10:00 a.m. AST"
kmarsden 5:46 am yesterday. Is that right?
kmarsden no 6:07
Bogomips_ Ya, that would be the time on it.
kmarsden Attempt to shutdown framework: DerbyNetClient
kmarsden 1a2,8
kmarsden > Begin ImplicitClose tests
kmarsden > 1
kmarsden > 2
kmarsden > Final call to ResultSet.next()
kmarsden > Call to ResultSet.next() thows the following exception:
kmarsden > org.apache.derby.client.am.SqlException: Invalid operation: result
set closed
kmarsden > ImplicitClose tests complete
kmarsden Test Failed.
kmarsden *** End: resultset jdk1.4.2_03 DerbyNetClient 2005-06-01 05:17:10 ***
kmarsden that's the output for client
Bogomips_ Sigh...
kmarsden that is good
kmarsden accidentally ran client first
Bogomips_ ok, good. Whew :-)
kmarsden *** Start: resultset jdk1.4.2_03 2005-06-01 05:18:21 ***
kmarsden 1a2,7
kmarsden > Begin ImplicitClose tests
kmarsden > 1
kmarsden > 2
kmarsden > Final call to ResultSet.next()
kmarsden > Call successful
kmarsden > ImplicitClose tests complete
kmarsden Test Failed.
kmarsden and for embedded
Bogomips_ good :-)
kmarsden For embedded looks good except it says it failed
Bogomips_ well, you still have to run ant, correct?
Bogomips_ for the new master file.
kmarsden but I think we just need to update the master
Bogomips_ agreed
kmarsden do you get the same result
kmarsden ?
Bogomips_ Before I modified the master, yes.
kmarsden so now mine passes
kmarsden OK. So lets look a bit at qryclsimp
kmarsden First looking at the specs
kmarsden http://www.opengroup.org/dbiop/
kmarsden There are three specifications.
kmarsden DRDA - Describes the overall flow
kmarsden FD:OCA describes the data type translation for different machines
kmarsden DDM describes the tokens that we send back and forth
kmarsden That is the 10,000 foot overview
kmarsden We are going to look at the DDM Manual C045
Bogomips_ Now to enter the orbit?
Bogomips_ k
kmarsden if you search for qryclsimp
kmarsden You will see I think that is only sent as part of the OPNQRY request.
Bogomips_ Alright
kmarsden looking at page 680 you can see the various values that can be sent
kmarsden 00 - Server choice
Bogomips_ 694 volume 3
kmarsden right.
kmarsden I think I gave you the page number at the bottom.
kmarsden 01 - Server has to close
kmarsden 02 - Server should not close
kmarsden the server should ignore the request for scrollable cursors even if 01
kmarsden The server indicates the end of data with SQLState 0200
kmarsden sorry 02000
kmarsden Do you see any more useful information here?
Bogomips_ looking
Bogomips_ If the query is scrollable, the server ignores the QRYCLSIMP value.
Bogomips_ There is also the definition of implicit closing but that is easily
understood.
kmarsden yes. So I think that what this tells us is that we need to add a test
for a scrollable cursor fetching past the end.
kmarsden to make sure it doesn't get closed
Bogomips_ Alright. Now to find out how to request a scrollable ResultSet
kmarsden
http://java.sun.com/j2se/1.4.2/docs/api/java/sql/Connection.html#prepareStatement(java.lang.String,
int, int)
Bogomips_ I believe the javadocs for java.sql.Connection.createStatement(int
resultSetType, int resultSetConcurrency) tell us.
Bogomips_ also true.
kmarsden I am guessing there are examples in resultset.java
Bogomips_ quite possibly.
kmarsden looking
Bogomips_ There is an example with SCROLL_INSENSITIVE, but I cannot find the
reverse.
kmarsden I think that would be fine for this.
kmarsden but we will look at the code and see if there is any special handling
for SCROLL_SENSITIVE and decide if we need two tests
kmarsden For now we can just make a note to add a test while we finish our
analysis
kmarsden I put todo tags in chats sometimes.
Bogomips_ Alright.
Bogomips_ Do you know which of these two options statements default to?
kmarsden TODO: Write tests fetching past end of scrollable resulsets.
kmarsden hmm.
Bogomips_ "Result sets created using the returned Statement object will by
default be type TYPE_FORWARD_ONLY and have a concurrency level of
CONCUR_READ_ONLY."
kmarsden Resultsets default to forward only
kmarsden Then you specifify SCROLL_SENSITIVE or SCROLL_INSENSITIVE explicitly
if you want scrollable.
kmarsden TYPE_SCROLL_INSENSITIVE
kmarsden - The constant indicating the type for a ResultSet object that is
scrollable but generally not sensitive to changes made by others.
kmarsden TYPE_SCROLL_SENSITIVE
kmarsden - The constant indicating the type for a ResultSet object that is
scrollable and generally sensitive to changes made by others.
kmarsden OK. So you said you think the client sends 01?
Bogomips_ On further analysis, I believe the DerbyNet framework sends 01 and
the DerbyNetClient sends 00.
kmarsden OK. good. Then probably what is going to happen with our
resultset.java test is that derbynet is going to keep its current output.
kmarsden Only derbynetclient will change behaviour since it sends SERVER_CHOICE
(00)
kmarsden Our changes will be on the server side.
kmarsden Let's look at DRDAStatement.java
Bogomips_ Understood. Beyond petitioning IBM for changes in the db2jcc.jar
there is little we can do for a value of 01.
Bogomips_ You may wish to verify my findings.
kmarsden We will verify. I figure we will change the SERVER_CHOICE option and
see what the behaviour is #:)
kmarsden As for petitioning IBM, I guess you could petition me since I work for
IBM.
kmarsden I will file a bug with JCC and hopefully they will fix.
kmarsden but we will just worry about the server and client for our changes.
Bogomips_ :-)
Bogomips_ Of importance before we start modifying codepoint values is the small
bug in the setOPNQRYoptions().
kmarsden yes
kmarsden The one thing that strikes me in looking at this is that QRYCLSIMP is
only sent on OPNQRY
kmarsden not on PRPSQLSTT or any other statement related call.
kmarsden I think that it is wrong that it is in DRDAStatement at all
kmarsden If we look at setOPNQRYoptions it is just updating the DRDAResultSet
values.
kmarsden and if we fix the bug you found. the value in DRDAStatement is not
being used at all I think.
Bogomips_ Quite possibly.
kmarsden If i check references for DRDAStatement.qryclsimp
kmarsden it is only used by setOPNQRYoptions, but in fact shouldn't be used
there at all.
Bogomips_ It is a protected variable so it can be called from other classes in
the package. Is it there for future compatibility?
kmarsden I don't think so.
Bogomips_ Or should the value always be specific to the DRDAResultSet?
Bogomips_ k
kmarsden It should always be specific to the resultset.
kmarsden I think it is just part of the general messiness of network server.
Bogomips_ Alright. A subsequent issue that might stem from our investigations
would be to clean up DRDAStatement.
kmarsden I think the same will apply to the other options as well.
kmarsden Yes!
Bogomips_ Alright.
Bogomips_ I guess we are out of time, do you have any homework assignments for
me :-)
kmarsden Yes. Search the rest of the DDM Manual and DRDA manual for qryclsimp
references and make sure we are not missing anything
kmarsden Think about adding a test for scrollable resultsets to our test.
kmarsden get your tests running
kmarsden I guess that last one is the most important.
Bogomips_ Indeed.
Bogomips_ Do you think we need to test ResultSets from both PreparedStatements
and Statements? If not I can remove that functionality and replace it with
Scrollability.
Bogomips_ If we do need both statements, I'll just layer the additional
functionality on top.
kmarsden I think that we don't need both prepared statements and statements
since they wil both go through the same code path
Bogomips_ k
kmarsden The general idea is that our tests optimally will provide 100% code
coverage of the areas we change.
Bogomips_ Since this is an area we will likely not touch it will not be needed.
Bogomips_ Alright, when would you next like to meet?
kmarsden Hold on. let me check something
kmarsden if we meet 9am my time tomorrow is that too late for you?
Bogomips_ No that should be fine.
kmarsden OK. good. until then.
Bogomips_ bye
* Bogomips_ ([EMAIL PROTECTED]) has left #derby
