Hi Kathey,
Thanks for all the guidance and help. Yes got one of my implicit questions right :-) of how to debug the tests in the framework.
I would definitely do a wrie up for this and put it up on the web when I am comfortable doing this.
Yes this sort of information would definitely hep users debug the tests in the framework.
Evan after running the tests with the properties that you had mentioned the tmp file did not have any major changes ( no stack tarces for the error I got ),
except for a few warning message relating to empty tables.
One interesting I noticed is that the select statement in the test that is prior to this is, does a similar operation and the result fetches two rows.
So I presume the delte should delete two rows am I missing something obvious here ?
I did a pass of code that I have changed, I just pull the correlation name from the query and pass it to the DeleteNode. Could this be a binding problem?
Looks like that from the error I am getting. I will look at this aspect more, any more pointers/comments on my code changes will definitely help.
~ Shreyas
Kathey Marsden wrote:
Shreyas Kaushik wrote:
Please see inline.
Kathey Marsden wrote:
Shreyas Kaushik wrote:
What did you find in anlalyzing your test failures? I have not lookedCan somebody reply on this?
~ Shreyas
Shreyas Kaushik wrote:
Did any1 have a chance to look at this?
~ Shreyas
Shreyas Kaushik wrote:
I did some changes to sqlgrammar.jj file to make a fix for this. With the patch , the following cases are taken care : 1) delete from <tablename> <correlationName> where <correlatioName>.<columnName> = <value> 2) delete from <tablename> AS <correlationName> where <correlatioName>.<columnName> = <value>
But when I ran derbyall I had a couple of failures. I am attaching the test failure diffs here as well.
Please comment.
~ Shreyas
at your diffs specifically, but one thing I have found helpful is that
often if there is a trace it is in the .tmp file, and has been filtered
out of the .out file.
No, nothing has been filtered out.
Take a look at the failures and try to determine how they might relate to your changes and let us know if you have some more specific questions, what is failing how you think it may or may not be related to your change.
w.r.t floattypes I don't see how my changes could have made any impact there. But the tests in refActions1 that have failed were earlier failing as there was no correlation name handling for delete statements. These were giving a different error as the diffs show. But now with the correlation name handling fixed ( hopefully ;-) ) I don't know whether the queries that are erroring out are correct ? It could be that I might have also missed something. I want people here to have a look at it so that they might spot something that I may have missed.
~ Shreyas
Kathey
Hi Shreyas,
Sounds like you are really stuck on this thing so I would like to try to
help.
Let me take a look at this refActions1 diff you sent.
** Start: refActions1 jdk1.5.0 derbyall:derbylang 2005-04-04 15:53:53 *** 6009 del < ERROR 42X01: Syntax error: Encountered "d" at line 1, column 26. 6009a6009
ERROR 42X04: Column 'DB2TEST.DEPT.DNO' is not in any table in the FROMlist or it appears within a join specification and is outside the scope
of the join specification or it appears in a HAVING clause and is not in
the GROUP BY list. If this is a CREATE or ALTER TABLE statement then
'DB2TEST.DEPT.DNO' is not a column in the target table.
Hmmm, we used to perform some sql command, and got a syntax error and now we perform the same sql command and get a different error. I can't really tell from this diff whether considering the changes you made ... a) The syntax error is correct. b) The new error is correct c) That sql command (whatever it is) should have completed just fine.
It seems to me maybe you have a bigger question of. "How do I debug a test failure"? And that I think is an excellent question, because our test harness is not always the easiest thing to figure out. So here are some quick notes on the topic to get you started. Nothing formal, so forgive typos and such. I hope they help you get past your current sticking point. I also hope that after you iterate over this process a few times and improve upon it, you will consider making a short write up for the web site on how to debug test failures. I think new folks would find it useful. I'll just stick to embedded here since that is the problem at hand, but certainly there are other tips and tricks for network server, so I would be happy to provide those if you are interested in making a web page with this info.
How to debug a test failure.
1) Run the test. take a look at ${derby.src}/java/testing/README for all sorts of good info on tests). I think I would run this test something like this.
java -Dij.exceptionTrace=true -Dkeepfiles=true org.apache.derbyTesting.functionTests.harness.RunTest lang/refActions1.sql
For network server add -Dframework=DerbyNet (OK 1 network server tip.)
2) Do a visual diff of the ouptut with the canon
It will give you more context in approaching the problem.
In the test output directory you will see refactions1.out (filitered test output)
refactions1.tmp (unfiltered test output - this one should have a trace)
To get the most information, I usually initially vdiff the tmp file
with the canon which is checked in under
java/testing/org/apache/derbyTesting/functionTests/master/ or
appropriate framework or jdk subdirectory.
3) Identify the sql statement or java code causing the diff. For sql scripts this is usually pretty obvious. For java programs you have to look at the test source to figure out what is going on.
4) Evaluate the diff. This of course starts the tricky and interesting
part.
Look carefully at the sql command or java code that caused the diff. Think about what it should do and how it relates to your change.
Decide what you think the right behaviour should be.
a) The old behaviour
b) The new behaviour
c) something else
If you have a trace, look at the the trace and see if it holds any clue.
If you are lucky it passes right through that code you changed.
Often it is helpful to put that one sql command or bit of java code in
a separate script or stand alone program, so you can evaluate it
independently outside of the harness and evaluate in the debugger.
Common results of the evaluation phase are:
a) An error in your code.
b) Someone elses test failure all together. Good to keep a clean client for testing this. c) A master update. Be careful with this one. make sure you have a
valid reason to update a master. Be especially cognizant of backward
compatibility. We don't want any real or perceived regressions to catch
us by surprise. Two good questions to ask your self and then answer when you post your
patch.
1) What is my valid reason for updating this master.
2) Might someone be surprised by this difference and perceive it as
a regression?
5) Resolve the issue. Here are some possible actions based on your evaluation in step 4
a) An error in your code.
Fix it #:)
b) Someone else's test failure all together. Look at the recent checkins and try to guess the culprit and post.
(See Dan's mail earlier today entitled jdbcapi/testRelative failing & Derby-186) for an example of this.)
c) A master update.
update the master and make sure you include an explanatio in
your patch.
If you get stuck along the way, please post to the list, but make sure
you include.
1) The small bit of sql or java code in question. 2) A description of the old and new behaviour and what you think the
right behavior should be.
3) The stack trace if there is one.
4) What you have learned so far from your own evaluation of 1-3 above.
5) A specific question that is not going to take a lot of research for
someone to answer that you think might send you back along your way if
answered.
Hope this helped. Good luck and happy debugging. I really do hope you will consider writing up the process as you become comfortable with it.
Thanks
Kathey
