Bugs item #1959269, was opened at 2008-05-07 09:16
Message generated for change (Comment added) made by vzzzbx
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=482468&aid=1959269&group_id=56967
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: PF/runtime
Group: Pathfinder CVS Head
>Status: Closed
Resolution: Fixed
Priority: 6
Private: No
Submitted By: Wouter Alink (vzzzbx)
Assigned to: Wouter Alink (vzzzbx)
Summary: XQ+JDBC: protocol violation
Initial Comment:
The MonetDB/XQuery of yesterday seems indeed more stable than the last version.
However, I have difficulty getting the java JDBC driver to work with MonetDB.
We use the JDBC connection extensively in the following way (which works
perfectly with the old backend):
=====================
Connection xqConn =
DriverManager.getConnection("jdbc:monetdb://localhost:50000/database?language=xquery",
"monetdb","monetdb");
Statement st = xqConn.createStatement();
ResultSet rs = st.executeQuery("1+1,\"aap\",<noot/>");
try {
rs.next();
if (rs.getLong(1) != 2) fail("failed to receive expected integer");
rs.next();
if (!rs.getString(1).equals("aap")) fail("failed to receive expected
string");
rs.next();
if (!rs.getString(1).equals("<noot/>")) fail("failed to receive
expected element");
} finally {
rs.close();
}
=====================
However, it doesn't work with the new backend, each query gives a "protocol
violated" SQLException.
----------------------------------------------------------------------
>Comment By: Wouter Alink (vzzzbx)
Date: 2009-11-15 16:34
Message:
added test (JDBC_5_results.SF-1959269). closing bug.
----------------------------------------------------------------------
Comment By: Stefan Manegold (stmane)
Date: 2008-05-28 16:09
Message:
Logged In: YES
user_id=572415
Originator: NO
As Fabian suggested, if this is a feature that you (Wouter) rely on, then
we should consider adding a respective test...
----------------------------------------------------------------------
Comment By: Fabian (mr-meltdown)
Date: 2008-05-14 14:43
Message:
Logged In: YES
user_id=963970
Originator: NO
I think it makes sense to add your specific code sample as test. There
are not so many JDBC tests in XQuery, and if you are heavily relying upon
that, it may be wise to test some more on what you're doing.
----------------------------------------------------------------------
Comment By: Wouter Alink (vzzzbx)
Date: 2008-05-14 14:37
Message:
Logged In: YES
user_id=621590
Originator: YES
sorry, yes i did check, and it now seems to work without problems. thanks
jan.
i guess no new tests need to be added. i'm closing this bug.
----------------------------------------------------------------------
Comment By: Stefan Manegold (stmane)
Date: 2008-05-14 13:06
Message:
Logged In: YES
user_id=572415
Originator: NO
Wouter,
did you check, whether Jan's fixes work for you?
If so, do you think, we can close this one?
The XQuery-JDBC tests seem to work fine with the ALG back-end, now:
http://monetdb.cwi.nl/testing/projects/monetdb/Current/pathfinder/.mTestsG103/GNU.32.32.d-Fedora8/tests_BugTracker/JDBC_250_results.SF-1730556.out.00.html
http://monetdb.cwi.nl/testing/projects/monetdb/Current/pathfinder/.mTestsG103/GNU.32.32.d-Fedora8/tests_BugTracker/JDBC_250_results.SF-1730556.err.00.html
http://monetdb.cwi.nl/testing/projects/monetdb/Current/pathfinder/.mTestsG103/GNU.32.32.d-Fedora8/tests_BugTracker/JDBC_0_results.SF-1886326.out.00.html
http://monetdb.cwi.nl/testing/projects/monetdb/Current/pathfinder/.mTestsG103/GNU.32.32.d-Fedora8/tests_BugTracker/JDBC_0_results.SF-1886326.err.00.html
.
----------------------------------------------------------------------
Comment By: Stefan Manegold (stmane)
Date: 2008-05-08 15:39
Message:
Logged In: YES
user_id=572415
Originator: NO
Running Mtest by hand (yes, that is still possible! ;-)), seems to show
that some (all?) Pathfinder-JDBC tests as well as some (all) of the
non-"xml" (i.e., "dm", "seq", etc.) serialization test now also seem to
work fine with the Algebra back-end.
More details will indeed we revealed by tomorrows TestWeb after tonight
test run.
Thanks to Jan (and all others involved) for yet another step forward
towards the upcoming "Algebra release".
----------------------------------------------------------------------
Comment By: Wouter Alink (vzzzbx)
Date: 2008-05-08 12:26
Message:
Logged In: YES
user_id=621590
Originator: YES
cross-posted with Jan.
(@Jan: I can try to check the nightly build tomorrow to see whether your
fix works)
As fabian already pointed out (and Stefan likes to hear): the testweb
should tell us whether the JDBC tests succeed :)
----------------------------------------------------------------------
Comment By: Wouter Alink (vzzzbx)
Date: 2008-05-08 12:03
Message:
Logged In: YES
user_id=621590
Originator: YES
i found a solution to the problem after reading all the helpful messages:
(thanks Fabian, Jan, Stefan)
it wasn't the 'dm' mode i was looking for, but the 'seq' mode. If i
compile MonetDB so that it generates genType="seq", then XQuery through
JDBC works fine with the algebra-backend.
this works for me at the moment, but i guess a permanent solution should
be found eventually. (lowered priority to 3)
----------------------------------------------------------------------
Comment By: Jan Rittinger (tsheyar)
Date: 2008-05-08 11:37
Message:
Logged In: YES
user_id=993208
Originator: NO
Stefan,
that's exactly what I already did some time ago :)
After some time I reached the core of the problem: In mps a single query
consists of a prologue, a query, and an epilogue. Trying to understand how
everything fits together definitely takes more time than I'm willing to
invest. (Somehow there is a static prologue whose pointer to variable
genType is looked up and later modified. In every query however there is
also a new prologue, that gets magically modified by Peters MIL
optimization. @Fabian: it's not Spaghetti but Maze code :))
...
The only feasible solution would be to build on the already active hacks,
take the value from the genType variable of the unused static prologue, and
use it as parameter for an explicit compiler argument. That's exactly what
I just checked in.
@Wouter: could you please check if your problem is now fixed.
@Peter: Is this complexity really necessary or isn't there a much simpler
solution?
----------------------------------------------------------------------
Comment By: Stefan Manegold (stmane)
Date: 2008-05-08 10:19
Message:
Logged In: YES
user_id=572415
Originator: NO
Jan,
while I agree that the genType (and alike!) HACK(s) should ("eventually")
be raplaced by proper runtime parameters,
I'm wondering, whether there is any reason that the genType HACK does not
work with the Algebra back-end other than that the algebra back-end does
not introduce and use a "genType" variable in the generated MIL code.
I.e., would/could simply adding a 'var genType := "xml";' and using
variable 'genType' instead of hard-wired "xml" in the generated
'print_result(...);' statement (and alike for 'genType = "...-timing-..."';
analogous to what MPS does) (at least temporarily) (and "quickly") "solve"
the problem?
Stefan
----------------------------------------------------------------------
Comment By: Jan Rittinger (tsheyar)
Date: 2008-05-08 09:33
Message:
Logged In: YES
user_id=993208
Originator: NO
Guys,
I think I did not make myself clear enough. There is no problem in the
serialization. In the algebra we can still use all serialization modes may
it be dm, mapi_dm, text, mapi_text, xml, mapi_xml, typed_xml,
mapi_typed_xml, xrpc_xml, mapi_xrpc_xml, or seq. The only problem is that
the MIL command print_result() decides which mode to use based on the value
of a parameter. The algebra code generation like mps always generates code
that uses the "xml" mode.
As the compiler (and the code generation) is decoupled from the runtime
part (client side querying) Peter introduced a quite ugly hack that takes
the internal representation of the generated MIL code, looks for a special
variable (genType), and replaces its content with the mode the client-side
querying asks for. This 'hack' does not work for the algebra generated
code---that's the problem here.
The cleanest solution would be to extend the compiler (PFcompile*()) with
an additional argument (client side options like the serialization mode)
that are then used to directly generate the correct code. As this might
break the mps code generation and is very low on 'my' priority list it is
not implemented so far.
----------------------------------------------------------------------
Comment By: Fabian (mr-meltdown)
Date: 2008-05-07 22:33
Message:
Logged In: YES
user_id=963970
Originator: NO
alternatively one could say that the BugTracker/JDBC* tests fail for
pathfinder/G and should be fixed one way or another ;)
----------------------------------------------------------------------
Comment By: Fabian (mr-meltdown)
Date: 2008-05-07 22:24
Message:
Logged In: YES
user_id=963970
Originator: NO
Thanks Wouter.
quote:
>>>
The second problem is a little bit more tricky and would require
more knowledge about the current mapi-protocol (which i do not have
anymore). I assume that Fabian is the one to ask here, but as he should
stay focused on his thesis. I guess with some documentation on (or
examples
of) the current mapi protocol and by reading serialize*.mx you or i could
do it in little time.
<<<
What needs to be done is trivial considering the protocol. Not much has
changed since we (Wouter and I) made that JDBC support in pathfinder.
Though, one needs to understand that what is emitted right now, IMHO does
not conform to anything. It may work, and may be fast, but it has no way
of reliably finding bounds (or doing sequences).
Therefore, regardless of JDBC, that situation needs to be fixed, unless
there is no interest in any API accessibility for XQuery of course (apart
from piping output of mclient).
That said, it's up to you guys to figure out your spaghetti ;) If you
need help to construct a valid "ResultSet" for JDBC, drop me an email. The
hard part of finding the sequence boundaries and serialising them in the
right form is up to you.
----------------------------------------------------------------------
Comment By: Lefteris Sidirourgos (lsidir)
Date: 2008-05-07 21:53
Message:
Logged In: YES
user_id=1856546
Originator: NO
Hi,
the reason I did this one priority 9 is because I understand that this is
a show-stopper bug and because I understood from our initial e-mails that
this problem emerged the last days, from some recent check-in.
Unfortunately my knowledge is limited on how to fix this bug.
----------------------------------------------------------------------
Comment By: Wouter Alink (vzzzbx)
Date: 2008-05-07 21:35
Message:
Logged In: YES
user_id=621590
Originator: YES
lefteris, i'm glad that you give this bug attention, but perhaps priority
9 is a little overdone.
some background information: fabian and i added the feature of
communicating with MonetDB/XQuery through JDBC about three years ago. back
then it looked to us as if XMLDB (the API proposed by others) wouldn't
survive, and it seems that we were right. therefore we added the special
serialization mode which would return a xquery sequence as a one-column sql
resultset. difference with sql was the fact that different types could
occur in a single column, but this didn't hurt mapi (as fabian already
mentioned, it doesn't really care what it transmits either way). It is
simple, it is simple, and it makes life easier. i don't know if there are
any other MonetDB/XQuery-users who abuse JDBC the same way.
The algebra backend from the beginning only supported the 'xml'
serialization (and this has been communicated). This serialization mode
implies that a result always is a constructed element, and never a
sequence. The reason for this bugreport is that I would like to see two
features being implemented in the algebra version in the same way they are
implemented in the mps version: 1) the 'dm' serialization mode, which
allows xquery sequences to be returned, and 2) the ability to do xquery
through JDBC.
I already tackled the first problem by compiling pathfinder after
replacing xquery(...,"xml",....) by xquery(..., "dm",...) in (i believe it
was) serialize_dflt.mx. I hope this can be made a (runtime?) configuration
option. The second problem is a little bit more tricky and would require
more knowledge about the current mapi-protocol (which i do not have
anymore). I assume that Fabian is the one to ask here, but as he should
stay focused on his thesis. I guess with some documentation on (or examples
of) the current mapi protocol and by reading serialize*.mx you or i could
do it in little time.
----------------------------------------------------------------------
Comment By: Lefteris Sidirourgos (lsidir)
Date: 2008-05-07 14:27
Message:
Logged In: YES
user_id=1856546
Originator: NO
Changed the priority to highest because this is an important bug.
----------------------------------------------------------------------
Comment By: Fabian (mr-meltdown)
Date: 2008-05-07 11:02
Message:
Logged In: YES
user_id=963970
Originator: NO
mapilib (mapiclient) is extremely lax towards the protocol. It can do
that as it usually just dumps exactly what it received on the stream, not
caring about the contents at all. JDBC on the other hand relies on the
correctness of the protocol, as it needs to determine correct actions for
each and every response it receives. Chances are big you just didn't
notice the protocol is violated, because mclient didn't warn you something
is wrong, but just dumped the output.
----------------------------------------------------------------------
Comment By: Jan Rittinger (tsheyar)
Date: 2008-05-07 10:57
Message:
Logged In: YES
user_id=993208
Originator: NO
Fabian is right. The problem arises in the server-side MIL code
generation.
The dirty tricks Peter played with the variable of name genType (modifying
the generated internal MIL tree) do not work for the algebra version (and I
don't know why). Because the strange side-effects to the genType MIL
variable do not affect the value of the variable the serialization always
uses the serialization mode 'xml' instead of the JDBC or MAPI mode. (If I
think some more I don't even know why client side querying works as the
special '=' characters at the beginning of the line are not generated.)
----------------------------------------------------------------------
Comment By: Fabian (mr-meltdown)
Date: 2008-05-07 10:49
Message:
Logged In: YES
user_id=963970
Originator: NO
here is the communication log for above example:
RD 1210082838966: read final block: 20 bytes
RX 1210082838966: :mserver:8:plain:LIT
RD 1210082838966: inserting prompt
TD 1210082838966: write final block: 43 bytes
TX 1210082838966: BIG:monetdb:{plain}monetdb:xquery:database:
RD 1210082838968: read final block: 69 bytes
RX 1210082838968: #MonetDB v4 doesn't have multiple database support,
argument
ignored
RD 1210082838968: inserting prompt
TD 1210082838968: write final block: 11 bytes
TX 1210082838968: Xoutput seq
RD 1210082838968: read final block: 0 bytes
RX 1210082838968:
RD 1210082838968: inserting prompt
TD 1210082838996: write final block: 4 bytes
TX 1210082838996: s1+1
RD 1210082839002: read final block: 2 bytes
RX 1210082839002: 2
RD 1210082839002: inserting prompt
----------------------------------------------------------------------
Comment By: Fabian (mr-meltdown)
Date: 2008-05-07 10:45
Message:
Logged In: YES
user_id=963970
Originator: NO
for the record:
it is a true violation, as the server returns:
2
which is clearly a violation, as it should have been
=2
or
[ 2 ]
or
[ "2" ]
NotMyBug.nl :)
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=482468&aid=1959269&group_id=56967
------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
Monetdb-bugs mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/monetdb-bugs
