Bugs item #1899258, was opened at 2008-02-22 02:06 Message generated for change (Comment added) made by yingying You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=482468&aid=1899258&group_id=56967
Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: PF/runtime Group: Pathfinder "stable" >Status: Closed >Resolution: Fixed Priority: 6 Private: No Submitted By: ju...@uva (marspluto) Assigned to: Ying Zhang (yingying) Summary: PF: Error message with Administrative Console Initial Comment: When I go to web admin console at http://127.0.0.1:50001/admin/, I am getting this error (see the screenshot): http://tinypic.com/view.php?pic=1ttoxf&s=3 It also has been attached. This occurs when I click on "List Collections" and "List All Documents". I am also getting the error when I click on "Backup Database". To summarize, the web admin console is not working. Could someone provide a fix, because I need to backup my index. Thanks! jz ---------------------------------------------------------------------- >Comment By: Ying Zhang (yingying) Date: 2009-08-06 14:48 Message: The problem with libxml2 is tested by the current admin Tests, and it has worked fine for sometime, so closing ---------------------------------------------------------------------- Comment By: Stefan Manegold (stmane) Date: 2008-09-12 16:50 Message: This is yet another reminder to no only thoroughly test the administrative interface interactively *before* a release, but also add (non-interactive) tests of both the functionality that the administrative interface provides and the functionality that the administrative interfaces requires from the MonetDB/XQuery server (incl. its built-in HTTP server and XRPC funtionality). See also https://sourceforge.net/mailarchive/forum.php?thread_name=20080618085309.GA2096%40cwi.nl&forum_name=monetdb-developers ---------------------------------------------------------------------- Comment By: Sjoerd Mullender (sjoerd) Date: 2008-09-12 16:35 Message: The current version (not yet released) of the installer contains the 2.6.32 version of libxml2. Reassigning to Jennie since the interface is broken in many respects. Just start clicking. View Database Statistics doesn't do anything, except perhaps causing other commands to then fail. Also try adding a document on Windows. It fails with a cryptic message saying that it can't open the file. Of course, you need to specify a URL (according to the label on the interface) but you can only specify a file since the text box is not editable. File names are not URLs. And on Windows they also don't look like one another. ---------------------------------------------------------------------- Comment By: Ying Zhang (yingying) Date: 2008-07-14 14:15 Message: Logged In: YES user_id=341633 Originator: NO Reopen this bug, as it hasn't completely disappeared yet. Similar problem still happens when using the windows installer (MonetDB4-XQuery-Installer-i386-20080624.msi) of the latest release on 32 bits Windows XP, although this installer contains an *older* version of libxml2 (i.e., 2.6.30). I overwrote the libxml2.dll that comes with the Windows installer with the one found in http://www.zlatkovic.com/libxml.en.html (libxml2-2.6.32+.win32.zip), and then it works fine. So I'd like to suggest to upgrade the libxml2 included in MXQ Windows installer. Regards, Jennie ---------------------------------------------------------------------- Comment By: Ying Zhang (yingying) Date: 2008-06-20 06:59 Message: Logged In: YES user_id=341633 Originator: NO This problem is solved in a newer version of libxml2 (> 2.6.31). ---------------------------------------------------------------------- Comment By: ju...@uva (marspluto) Date: 2008-02-25 17:47 Message: Logged In: YES user_id=2016221 Originator: YES Thanks Jennie (did I meet you at the SIGIR bag packing event?) and everyone else! I will try out this work-around. I will let you if it works for me. Best, Junte ---------------------------------------------------------------------- Comment By: Ying Zhang (yingying) Date: 2008-02-25 17:40 Message: Logged In: YES user_id=341633 Originator: NO The concequence of the bug in libxml2-2.6.31 is that MonetDB/XQuery is not able to load module definition files from an HTTP server that is running on a port other than 80 (this problem does not occur if libxml2-2.6.30 is used ). What the admin console does is call pre-defined XQuery functions (defined in admin.xq) to do backup/restore work. Those functions can also be executed using other MonetDB clients, for example, 'mclient'. A workaround for users of libxml2-2.6.31 is thus as the following: (IMPORTANT: this workaround has been tested on the latest release, nl. version 0.22. It is highly recommended to first test the following queries on a dummy database!) The module definition file "admin.xq" can be found in '$prefix/share/MonetDB/xrpc/admin/', where '$prefix' is the prefix used to install MonetDB. I copied it to an easy to use place: $ cp $prefix/share/MonetDB/xrpc/admin/admin.xq /tmp/ Call functions in "/tmp/admin.xq" a 'mclient' session to first backup your XML database: $ mclient -lx xquery>pf:collections() more><> <collection updatable="true" size="102 KiB" numDocs="1">test</collection> xquery>pf:documents() more><> <document updatable="true" url="/tmp/test.xml" collection="test">test.xml</document> xquery>pf:documents("test") more><> <document updatable="true" url="/tmp/test.xml" collection="test">test.xml</document> xquery>import module namespace adm = "http://monetdb.cwi.nl/XQuery/admin/"; at "/tmp/admin.xq"; adm:backup("backup1") more><> Here we see that with 'adm:backup', a backup of *all* collections in the database (in this case, one collection "test" with one document "test.xml" in the database 'demo') are made in directory '$prefix/var/MonetDB4/dbfarm/demo/backup/backup1/'. (IMPORTANT: it is HIGHLY recommended to check now if all your documents are in the directory '$prefix/var/MonetDB4/dbfarm/<dbname>/backup/<backup-name>/<coll-name>', before executing the next query!) xquery>import module namespace adm = "http://monetdb.cwi.nl/XQuery/admin/"; at "/tmp/admin.xq"; adm:del-col("test") more><> xquery>pf:documents() more><> Then, one can use 'adm:del-col()' to clean up the database. Now the previous version of MonetDB/XQuery can be de-installed, and the new version can be installed, after which, the following query can be used to restore the database: $ mclient -lx xquery>import module namespace adm = "http://monetdb.cwi.nl/XQuery/admin/"; at "/tmp/admin.xq"; adm:restore("backup1", 10) more><> xquery>pf:documents() more><> <document updatable="true" url="backup/backup1/test/10000/1.xml" collection="test">test.xml</document> Hope this helps. Regards, Jennie ---------------------------------------------------------------------- Comment By: Stefan Manegold (stmane) Date: 2008-02-25 15:43 Message: Logged In: YES user_id=572415 Originator: NO It appears that we're hitting this bug in libxml2-2.6.31: http://bugzilla.gnome.org/show_bug.cgi?id=514521 ---------------------------------------------------------------------- Comment By: Fabian (mr-meltdown) Date: 2008-02-25 15:25 Message: Logged In: YES user_id=963970 Originator: NO FYI: With the following version of libxml2 it works fine: % qlist -Iv libxml2 dev-libs/libxml2-2.6.30-r1 ---------------------------------------------------------------------- Comment By: Ying Zhang (yingying) Date: 2008-02-22 19:12 Message: Logged In: YES user_id=341633 Originator: NO After a lot of debugging, I found what is exactly happening, which is *very* strange. The reason that clicking on buttons like "list collections" and "list all documents" (which triggers an xrpc request message to be send to the xrpc server to execute the corresponding function) does not respond, is that the HTTP server (sttpd) hangs in trying to receive the complete request message send by the function xmlParserInputBufferCreateFilename() (in PFurlcache()). The error "could not load module" is given after the connection has expired. The syntax error "unexpected $end on line 1, column 0" is caused by that xmlParserInputBufferCreateFilename() is not able to retrieve the module file, which results in an emtpy string to be parsed. When xrpc server receives a request, e.g. "list collections", it first load the module in which the function "collections()" is defined. in my test case, it is: http://andorea.ins.cwi.nl:8080/admin/admin.xq Loading the module is done by the function xmlParserInputBufferCreateFilename (in PFurlcache(), in compiler.c), which sends an HTTP GET request to andorea.ins.cwi.nl:8080 to retrieve the module file. Since this is a file request, it is handled by the HTTP server (shttpd) itself (not by the xrpc server). What shttpd does is simple: buffer the full request message (i.e. the request message contains the chars '\n\r\n'), then handle it. But here, strange thing happens: it seems that the request sent by xmlParserInputBufferCreateFilename() is not complete/valid !? After shttd has tried to read the request message several times, the buffer still only contains: (gdb) p buf $21 = 0x2aaac0005258 "GET /admin/admin.xq HTTP/1.0\r\nHost: andorea.ins.cwi.nl:8080\r\nAccept-Encoding: gzip" Note that the message is not ended by '\r\n\r\n', according to HTTP 1.1 To prove that shttpd works, using wget to retrieve the same file from shttpd succeeds: $ wget http://andorea.ins.cwi.nl:8080/admin/admin.xq ... 18:41:36 (156.47 MB/s) - `admin.xq' saved [4950/4950] $ cat admin.xq module namespace admin = 'http://monetdb.cwi.nl/XQuery/admin/'; ... the request sent by wget is: (gdb) p buf $20 = 0x2aaac0005258 "GET /admin/admin.xq HTTP/1.0\r\nUser-Agent: Wget/1.10.2 (Red Hat modified)\r\nAccept: */*\r\nHost: andorea.ins.cwi.nl:8080\r\nConnection: Keep-Alive\r\n\r\n" which is obviously complete/correct. However, another HTTP server (e.g. www.cwi.nl) does not have any problem with the request message sent by xmlParserInputBufferCreateFilename(): $ cat test.xq import module namespace t = "http://www.monetdb.nl/basicfunctions"; at "http://homepages.cwi.nl/~zhang/xrpc/basicfunctions.xq";; t:convert(23) $ mclient -lx test.xq 50.685330 For the query above, xmlParserInputBufferCreateFilename() is used to GET the file http://homepages.cwi.nl/~zhang/xrpc/basicfunctions.xq, which succeeded. But, if I change the query to use a module file served by my Apache Tomcat/5.5.9 server, the same problem occurs. mclient hangs for a while (note the time in the prompt), and gives the same error: $ cat test.xq (: import module namespace t = "http://www.monetdb.nl/basicfunctions"; at "http://homepages.cwi.nl/~zhang/xrpc/basicfunctions.xq";; :) import module namespace t = "http://www.monetdb.nl/basicfunctions"; at "http://andorea.ins.cwi.nl:8080/basicfunctions.xq";; t:convert(23) [Fri Feb 22 19:06:02 CET 2008] [andorea:/ufs/zhang] $ mclient -lx test.xq MAPI = mone...@localhost:50000 QUERY = (: ERROR = !parse error: syntax error, unexpected $end on line 1, column 0 !parse error: XQuery parsing failed !# halted in /ufs/zhang/monet/stable/pathfinder/compiler/parser/parser.y (PFparse), line 2902 !parse error: syntax error, unexpected $end on line 1, column 0 !parse error: error parsing module from http://andorea.ins.cwi.nl:8080/basicfunctions.xq !# halted in /ufs/zhang/monet/stable/pathfinder/compiler/parser/parser.y (parse_module), line 2939 [Fri Feb 22 19:06:23 CET 2008] [andorea:/ufs/zhang] $ The code in shttpd.c to hand file request hasn't been changed since 2006, and the admin page has workt before. So, am I able to conclude that: some thing is wrong with xmlParserInputBufferCreateFilename(), but maybe some HTTP server is not very strict with if the request message is correctly ended with a '\r\n\r\n'? Are we using a newer version of libxml2 and since when? Thanks in advance for any suggestions/comments, Jennie ---------------------------------------------------------------------- Comment By: Ying Zhang (yingying) Date: 2008-02-22 13:59 Message: Logged In: YES user_id=341633 Originator: NO Stefan, Thanks for the pictures. I now also get similar problem. I'm searching for the reason why it can't find the module. You are right, if it is used for such important task, it should be tested more often. Nightly test would be great, but I won't know how... I'm willing to test it manually before each release. ---------------------------------------------------------------------- Comment By: Stefan Manegold (stmane) Date: 2008-02-22 13:47 Message: Logged In: YES user_id=572415 Originator: NO File Added: Error3.png ---------------------------------------------------------------------- Comment By: Stefan Manegold (stmane) Date: 2008-02-22 13:46 Message: Logged In: YES user_id=572415 Originator: NO File Added: Error2.png ---------------------------------------------------------------------- Comment By: Stefan Manegold (stmane) Date: 2008-02-22 13:45 Message: Logged In: YES user_id=572415 Originator: NO Jennie, I currently only have the latest Stable (0.22) but not the former one (0.20) running. With the latest Stable, I get the following (virgin DB with 1 document loaded): - first click on "List Collections": No reaction - second click on "List Collections": Attached Error1.png - first click on "List All Documents": No reaction - second click on "List All Documents": Attached Error2.png - click on "Backup Database", fill in "Backup ID", click "Backup": Attached Error3.png ... I'm afraid, there is major need for testing this functionality --- preferably even automated (nightly!) testing, or mandatory "by hand" testing by a confirmed maintainer before each future release ... File Added: Error1.png ---------------------------------------------------------------------- Comment By: Ying Zhang (yingying) Date: 2008-02-22 12:57 Message: Logged In: YES user_id=341633 Originator: NO Sure, I'm looking at this problem. On my mac, I get neigher results nor error :( Stefan, if you use the admin page, will you get similar error? Jennie ---------------------------------------------------------------------- Comment By: Stefan Manegold (stmane) Date: 2008-02-22 07:14 Message: Logged In: YES user_id=572415 Originator: NO Jennie, could you please have a look at this one (possibly together with Peter)? In particular the backup functionality in MonetDB/XQuery 0.20 (and restore in 0.22) is required to keep existing databases when upgrading from 0.20 to 0.22. Thanks! Stefan ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=482468&aid=1899258&group_id=56967 ------------------------------------------------------------------------------ Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ Monetdb-bugs mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/monetdb-bugs
