Bugs item #2859986, was opened at 2009-09-16 15:24 Message generated for change (Comment added) made by yingying You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=482468&aid=2859986&group_id=56967
Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. >Category: PF/runtime >Group: Pathfinder "stable" Status: Open Resolution: None Priority: 5 Private: No Submitted By: Axel Belinfante (axelbel) >Assigned to: Ying Zhang (yingying) Summary: xrpc use messes up mclient use Initial Comment: doing an xquery via mclient works immediately after starting Monetdb. doing then a couple of xrpc requests somehow messes up some state in the server, such that then doing the same xquery request via mclient once more yields an ERROR result, or an empty result. I have seen cases where at some point mclient continuously returns ERROR for all xquery queries, even for simple ones like <p>aap</p> . in other cases I just get an empty result. it is easy to reproduce on our system. but turned out to be harder to isolate such that it still is reproducable. I hope I succeeded in that. server: Aug2009 Monetdb release, running on (uname -a output): Linux ewi865 2.6.16.21-0.8-smp #1 SMP Mon Jul 3 18:25:39 UTC 2006 x86_64 x86_64 x86_64 GNU/Linux ---------------------------------------------------------------------- >Comment By: Ying Zhang (yingying) Date: 2009-09-25 14:20 Message: Hello Axel, I have run your queries, and I can reproduce it (the xrpc request messages were first translated into xquery queries with xrpc calls). I think it is a bug in the function cache in MPS. Function cache in pathfinder: in the (old) MPS front-end, the whole module file is compiled, when it is 'import'-ed by a query. The compiler output of each function (i.e., the MIL-tree) in the module file is cached, so that a subsequent call to a function in the same module could be compiled (much) faster. So, in an Mserver session, each module file is only compiled once (inline, user-defined-functions are not cached). Function cache only works with MPS front-end, not with the (new) algebra front-end. For all functions called over XRPC, function cache is used. For normal queries (i.e., without any xrpc statement), function cache is used, if we can detect that the query contains a *single* call to a predefined function, with *only* simple literal paramters (i.e., each parameter is a single atomic-value). In Pathfinder runtime, we always check if function cache could be used, even if mclient has been started to use the algebra front-end. Once we found out that a query could be handeled with cached function, we automatically switch to the MPS front-end. And this is what has happend to your mclient-query06.xq, since it calls one module function with a single string parameter. Hence, both your normal query and your xrpc queries use the same cached MIL-trees, and most probably, the problem is caused by a not completely/correctly cleaned-up environment. I don't know how to fix this problem yet. I only have some suggestions as workaround of this problem. I hope they could be helpful for you, so that you could continue your work. Basically, what we want to do here, is to prevent pathfinder from using function cache for your normal query. This could be done in different ways (I have tried both and it seems working): 1. do not use module function in your normal query, but directly inline the function bodies in the query. For instance, I have replaced function calls in mclient-query06.xq as the following: $ cat mclient-query06.xq (: import module namespace ecv = "http://www.cs.utwente.nl/~belinfan/ecv"; at "/ufs/zhang/tmp/axelbel/ECVtest/xrpc/export/dyn/repo/dynxquery/axel/zambon/0"; ecv:view-from-save("eprints-export-conv-fmt.xml") :) let $eps := for $ep in doc("eprints-export-conv-fmt.xml")//tuple where ( ( ( ( some $c in $ep//creators_family satisfies contains(upper-case($c), "ZAMBON")) ) ) ) return $ep return <div>{ <div class="items">{$eps//fmt/div}</div> }</div> advantage: query compilation time is short (~30ms with MPS, ~20ms with algebra), as no unused functions are compiled. disadvantage: query maintainance, as you must update the normal query, if the corresponding functions in the module file have been changed. 2. this is very hacky: pass more complex parameter to the function call, so that it cannot be detected by the pathfinder runtime, as for which function cache could be used. This could be done very easily, for example: import module namespace ecv = "http://www.cs.utwente.nl/~belinfan/ecv"; at "/ufs/zhang/tmp/axelbel/ECVtest/xrpc/export/dyn/repo/dynxquery/axel/zambon/0"; ecv:view-from-save( ("eprints-export-conv-fmt.xml") ) ^ ^ Note the extra parentheses, now the parameter has become a sequence containing one string value. It is still valid, but will prevent the pathfinder runtime from using function cache :P advantage: ease of maintainance, minimal change of code. disadvantage: long compile time! For each normal query, the complete module must be load and compiled. With algebra, this costs ~300ms; with MPS, ~700ms. Would you please let us know if you can live with any of my suggestions, so that we can increase the priority of this bug accordingly? Regards, Jennie ---------------------------------------------------------------------- Comment By: Ying Zhang (yingying) Date: 2009-09-17 18:10 Message: Hello Axel, Thank you very much for reporting this problem. In the very beginning when xrpc was implemented, we have notices that there are situations, in which xrpc conflicts with mclient. We have tried to solve the cases we have noticed, but we could have missed some cases. I will look into your problem soon and post more information here, as soon as I know more. Regards, Jennie ---------------------------------------------------------------------- Comment By: Axel Belinfante (axelbel) Date: 2009-09-17 10:11 Message: I wrote: "however, if I then do the same request via xrpc, by repeating the third xrpc request, I do get a correct result..." this is not always the case. sometimes then such xrpc request returns an empty result ( <div><div class="items"/></div> ). Sometimes when the mclient request fails, it returns 'ERROR', together with almost the correct result: the result is correct, but the first couple of characters are missing (overwritten by the ERROR?). Finally, I don't know whether it is related to this issue, it happens that the server gets into a state where a pf:add-doc that is done via mclient just hangs, forever, until I kill the server to restart it. This issue was already present in the May-2009 release, and is still present in the Aug-2009 release. ---------------------------------------------------------------------- Comment By: Axel Belinfante (axelbel) Date: 2009-09-16 15:42 Message: I atdded the files that I used to set up the database at http://library.cs.utwente.nl/ecv/files-for-bugreport-2859986.tgz (size: 22861872 bytes) the file start-mxq-test.mil in the .tgz shows how I initialized the database. I invoked the xrpc commands by using telnet from within an xterm, and pasting the contents of the attached files. I invoked mclient as: mclient -lx -p myportnumber mclient-query06 so, when I cleanup the database 'var' directory, and restart it the mclient query works fine after I have done the first xrpc query, the mclient query still works fine after I have done the second xrpc query, the mclient query still works fine after I have done the third xrpc query, the mclient query returns an 'empty' result ( <div><div class="items"/></div> ) however, if I then do the same request via xrpc, by repeating the third xrpc request, I do get a correct result... ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=482468&aid=2859986&group_id=56967 ------------------------------------------------------------------------------ Come build with us! The BlackBerry® Developer Conference in SF, CA is the only developer event you need to attend this year. Jumpstart your developing skills, take BlackBerry mobile applications to market and stay ahead of the curve. Join us from November 9-12, 2009. Register now! http://p.sf.net/sfu/devconf _______________________________________________ Monetdb-bugs mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/monetdb-bugs
