Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/new-jsoniq into lp:zorba
Ghislain, I've fixed the null/true/false etc and select issues (points 5 and 8). -- -- https://code.launchpad.net/~zorba-coders/zorba/new-jsoniq/+merge/183640 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/new-jsoniq into lp:zorba
Review: Approve -- https://code.launchpad.net/~zorba-coders/zorba/new-jsoniq/+merge/183640 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/new-jsoniq into lp:zorba
1. Parser.y, line 2259: is || (block != NULL block-isEmpty()) really needed? Will block not always be null if a BlockExpr is {} (see StatementsAndOptionalExpr nonterminal)? Nope, you can have a nested empty block {{}}. 2. Why is the OBJECT token treated specially and not mentioned in GeneralizedAtomicType like array/item/structured-item? Note that jn:object is no longer a function, as it was removed (I thought Markos removed it from Zorba, too). Then object should also be moved back to invalid function names. Suggested test: object() throws a parsing error. I was not aware the function was removed. I've moved the token. 3. VersionDecl should not allow xquery in the JSONiq parser. Now, it seems to be allowed. A JSONiq query version declaration must be jsoniq (if a file begins with xquery the XQuery parser will be used anyway, so there is no way to test). Yes, it makes sense. Fixed. 4. Why is append not authorized as a function? I thought it would be together with insert/rename/replace/delete. Suggested test: local definition for local:append + default function namespace and query append(). Fixed, but the function call can only be made through the prefixed form, because in jsoniq you can have: append () into ... 5. From instead of For is implemented, but where is select instead of return? (this might require discussion about getting rid of from, actually. I am not sure if the from/select decision was not reverted at some point.) Suggested test: from $i in 1 to 10 select $i select is handled in the scanner. There already are plenty of tests in zorba/jsoniq. Though I think select handling should be moved from the scanner to the parser if we are to allow it as a qname/function name. 6. Can you give me details and examples about the two new conflicts? Which two new conflicts? The conflicts count has not changed. 7. How is [[]] array handled? Is it parsed as a predicate with an array constructor and handled in the translator? Is it actually handled? Quite interestingly this: let $var := [[1, 2, 3]] return $var[[]] segfaults in the current trunk. 8. TRUE/FALSE/NULL/FROM(/SELECT?) should appear in the FUNCTION_NAME rule in a JSONIQ_PARSER #ifdef so that they can be used as function names. Or is this handled somewhere else? Suggested test: false(), true(), null() -- from() with a local definition (see append above). It is not possible to add them to the qnames because they would completely interfere with the path expressions. One way to go around it would be to remove the keywords from the Literals production and then check the value of the RelativePathExprs in the rule action. It might work, but I would have to test it. -- https://code.launchpad.net/~zorba-coders/zorba/new-jsoniq/+merge/183640 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/new-jsoniq into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/new-jsoniq into lp:zorba has been updated. Status: Needs review = Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/new-jsoniq/+merge/183479 -- https://code.launchpad.net/~zorba-coders/zorba/new-jsoniq/+merge/183479 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~nbrinza/zorba/trunk-test into lp:zorba
The proposal to merge lp:~nbrinza/zorba/trunk-test into lp:zorba has been updated. Status: Needs review = Approved For more details, see: https://code.launchpad.net/~nbrinza/zorba/trunk-test/+merge/183495 -- https://code.launchpad.net/~nbrinza/zorba/trunk-test/+merge/183495 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~nbrinza/zorba/trunk-test into lp:zorba
Nicolae Brinza has proposed merging lp:~nbrinza/zorba/trunk-test into lp:zorba. Commit message: Test merge. Requested reviews: Nicolae Brinza (nbrinza) For more details, see: https://code.launchpad.net/~nbrinza/zorba/trunk-test/+merge/183495 Test merge. -- https://code.launchpad.net/~nbrinza/zorba/trunk-test/+merge/183495 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~nbrinza/zorba/trunk-test into lp:zorba
The proposal to merge lp:~nbrinza/zorba/trunk-test into lp:zorba has been updated. Status: Merged = Rejected For more details, see: https://code.launchpad.net/~nbrinza/zorba/trunk-test/+merge/183495 -- https://code.launchpad.net/~nbrinza/zorba/trunk-test/+merge/183495 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~nbrinza/zorba/trunk-test2 into lp:zorba
The proposal to merge lp:~nbrinza/zorba/trunk-test2 into lp:zorba has been updated. Status: Needs review = Approved For more details, see: https://code.launchpad.net/~nbrinza/zorba/trunk-test2/+merge/183504 -- https://code.launchpad.net/~nbrinza/zorba/trunk-test2/+merge/183504 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~nbrinza/zorba/trunk-test2 into lp:zorba
Nicolae Brinza has proposed merging lp:~nbrinza/zorba/trunk-test2 into lp:zorba. Commit message: Test merge. Requested reviews: Nicolae Brinza (nbrinza) For more details, see: https://code.launchpad.net/~nbrinza/zorba/trunk-test2/+merge/183504 Test merge. -- https://code.launchpad.net/~nbrinza/zorba/trunk-test2/+merge/183504 Your team Zorba Coders is subscribed to branch lp:zorba. === modified file 'ChangeLog' --- ChangeLog 2013-08-21 10:25:42 + +++ ChangeLog 2013-09-02 16:36:07 + @@ -1,5 +1,6 @@ Zorba - The XQuery Processor +#test version 3.0 -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~nbrinza/zorba/trunk-test2 into lp:zorba
The proposal to merge lp:~nbrinza/zorba/trunk-test2 into lp:zorba has been updated. Status: Approved = Rejected For more details, see: https://code.launchpad.net/~nbrinza/zorba/trunk-test2/+merge/183504 -- https://code.launchpad.net/~nbrinza/zorba/trunk-test2/+merge/183504 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~nbrinza/zorba/trunk-test2 into lp:zorba
The proposal to merge lp:~nbrinza/zorba/trunk-test2 into lp:zorba has been updated. Status: Rejected = Approved For more details, see: https://code.launchpad.net/~nbrinza/zorba/trunk-test2/+merge/183504 -- https://code.launchpad.net/~nbrinza/zorba/trunk-test2/+merge/183504 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~nbrinza/zorba/trunk-test2 into lp:zorba
The proposal to merge lp:~nbrinza/zorba/trunk-test2 into lp:zorba has been updated. Status: Approved = Rejected For more details, see: https://code.launchpad.net/~nbrinza/zorba/trunk-test2/+merge/183504 -- https://code.launchpad.net/~nbrinza/zorba/trunk-test2/+merge/183504 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~nbrinza/zorba/error-messages into lp:zorba
The proposal to merge lp:~nbrinza/zorba/error-messages into lp:zorba has been updated. Commit Message changed to: Fixed bugs #1210320 and #1212423 - misplaced compiler error For more details, see: https://code.launchpad.net/~nbrinza/zorba/error-messages/+merge/181327 -- https://code.launchpad.net/~nbrinza/zorba/error-messages/+merge/181327 Your team Zorba Coders is requested to review the proposed merge of lp:~nbrinza/zorba/error-messages into lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/use-dataguide into lp:zorba
Paul, I've also addressed your comments on the merge proposal. -- -- https://code.launchpad.net/~zorba-coders/zorba/use-dataguide/+merge/176385 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/use-dataguide into lp:zorba
I have fixed issues 1 through 3. Regarding point 4: your approach, if I understood it correctly, will only build the dataflow information, but not the dataguide itself. To compute it, it would require an additional pass through the expression tree and an additional data structure. Regarding the copying in the current code -- first of all, the expr class has a dataguide_cb_t which is only an rchandle, so most of copying you see is just copying of pointers. In cases where data from json sources cannot reach expressions, just NULL pointers are propagated. The operation that is costly in the current code is the cloning of dataguides, which is done in a clone-on-modify fashion. That portion could be improved through saving only incremental information instead of cloning and modifying an entire dataguide. -- -- https://code.launchpad.net/~zorba-coders/zorba/use-dataguide/+merge/176385 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/skip-items into lp:zorba
I've fixed the crash and added the query as zorba/eval/eval16.xq. -- -- https://code.launchpad.net/~zorba-coders/zorba/skip-items/+merge/174723 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/use-dataguide into lp:zorba
I don't understand why. In my example it wouldn't. The count functions could count the number of empty objects or objects that contain only the STREET field. I've looked into the test and here are the issues: 1) The fn:count() does not have the %explores-json annotation. I've confused it with the JSONiq array size() function. 2) Currently the dataguide does not distinguish between any of the fn: functions. So for example if you modify the query: for $obj in parse-json(f:read-text(fn:resolve-uri(citylots-small.json))) group by $s := $obj.STREET return { street : $s, count : count($obj), seq : subsequence($obj, 1, 100) } then if you prune the input to have only the .STREET field you'll get an incorrect result. So currently the argument to subsequence() is explored and so the dataguide is * in this case. The same happens with count(). 3) fn:count() is a special case as it only cares for the number of objects and not their contents. I think another annotation could be added, which would be the opposite of %explores-json and which would indicate that the function ignores the details of the object. Or I could just add some code in the dataguide collection to handle fn:count() separately. -- -- https://code.launchpad.net/~zorba-coders/zorba/use-dataguide/+merge/176385 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/use-dataguide into lp:zorba
Ok, it wasn't too much work -- I've added a special handler for fn:count() in the dataguide code and now the example you gave prunes all the fields except price. -- -- https://code.launchpad.net/~zorba-coders/zorba/use-dataguide/+merge/176385 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/use-dataguide into lp:zorba
I meant the STREET field. I've used price in the testcase. -- https://code.launchpad.net/~zorba-coders/zorba/use-dataguide/+merge/176385 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/use-dataguide into lp:zorba
I have no idea what problem this MP is supposed to solve; nor do I know what a data guide is. Paul, I've made some changes to the JSON loader so that it skips creating nodes that are not in a given template (== dataguide). Since you've written the loader, could you please review only the changes to it and let me know if you find any problems, issues or anything else? -- -- https://code.launchpad.net/~zorba-coders/zorba/use-dataguide/+merge/176385 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/use-dataguide into lp:zorba
The way I see this is not a problem with the fn functions. The problem is related to the fact how the objects are used. In this case, the objects are used to construct a new object. The dataguide needs to handle that. Similar to serialization, this means that the entire object is needed. Yes, this is being handled and the usage of objects is being tracked. The process() function in the dataguide computation has a parameter for that -- propagates_to_output. You've actually seen it in action -- it is pessimistic and assumes all fn: functions propagate their input to the output. That is why the parameter to count() was assumed to be used and you've seen no pruning. I've now made fn:count() a special case and it's input is no longer considered used for output and its parameter is allowed to be pruned. All the other functions are handled as if they propagate their input and skipping is allowed only where it's indeed possible. -- -- https://code.launchpad.net/~zorba-coders/zorba/use-dataguide/+merge/176385 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/use-dataguide into lp:zorba
I have tried some basic queries and there is a huge performance improvement. This is great. However, I detected a memory leak and have one question. - memory leak in translator.cpp:4823 I've fixed it. - In the following query, no dataguide seems to be pushed into the parser. Why? The count() function is marked with the %explores-json annotation, because pruning objects that reach the function will modify the returned result. That is why the dataguide is empty. -- -- https://code.launchpad.net/~zorba-coders/zorba/use-dataguide/+merge/176385 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/skip-items into lp:zorba
- In ZorbaCollectionIterator::initCollection lines 307 and 326, you eventually add skipCount twice. Something smells spooky here. Yes, it seems it was added twice. I've fixed it. I've also added the skip() function to the EvalIterator. -- -- https://code.launchpad.net/~zorba-coders/zorba/skip-items/+merge/174723 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/skip-items into lp:zorba
I've looked into the index probe iterators but they already optimize any skipping. They take a Skip parameter and then they push it into an underlying iterator e.g. ProbeValueTreeIndexIterator which handles the skip internally. So there is nothing to be done there. I've pushed the pending fixes and the branch is ready for merging. -- https://code.launchpad.net/~zorba-coders/zorba/skip-items/+merge/174723 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/dataguide into lp:zorba
I've done some additional testing, and these are the results: For the xray query, the largest that we have in the testsuite, compilation time with --compile-only is pretty much the same with and without the dataguide computaiton, at around ~0.08 sec. With a specially constructed query that looks like this: (see dataguide-29.jq test) let $col := dml:collection() let $col2 := ($col.cat1, $col.cat2, ... , $col.cat10) return $col2.category.category.category ... category (repeated ~2000 times) the compilation time goes from ~0.7s without the dataguide to ~10s with the dataguide enabled, so it is significant. But this is a worst-case scenario. The resulting dataguide is an object 2000-levels deep. The compilation can be improved significantly by: - keeping track of the leaves nodes in the dataguide tree - rewriting a bit the dataguide structure to store the trees incrementally instead of cloning them - adding a depth cutoff What do you think? -- https://code.launchpad.net/~zorba-coders/zorba/dataguide/+merge/173026 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/use-dataguide into lp:zorba
Review: Approve -- https://code.launchpad.net/~zorba-coders/zorba/use-dataguide/+merge/176385 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/skip-items into lp:zorba
- Shouldn't skip take an unsigned integer? The parameters to fn:subsequence and co are all signed. E.g. you can pass a negative skip. - Why is the while loop in collections_impl.cpp:397 necessary if initCollection is called before? I think there might be a bug in initCollection. Specifically, the else block in line 298 is never called because the CollectionIterator is not rewritten anymore (i.e. theChildren.size() will always return one) You're very right. It was not really a bug, but the skip optimization didn't really happen because it was the while() that was doing the skipping instead of the getIterator(skip). I've fixed that. But the else block at line 298 cannot be deleted as the 3-parameter function was not only used by the rewriter, e.g. the collections/paging_1 to _5 use it. - state-theIteratorOpened == false = !state-theIteratorOpened Fixed. - could we implement and test skip for the EvalIterator as well? All iterators have the default implementation of skip in plan_iterator.cpp. How can we improve that implementation for EvalIterator? - What about the index probe iterators (e.g. ProbeIndexPointValueIterator), they also skip I was not aware they do. I'll write an optimization for them as well. -- -- https://code.launchpad.net/~zorba-coders/zorba/skip-items/+merge/174723 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/dataguide into lp:zorba
DataGuides serve as dynamic schemas, generated from the database. What we generate is a schema from the query. Still, it is a data schema, not a query schema. The one in the paper would be a Database DataGuide and ours would be Query DataGuide. I would agree to change it to QueryDataguide but I don't think there would be any confusions if it was simply called Dataguide. I think we will run into a problem. 28msec has only one buffer that is accessed by all db:collection() calls in a query. Hence, the information needs to be the union. If there is no way of removing that limitation then we can overcome this by doing an union on all db:collection() dataguides and this will ensure correctness. But it would be a pity to loose the individually computed dataguides for each separate call. Still, if the name of fields of different collections are mostly disjoint sets, then we won't loose much of the improvement. Again I suggest leaving this until I start implementing the push-down of projection info into the db:collection() calls. It has no impact on jn:parse() -- these dataguides can still be computed and kept individually for each call even if we do an union on db:collection() calls. -- -- https://code.launchpad.net/~zorba-coders/zorba/dataguide/+merge/173026 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/dataguide into lp:zorba
- I find the name dataguide misleading because it's a guide on the query and not on the data. Maybe QueryPruneGuide would be more meaningful The query itself is not pruned, the data is. I think dataguide is the established term -- see for example this paper: http://ilpubs.stanford.edu:8090/264/1/1997-50.pdf . - Can the user also use the zann_explores_json annotation? Yes, the users can use it as well. But does it make sense for them to use it? If they have an external function -- it is automatically handled as if it has the annotation. For a UDF it doesn't really make any sense to add it. - Why is the dataguide parameter on the Store's getCollection() function? Shouldn't it be on the function that returns the iterator? The problem is that a Collection object within the simplestore exists only once per collection. What's the semantics if multiple queries access the collection (possibly in parallel)? It very much depends on how the collections are handled. Currently for Zorba collections it doesn't make sense to have any dataguides at all, because they're in-memory collections. I have not taken a look at the Sausalito code and have not seen how e.g. the MongoDB collections are managed. getCollection() seemed the most logical place where it should be passed, but the dataguide parameter could be easily propagated to any Store class, including the function that returns the iterator. Currently each and every db:collection() call has its own dataguide, even if they might refer to the same collection. If the collection manager currently caches or reuses the collection iterators, then it might make sense to forbid that so that the dataguide for each individual db:collection call could be used. Or alternatively, an union on the dataguides that refer to the same collection could be performed. But I think it is not always possible to determine if that is the case. I think this could be investigated and decided upon when implementing the Dataguide push-down into MongoDB or when I would take a better look at the Sausalito's collection manager code. - Did you measure the performance impact of the optimizer on some larger queries? The expression tree is traversed in its entirety once and only once, visiting each node, so the performance should not be very different from any other dataflow computation, e.g. ignores sorts/order/etc. If there are no sources, i.e. db:collection() or jn:parse() calls, then the dataguide computation just propagates NULLs, doing no calculations and almost no memory allocations (at most one dataguide_cb allocation per fo_exprs and several others). If there are sources in the tree -- there will be some union operations being performed for some of the nodes. I will check if any of our larger queries have longer compilation times, but because none of them have db:collection() or jn:parse() calls, I do not expect any differences. It would make sense to have a specially constructed query that would do a stress-test of the dataguide code -- e.g. a db:collection().navigation.navigation. ... .navigation several thousand times or something similar. I will try that out and see if it manages to slow down the compilation. -- -- https://code.launchpad.net/~zorba-coders/zorba/dataguide/+merge/173026 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/skip-items into lp:zorba
Review: Approve -- https://code.launchpad.net/~zorba-coders/zorba/skip-items/+merge/174723 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/dataguide into lp:zorba
Review: Approve -- https://code.launchpad.net/~zorba-coders/zorba/dataguide/+merge/173026 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/skip-items into lp:zorba
Nicolae Brinza has proposed merging lp:~zorba-coders/zorba/skip-items into lp:zorba. Commit message: Implemented the Skip-items facility Requested reviews: Nicolae Brinza (nbrinza) For more details, see: https://code.launchpad.net/~zorba-coders/zorba/skip-items/+merge/174723 Implemented the Skip-items facility -- https://code.launchpad.net/~zorba-coders/zorba/skip-items/+merge/174723 Your team Zorba Coders is subscribed to branch lp:zorba. === modified file 'src/functions/func_sequences_impl.cpp' --- src/functions/func_sequences_impl.cpp 2013-05-14 05:21:11 + +++ src/functions/func_sequences_impl.cpp 2013-07-15 10:38:33 + @@ -43,127 +43,6 @@ /*** - -/ -bool rewriteSubsequenceCollection( -static_context* aSctx, -const QueryLoc aLoc, -std::vectorPlanIter_t aArgs, -bool aIsIntSubsequence) -{ - ZorbaCollectionIterator* collIter = - dynamic_castZorbaCollectionIterator*(aArgs[0].getp()); - assert(collIter); - - std::vectorPlanIter_t lCollectionArgs = collIter-getChildren(); - - SingletonIterator* lPosIter = dynamic_castSingletonIterator*(aArgs[1].getp()); - if (lPosIter == NULL) - { -return false; - } - - xs_long pos; - const store::Item_t lPosItem = lPosIter-getValue(); - - try - { -if (aIsIntSubsequence) -{ - pos = lPosItem-getLongValue(); -} -else -{ - xs_double dpos = lPosItem-getDoubleValue().round(); - xs_integer ipos(dpos.getNumber()); - pos = to_xs_long(ipos); -} - } - catch (std::exception const) - { -return false; - } - - if (pos = 1) - { -// if the start position is less than 1 we can't push this down into -// the collection skip parameter because the result won't be equivalent. -return false; - } - - // prepare helper - store::Item_t lItemOne; - GENV_ITEMFACTORY-createInteger(lItemOne, Integer(1)); - - int lNumCollArgs = lCollectionArgs.size(); - if (lNumCollArgs == 1) - { -// argument is of type collection(qname) -// simply move the (pos-1) of subsequence into the skip of -// collection function -// subsequence(collection(qname), 10, 20) -// - subsequence(collection(qname, 10-1), 1, 20) -PlanIter_t lNewCollSkipIter = aArgs[1]; -PlanIter_t lOneIter = new SingletonIterator(aSctx, aLoc, lItemOne); - -lCollectionArgs.push_back( - new NumArithIteratorzorba::SubtractOperation(aSctx, - collIter-getLocation(), - lNewCollSkipIter, - lOneIter) - ); - } - else if (lNumCollArgs = 3 lNumCollArgs != 0) - { -// argument is of type collection(qname,skip) or -// collection(qname,start_uri,skip) -int lSkipPosition = 1; -if (lNumCollArgs == 3) -{ - // collection function with start reference - skip is the 3rd param - lSkipPosition = 2; -} - -// add position-1 of subsequence to collection skip -// subsequence(collection(qname, 10), 10, 20) -// - subsequence(collection(qname, 10+10-1), 10, 20) -PlanIter_t lOldCollSkipIter = lCollectionArgs[lSkipPosition]; -PlanIter_t lOneIter = new SingletonIterator (aSctx, aLoc, lItemOne); -PlanIter_t lSubseqPosIter = aArgs[1]; - -PlanIter_t lCollSkipAdditionIter = -new NumArithIteratorzorba::SubtractOperation(aSctx, - collIter-getLocation(), - lSubseqPosIter, - lOneIter); -lCollectionArgs[lSkipPosition] = -new NumArithIteratorzorba::AddOperation(aSctx, - collIter-getLocation(), - lOldCollSkipIter, - lCollSkipAdditionIter); - } - else - { -// no collection function with 0 or 3 params -assert(false); - } - - aArgs[0] = new ZorbaCollectionIterator(aSctx, - collIter-getLocation(), - lCollectionArgs, - collIter-isDynamic()); - - // after pushing the position param down we need to rewrite the actual - // position to 1: - // subsequence(collection(qname, 10+10-1), 10, 20) - // - subsequence(collection(qname, 10+10-1), 1, 20) - aArgs[1] = new SingletonIterator(aSctx, aLoc, lItemOne); - - return true; -} - - -/*** / xqtref_t fn_unordered::getReturnType(const fo_expr* caller) const { @@ -573,8 +452,7 @@ const QueryLoc aLoc
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module
The proposal to merge lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module has been updated. Status: Needs review = Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/171824 -- https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/171824 Your team Zorba Coders is subscribed to branch lp:zorba/process-module. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/dataguide into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/dataguide into lp:zorba has been updated. Status: Needs review = Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/dataguide/+merge/173026 -- https://code.launchpad.net/~zorba-coders/zorba/dataguide/+merge/173026 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module
The proposal to merge lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module has been updated. Commit Message changed to: Changed module's errors to the new modules guidelines; Potential fix for execvpe() on Macs For more details, see: https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/171824 -- https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/171824 Your team Zorba Coders is subscribed to branch lp:zorba/process-module. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module
Nicolae Brinza has proposed merging lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module. Commit message: Changed module's errors to the new modules guidelines; Potential fix for execvpe() on Macs Requested reviews: Nicolae Brinza (nbrinza) Ghislain Fourny (gislenius) Related bugs: Bug #1188053 in Zorba: Update non-core module process https://bugs.launchpad.net/zorba/+bug/1188053 For more details, see: https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/171824 Changed module's errors to the new modules guidelines; Potential fix for execvpe() on Macs -- https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/171824 Your team Zorba Coders is subscribed to branch lp:zorba/process-module. === modified file 'src/process-2.xq' --- src/process-2.xq 2013-06-15 19:46:05 + +++ src/process-2.xq 2013-06-27 14:46:36 + @@ -26,8 +26,8 @@ : p : Example: : pre - : import module namespace proc = http://zorba.io/modules/process;; - : proc:exec(ls) + : import module namespace p = http://zorba.io/modules/process;; + : p:exec(ls) : /pre : /p : @@ -86,7 +86,7 @@ : : @return the result of the execution as an object : - : @error p:PROC01 if an error occurred while communicating with the process. + : @error p:COMMUNICATION if an error occurred while communicating with the process. :) declare %an:sequential function p:exec( $filename as string @@ -113,7 +113,7 @@ : : @return the result of the execution as an object : - : @error p:PROC01 if an error occurred while communicating with the process. + : @error p:COMMUNICATION if an error occurred while communicating with the process. :) declare %an:sequential function p:exec( $filename as string, @@ -147,7 +147,7 @@ : : @return the result of the execution as an object : - : @error p:PROC01 if an error occurred while communicating with the process. + : @error p:COMMUNICATION if an error occurred while communicating with the process. :) declare %an:sequential function p:exec( $filename as string, @@ -168,7 +168,7 @@ : : @return the result of the execution as an object : - : @error p:PROC01 if an error occurred while communicating with the process. + : @error p:COMMUNICATION if an error occurred while communicating with the process. :) declare %an:sequential function p:exec-command( $cmd as string @@ -189,7 +189,7 @@ : : @return the result of the execution as an object : - : @error p:PROC01 if an error occurred while communicating with the process. + : @error p:COMMUNICATION if an error occurred while communicating with the process. :) declare %an:sequential function p:exec-command( $cmd as string, === modified file 'src/process-2.xq.src/process.cpp' --- src/process-2.xq.src/process.cpp 2013-06-21 03:37:04 + +++ src/process-2.xq.src/process.cpp 2013-06-27 14:46:36 + @@ -51,12 +51,21 @@ #ifndef WIN32 int execvpe(const char *program, char **argv, char **envp) { + TREE extern char **environ; char **saved = environ; int rc; environ = envp; rc = execvp(program, argv); environ = saved; +=== + clearenv(); + int i = 0; + while (envp[i] != NULL) +putenv(envp[i++]); + + int rc = execvp(program, argv); + MERGE-SOURCE return rc; } #endif @@ -105,8 +114,7 @@ wsprintf(lErrorBuffer,TEXT(Process Error Code: %d - Message= %s),GetLastError(), (TCHAR *)lpvMessageBuffer); LocalFree(lpvMessageBuffer); Item lQName = ProcessModule::getItemFactory()-createQName( -http://www.zorba-xquery.com/modules/process;, -PROC01); +http://zorba.io/modules/process;, COMMUNICATION); #ifdef UNICODE char error_str[1024]; WideCharToMultiByte(CP_UTF8, 0, lErrorBuffer, -1, error_str, sizeof(error_str), NULL, NULL); @@ -239,7 +247,7 @@ || !CreatePipe(lErrRead,lStdErr,lSecurityAttributes,1024*1024) // std::cerr lErrRead ){ Item lQName = ProcessModule::getItemFactory()-createQName( - http://www.zorba-xquery.com/modules/process;, PROC01); + http://zorba.io/modules/process;, COMMUNICATION); throw USER_EXCEPTION(lQName, Couldn't create one of std::cout/std::cerr pipe for child process execution. ); @@ -261,7 +269,7 @@ lErrorMsg Couldn't get exit code from child process. Executed command: ' aCommand '.; Item lQName = ProcessModule::getItemFactory()-createQName( -http://www.zorba-xquery.com/modules/process;, PROC01); +http://zorba.io/modules/process;, COMMUNICATION); throw USER_EXCEPTION(lQName, lErrorMsg.str().c_str()); } @@ -451,7 +459,7 @@ std::stringstream lErrorMsg; lErrorMsg Failed to execute the command ( code ); Item lQName = ProcessModule::getItemFactory()-createQName( - http://www.zorba-xquery.com/modules/process;, PROC01); + http://zorba.io/modules/process;, COMMUNICATION); throw USER_EXCEPTION(lQName, lErrorMsg.str().c_str()); } exit_code = code; @@ -489,7 +497,7 @@ std
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module
Review: Approve -- https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/171824 Your team Zorba Coders is subscribed to branch lp:zorba/process-module. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module
The proposal to merge lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module has been updated. Status: Needs review = Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/171824 -- https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/171824 Your team Zorba Coders is subscribed to branch lp:zorba/process-module. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module
The proposal to merge lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module has been updated. Status: Needs review = Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/171824 -- https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/171824 Your team Zorba Coders is subscribed to branch lp:zorba/process-module. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module
The proposal to merge lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module has been updated. Status: Needs review = Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/164415 -- https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/164415 Your team Zorba Coders is subscribed to branch lp:zorba/process-module. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module
The proposal to merge lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module has been updated. Status: Needs review = Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/164415 -- https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/164415 Your team Zorba Coders is subscribed to branch lp:zorba/process-module. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module
Review: Approve -- https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/164415 Your team Zorba Coders is subscribed to branch lp:zorba/process-module. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module
I've merged Paul's change -- it was only a one line change. -- https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/164415 Your team Zorba Coders is subscribed to branch lp:zorba/process-module. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/new-jsoniq into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/new-jsoniq into lp:zorba has been updated. Status: Needs review = Rejected For more details, see: https://code.launchpad.net/~zorba-coders/zorba/new-jsoniq/+merge/167907 -- https://code.launchpad.net/~zorba-coders/zorba/new-jsoniq/+merge/167907 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/new-jsoniq into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/new-jsoniq into lp:zorba has been updated. Status: Needs review = Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/new-jsoniq/+merge/168471 -- https://code.launchpad.net/~zorba-coders/zorba/new-jsoniq/+merge/168471 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/new-jsoniq into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/new-jsoniq into lp:zorba has been updated. Status: Needs review = Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/new-jsoniq/+merge/168471 -- https://code.launchpad.net/~zorba-coders/zorba/new-jsoniq/+merge/168471 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/new-jsoniq into lp:zorba
Review: Approve -- https://code.launchpad.net/~zorba-coders/zorba/new-jsoniq/+merge/168471 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/new-jsoniq into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/new-jsoniq into lp:zorba has been updated. Status: Needs review = Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/new-jsoniq/+merge/168471 -- https://code.launchpad.net/~zorba-coders/zorba/new-jsoniq/+merge/168471 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/new-jsoniq into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/new-jsoniq into lp:zorba has been updated. Status: Needs review = Rejected For more details, see: https://code.launchpad.net/~zorba-coders/zorba/new-jsoniq/+merge/167824 -- https://code.launchpad.net/~zorba-coders/zorba/new-jsoniq/+merge/167824 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/new-jsoniq into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/new-jsoniq into lp:zorba has been updated. Status: Needs review = Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/new-jsoniq/+merge/167907 -- https://code.launchpad.net/~zorba-coders/zorba/new-jsoniq/+merge/167907 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/array-unboxing into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/array-unboxing into lp:zorba has been updated. Status: Needs review = Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/array-unboxing/+merge/168099 -- https://code.launchpad.net/~zorba-coders/zorba/array-unboxing/+merge/168099 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/new-jsoniq into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/new-jsoniq into lp:zorba has been updated. Status: Needs review = Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/new-jsoniq/+merge/167824 -- https://code.launchpad.net/~zorba-coders/zorba/new-jsoniq/+merge/167824 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/new-jsoniq into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/new-jsoniq into lp:zorba has been updated. Status: Needs review = Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/new-jsoniq/+merge/167824 -- https://code.launchpad.net/~zorba-coders/zorba/new-jsoniq/+merge/167824 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/new-jsoniq into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/new-jsoniq into lp:zorba has been updated. Status: Needs review = Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/new-jsoniq/+merge/167824 -- https://code.launchpad.net/~zorba-coders/zorba/new-jsoniq/+merge/167824 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module
For some reason, the commits to my checked-out branch are not pushed automatically, and I forgot to do it manually. It is pushed now. I've also fixed all issues you reported except for the namespace change to zorba.io. As far as I understood, it will be done for all modules at once -- it is better than changing the namespace for one module at random, so I think it can be merged now with the current namespace. -- https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/164415 Your team Zorba Coders is subscribed to branch lp:zorba/process-module. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/fix_bug_1182910 into lp:zorba
Review: Approve -- https://code.launchpad.net/~zorba-coders/zorba/fix_bug_1182910/+merge/167369 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module
Matthias, They URI messages won't appear anymore, but I've done it by calling abort() instead of exit() in the child fork() in the process module, because: -- it will keep the Zorba URIs leaks messages in the Release build -- the exit code in the case of an error is lost anyway due to the thrown exception in Zorba -- it is not actually correct to deallocate the Zorba store twice (once in the child and the second time in the parent). It could lead to problems if store is not an in-memory store. I've added a lengthy comment in the code describing what happens and where the problem lies. If the error exit code could be sent to the main process (e.g. through another opened pipe) then it would work perfectly. -- https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/164415 Your team Zorba Coders is subscribed to branch lp:zorba/process-module. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module
The proposal to merge lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module has been updated. Status: Needs review = Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/164415 -- https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/164415 Your team Zorba Coders is subscribed to branch lp:zorba/process-module. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module
I've fixed all the issues except for the leaks problem. I have not removed the printing of the URIs because it would disable them for the entire Zorba engine. Even if the messages are removed there is still the problem of the exit code, which cannot be fixed easily now. -- -- https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/164415 Your team Zorba Coders is subscribed to branch lp:zorba/process-module. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module
Regarding the referenced URI messages -- these are indeed coming from the StringPool as leaked entries. When the main process is fork()'ed, the child process is basically a copy of the parent process with all the statically allocated objects duplicated. When the child process calls exit(), those objects are destroyed. Since Zorba doesn't go through a normal shutdown, the StringPool still contains those entries and it prints the error messages and even throws an error. As a result the exit code of the process that was executed is actually lost. Even if the StringPool errors are bypassed somehow, another error is raised in the Lock() class. This problem was present in 1.0 version as well. The root of the problem is the bad design of engine and store instantiation: they are static singletons which require explicit calls to shutdown. Some possible solutions are: 1) Call abort() instead of exit() -- this will bypass all the static object destruction but I will need to find a way to pass the exit code to the parent process. 2) Pass the engine and store pointers to the process module and down to the child process so that it can call shutdown on them. This has the added problem of shutting down the store twice -- it is not an issue for an in-memory store, but could be for other stores. -- https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/164415 Your team Zorba Coders is subscribed to branch lp:zorba/process-module. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module
Nicolae Brinza has proposed merging lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module. Commit message: Version 2.0 of the process module, allows running executables directly, without invoking bash/cmd.exe Requested reviews: Nicolae Brinza (nbrinza) Matthias Brantner (matthias-brantner) For more details, see: https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/164354 Version 2.0 of the process module, allows running executables directly, without invoking bash/cmd.exe -- https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/164354 Your team Zorba Coders is subscribed to branch lp:zorba/process-module. === modified file 'src/com/zorba-xquery/www/modules/CMakeLists.txt' --- src/com/zorba-xquery/www/modules/CMakeLists.txt 2011-07-01 09:24:09 + +++ src/com/zorba-xquery/www/modules/CMakeLists.txt 2013-05-17 12:29:26 + @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -DECLARE_ZORBA_MODULE (URI http://www.zorba-xquery.com/modules/process; VERSION 1.0 FILE process.xq) +DECLARE_ZORBA_MODULE (URI http://www.zorba-xquery.com/modules/process; VERSION 2.0 FILE process.xq) === modified file 'src/com/zorba-xquery/www/modules/process.xq' --- src/com/zorba-xquery/www/modules/process.xq 2012-12-06 02:13:28 + +++ src/com/zorba-xquery/www/modules/process.xq 2013-05-17 12:29:26 + @@ -28,14 +28,17 @@ : : Potential result: : pre class=ace-static ace-mode=xquery![CDATA[ - : result xmlns=http://www.zorba-xquery.com/modules/process; - : stdoutmyfile.txt/stout - : stderr/ - : exit-code0/exit-code - : /result + : { + : exit-code: 0, + : stdout: myfile.txt, + : stderr: + : } : ]]/pre : - : @author Cezar Andrei + : The exec-command() set of functions allows execution of commands through the operating + : system's command line interpreter, such as sh on Linux or cmd.exe on Windows. + : + : @author Cezar Andrei, Nicolae Brinza : @project Zorba/IO/Process : :) @@ -44,7 +47,100 @@ declare namespace an = http://www.zorba-xquery.com/annotations;; declare namespace ver = http://www.zorba-xquery.com/options/versioning;; -declare option ver:module-version 1.0; +declare option ver:module-version 2.0; + + +(:~ + : Executes the specified program in a separate process. + : This function does not allow arguments to be passed to + : the command. The $filename parameter can contain the full path to the + : executable. On Linux systems, if the specified filename does not contain + : a slash /, the function duplicates the actions of the shell in searching + : for an executable file. The file is sought in the colon-separated list of + : directory pathnames specified in the PATH environment variable. If this + : variable isn't defined, the path list defaults to the current directory + : followed by the list of directories returned by the operating system. + : + : @param $filename the name of program to be executed + : + : @return the result of the execution as an object as + : shown in the documentation of this module. The exit-code + : returns the exit code of the child process. + : For POSIX compliant platforms: returns the program exit code. If the program is + : terminated or stopped: 128 + termination signal code. + : For Windows platforms: returns the return value of the program or the exit + : or terminate process specified value. + : + : @error process:PROC01 if an error occurred while communicating + : with the executed process. + :) +declare %an:sequential function process:exec( + $filename as xs:string +) as object() external; + +(:~ + : Executes the specified program in a separate process. + : The $filename parameter can contain the full path to the + : executable. On Linux systems, if the specified filename does not contain + : a slash /, the function duplicates the actions of the shell in searching + : for an executable file. The file is sought in the colon-separated list of + : directory pathnames specified in the PATH environment variable. If this + : variable isn't defined, the path list defaults to the current directory + : followed by the list of directories returned by the operating system. + : The $args parameters will be passed to the executable file as arguments. + : + : @param $filename the name of program to be executed + : @param $args arguments to be passed to the executable + : + : @return the result of the execution as an object as + : shown in the documentation of this module. The exit-code + : returns the exit code of the child process. + : For POSIX compliant platforms: returns the program exit code. If the program is + : terminated or stopped: 128 + termination signal code. + : For Windows platforms: returns the return value of the program or the exit + : or terminate process specified value. + : + : @error process:PROC01 if an error occurred while communicating + : with the executed process
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module
The proposal to merge lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module has been updated. Status: Needs review = Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/164354 -- https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/164354 Your team Zorba Coders is subscribed to branch lp:zorba/process-module. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module
Review: Approve -- https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/164354 Your team Zorba Coders is subscribed to branch lp:zorba/process-module. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module
The proposal to merge lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module has been updated. Status: Needs review = Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/164354 -- https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/164354 Your team Zorba Coders is subscribed to branch lp:zorba/process-module. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module
The proposal to merge lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module has been updated. Status: Needs review = Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/164354 -- https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/164354 Your team Zorba Coders is subscribed to branch lp:zorba/process-module. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module
The proposal to merge lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module has been updated. Status: Needs review = Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/164354 -- https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/164354 Your team Zorba Coders is subscribed to branch lp:zorba/process-module. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module
The proposal to merge lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module has been updated. Status: Needs review = Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/164354 -- https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/164354 Your team Zorba Coders is subscribed to branch lp:zorba/process-module. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module
The proposal to merge lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module has been updated. Status: Needs review = Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/164354 -- https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/164354 Your team Zorba Coders is subscribed to branch lp:zorba/process-module. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module
The proposal to merge lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module has been updated. Status: Needs review = Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/164354 -- https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/164354 Your team Zorba Coders is subscribed to branch lp:zorba/process-module. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module
Review: Approve -- https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/164354 Your team Zorba Coders is subscribed to branch lp:zorba/process-module. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module
Nicolae Brinza has proposed merging lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module. Commit message: Version 2.0 of the process module, allows running executables directly, without invoking bash/cmd.exe Requested reviews: Nicolae Brinza (nbrinza) Matthias Brantner (matthias-brantner) For more details, see: https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/164415 Version 2.0 of the process module, allows running executables directly, without invoking bash/cmd.exe -- https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/164415 Your team Zorba Coders is subscribed to branch lp:zorba/process-module. === modified file 'src/com/zorba-xquery/www/modules/CMakeLists.txt' --- src/com/zorba-xquery/www/modules/CMakeLists.txt 2011-07-01 09:24:09 + +++ src/com/zorba-xquery/www/modules/CMakeLists.txt 2013-05-17 14:50:32 + @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -DECLARE_ZORBA_MODULE (URI http://www.zorba-xquery.com/modules/process; VERSION 1.0 FILE process.xq) +DECLARE_ZORBA_MODULE (URI http://www.zorba-xquery.com/modules/process; VERSION 2.0 FILE process.xq) === modified file 'src/com/zorba-xquery/www/modules/process.xq' --- src/com/zorba-xquery/www/modules/process.xq 2012-12-06 02:13:28 + +++ src/com/zorba-xquery/www/modules/process.xq 2013-05-17 14:50:32 + @@ -28,14 +28,17 @@ : : Potential result: : pre class=ace-static ace-mode=xquery![CDATA[ - : result xmlns=http://www.zorba-xquery.com/modules/process; - : stdoutmyfile.txt/stout - : stderr/ - : exit-code0/exit-code - : /result + : { + : exit-code: 0, + : stdout: myfile.txt, + : stderr: + : } : ]]/pre : - : @author Cezar Andrei + : The exec-command() set of functions allows execution of commands through the operating + : system's command line interpreter, such as sh on Linux or cmd.exe on Windows. + : + : @author Cezar Andrei, Nicolae Brinza : @project Zorba/IO/Process : :) @@ -44,7 +47,100 @@ declare namespace an = http://www.zorba-xquery.com/annotations;; declare namespace ver = http://www.zorba-xquery.com/options/versioning;; -declare option ver:module-version 1.0; +declare option ver:module-version 2.0; + + +(:~ + : Executes the specified program in a separate process. + : This function does not allow arguments to be passed to + : the command. The $filename parameter can contain the full path to the + : executable. On Linux systems, if the specified filename does not contain + : a slash /, the function duplicates the actions of the shell in searching + : for an executable file. The file is sought in the colon-separated list of + : directory pathnames specified in the PATH environment variable. If this + : variable isn't defined, the path list defaults to the current directory + : followed by the list of directories returned by the operating system. + : + : @param $filename the name of program to be executed + : + : @return the result of the execution as an object as + : shown in the documentation of this module. The exit-code + : returns the exit code of the child process. + : For POSIX compliant platforms: returns the program exit code. If the program is + : terminated or stopped: 128 + termination signal code. + : For Windows platforms: returns the return value of the program or the exit + : or terminate process specified value. + : + : @error process:PROC01 if an error occurred while communicating + : with the executed process. + :) +declare %an:sequential function process:exec( + $filename as xs:string +) as object() external; + +(:~ + : Executes the specified program in a separate process. + : The $filename parameter can contain the full path to the + : executable. On Linux systems, if the specified filename does not contain + : a slash /, the function duplicates the actions of the shell in searching + : for an executable file. The file is sought in the colon-separated list of + : directory pathnames specified in the PATH environment variable. If this + : variable isn't defined, the path list defaults to the current directory + : followed by the list of directories returned by the operating system. + : The $args parameters will be passed to the executable file as arguments. + : + : @param $filename the name of program to be executed + : @param $args arguments to be passed to the executable + : + : @return the result of the execution as an object as + : shown in the documentation of this module. The exit-code + : returns the exit code of the child process. + : For POSIX compliant platforms: returns the program exit code. If the program is + : terminated or stopped: 128 + termination signal code. + : For Windows platforms: returns the return value of the program or the exit + : or terminate process specified value. + : + : @error process:PROC01 if an error occurred while communicating + : with the executed process
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module
The proposal to merge lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module has been updated. Commit Message changed to: Version 2.0 of the process module, allows running executables directly, without invoking bash/cmd.exe For more details, see: https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/164415 -- https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/164415 Your team Zorba Coders is subscribed to branch lp:zorba/process-module. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module
The proposal to merge lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module has been updated. Status: Needs review = Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/164415 -- https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/164415 Your team Zorba Coders is subscribed to branch lp:zorba/process-module. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module
Review: Approve -- https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/164415 Your team Zorba Coders is subscribed to branch lp:zorba/process-module. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module
The proposal to merge lp:~zorba-coders/zorba/process-2 into lp:zorba/process-module has been updated. Status: Needs review = Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/164415 -- https://code.launchpad.net/~zorba-coders/zorba/process-2/+merge/164415 Your team Zorba Coders is subscribed to branch lp:zorba/process-module. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/new-exec into lp:zorba
Nicolae Brinza has proposed merging lp:~zorba-coders/zorba/new-exec into lp:zorba. Commit message: Modified files that use the process module to request version 1.0, in preparation of the new version of the module. Requested reviews: Nicolae Brinza (nbrinza) Matthias Brantner (matthias-brantner) For more details, see: https://code.launchpad.net/~zorba-coders/zorba/new-exec/+merge/164166 Modified files that use the process module to request version 1.0, in preparation of the new version of the module. -- https://code.launchpad.net/~zorba-coders/zorba/new-exec/+merge/164166 Your team Zorba Coders is subscribed to branch lp:zorba. === modified file 'test/fots_driver/tools/process.xq' --- test/fots_driver/tools/process.xq 2013-02-25 15:39:39 + +++ test/fots_driver/tools/process.xq 2013-05-16 11:54:46 + @@ -23,7 +23,7 @@ import module namespace proc = - http://www.zorba-xquery.com/modules/process;; + http://www.zorba-xquery.com/modules/process#1.0;; import module namespace util = http://www.zorba-xquery.com/fots-driver/util; at ../util.xq; import module namespace d = === modified file 'test/zperf/src/start.xq' --- test/zperf/src/start.xq 2013-02-07 17:24:36 + +++ test/zperf/src/start.xq 2013-05-16 11:54:46 + @@ -17,7 +17,7 @@ (: XQuery main module :) import module namespace file=http://expath.org/ns/file;; -import module namespace proc=http://www.zorba-xquery.com/modules/process;; +import module namespace proc=http://www.zorba-xquery.com/modules/process#1.0;; import module namespace tims=http://www.example.com/timings; at timings.xq; import module namespace sys =http://www.zorba-xquery.com/modules/system;; import module namespace http=http://expath.org/ns/http-client;; -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/new-exec into lp:zorba
Review: Approve -- https://code.launchpad.net/~zorba-coders/zorba/new-exec/+merge/164166 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/new-exec into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/new-exec into lp:zorba has been updated. Status: Needs review = Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/new-exec/+merge/164166 -- https://code.launchpad.net/~zorba-coders/zorba/new-exec/+merge/164166 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/null-type-fix into lp:zorba
Review: Approve -- https://code.launchpad.net/~zorba-coders/zorba/null-type-fix/+merge/163883 Your team Zorba Coders is subscribed to branch lp:~zorba-coders/zorba/new-jsoniq. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/new-jsoniq into lp:zorba
Review: Approve -- https://code.launchpad.net/~zorba-coders/zorba/new-jsoniq/+merge/162375 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/new-jsoniq into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/new-jsoniq into lp:zorba has been updated. Status: Needs review = Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/new-jsoniq/+merge/162375 -- https://code.launchpad.net/~zorba-coders/zorba/new-jsoniq/+merge/162375 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/null-type-fix into lp:~zorba-coders/zorba/new-jsoniq
Review: Approve -- https://code.launchpad.net/~zorba-coders/zorba/null-type-fix/+merge/163902 Your team Zorba Coders is subscribed to branch lp:~zorba-coders/zorba/new-jsoniq. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/new-jsoniq into lp:zorba
I've added a mention in the ChangeLog -- https://code.launchpad.net/~zorba-coders/zorba/new-jsoniq/+merge/162375 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/new-jsoniq into lp:zorba
Review: Approve -- https://code.launchpad.net/~zorba-coders/zorba/new-jsoniq/+merge/162375 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/new-jsoniq into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/new-jsoniq into lp:zorba has been updated. Status: Needs review = Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/new-jsoniq/+merge/162375 -- https://code.launchpad.net/~zorba-coders/zorba/new-jsoniq/+merge/162375 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/new-jsoniq into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/new-jsoniq into lp:zorba has been updated. Status: Approved = Rejected For more details, see: https://code.launchpad.net/~zorba-coders/zorba/new-jsoniq/+merge/162375 -- https://code.launchpad.net/~zorba-coders/zorba/new-jsoniq/+merge/162375 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/new-jsoniq into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/new-jsoniq into lp:zorba has been updated. Status: Rejected = Needs review For more details, see: https://code.launchpad.net/~zorba-coders/zorba/new-jsoniq/+merge/162375 -- https://code.launchpad.net/~zorba-coders/zorba/new-jsoniq/+merge/162375 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/new-jsoniq into lp:zorba
Done -- https://code.launchpad.net/~zorba-coders/zorba/new-jsoniq/+merge/162375 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/new-jsoniq into lp:zorba
Review: Approve -- https://code.launchpad.net/~zorba-coders/zorba/new-jsoniq/+merge/162375 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/new-jsoniq into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/new-jsoniq into lp:zorba has been updated. Status: Needs review = Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/new-jsoniq/+merge/162375 -- https://code.launchpad.net/~zorba-coders/zorba/new-jsoniq/+merge/162375 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/new-jsoniq into lp:zorba
Yeah, I just fixed that unit test as well... -- https://code.launchpad.net/~zorba-coders/zorba/new-jsoniq/+merge/162375 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/new-jsoniq into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/new-jsoniq into lp:zorba has been updated. Status: Needs review = Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/new-jsoniq/+merge/162375 -- https://code.launchpad.net/~zorba-coders/zorba/new-jsoniq/+merge/162375 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/new-jsoniq into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/new-jsoniq into lp:zorba has been updated. Status: Needs review = Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/new-jsoniq/+merge/162375 -- https://code.launchpad.net/~zorba-coders/zorba/new-jsoniq/+merge/162375 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/new-jsoniq into lp:zorba
Matthias, your suggestions for two improved warnings do not change the messages: you just switched the order of the words. The warnings are built around the template: feature not supported by the common language grammar{: $1} So it is not easy to make this swap. I could probably do it, but then I would have to change all the other messages. Do you think it is worth it? Other than that, I have fixed all of the issues raised by you and Ghislain. There has been a confusion which resulted in many warnings not working for you and Ghislain even though they were implemented. The problem was that you both used a qualified string for the op:enable option, i.e. you used f:common-language. Why would you need a qualified value if the option itself is already qualified? As we discussed, it is not possible to resolve a prefix during the Bison parsing, and that is why the initial code did not allow for a prefix in the value. I have changed it, and now it allows a prefix but it will be ignored during the parsing. -- 1. improve warning messages: feature not supported by the common language grammar: JSONiq dot object lookup = jsoniq dot object lookup not supported by common language feature not supported by the common language grammar: context item expression; use zorba-util:context-item() = context item expression not supported by common language (use util:context- item() instead) 2. Query doesn't raise a warning: jsoniq version 1.0; declare namespace op = http://www.zorba-xquery.com/options/features;; declare namespace f = http://www.zorba-xquery.com/features;; declare option op:enable f:common-language; {} 3. Query without version declaration doesn't raise a warning 4. Query returns warning with wrong warning location, i.e. 8,13 jsoniq version 1.0; declare namespace op = http://www.zorba-xquery.com/options/features;; declare namespace f = http://www.zorba-xquery.com/features;; declare option op:enable f:common-language; let $foo := { f.oo : bar }.f.oo return $foo /home/mbrantner/zorba/build/tmp.xq:8,13: Zorba static warning [zwarn:ZWST0009]: feature not supported by the common language grammar: JSONiq dot object lookup /home/mbrantner/zorba/build/tmp.xq:8,13: Zorba static warning [zwarn:ZWST0009]: feature not supported by the common language grammar: JSONiq dot object lookup 5. The following query doesn't raise a warning xquery version 1.0; declare namespace op = http://www.zorba-xquery.com/options/features;; declare namespace f = http://www.zorba-xquery.com/features;; declare option op:enable f:common-language; let $foo := 'foo' return $foo 6. The following query doesn't raise a warning jsoniq version 1.0; declare namespace op = http://www.zorba-xquery.com/options/features;; declare namespace f = http://www.zorba-xquery.com/features;; declare option op:enable f:common-language; let $foo := { foo : bar } return insert { bar : 42 } into $foo; -- https://code.launchpad.net/~zorba-coders/zorba/new-jsoniq/+merge/162375 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/new-jsoniq into lp:zorba
Why would you need a qualified value if the option itself is already qualified? I guess because the option name and the option value may not live in the same namespace. Why not? It's not as if you do schema validation on the options and their values. The option needs to be qualified to distinguish it from XQuery options, but the value needs not. And the validation is done entirely in the code so there is no schema validation. -- https://code.launchpad.net/~zorba-coders/zorba/new-jsoniq/+merge/162375 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/new-jsoniq into lp:zorba
(by the way, character entity references do not exist in XML. There are character references (#...;) and entity references (...;), but not both at the same time if I am correct :-) ) Indeed you're right. I trusted Wikipedia which uses the term character entity references and says ``The XML specification does not use the term character entity or character entity reference. '' -- http://en.wikipedia.org/wiki/Character_entity_reference#Predefined_entities_in_XML . I've checked now the XML spec and it does seem to have the definitions: http://www.w3.org/TR/xml/#sec-references . I'll change the messages to what you suggested, but I'll keep both of them -- it is much better to have them as customized as possible. -- https://code.launchpad.net/~zorba-coders/zorba/new-jsoniq/+merge/162375 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/new-jsoniq into lp:zorba
Fixed all the raised issues. -- https://code.launchpad.net/~zorba-coders/zorba/new-jsoniq/+merge/162375 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/bug-1176038 into lp:zorba
Nicolae Brinza has proposed merging lp:~zorba-coders/zorba/bug-1176038 into lp:zorba. Commit message: Fixed a bug in the parser that was revealed by bison 2.7 + #include clean-up Requested reviews: Nicolae Brinza (nbrinza) Paul J. Lucas (paul-lucas) Related bugs: Bug #1176038 in Zorba: pointer being freed was not allocated https://bugs.launchpad.net/zorba/+bug/1176038 For more details, see: https://code.launchpad.net/~zorba-coders/zorba/bug-1176038/+merge/162953 Fixed a bug in the parser that was revealed by bison 2.7 + #include clean-up -- https://code.launchpad.net/~zorba-coders/zorba/bug-1176038/+merge/162953 Your team Zorba Coders is subscribed to branch lp:zorba. === modified file 'ChangeLog' --- ChangeLog 2013-05-08 03:22:27 + +++ ChangeLog 2013-05-08 10:58:45 + @@ -1,5 +1,15 @@ Zorba - The XQuery Processor +version 3.x + +New Features: + +Optimizations: + +Bug Fixes/Other Changes: + * Fixed bug #1176038 (memory error in parser revealed by Bison 2.7) + + version 2.9 New Features: === modified file 'src/api/dynamiccontextimpl.cpp' --- src/api/dynamiccontextimpl.cpp 2013-05-08 03:19:57 + +++ src/api/dynamiccontextimpl.cpp 2013-05-08 10:58:45 + @@ -27,6 +27,7 @@ #include types/typemanager.h #include types/root_typemanager.h #include types/schema/validate.h +#include zorbatypes/integer.h #include api/unmarshaller.h #include api/zorbaimpl.h === modified file 'src/api/item.cpp' --- src/api/item.cpp 2013-02-26 04:12:43 + +++ src/api/item.cpp 2013-05-08 10:58:45 + @@ -38,7 +38,9 @@ #include store/api/iterator.h #include store/api/collection.h -#include zorbatypes/numconversions.h +#include zorbatypes/floatimpl.h +#include zorbatypes/integer.h +#include zorbatypes/numconversions.h namespace zorba { === modified file 'src/api/itemfactoryimpl.cpp' --- src/api/itemfactoryimpl.cpp 2013-03-27 03:30:14 + +++ src/api/itemfactoryimpl.cpp 2013-05-08 10:58:45 + @@ -25,14 +25,17 @@ #include api/itemfactoryimpl.h -#include zorbatypes/duration.h +#include zorbatypes/decimal.h +#include zorbatypes/floatimpl.h +#include zorbatypes/integer.h +#include zorbatypes/schema_types.h #include system/globalenv.h +#include store/api/copymode.h +#include store/api/item.h #include store/api/item_factory.h #include store/api/store.h -#include store/api/copymode.h -#include store/api/item.h #include api/unmarshaller.h === modified file 'src/api/serialization/serializer.cpp' --- src/api/serialization/serializer.cpp 2013-04-25 02:05:20 + +++ src/api/serialization/serializer.cpp 2013-05-08 10:58:45 + @@ -39,6 +39,7 @@ #include system/globalenv.h #include zorbamisc/ns_consts.h +#include zorbatypes/integer.h #include zorbatypes/numconversions.h #include store/api/iterator.h === modified file 'src/capi/csequence.cpp' --- src/capi/csequence.cpp 2013-02-07 17:24:36 + +++ src/capi/csequence.cpp 2013-05-08 10:58:45 + @@ -18,16 +18,17 @@ #include capi/csequence.h #include cassert -#include string.h +#include cstring #include zorba/zorba.h #include zorba/diagnostic_list.h #include zorba/iterator.h #include zorba/store_consts.h -#include zorbamisc/ns_consts.h -#include zorbatypes/numconversions.h #include util/string_util.h +#include zorbamisc/ns_consts.h +#include zorbatypes/floatimpl.h +#include zorbatypes/numconversions.h #include error.h === modified file 'src/compiler/expression/expr.cpp' --- src/compiler/expression/expr.cpp 2013-03-07 10:10:10 + +++ src/compiler/expression/expr.cpp 2013-05-08 10:58:45 + @@ -49,6 +49,9 @@ #include store/api/store.h #include store/api/item_factory.h +#include zorbatypes/decimal.h +#include zorbatypes/floatimpl.h +#include zorbatypes/integer.h namespace zorba === modified file 'src/compiler/expression/expr_manager.cpp' --- src/compiler/expression/expr_manager.cpp 2013-04-16 20:06:08 + +++ src/compiler/expression/expr_manager.cpp 2013-05-08 10:58:45 + @@ -15,22 +15,24 @@ */ #include stdafx.h + +#include zorbatypes/decimal.h +#include zorbatypes/integer.h + +#include expr.h #include expr_manager.h - -#include mem_manager.h - -#include expr.h -#include ftnode.h -#include var_expr.h #include flwor_expr.h #include fo_expr.h #include ft_expr.h +#include ftnode.h #include function_item_expr.h +#include json_exprs.h +#include mem_manager.h #include path_expr.h +#include pragma.h #include script_exprs.h #include update_exprs.h -#include json_exprs.h -#include pragma.h +#include var_expr.h namespace zorba { === modified file 'src/compiler/parser/symbol_table.cpp' --- src/compiler/parser/symbol_table.cpp 2013-03-25 19:47:18 + +++ src/compiler/parser/symbol_table.cpp 2013-05-08 10:58:45 + @@ -15,6 +15,9 @@ */ #include stdafx.h +#include zorbatypes/decimal.h +#include zorbatypes/floatimpl.h +#include zorbatypes/integer.h #include zorbatypes/numconversions.h #include compiler/parser/symbol_table.h === modified file 'src/compiler
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/bug-1176038 into lp:zorba
Review: Approve -- https://code.launchpad.net/~zorba-coders/zorba/bug-1176038/+merge/162953 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/bug-1176038 into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/bug-1176038 into lp:zorba has been updated. Status: Needs review = Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/bug-1176038/+merge/162953 -- https://code.launchpad.net/~zorba-coders/zorba/bug-1176038/+merge/162953 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp