Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/bug-1026250 into lp:zorba
Review: Approve I did not like very much the versioning solution for the collection deadlock. It certainly does not work with multi-threading. But this is not the only thing that is wrong for multi-threading, so I will approve. I think a better solution would be that any dml:collection() function that is executed within a nonmaterialize pragma acquires the collection lock in exclussive mode. Of course that would require changes in the compiler, runtime, and store api. -- https://code.launchpad.net/~zorba-coders/zorba/bug-1026250/+merge/138044 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/bug-1026250 into lp:zorba
Review: Approve I did not like very much the versioning solution for the collection deadlock. It certainly does not work with multi-threading. But this is not the only thing that is wrong for multi-threading, so I will approve. I think a better solution would be that any dml:collection() function that is executed within a nonmaterialize pragma acquires the collection lock in exclussive mode. Of course that would require changes in the compiler, runtime, and store api. -- https://code.launchpad.net/~zorba-coders/zorba/bug-1026250/+merge/138044 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/bug-1026250 into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/bug-1026250 into lp:zorba has been updated. Status: Needs review = Merged For more details, see: https://code.launchpad.net/~zorba-coders/zorba/bug-1026250/+merge/138044 -- https://code.launchpad.net/~zorba-coders/zorba/bug-1026250/+merge/138044 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/bug-1026250 into lp:zorba
Validation queue succeeded - proposal merged! -- https://code.launchpad.net/~zorba-coders/zorba/bug-1026250/+merge/138044 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/bug-1026250 into lp:zorba
Validation queue starting for the following merge proposals: https://code.launchpad.net/~zorba-coders/zorba/bug-1026250/+merge/138044 Progress dashboard at http://jenkins.lambda.nu/view/ValidationQueue -- https://code.launchpad.net/~zorba-coders/zorba/bug-1026250/+merge/138044 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/bug-1026250 into lp:zorba
Thanks for the feedback. Merging -- https://code.launchpad.net/~zorba-coders/zorba/bug-1026250/+merge/138044 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/bug-1026250 into lp:zorba
Thanks for the review. Can you explain me why SYNC_CODE(theCollection-theLatch.rlock();) and SYNC_CODE(theCollection-theLatch.unlock();) has been commented in this branch? They were already commented in the branch when I started working on it. They were causing a deadlock when the collection being read from was modified. This is now fixed using collection versioning. The two lines can be removed. -- https://code.launchpad.net/~zorba-coders/zorba/bug-1026250/+merge/138044 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/bug-1026250 into lp:zorba
Thanks for the review. Can you explain me why SYNC_CODE(theCollection-theLatch.rlock();) and SYNC_CODE(theCollection-theLatch.unlock();) has been commented in this branch? They were already commented in the branch when I started working on it. -- https://code.launchpad.net/~zorba-coders/zorba/bug-1026250/+merge/138044 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/bug-1026250 into lp:zorba
Ready for review -- https://code.launchpad.net/~zorba-coders/zorba/bug-1026250/+merge/138044 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/bug-1026250 into lp:zorba
One of the tests uses ext:nomaterialization without dash. Is this intented? -- https://code.launchpad.net/~zorba-coders/zorba/bug-1026250/+merge/138044 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/bug-1026250 into lp:zorba
nope, let me fix it On Wed, Aug 14, 2013 at 3:14 PM, Ghislain Fourny ghislain.fou...@28msec.com wrote: One of the tests uses ext:nomaterialization without dash. Is this intented? -- https://code.launchpad.net/~zorba-coders/zorba/bug-1026250/+merge/138044 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp -- https://code.launchpad.net/~zorba-coders/zorba/bug-1026250/+merge/138044 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/bug-1026250 into lp:zorba
Fixed and pushed On Wed, Aug 14, 2013 at 9:36 PM, Federico Cavalieri federico.cavali...@28msec.com wrote: nope, let me fix it On Wed, Aug 14, 2013 at 3:14 PM, Ghislain Fourny ghislain.fou...@28msec.com wrote: One of the tests uses ext:nomaterialization without dash. Is this intented? -- https://code.launchpad.net/~zorba-coders/zorba/bug-1026250/+merge/138044 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp -- https://code.launchpad.net/~zorba-coders/zorba/bug-1026250/+merge/138044 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/bug-1026250 into lp:zorba
Nice change. The modifications in the following files don't seem to be needed: - test/rbkt/Queries/zorba/updates/flwor1.xq - test/rbkt/Queries/zorba/scripting/flwor6.xq - test/rbkt/Queries/zorba/updates/flwor1.xq - test/rbkt/Queries/zorba/no-copy/dataguide-c.xq I'll add Markos as a second reviewer. -- https://code.launchpad.net/~zorba-coders/zorba/bug-1026250/+merge/138044 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/bug-1026250 into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/bug-1026250 into lp:zorba has been updated. Description changed to: - annotation to turn on sequential flwor streaming - introduce support for pragmas and allows the body of a pragma to be a statement (recent scripting extension) - collections are versioned in order to avoid deadlocks when modifying a collection that is currently read from (instead raise an error) For more details, see: https://code.launchpad.net/~zorba-coders/zorba/bug-1026250/+merge/138044 -- https://code.launchpad.net/~zorba-coders/zorba/bug-1026250/+merge/138044 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/bug-1026250 into lp:zorba
Review: Approve -- https://code.launchpad.net/~zorba-coders/zorba/bug-1026250/+merge/138044 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/bug-1026250 into lp:zorba
Matthias will get this scheduled within 28msec, hopefully soon. -- https://code.launchpad.net/~zorba-coders/zorba/bug-1026250/+merge/138044 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/bug-1026250 into lp:zorba
Please put a more descriptive commit message for this change. -- https://code.launchpad.net/~zorba-coders/zorba/bug-1026250/+merge/138044 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/bug-1026250 into lp:zorba
I modified test flwor10.xq to be: declare namespace ext = http://www.zorba-xquery.com/extensions;; variable $x := 0; (# ext:materialize #) { for $i in (1, 2, 3) let $y := { $x := $x + $i; () } return $x } ran it and got: Zorba error [zerr:ZXQP0002]: false: assertion failed with the message: ClauseKind = materialize_clause; raised at /Users/pjl/src/flwor/zorba/repo/bug-1026250/zorba/src/compiler/codegen/plan_visitor.cpp:1028 The switch statement doesn't have a case for materialize_clause. Either it should, or it should never be the case that a materialize_clause should occur here. FYI: I modified the code on the branch to overload operator for ClauseKind -- makes it handy for debugging. -- https://code.launchpad.net/~zorba-coders/zorba/bug-1026250/+merge/138044 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/bug-1026250 into lp:zorba
Of the failing tests, only some have been modified. If you changed the default behavior, shouldn't you also have to change every affected test by adding the pragma? -- https://code.launchpad.net/~zorba-coders/zorba/bug-1026250/+merge/138044 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/bug-1026250 into lp:zorba
Matthias Brantner has proposed merging lp:~zorba-coders/zorba/bug-1026250 into lp:zorba. Requested reviews: Matthias Brantner (matthias-brantner) For more details, see: https://code.launchpad.net/~zorba-coders/zorba/bug-1026250/+merge/138044 -- https://code.launchpad.net/~zorba-coders/zorba/bug-1026250/+merge/138044 Your team Zorba Coders is subscribed to branch lp:zorba. === modified file 'include/zorba/pregenerated/diagnostic_list.h' --- include/zorba/pregenerated/diagnostic_list.h 2012-11-12 16:17:02 + +++ include/zorba/pregenerated/diagnostic_list.h 2012-12-05 01:28:26 + @@ -640,6 +640,8 @@ extern ZORBA_DLL_PUBLIC ZorbaErrorCode ZDDY0036_NON_ROOT_NODE_DELETION; +extern ZORBA_DLL_PUBLIC ZorbaErrorCode ZDDY0037_CONCURRENT_MODIFICATION; + extern ZORBA_DLL_PUBLIC ZorbaErrorCode ZDST0001_COLLECTION_ALREADY_DECLARED; extern ZORBA_DLL_PUBLIC ZorbaErrorCode ZDST0002_COLLECTION_ALREADY_IMPORTED; === modified file 'modules/com/zorba-xquery/www/modules/pregenerated/errors.xq' --- modules/com/zorba-xquery/www/modules/pregenerated/errors.xq 2012-11-08 05:38:30 + +++ modules/com/zorba-xquery/www/modules/pregenerated/errors.xq 2012-12-05 01:28:26 + @@ -566,6 +566,10 @@ (:~ :) +declare variable $zerr:ZDDY0037 as xs:QName := fn:QName($zerr:NS, zerr:ZDDY0037); + +(:~ +:) declare variable $zerr:ZDST0001 as xs:QName := fn:QName($zerr:NS, zerr:ZDST0001); (:~ === modified file 'src/compiler/codegen/plan_visitor.cpp' --- src/compiler/codegen/plan_visitor.cpp 2012-10-29 11:41:36 + +++ src/compiler/codegen/plan_visitor.cpp 2012-12-05 01:28:26 + @@ -883,6 +883,9 @@ if (v.is_sequential()) { +pragma* pr = 0; +theCCB-lookup_pragma(v, materialize, pr); + if (!isGeneral) { if (v.has_sequential_clauses()) @@ -929,17 +932,11 @@ } } - // Note: a materialize clause may exist already in case plan serialization - // is on (see comment in materialize_clause::clone) - if (!isGeneral - v.get_return_expr()-is_sequential() - v.get_clause(numClauses-1)-get_kind() != flwor_clause::materialize_clause - (v.get_order_clause() != NULL || v.get_group_clause() == NULL)) + if (pr) { materialize_clause* mat = theCCB-theEM-create_materialize_clause(v.get_sctx(), v.get_return_expr()-get_loc()); - v.add_clause(mat); ++numClauses; } @@ -975,7 +972,7 @@ ++numForClauses; } - if (domExpr-is_sequential()) + if (pr domExpr-is_sequential()) { if (k == flwor_clause::for_clause || k == flwor_clause::window_clause || === modified file 'src/compiler/expression/flwor_expr.cpp' --- src/compiler/expression/flwor_expr.cpp 2012-10-26 07:13:42 + +++ src/compiler/expression/flwor_expr.cpp 2012-12-05 01:28:26 + @@ -699,7 +699,7 @@ expr::substitution_t subst) const { // we will reach here under the following scenario: - // 1. We do plan seriazation + // 1. We do plan serialization // 2. getPlan is called on udf A; this causes a mat clause to be created //during the codegen on A's body // 3. getPlan is called on udf B, which invokes A, and A's body is === modified file 'src/compiler/translator/translator.cpp' --- src/compiler/translator/translator.cpp 2012-11-20 18:17:10 + +++ src/compiler/translator/translator.cpp 2012-12-05 01:28:26 + @@ -2331,6 +2331,31 @@ } +/*** +/ +void +recognizePragma(expr* e, const zstring aLocalName) +{ + for (std::vectorpragma*::const_iterator lIter = theScopedPragmas.begin(); + lIter != theScopedPragmas.end(); + ++lIter) + { +pragma* p = *lIter; +if (p-theQName-getNamespace() == ZORBA_EXTENSIONS_NS) +{ + if (p-theQName-getLocalName() == aLocalName) + { +e-get_ccb()-add_pragma(e, p); +e-setContainsPragma(ANNOTATION_TRUE); +break; + } +} + } +} + + + + / // // // Module, VersionDecl, MainModule, LibraryModule, ModuleDecl // @@ -6290,6 +6315,8 @@ theFlworClausesStack.resize(curClausePos); + recognizePragma(flwor, materialize); + push_nodestack(flwor); } === modified file 'src/diagnostics/diagnostic_en.xml' --- src/diagnostics/diagnostic_en.xml 2012-12-01 01:04:39 + +++ src/diagnostics/diagnostic_en.xml 2012-12-05 01:28:26 + @@ -2255,6 +2255,10 @@ valueattempt to delete non-root node from collection $1/value /diagnostic +diagnostic code=ZDDY0037 name=CONCURRENT_MODIFICATION + value$1: collection was modified while reading/value +
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/bug-1026250 into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/bug-1026250 into lp:zorba has been updated. Commit Message changed to: fix for bug lp:1026250 For more details, see: https://code.launchpad.net/~zorba-coders/zorba/bug-1026250/+merge/138044 -- https://code.launchpad.net/~zorba-coders/zorba/bug-1026250/+merge/138044 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/bug-1026250 into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/bug-1026250 into lp:zorba has been updated. Status: Needs review = Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/bug-1026250/+merge/138044 -- https://code.launchpad.net/~zorba-coders/zorba/bug-1026250/+merge/138044 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/bug-1026250 into lp:zorba
The attempt to merge lp:~zorba-coders/zorba/bug-1026250 into lp:zorba failed. Below is the output from the failed tests. CMake Error at /home/ceej/zo/testing/zorbatest/tester/TarmacLander.cmake:275 (message): Validation queue job bug-1026250-2012-12-05T01-32-06.304Z is finished. The final status was: 13 tests did not succeed - changes not commited. Error in read script: /home/ceej/zo/testing/zorbatest/tester/TarmacLander.cmake -- https://code.launchpad.net/~zorba-coders/zorba/bug-1026250/+merge/138044 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/bug-1026250 into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/bug-1026250 into lp:zorba has been updated. Status: Approved = Needs review For more details, see: https://code.launchpad.net/~zorba-coders/zorba/bug-1026250/+merge/138044 -- https://code.launchpad.net/~zorba-coders/zorba/bug-1026250/+merge/138044 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp