Bugs item #1991738, was opened at 2008-06-12 11:30
Message generated for change (Comment added) made by tsheyar
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=482468&aid=1991738&group_id=56967
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: PF/compiler
Group: Pathfinder 0.24
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Wouter Alink (vzzzbx)
Assigned to: Nobody/Anonymous (nobody)
Summary: XQ: compilation or runtime choice
Initial Comment:
(stable nightly build, compiled from supersourcetarball, June 11, 2008)
The following query takes about 35 minutes to compile (and I have reason to
believe it was also the reason why the machine crashed after it was executed a
few times):
for $run in pf:collection("log.xml")//run
let $successCount := if ($run/success) then count($run/success/task) else 0
let $failureCount := if ($run/failure) then count($run/failure/task) else 0
let $nooutputCount := if ($run/nooutput) then (if ($run/nooutput/@count) then
zero-or-one($run/nooutput/@count) else 0) + count($run/nooutput/task) else 0
let $totalTasks := $successCount + $failureCount + $nooutputCount
return <run>
{ $run/@tool }
{ for $v in $run/prepared/date/@unixtime[. > 0] return attribute
preparation {$v - exactly-one($run/started/date/@unixtime)} }
{ for $v in $run/executed/date/@unixtime[. > 0] return attribute
execution {$v - exactly-one($run/prepared/date/@unixtime)} }
{ for $v in $run/validated/date/@unixtime[. > 0] return attribute
validation {$v - exactly-one($run/executed/date/@unixtime)} }
{ for $v in $run/committed/date/@unixtime[. > 0] return attribute
commit {$v - exactly-one($run/validated/date/@unixtime)} }
{ for $v in $run/prepared/date/@unixtime[. > 0 and $totalTasks > 0]
return attribute preparationPerTask {($v -
exactly-one($run/started/date/@unixtime)) div $totalTasks} }
{ for $v in $run/executed/date/@unixtime[. > 0 and $totalTasks > 0]
return attribute executionPerTask {($v -
exactly-one($run/prepared/date/@unixtime)) div $totalTasks} }
{ for $v in $run/validated/date/@unixtime[. > 0 and $totalTasks > 0]
return attribute validationPerTask {($v -
exactly-one($run/executed/date/@unixtime)) div $totalTasks} }
{ for $v in $run/committed/date/@unixtime[. > 0 and $totalTasks > 0]
return attribute commitPerTask {($v -
exactly-one($run/validated/date/@unixtime)) div $totalTasks} }
{attribute allowedExecution {exactly-one($run/allowedExecutionTime)
* 1000}}
{attribute successCount {$successCount}}
{attribute failureCount {$failureCount}}
{attribute nooutputCount {$nooutputCount}}
{attribute totalTasks {$totalTasks}}
</run>
but, as you might have noticed the query is not written in a very nice way...
Therefore i tried to rewrite the query to the (hopefully) equivalent:
for $run in pf:collection("log.xml")//run
let $successCount := count($run/success/task)
let $failureCount := count($run/failure/task)
let $nooutputCount := if ($run/nooutput/@count) then
exactly-one($run/nooutput/@count) cast as xs:integer else 0 +
count($run/nooutput/task)
let $totalTasks := ($successCount + $failureCount + $nooutputCount) cast as
xs:integer
let $started := $run/started/date/@unixtime cast as xs:integer?
let $prepared := $run/prepared/date/@unixtime cast as xs:integer?
let $executed := $run/executed/date/@unixtime cast as xs:integer?
let $validated := $run/validated/date/@unixtime cast as xs:integer?
let $committed := $run/committed/date/@unixtime cast as xs:integer?
return element { "run" } {
$run/@tool,
if ($prepared > 0) then attribute preparation { $prepared - $started } else
(),
if ($executed > 0) then attribute execution { $executed - $prepared } else
(),
if ($validated > 0) then attribute validation { $validated - $executed }
else (),
if ($committed > 0) then attribute commit { $committed - $validated } else
(),
if ($totalTasks > 0)
then
(if ($prepared > 0) then attribute preparationPerTask { ($prepared -
$started) div $totalTasks } else (),
if ($executed > 0) then attribute executionPerTask { ($executed -
$prepared) div $totalTasks } else (),
if ($validated > 0) then attribute vaalidationPerTask { ($validated -
$executed) div $totalTasks } else (),
if ($committed > 0) then attribute commitPerTask { ($committed -
$validated) div $totalTasks } else ())
else
(),
attribute allowedExecution {exactly-one($run/allowedExecutionTime) * 1000},
attribute successCount {$successCount},
attribute failureCount {$failureCount},
attribute nooutputCount {$nooutputCount},
attribute totalTasks {$totalTasks}
}
This query compiles within seconds, but then the runtime environment complains:
MAPI = [EMAIL PROTECTED]
QUERY = for $run in pf:collection("log.xml")//run
ERROR = !ERROR: merged_union: tail of first BAT/column of table 2 must be
sorted.
!ERROR: CMDmerged_union: operation failed.
Unfortunately I cannot provide the log.xml-document, but when leaving out the
if-statement:
"if ($totalTasks > 0) then ... else ()"
the query executes fine.
p.s. This workaround works for us at the moment.
----------------------------------------------------------------------
>Comment By: Jan Rittinger (tsheyar)
Date: 2008-06-16 13:48
Message:
Logged In: YES
user_id=993208
Originator: NO
Wouter, Am I right that adding the line
assert_order(a0090);
before the failing merged_union fixes the problem for this call to
merged_union?
----------------------------------------------------------------------
Comment By: Wouter Alink (vzzzbx)
Date: 2008-06-16 12:54
Message:
Logged In: YES
user_id=621590
Originator: YES
The error:
ERROR = !ERROR: merged_union: tail of first BAT/column of table 2 must be
sorted.
!ERROR: CMDmerged_union: operation failed.
happens in the merged_union on line 1854:
a0051 := unused;
a0007 := unused;
a0007 := [lng](a0039);
if (a0007.reverse ().exist (lng(nil))) {
ERROR ("err:FORG0001.");
} else {
}
a0051 := unused;
a0051 := [+](a0107, a0108);
a0107 := unused;
a0108 := unused;
a0039 := unused;
a0039 := merged_union (a0103, a0090, a0097, a0075, a0098, a0076,
a0099, a0077, a0100, a0087, a0101, a0088, a0102, a0089, a0007, a0047,
a0051, a0050);
a0056 := a0039.fetch (0);
a0057 := a0039.fetch (1);
a0058 := a0039.fetch (2);
a0059 := a0039.fetch (3);
a0060 := a0039.fetch (4);
a0073 := a0039.fetch (6);
a0074 := a0039.fetch (7);
a0091 := a0039.fetch (8);
a0039 := unused;
a0103 := unused;
a0097 := unused;
a0098 := unused;
----------------------------------------------------------------------
Comment By: Wouter Alink (vzzzbx)
Date: 2008-06-13 18:53
Message:
Logged In: YES
user_id=621590
Originator: YES
Hello Jan,
There is not much that I expect out of this bugreport. We found a viable
workaround for the given query, so there is certainly no hurry in fixing
this. I wanted to indicate that there were problems with running this
query. I will try to provide the failing line in the milscript on monday.
For the first query: i knew monetdb was memory-hungry, however it might be
nice to have some 'detection-mechanism' that stops optimizing when more
than 2?GB of memory is spent by the pathfinder compiler alone trying to
come up with a query plan... but i guess that is probably a feature request
for a next release :)
----------------------------------------------------------------------
Comment By: Jan Rittinger (tsheyar)
Date: 2008-06-13 09:42
Message:
Logged In: YES
user_id=993208
Originator: NO
Hi Wouter,
I'm not sure what you expect from this bug report.
The optimization of the first query screws up because the optimizer wants
to get rid of all ``for loops'' and has a harder time reaching this goal
than in the second case. If you however look at the query heavy
optimization is not needed to provide a good performance. Knowing this it
could be perhaps helpful to state that you only want to apply ``light''
optimization for this query (in comparison to others).
For the second query I cannot see anything wrong in the code generation.
To solve this problem I would need some help from you.
If you generate the query with pf -A, profile it, give the position in the
MIL code (the merged_union that is going to fail; probably one of the 3
calls with 10 arguments), and perhaps a characteristic snippet of the
second input BAT (it should be an iter/oid column), then it should be
possible to decide whether the underlying string_join() operator or an
incorrect property inference in MonetDB might be the reason for the failing
merged_union() call.
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=482468&aid=1991738&group_id=56967
-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
Monetdb-bugs mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/monetdb-bugs
