Bugs item #2836863, was opened at 2009-08-13 13:47
Message generated for change (Comment added) made by stmane
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=482468&aid=2836863&group_id=56967
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: SQL/Core
Group: SQL "stable"
>Status: Closed
Resolution: Fixed
Priority: 6
Private: No
Submitted By: Stefan Manegold (stmane)
Assigned to: Stefan Manegold (stmane)
Summary: SQL: ORDER BY tests fail on 64-bit systems with 32-bit OIDs
Initial Comment:
Most probably since this checkin
===================================================================
2009/06/09 - nielsnes: sql/src/backends/monet5/sql_gencode.mx,1.319.2.4
sql/src/backends/monet5/sql_optimizer.mx,1.218.2.2
sql/src/server/rel_bin.mx,1.78.2.1
sql/src/server/sql_rel2bin.mx,1.132.2.1
sql/src/server/sql_statement.mx,1.184.2.1
ng problems with 'order' dependend statements (such as mkey xor and batcalc*
)
this solves problems with alter_table_drop_constraint as well as
SQL: test type_dump_test.SF-989257.SQL.sh fails since Feb 28
projection joins are now handled by leftjoins
===================================================================
order-by tests fail on 64-bit systems with 32-bit OIDs (the advised default
used with our 64-bit Windows installers); cf.
http://monetdb.cwi.nl/testing/projects/monetdb/Stable/sql/.mTests103/Int.64.32.d.1-Windows5.2/src_benchmarks_ATIS/select_group.out.00.html
http://monetdb.cwi.nl/testing/projects/monetdb/Stable/sql/.mTests103/GNU.64.32.d.1-Fedora10/src_test_ADT2006/slam.out.00.html
http://monetdb.cwi.nl/testing/projects/monetdb/Stable/sql/.mTests103/GNU.64.32.d.1-Fedora10/src_test_Dependencies/Dependencies.out.00.html
http://monetdb.cwi.nl/testing/projects/monetdb/Stable/sql/.mTests103/GNU.64.32.d.1-Fedora10/src_test_UserDump/create.out.00.html
http://monetdb.cwi.nl/testing/projects/monetdb/Stable/sql/.mTests103/GNU.64.32.d.1-Fedora10/src_test_bugs/subselect_multiple_unionall_where_1=1-bug-sf-1005596.out.00.html
The problem might be related to these comments in the above checkin in
sql/src/server/sql_rel2bin.mx
"
/* TODO the semijoin may break the order!! */
/* TODO semijoin may break order */
"
maybe, we need to introduce an order-preserving "left"-semijoin?
----------------------------------------------------------------------
>Comment By: Stefan Manegold (stmane)
Date: 2009-08-18 17:58
Message:
testing confirmed fix. closing.
----------------------------------------------------------------------
Comment By: Stefan Manegold (stmane)
Date: 2009-08-18 00:10
Message:
Most probably fixed by
===================================================================
2009/08/17 - stmane: MonetDB/src/gdk/gdk_batop.mx,1.173.2.3
Added missing GDK_VARSHIFT for string heap access in BATordered() .
This should fix
ID: 2836863 "SQL: ORDER BY tests fail on 64-bit systems with 32-bit OIDs"
https://sourceforge.net/tracker/?func=detail&aid=2836863&group_id=56967&atid=482468
===================================================================
To be closed once testing has confirmed the fix.
----------------------------------------------------------------------
Comment By: Stefan Manegold (stmane)
Date: 2009-08-17 09:18
Message:
Ok, running the failing query
select category,count(*) as totalnr from aircraft where engines=2 group by
category having count(*)>4 order by category;
in isolation against a freshly restarted mserver5 (after loading the data)
reveals that the first run gives the wrong order as in the test, but any
subsequent run (against the same running mserver5) yield the correctly
ordered result.
TRACEing the query and enabling algorithm tracing in the server
(--algorithm) reveals as (only) significant difference between the 1st &
2nd run, that the 1st run creates some hash table before showing any other
tracing output, while the 2nd does not:
#BAThash: create hash(3);
... I cannot say that this clams me down, much ...
... to be continued ...
----------------------------------------------------------------------
Comment By: Stefan Manegold (stmane)
Date: 2009-08-17 08:50
Message:
Unfortunately, the problem is even stranger:
Looking more detailed at the first failing test
http://monetdb.cwi.nl/testing/projects/monetdb/Stable/sql/.mTests103/Int.64
.32.d.1-Windows5.2/src_benchmarks_ATIS/select_group.out.00.html,
it appears that semijoin (BATsemijoin) is not even used; hence, what
seemed to be a fix (see my prev. comment) was rather by coincidence that
the test worked.
In fact, when run single-threaded (gdk_nr_threads=1), the test always
fails (only on 64-bit systems with 32-bit OIDs); however when run
multi-threaded (ghk_nr_thread>1) (on a multi-core system), the test seems
to work randomly in roughtly 50% of the attempts ...
... I'm afraid it will be a nasty task to debug and eventually fix this
one ... |-(
----------------------------------------------------------------------
Comment By: Stefan Manegold (stmane)
Date: 2009-08-16 23:27
Message:
A quick test indeed suggests that the kernel (GDK) seems to choose a
"reversed" and hence not order-preserving implementation of semijoin, and
this results in incorrect order --- at least disabling the reversed
semijoin implementation does yield the correct results.
Hence, introducing (and using) an order-preserving "left"semijoin
(basically a semijoin that is not allowed to internally use the "reversed"
implementation that loops over the right iso. the left input) should help.
Open question is, why the problems only occur on 64-bit systems with
32-bit OIDs ...
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=482468&aid=2836863&group_id=56967
------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
Monetdb-bugs mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/monetdb-bugs
