Changeset: 1d33143ec6a2 for MonetDB URL: https://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=1d33143ec6a2 Added Files: monetdb5/modules/mosaic/Tests/mosaic_bit_1.delta.MAL.py monetdb5/modules/mosaic/Tests/mosaic_bit_1.delta.options5 monetdb5/modules/mosaic/Tests/mosaic_bit_1.delta.stable.err monetdb5/modules/mosaic/Tests/mosaic_bit_1.delta.stable.out monetdb5/modules/mosaic/Tests/mosaic_bit_with_nills_1.delta.MAL.py monetdb5/modules/mosaic/Tests/mosaic_bit_with_nills_1.delta.options5 monetdb5/modules/mosaic/Tests/mosaic_bit_with_nills_1.delta.stable.err monetdb5/modules/mosaic/Tests/mosaic_bit_with_nills_1.delta.stable.out monetdb5/modules/mosaic/Tests/mosaic_date_with_nills_1.delta.MAL.py monetdb5/modules/mosaic/Tests/mosaic_date_with_nills_1.delta.options5 monetdb5/modules/mosaic/Tests/mosaic_date_with_nills_1.delta.stable.err monetdb5/modules/mosaic/Tests/mosaic_date_with_nills_1.delta.stable.out monetdb5/modules/mosaic/Tests/mosaic_lng_with_nills_1.delta.MAL.py monetdb5/modules/mosaic/Tests/mosaic_lng_with_nills_1.delta.options5 monetdb5/modules/mosaic/Tests/mosaic_lng_with_nills_1.delta.stable.err monetdb5/modules/mosaic/Tests/mosaic_lng_with_nills_1.delta.stable.out monetdb5/modules/mosaic/Tests/mosaic_oid_with_nills_1.delta.MAL.py monetdb5/modules/mosaic/Tests/mosaic_oid_with_nills_1.delta.options5 monetdb5/modules/mosaic/Tests/mosaic_oid_with_nills_1.delta.stable.err monetdb5/modules/mosaic/Tests/mosaic_oid_with_nills_1.delta.stable.out monetdb5/modules/mosaic/Tests/mosaic_timestamp_1.delta.stable.err monetdb5/modules/mosaic/Tests/mosaic_timestamp_with_nills_1.delta.MAL.py monetdb5/modules/mosaic/Tests/mosaic_timestamp_with_nills_1.delta.options5 monetdb5/modules/mosaic/Tests/mosaic_timestamp_with_nills_1.delta.stable.err monetdb5/modules/mosaic/Tests/mosaic_timestamp_with_nills_1.delta.stable.out Modified Files: monetdb5/modules/mosaic/TODO_MOSAIC.txt monetdb5/modules/mosaic/Tests/All monetdb5/modules/mosaic/Tests/mosaic_timestamp_1.delta.stable.out monetdb5/modules/mosaic/mosaic.c monetdb5/modules/mosaic/mosaic_delta.c monetdb5/modules/mosaic/mosaic_delta.h sql/test/mosaic/Tests/xqueries_delta.stable.err Branch: mosaic Log Message:
Delta encoding: nil semantics, code clean up, more tests. diffs (truncated from 12878 to 300 lines): diff --git a/monetdb5/modules/mosaic/TODO_MOSAIC.txt b/monetdb5/modules/mosaic/TODO_MOSAIC.txt --- a/monetdb5/modules/mosaic/TODO_MOSAIC.txt +++ b/monetdb5/modules/mosaic/TODO_MOSAIC.txt @@ -1,9 +1,8 @@ ####technical depth#### FIX HAS_NIL optimization in join algorithms -Use bit vectors in mosaic_delta --- then resolve test mosaic_delta_timestamp --- and possibly other compression techniques as well.(2) Finish null semantics plus tests Add test for join nil_matches set for non-nill tests. -make sure candidate lists are working properly for select, theta_select and join +make sure candidate lists are working properly for select, theta_select and join (2) Be sure alignment is correct. Use asserts for this. Add test for float/double Add test for huge @@ -14,7 +13,6 @@ Remove floating point support in prefix Apply compressed operations directly on keys in dictionary compression. consider packing mosaic_block_header's more efficiently. -delta compression: check if the hardcoded compression expression are correct. (2*) Add asserts on type degeneration for bit => bte daytime => lng timestmap => lng date => int. benchmark @@ -24,6 +22,5 @@ Update copyright. create dedicated type for bitvector internal type which is currently unsignned int. E.g. introduce #define BitVectorChunkType unsigned int. create a common header/source file for mosaic for common tasks e.g. to house a generic IS_NIL macro, IPtpe, etc. move mask, bits, framebits from MosaicHdr to MosaicBlkHeader_frame_t. -mosaic_delta.c: improve the macro's: for instance get rid of hardcoded expression arguments in DELTAcompress. (2) mosaic_<compression>.c: for type macro's use macro-wised inline functions. -Improve the administration of the various hard coded compression techniques. (2*) +Improve the administration of the various hard coded compression techniques, e.g. MOSskipit skip (2*) diff --git a/monetdb5/modules/mosaic/Tests/All b/monetdb5/modules/mosaic/Tests/All --- a/monetdb5/modules/mosaic/Tests/All +++ b/monetdb5/modules/mosaic/Tests/All @@ -69,10 +69,16 @@ mosaic_oid_with_nills_1.capped mosaic_date_with_nills_1.capped mosaic_timestamp_with_nills_1.capped +mosaic_bit_1.delta mosaic_lng_1.delta mosaic_oid_1.delta mosaic_date_1.delta -#mosaic_timestamp_1.delta +mosaic_timestamp_1.delta +mosaic_bit_with_nills_1.delta +mosaic_lng_with_nills_1.delta +mosaic_oid_with_nills_1.delta +mosaic_date_with_nills_1.delta +mosaic_timestamp_with_nills_1.delta mosaic_bit_2.linear mosaic_lng_2.linear diff --git a/monetdb5/modules/mosaic/Tests/mosaic_bit_1.prefix.MAL.py b/monetdb5/modules/mosaic/Tests/mosaic_bit_1.delta.MAL.py copy from monetdb5/modules/mosaic/Tests/mosaic_bit_1.prefix.MAL.py copy to monetdb5/modules/mosaic/Tests/mosaic_bit_1.delta.MAL.py diff --git a/monetdb5/modules/mosaic/Tests/mosaic_bit_1.prefix.options5 b/monetdb5/modules/mosaic/Tests/mosaic_bit_1.delta.options5 copy from monetdb5/modules/mosaic/Tests/mosaic_bit_1.prefix.options5 copy to monetdb5/modules/mosaic/Tests/mosaic_bit_1.delta.options5 diff --git a/monetdb5/modules/mosaic/Tests/mosaic_bit_1.prefix.stable.err b/monetdb5/modules/mosaic/Tests/mosaic_bit_1.delta.stable.err copy from monetdb5/modules/mosaic/Tests/mosaic_bit_1.prefix.stable.err copy to monetdb5/modules/mosaic/Tests/mosaic_bit_1.delta.stable.err --- a/monetdb5/modules/mosaic/Tests/mosaic_bit_1.prefix.stable.err +++ b/monetdb5/modules/mosaic/Tests/mosaic_bit_1.delta.stable.err @@ -1,4 +1,4 @@ -stderr of test 'mosaic_bit_1.prefix` in directory 'monetdb5/modules/mosaic` itself: +stderr of test 'mosaic_bit_1.delta` in directory 'monetdb5/modules/mosaic` itself: # 13:33:26 > diff --git a/monetdb5/modules/mosaic/Tests/mosaic_bit_1.prefix.stable.out b/monetdb5/modules/mosaic/Tests/mosaic_bit_1.delta.stable.out copy from monetdb5/modules/mosaic/Tests/mosaic_bit_1.prefix.stable.out copy to monetdb5/modules/mosaic/Tests/mosaic_bit_1.delta.stable.out --- a/monetdb5/modules/mosaic/Tests/mosaic_bit_1.prefix.stable.out +++ b/monetdb5/modules/mosaic/Tests/mosaic_bit_1.delta.stable.out @@ -1,4 +1,4 @@ -stdout of test 'mosaic_bit_1.prefix` in directory 'monetdb5/modules/mosaic` itself: +stdout of test 'mosaic_bit_1.delta` in directory 'monetdb5/modules/mosaic` itself: # 16:26:45 > @@ -16,108 +16,103 @@ stdout of test 'mosaic_bit_1.prefix` in # Listening for connection requests on mapi:monetdb://aris-XPS-13-9380:34578/ # Listening for UNIX domain connection requests on mapi:monetdb:///var/tmp/mtest-26512/.s.monetdb.34578 # MonetDB/SQL module loaded - -# 16:26:45 > -# 16:26:45 > "mclient" "-lmal" "-ftest" "-tnone" "-Eutf-8" "--host=/var/tmp/mtest-26512" "--port=34578" -# 16:26:45 > +MOScompress_delta +#BATmosaic: mosaic construction 22 usec +MOSdecompress_delta +MOSselect_delta +MOSselect_delta +MOSthetaselect_delta +MOSselect_delta +MOSthetaselect_delta +MOSselect_delta +MOSthetaselect_delta +MOSselect_delta +MOSselect_delta +MOSthetaselect_delta +MOSselect_delta +MOSselect_delta +MOSselect_delta +MOSselect_delta +MOSthetaselect_delta +MOSselect_delta +MOSthetaselect_delta +MOSselect_delta +MOSthetaselect_delta +MOSselect_delta +MOSselect_delta +MOSthetaselect_delta +MOSselect_delta +MOSselect_delta +MOSselect_delta +MOSselect_delta +MOSselect_delta +MOSselect_delta +MOSthetaselect_delta +MOSselect_delta +MOSthetaselect_delta +MOSselect_delta +MOSthetaselect_delta +MOSselect_delta +MOSselect_delta +MOSthetaselect_delta +MOSselect_delta +MOSselect_delta +MOSselect_delta +MOSthetaselect_delta +MOSselect_delta +MOSthetaselect_delta +MOSselect_delta +MOSthetaselect_delta +MOSselect_delta +MOSselect_delta +MOSthetaselect_delta +MOSselect_delta +MOSselect_delta +MOSselect_delta +MOSselect_delta +MOSselect_delta +MOSselect_delta +MOSselect_delta +MOSselect_delta +MOSselect_delta +MOSselect_delta +MOSselect_delta +MOSselect_delta +MOSselect_delta +MOSselect_delta +MOSselect_delta +MOSselect_delta +MOSselect_delta +MOSselect_delta +MOSselect_delta +MOSselect_delta +MOSselect_delta +MOSselect_delta +MOSselect_delta +MOSselect_delta +MOSselect_delta +MOSselect_delta +MOSselect_delta +MOSselect_delta +MOSselect_delta +MOSselect_delta +MOSselect_delta +MOSselect_delta +MOSselect_delta +MOSselect_delta +MOSselect_delta +MOSselect_delta +MOSselect_delta +MOSselect_delta +MOSselect_delta +MOSjoin_delta +MOSthetaselect_delta +MOSprojection_delta +MOSprojection_delta -MOScompress_prefix -#BATmosaic: mosaic construction 15 usec -MOSdecompress_prefix -MOSselect_prefix -MOSselect_prefix -MOSthetaselect_prefix -MOSselect_prefix -MOSthetaselect_prefix -MOSselect_prefix -MOSthetaselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSthetaselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSthetaselect_prefix -MOSselect_prefix -MOSthetaselect_prefix -MOSselect_prefix -MOSthetaselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSthetaselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSthetaselect_prefix -MOSselect_prefix -MOSthetaselect_prefix -MOSselect_prefix -MOSthetaselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSthetaselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSthetaselect_prefix -MOSselect_prefix -MOSthetaselect_prefix -MOSselect_prefix -MOSthetaselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSthetaselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSselect_prefix -MOSjoin_prefix -MOSthetaselect_prefix -MOSprojection_prefix -MOSprojection_prefix - -# 16:33:29 > -# 16:33:29 > "/usr/bin/python3" "mosaic_bit_1.prefix.MAL.py" "mosaic_bit_1.prefix" -# 16:33:29 > +# 11:35:46 > +# 11:35:46 > "/usr/bin/python3" "mosaic_bit_1.delta.MAL.py" "mosaic_bit_1.delta" +# 11:35:46 > #--------------------------# # t t t # name diff --git a/monetdb5/modules/mosaic/Tests/mosaic_bit_with_nills_1.prefix.MAL.py b/monetdb5/modules/mosaic/Tests/mosaic_bit_with_nills_1.delta.MAL.py copy from monetdb5/modules/mosaic/Tests/mosaic_bit_with_nills_1.prefix.MAL.py copy to monetdb5/modules/mosaic/Tests/mosaic_bit_with_nills_1.delta.MAL.py diff --git a/monetdb5/modules/mosaic/Tests/mosaic_bit_with_nills_1.prefix.options5 b/monetdb5/modules/mosaic/Tests/mosaic_bit_with_nills_1.delta.options5 copy from monetdb5/modules/mosaic/Tests/mosaic_bit_with_nills_1.prefix.options5 copy to monetdb5/modules/mosaic/Tests/mosaic_bit_with_nills_1.delta.options5 diff --git a/monetdb5/modules/mosaic/Tests/mosaic_bit_with_nills_1.prefix.stable.err b/monetdb5/modules/mosaic/Tests/mosaic_bit_with_nills_1.delta.stable.err copy from monetdb5/modules/mosaic/Tests/mosaic_bit_with_nills_1.prefix.stable.err copy to monetdb5/modules/mosaic/Tests/mosaic_bit_with_nills_1.delta.stable.err --- a/monetdb5/modules/mosaic/Tests/mosaic_bit_with_nills_1.prefix.stable.err +++ b/monetdb5/modules/mosaic/Tests/mosaic_bit_with_nills_1.delta.stable.err @@ -1,4 +1,4 @@ -stderr of test 'mosaic_bit_with_nills_1.prefix` in directory 'monetdb5/modules/mosaic` itself: +stderr of test 'mosaic_bit_with_nills_1.delta` in directory 'monetdb5/modules/mosaic` itself: _______________________________________________ checkin-list mailing list [email protected] https://www.monetdb.org/mailman/listinfo/checkin-list
