MonetDB: mosaic - Update TODO's.
Changeset: ce16cfb4ebdb for MonetDB URL: https://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=ce16cfb4ebdb Modified Files: monetdb5/modules/mosaic/TODO_MOSAIC.txt Branch: mosaic Log Message: Update TODO's. diffs (15 lines): diff --git a/monetdb5/modules/mosaic/TODO_MOSAIC.txt b/monetdb5/modules/mosaic/TODO_MOSAIC.txt --- a/monetdb5/modules/mosaic/TODO_MOSAIC.txt +++ b/monetdb5/modules/mosaic/TODO_MOSAIC.txt @@ -1,10 +1,7 @@ technical depth -Make sure layout and analysis work on propper sql object identifiers ,e.g. schema, table column. -Add layout summary in analysis. -Fix/check/test MOSlayout. +Figure out why mix compression don't seem to work that well. Add the compression on OID heap for string valued columns too. -Figure out why mix compression don't seem to work that well. benchmark using TPC-H altering the storage doesn't seem to work. Add negative values to tests ___ checkin-list mailing list checkin-list@monetdb.org https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: mosaic - Update TODO's.
Changeset: deaf3219b04a for MonetDB URL: https://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=deaf3219b04a Modified Files: monetdb5/modules/mosaic/TODO_MOSAIC.txt Branch: mosaic Log Message: Update TODO's. diffs (18 lines): diff --git a/monetdb5/modules/mosaic/TODO_MOSAIC.txt b/monetdb5/modules/mosaic/TODO_MOSAIC.txt --- a/monetdb5/modules/mosaic/TODO_MOSAIC.txt +++ b/monetdb5/modules/mosaic/TODO_MOSAIC.txt @@ -1,6 +1,5 @@ technical depth -Expand the select interface to handle boolean unknown parameter. Add layout summary in analysis. Fix/check/test MOSlayout. Add the compression on OID heap for string valued columns too. @@ -27,6 +26,7 @@ Find a more 'generic' model to estimate Create a plug-in estimation model framework. code issues +Straighten out semantics MOSselect. Make them exact to BATselect. See if the iteration algorithms can be improved. Use GDK naming conventions, e.g. o for stuff coming out if canditer_next. Update copyright in 2020. ___ checkin-list mailing list checkin-list@monetdb.org https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: mosaic - Update TODO's.
Changeset: 8cd342e7dd82 for MonetDB URL: https://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=8cd342e7dd82 Modified Files: monetdb5/modules/mosaic/TODO_MOSAIC.txt Branch: mosaic Log Message: Update TODO's. diffs (31 lines): diff --git a/monetdb5/modules/mosaic/TODO_MOSAIC.txt b/monetdb5/modules/mosaic/TODO_MOSAIC.txt --- a/monetdb5/modules/mosaic/TODO_MOSAIC.txt +++ b/monetdb5/modules/mosaic/TODO_MOSAIC.txt @@ -1,11 +1,15 @@ technical depth + +Expand the select interface to handle boolean unknown parameter. +Add layout summary in analysis. +Fix/check/test MOSlayout. +Add the compression on OID heap for string valued columns too. +Figure out why mix compression don't seem to work that well. +benchmark using TPC-H Refactor macro-ised functions to real functions in header files and apply the include trick -benchmark using TPC-H -Fix/check/test MOSlayout. Add negative values to tests Add test for huge create mix compression test cases particularly for dict/dict256 combo. -Introduce a custom pre-processor to expand macro's efficiently for debugging purposes. Introduce mosaic into gdk: create hooks in scanselect functions/macro's, joins and projections. make mitosis work with mosaic and see how this relates to the previous landmark stuff. Transform control dependency into data dependency select loops. @@ -18,7 +22,7 @@ consider packing mosaic_block_header's m Add asserts on type degeneration for bit => bte daytime => lng timestmap => lng date => int. DO nil checking more efficient similar to gdk_select Timing in the context of MOSanalysis -Add the compression on OID heap for string valued columns too. + Find a more 'generic' model to estimate and explore compression space perhaps Monte Carlo based. Create a plug-in estimation model framework. ___ checkin-list mailing list checkin-list@monetdb.org https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: mosaic - Update TODO's.
Changeset: 7f1c5a834ac0 for MonetDB URL: https://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=7f1c5a834ac0 Modified Files: monetdb5/modules/mosaic/TODO_MOSAIC.txt Branch: mosaic Log Message: Update TODO's. diffs (10 lines): diff --git a/monetdb5/modules/mosaic/TODO_MOSAIC.txt b/monetdb5/modules/mosaic/TODO_MOSAIC.txt --- a/monetdb5/modules/mosaic/TODO_MOSAIC.txt +++ b/monetdb5/modules/mosaic/TODO_MOSAIC.txt @@ -1,5 +1,6 @@ technical depth Refactor macro-ised functions to real functions in header files and apply the include trick +Find SF-5 bug. benchmark using TPC-H Fix/check/test MOSlayout. Add negative values to tests ___ checkin-list mailing list checkin-list@monetdb.org https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: mosaic - Update TODO's.
Changeset: 25ea4a66f869 for MonetDB URL: https://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=25ea4a66f869 Modified Files: monetdb5/modules/mosaic/TODO_MOSAIC.txt Branch: mosaic Log Message: Update TODO's. diffs (11 lines): diff --git a/monetdb5/modules/mosaic/TODO_MOSAIC.txt b/monetdb5/modules/mosaic/TODO_MOSAIC.txt --- a/monetdb5/modules/mosaic/TODO_MOSAIC.txt +++ b/monetdb5/modules/mosaic/TODO_MOSAIC.txt @@ -1,5 +1,6 @@ technical depth -Create a custom preprocessor to expand selected macro's and respecting new lines. +Find mosaic memory-leak +Refactor macro-ised functions to real functions in header files and apply the include trick benchmark using TPC-H Fix/check/test MOSlayout. Add negative values to tests ___ checkin-list mailing list checkin-list@monetdb.org https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: mosaic - Update TODO's.
Changeset: 799c7b16e38b for MonetDB URL: https://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=799c7b16e38b Modified Files: monetdb5/modules/mosaic/TODO_MOSAIC.txt Branch: mosaic Log Message: Update TODO's. diffs (12 lines): diff --git a/monetdb5/modules/mosaic/TODO_MOSAIC.txt b/monetdb5/modules/mosaic/TODO_MOSAIC.txt --- a/monetdb5/modules/mosaic/TODO_MOSAIC.txt +++ b/monetdb5/modules/mosaic/TODO_MOSAIC.txt @@ -1,4 +1,8 @@ technical depth +Improve the dictionary creation algorithms get rid of bubble sort +MERGE consecutive dictionary estimates. +Analysis: bug not all compression pairs are constructed +Create a custom preprocessor to expand selected macro's and respecting new lines. benchmark using TPC-H Fix/check/test MOSlayout. Add negative values to tests ___ checkin-list mailing list checkin-list@monetdb.org https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: mosaic - Update TODO's.
Changeset: fcc2d898c9f2 for MonetDB URL: https://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=fcc2d898c9f2 Modified Files: monetdb5/modules/mosaic/TODO_MOSAIC.txt Branch: mosaic Log Message: Update TODO's. diffs (11 lines): diff --git a/monetdb5/modules/mosaic/TODO_MOSAIC.txt b/monetdb5/modules/mosaic/TODO_MOSAIC.txt --- a/monetdb5/modules/mosaic/TODO_MOSAIC.txt +++ b/monetdb5/modules/mosaic/TODO_MOSAIC.txt @@ -4,6 +4,7 @@ Fix/check/test MOSlayout. Add negative values to tests Add test for huge create mix compression test cases. +Introduce a custom pre-processor to expand macro's efficiently for debugging purposes. Introduce mosaic into gdk: create hooks in scanselect functions/macro's, joins and projections. make mitosis work with mosaic and see how this relates to the previous landmark stuff. Transform control dependency into data dependency select loops. ___ checkin-list mailing list checkin-list@monetdb.org https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: mosaic - Update TODO's.
Changeset: be5ce5898735 for MonetDB URL: https://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=be5ce5898735 Modified Files: monetdb5/modules/mosaic/TODO_MOSAIC.txt Branch: mosaic Log Message: Update TODO's. diffs (22 lines): diff --git a/monetdb5/modules/mosaic/TODO_MOSAIC.txt b/monetdb5/modules/mosaic/TODO_MOSAIC.txt --- a/monetdb5/modules/mosaic/TODO_MOSAIC.txt +++ b/monetdb5/modules/mosaic/TODO_MOSAIC.txt @@ -1,9 +1,9 @@ technical depth +benchmark using TPC-H Fix/check/test MOSlayout. Add negative values to tests Add test for huge create mix compression test cases. -benchmark using TPC-H Introduce mosaic into gdk: create hooks in scanselect functions/macro's, joins and projections. make mitosis work with mosaic and see how this relates to the previous landmark stuff. Transform control dependency into data dependency select loops. @@ -20,6 +20,7 @@ Find a more 'generic' model to estimate Create a plug-in estimation model framework. code issues +Get rid of unsused functions. limit/blk etc. See if the iteration algorithms can be improved. Use GDK naming conventions, e.g. o for stuff coming out if canditer_next. Update copyright in 2020. ___ checkin-list mailing list checkin-list@monetdb.org https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: mosaic - Update TODO's.
Changeset: a1535a74e9fe for MonetDB URL: https://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=a1535a74e9fe Modified Files: monetdb5/modules/mosaic/TODO_MOSAIC.txt Branch: mosaic Log Message: Update TODO's. diffs (18 lines): diff --git a/monetdb5/modules/mosaic/TODO_MOSAIC.txt b/monetdb5/modules/mosaic/TODO_MOSAIC.txt --- a/monetdb5/modules/mosaic/TODO_MOSAIC.txt +++ b/monetdb5/modules/mosaic/TODO_MOSAIC.txt @@ -1,5 +1,4 @@ technical depth -Add raw to technique overview in analysis Rename 'capped' and 'var' to 'dict256' and 'dict' respectively. Fix/check/test MOSlayout. Add negative values to tests @@ -18,6 +17,8 @@ Add asserts on type degeneration for bit DO nil checking more efficient similar to gdk_select Timing in the context of MOSanalysis Add the compression on OID heap for string valued columns too. +Find a more 'generic' model to estimate and explore compression space perhaps Monte Carlo based. +Create a plug-in estimation model framework. code issues # Improve macro code style: remove do_ from do_OPERATION-like macro's, NAME -> TECHNIQUE, small caps technique identifier -> CAPITAL CAPS technique identifier. ___ checkin-list mailing list checkin-list@monetdb.org https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: mosaic - Update TODO's.
Changeset: 14806304629f for MonetDB URL: https://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=14806304629f Modified Files: monetdb5/modules/mosaic/TODO_MOSAIC.txt Branch: mosaic Log Message: Update TODO's. diffs (12 lines): diff --git a/monetdb5/modules/mosaic/TODO_MOSAIC.txt b/monetdb5/modules/mosaic/TODO_MOSAIC.txt --- a/monetdb5/modules/mosaic/TODO_MOSAIC.txt +++ b/monetdb5/modules/mosaic/TODO_MOSAIC.txt @@ -1,5 +1,8 @@ technical depth Fix estimation. Use a differential approach? +Add raw to technique overview in analysis +Rename 'capped' and 'var' to 'dict256' and 'dict' respectively. +Make header smaller: add padding to outer struct Fix/check/test MOSlayout. Add negative values to tests Add test for huge ___ checkin-list mailing list checkin-list@monetdb.org https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: mosaic - Update TODO's.
Changeset: 77efbb35878a for MonetDB URL: https://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=77efbb35878a Modified Files: monetdb5/modules/mosaic/TODO_MOSAIC.txt Branch: mosaic Log Message: Update TODO's. diffs (27 lines): diff --git a/monetdb5/modules/mosaic/TODO_MOSAIC.txt b/monetdb5/modules/mosaic/TODO_MOSAIC.txt --- a/monetdb5/modules/mosaic/TODO_MOSAIC.txt +++ b/monetdb5/modules/mosaic/TODO_MOSAIC.txt @@ -2,12 +2,14 @@ Be sure alignment is correct. Use asserts and type specialized block headers for this.(1*) Check MOSanalysis and its test. Fix/check/test MOSlayout. +Add negative values to tests +Add test for huge create mix compression test cases. -Add test for huge Introduce mosaic into gdk: create hooks in scanselect functions/macro's, joins and projections. benchmark using TPC-H make mitosis work with mosaic and see how this relates to the previous landmark stuff. Transform control dependency into data dependency select loops. +Apply compressed operations more directly: on constant value in runlength compression Apply compressed operations more directly: on keys in dictionary compression Apply compressed operations more directly: on prefix|suffix in prefix compression Apply compressed operations more directly: on delta's and frame in frame compression @@ -17,6 +19,7 @@ DO nil checking more efficient similar t Timing in the context of MOSanalysis code issues +# Improve macro code style: remove do_ from do_OPERATION-like macro's, NAME -> TECHNIQUE, small caps technique identifier -> CAPITAL CAPS technique identifier. See if the iteration algorithms can be improved. Get rid of (most of) MOStask and/or don't allocate it on the heap. Make sure the parameters.bits fields are always char. ___ checkin-list mailing list checkin-list@monetdb.org https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: mosaic - Update TODO's.
Changeset: 1708ff8d4abf for MonetDB URL: https://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=1708ff8d4abf Modified Files: monetdb5/modules/mosaic/TODO_MOSAIC.txt Branch: mosaic Log Message: Update TODO's. diffs (22 lines): diff --git a/monetdb5/modules/mosaic/TODO_MOSAIC.txt b/monetdb5/modules/mosaic/TODO_MOSAIC.txt --- a/monetdb5/modules/mosaic/TODO_MOSAIC.txt +++ b/monetdb5/modules/mosaic/TODO_MOSAIC.txt @@ -1,8 +1,9 @@ technical depth -Add test for huge Be sure alignment is correct. Use asserts and type specialized block headers for this.(1*) Check MOSanalysis and its test. Fix/check/test MOSlayout. +create mix compression test cases. +Add test for huge Introduce mosaic into gdk: create hooks in scanselect functions/macro's, joins and projections. benchmark using TPC-H make mitosis work with mosaic and see how this relates to the previous landmark stuff. @@ -14,7 +15,6 @@ consider packing mosaic_block_header's m Add asserts on type degeneration for bit => bte daytime => lng timestmap => lng date => int. DO nil checking more efficient similar to gdk_select Timing in the context of MOSanalysis -create mix compression test cases. code issues See if the iteration algorithms can be improved. ___ checkin-list mailing list checkin-list@monetdb.org https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: mosaic - Update TODO's.
Changeset: ff93b1c8d301 for MonetDB URL: https://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=ff93b1c8d301 Modified Files: monetdb5/modules/mosaic/TODO_MOSAIC.txt Branch: mosaic Log Message: Update TODO's. diffs (9 lines): diff --git a/monetdb5/modules/mosaic/TODO_MOSAIC.txt b/monetdb5/modules/mosaic/TODO_MOSAIC.txt --- a/monetdb5/modules/mosaic/TODO_MOSAIC.txt +++ b/monetdb5/modules/mosaic/TODO_MOSAIC.txt @@ -1,5 +1,4 @@ technical depth -Add test for float/double Add test for huge Be sure alignment is correct. Use asserts and type specialized block headers for this.(1*) Check MOSanalysis and its test. ___ checkin-list mailing list checkin-list@monetdb.org https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: mosaic - Update TODO's.
Changeset: 54e4a89c6953 for MonetDB URL: https://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=54e4a89c6953 Modified Files: monetdb5/modules/mosaic/TODO_MOSAIC.txt Branch: mosaic Log Message: Update TODO's. diffs (25 lines): diff --git a/monetdb5/modules/mosaic/TODO_MOSAIC.txt b/monetdb5/modules/mosaic/TODO_MOSAIC.txt --- a/monetdb5/modules/mosaic/TODO_MOSAIC.txt +++ b/monetdb5/modules/mosaic/TODO_MOSAIC.txt @@ -1,4 +1,5 @@ technical depth +Remove debug flags from test runs and restore pound sign in debug messages Add test for float/double Add test for huge Be sure alignment is correct. Use asserts and type specialized block headers for this.(1*) @@ -7,6 +8,7 @@ Fix/check/test MOSlayout. Introduce mosaic into gdk: create hooks in scanselect functions/macro's, joins and projections. benchmark using TPC-H make mitosis work with mosaic and see how this relates to the previous landmark stuff. +Transform control dependency into data dependency select loops. Apply compressed operations more directly: on keys in dictionary compression Apply compressed operations more directly: on prefix|suffix in prefix compression Apply compressed operations more directly: on delta's and frame in frame compression @@ -14,6 +16,7 @@ consider packing mosaic_block_header's m Add asserts on type degeneration for bit => bte daytime => lng timestmap => lng date => int. DO nil checking more efficient similar to gdk_select Timing in the context of MOSanalysis +create mix compression test cases. code issues See if the iteration algorithms can be improved. ___ checkin-list mailing list checkin-list@monetdb.org https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: mosaic - Update TODO's.
Changeset: 1478513258aa for MonetDB URL: https://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=1478513258aa Modified Files: monetdb5/modules/mosaic/TODO_MOSAIC.txt Branch: mosaic Log Message: Update TODO's. diffs (30 lines): diff --git a/monetdb5/modules/mosaic/TODO_MOSAIC.txt b/monetdb5/modules/mosaic/TODO_MOSAIC.txt --- a/monetdb5/modules/mosaic/TODO_MOSAIC.txt +++ b/monetdb5/modules/mosaic/TODO_MOSAIC.txt @@ -2,15 +2,15 @@ Add lots of tests with candidate lists for all Add test for float/double Add test for huge -Be sure alignment is correct. Use asserts for this. +Be sure alignment is correct. Use asserts and type specialized block headers for this.(1*) Check MOSanalysis and its test. Fix/check/test MOSlayout. Introduce mosaic into gdk: create hooks in scanselect functions/macro's, joins and projections. benchmark using TPC-H make mitosis work with mosaic and see how this relates to the previous landmark stuff. -Apply compressed operations more directly: on keys in dictionary compression (1) -Apply compressed operations more directly: on prefix|suffix in prefix compression (1) -Apply compressed operations more directly: on delta's and frame in frame compression (1) +Apply compressed operations more directly: on keys in dictionary compression +Apply compressed operations more directly: on prefix|suffix in prefix compression +Apply compressed operations more directly: on delta's and frame in frame compression consider packing mosaic_block_header's more efficiently. Add asserts on type degeneration for bit => bte daytime => lng timestmap => lng date => int. DO nil checking more efficient similar to gdk_select @@ -21,5 +21,5 @@ See if the iteration algorithms can be i Get rid of (most of) MOStask and/or don't allocate it on the heap. Make sure the parameters.bits fields are always char. Use GDK naming conventions, e.g. o for stuff coming out if canditer_next. -Make block headers specific by having type parametrized blockheader struct's. +Make block headers specific by having type parametrized blockheader struct's.(1) Update copyright in 2020. ___ checkin-list mailing list checkin-list@monetdb.org https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: mosaic - Update TODO's.
Changeset: aa7733abfe89 for MonetDB URL: https://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=aa7733abfe89 Modified Files: monetdb5/modules/mosaic/TODO_MOSAIC.txt Branch: mosaic Log Message: Update TODO's. diffs (20 lines): diff --git a/monetdb5/modules/mosaic/TODO_MOSAIC.txt b/monetdb5/modules/mosaic/TODO_MOSAIC.txt --- a/monetdb5/modules/mosaic/TODO_MOSAIC.txt +++ b/monetdb5/modules/mosaic/TODO_MOSAIC.txt @@ -1,9 +1,9 @@ technical depth Create test for swapped join. +Add test for default join and COUI case both in MAL and in SQL. Add lots of tests with candidate lists for all Add test for float/double Add test for huge -Add test for default join and COUI case both in MAL and in SQL. Be sure alignment is correct. Use asserts for this. Check MOSanalysis and its test. Fix/check/test MOSlayout. @@ -24,5 +24,4 @@ Get rid of (most of) MOStask and/or don' Make sure the parameters.bits fields are always char. Use GDK naming conventions, e.g. o for stuff coming out if canditer_next. Make block headers specific by having type parametrized blockheader struct's. -create a common header/source file for mosaic for common tasks e.g. to house a generic IS_NIL macro, IPtpe, etc. Update copyright in 2020. ___ checkin-list mailing list checkin-list@monetdb.org https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: mosaic - Update TODO's.
Changeset: 870983ea9c3c for MonetDB URL: https://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=870983ea9c3c Modified Files: monetdb5/modules/mosaic/TODO_MOSAIC.txt Branch: mosaic Log Message: Update TODO's. diffs (40 lines): diff --git a/monetdb5/modules/mosaic/TODO_MOSAIC.txt b/monetdb5/modules/mosaic/TODO_MOSAIC.txt --- a/monetdb5/modules/mosaic/TODO_MOSAIC.txt +++ b/monetdb5/modules/mosaic/TODO_MOSAIC.txt @@ -6,29 +6,29 @@ Create test for swapped join. Add lots of tests with candidate lists for all Add test for float/double Add test for huge -Introduce mosaic into gdk: create hooks in scanselect functions/macro's, joins and projections. -Apply compressed operations more directly: on keys in dictionary compression (1) -Apply compressed operations more directly: on prefix|suffix in prefix compression (1) -Apply compressed operations more directly: on delta's and frame in frame compression (1) +Add test for default join and COUI case both in MAL and in SQL. Be sure alignment is correct. Use asserts for this. Check MOSanalysis and its test. Fix/check/test MOSlayout. +Introduce mosaic into gdk: create hooks in scanselect functions/macro's, joins and projections. benchmark using TPC-H make mitosis work with mosaic and see how this relates to the previous landmark stuff. +Apply compressed operations more directly: on keys in dictionary compression (1) +Apply compressed operations more directly: on prefix|suffix in prefix compression (1) +Apply compressed operations more directly: on delta's and frame in frame compression (1) consider packing mosaic_block_header's more efficiently. Add asserts on type degeneration for bit => bte daytime => lng timestmap => lng date => int. DO nil checking more efficient similar to gdk_select +Timing in the context of MOSanalysis code issues +See if the iteration algorithms can be improved. Get rid of (most of) MOStask and/or don't allocate it on the heap. Make sure the parameters.bits fields are always char. Use GDK naming conventions, e.g. o for stuff coming out if canditer_next. Make block headers specific by having type parametrized blockheader struct's. atomic base type of an oid bat is lng. So we can remove that one. It is a bit cleaner if MOSadvance call's are done outside of select functions. -Refactor frame compressions code to be more inline with the other compression algortihm code.(1) create a common header/source file for mosaic for common tasks e.g. to house a generic IS_NIL macro, IPtpe, etc. -mosaic_.c: for type macro's use macro-wised inline functions. (1) -Factor out more boilerplate code in various compression techniques. All compression follow similar patterns. (1*) Improve the administration of the various hard coded compression techniques, e.g. MOSskipit skip (2*) Update copyright in 2020. ___ checkin-list mailing list checkin-list@monetdb.org https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: mosaic - Update TODO's.
Changeset: 71fa1f286431 for MonetDB URL: https://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=71fa1f286431 Modified Files: monetdb5/modules/mosaic/TODO_MOSAIC.txt Branch: mosaic Log Message: Update TODO's. diffs (19 lines): diff --git a/monetdb5/modules/mosaic/TODO_MOSAIC.txt b/monetdb5/modules/mosaic/TODO_MOSAIC.txt --- a/monetdb5/modules/mosaic/TODO_MOSAIC.txt +++ b/monetdb5/modules/mosaic/TODO_MOSAIC.txt @@ -1,5 +1,6 @@ technical depth Adapt project and join to use candidate lists properly. +Create test for swapped join. Factor out boilerplate code mosaic code: resulting mosaic boilerplate is for testing purposes. Add lots of tests with candidate lists for all Introduce mosaic into gdk: create hooks in scanselect functions/macro's, joins and projections. @@ -20,6 +21,8 @@ DO nil checking more efficient similar t code issues Replace the MOSskip functions by MOSadvance functions +Get rid of (most of) MOStask. +Make sure the parameters.bits fields are always char. Use GDK naming conventions, e.g. o for stuff coming out if canditer_next. Make block headers specific by having type parametrized blockheader struct's. atomic base type of an oid bat is lng. So we can remove that one. ___ checkin-list mailing list checkin-list@monetdb.org https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: mosaic - Update TODO's.
Changeset: f713b1976b7c for MonetDB URL: https://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=f713b1976b7c Modified Files: monetdb5/modules/mosaic/TODO_MOSAIC.txt Branch: mosaic Log Message: Update TODO's. diffs (21 lines): diff --git a/monetdb5/modules/mosaic/TODO_MOSAIC.txt b/monetdb5/modules/mosaic/TODO_MOSAIC.txt --- a/monetdb5/modules/mosaic/TODO_MOSAIC.txt +++ b/monetdb5/modules/mosaic/TODO_MOSAIC.txt @@ -11,15 +11,16 @@ Add test for float/double Add test for huge Check MOSanalysis and its test. Fix/check/test MOSlayout. +benchmark using TPC-H make mitosis work with mosaic and see how this relates to the previous landmark stuff. Remove floating point support in prefix compression and use prefix-like-frame semantics consider packing mosaic_block_header's more efficiently. Add asserts on type degeneration for bit => bte daytime => lng timestmap => lng date => int. -benchmark DO nil checking more efficient similar to gdk_select code issues Replace the MOSskip functions by MOSadvance functions +Use GDK naming conventions, e.g. o for stuff coming out if canditer_next. Make block headers specific by having type parametrized blockheader struct's. atomic base type of an oid bat is lng. So we can remove that one. It is a bit cleaner if MOSadvance call's are done outside of select functions. ___ checkin-list mailing list checkin-list@monetdb.org https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: mosaic - Update TODO's.
Changeset: 564baccf6787 for MonetDB URL: https://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=564baccf6787 Modified Files: monetdb5/modules/mosaic/TODO_MOSAIC.txt Branch: mosaic Log Message: Update TODO's. diffs (28 lines): diff --git a/monetdb5/modules/mosaic/TODO_MOSAIC.txt b/monetdb5/modules/mosaic/TODO_MOSAIC.txt --- a/monetdb5/modules/mosaic/TODO_MOSAIC.txt +++ b/monetdb5/modules/mosaic/TODO_MOSAIC.txt @@ -1,8 +1,11 @@ technical depth +Adapt project and join to use candidate lists properly. Factor out boilerplate code mosaic code: resulting mosaic boilerplate is for testing purposes. -make sure that dense candidate lists are not transparently skipped and pushed to algebra module operators. -Introduce mosaic into gdk: create hooks in pointselect and scanselect functions/macro's. -make sure candidate lists are working properly (2) +Add lots of tests with candidate lists for all +Introduce mosaic into gdk: create hooks in scanselect functions/macro's, joins and projections. +Apply compressed operations more directly: on keys in dictionary compression (1) +Apply compressed operations more directly: on prefix|suffix in prefix compression (1) +Apply compressed operations more directly: on linear in prefix compression (1) Be sure alignment is correct. Use asserts for this. Add test for float/double Add test for huge @@ -10,9 +13,6 @@ Check MOSanalysis and its test. Fix/check/test MOSlayout. make mitosis work with mosaic and see how this relates to the previous landmark stuff. Remove floating point support in prefix compression and use prefix-like-frame semantics -Apply compressed operations more directly: on keys in dictionary compression (1) -Apply compressed operations more directly: on prefix|suffix in prefix compression (1) -Apply compressed operations more directly: on linear in prefix compression (1) consider packing mosaic_block_header's more efficiently. Add asserts on type degeneration for bit => bte daytime => lng timestmap => lng date => int. benchmark ___ checkin-list mailing list checkin-list@monetdb.org https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: mosaic - Update TODO's.
Changeset: d688f66b27a4 for MonetDB URL: https://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=d688f66b27a4 Modified Files: monetdb5/modules/mosaic/TODO_MOSAIC.txt Branch: mosaic Log Message: Update TODO's. diffs (23 lines): diff --git a/monetdb5/modules/mosaic/TODO_MOSAIC.txt b/monetdb5/modules/mosaic/TODO_MOSAIC.txt --- a/monetdb5/modules/mosaic/TODO_MOSAIC.txt +++ b/monetdb5/modules/mosaic/TODO_MOSAIC.txt @@ -1,5 +1,7 @@ technical depth -make sure that dense candidate lists are not transparently skipped and pushed to algebra module operators. Perhaps improve this on an optimizer level. +Factor out boilerplate code mosaic code: resulting mosaic boilerplate is for testing purposes. +make sure that dense candidate lists are not transparently skipped and pushed to algebra module operators. +Introduce mosaic into gdk: create hooks in pointselect and scanselect functions/macro's. make sure candidate lists are working properly (2) Be sure alignment is correct. Use asserts for this. Add test for float/double @@ -17,6 +19,10 @@ benchmark DO nil checking more efficient similar to gdk_select code issues +Replace the MOSskip functions by MOSadvance functions +Make block headers specific by having type parametrized blockheader struct's. +atomic base type of an oid bat is lng. So we can remove that one. +It is a bit cleaner if MOSadvance call's are done outside of select functions. Refactor frame compressions code to be more inline with the other compression algortihm code.(1) create a common header/source file for mosaic for common tasks e.g. to house a generic IS_NIL macro, IPtpe, etc. mosaic_.c: for type macro's use macro-wised inline functions. (1) ___ checkin-list mailing list checkin-list@monetdb.org https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: mosaic - Update TODO's.
Changeset: 254847b60365 for MonetDB URL: https://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=254847b60365 Modified Files: monetdb5/modules/mosaic/TODO_MOSAIC.txt Branch: mosaic Log Message: Update TODO's. diffs (17 lines): diff --git a/monetdb5/modules/mosaic/TODO_MOSAIC.txt b/monetdb5/modules/mosaic/TODO_MOSAIC.txt --- a/monetdb5/modules/mosaic/TODO_MOSAIC.txt +++ b/monetdb5/modules/mosaic/TODO_MOSAIC.txt @@ -1,4 +1,5 @@ technical depth +make sure that dense candidate lists are not transparently skipped and pushed to algebra module operators. Perhaps improve this on an optimizer level. make sure candidate lists are working properly (2) Be sure alignment is correct. Use asserts for this. Add test for float/double @@ -13,6 +14,7 @@ Apply compressed operations more directl consider packing mosaic_block_header's more efficiently. Add asserts on type degeneration for bit => bte daytime => lng timestmap => lng date => int. benchmark +DO nil checking more efficient similar to gdk_select code issues Refactor frame compressions code to be more inline with the other compression algortihm code.(1) ___ checkin-list mailing list checkin-list@monetdb.org https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: mosaic - Update TODO's.
Changeset: ada75509bbe2 for MonetDB URL: https://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=ada75509bbe2 Modified Files: monetdb5/modules/mosaic/TODO_MOSAIC.txt Branch: mosaic Log Message: Update TODO's. diffs (15 lines): diff --git a/monetdb5/modules/mosaic/TODO_MOSAIC.txt b/monetdb5/modules/mosaic/TODO_MOSAIC.txt --- a/monetdb5/modules/mosaic/TODO_MOSAIC.txt +++ b/monetdb5/modules/mosaic/TODO_MOSAIC.txt @@ -12,10 +12,10 @@ Apply compressed operations more directl Apply compressed operations more directly: on linear in prefix compression (1) consider packing mosaic_block_header's more efficiently. Add asserts on type degeneration for bit => bte daytime => lng timestmap => lng date => int. - benchmark code issues +Refactor frame compressions code to be more inline with the other compression algortihm code.(1) create a common header/source file for mosaic for common tasks e.g. to house a generic IS_NIL macro, IPtpe, etc. mosaic_.c: for type macro's use macro-wised inline functions. (1) Factor out more boilerplate code in various compression techniques. All compression follow similar patterns. (1*) ___ checkin-list mailing list checkin-list@monetdb.org https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: mosaic - Update TODO's.
Changeset: 32cc05025bd2 for MonetDB URL: https://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=32cc05025bd2 Modified Files: monetdb5/modules/mosaic/TODO_MOSAIC.txt Branch: mosaic Log Message: Update TODO's. diffs (9 lines): diff --git a/monetdb5/modules/mosaic/TODO_MOSAIC.txt b/monetdb5/modules/mosaic/TODO_MOSAIC.txt --- a/monetdb5/modules/mosaic/TODO_MOSAIC.txt +++ b/monetdb5/modules/mosaic/TODO_MOSAIC.txt @@ -1,4 +1,5 @@ technical depth +Fix memory leak make sure candidate lists are working properly (2) Be sure alignment is correct. Use asserts for this. Add test for float/double ___ checkin-list mailing list checkin-list@monetdb.org https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: mosaic - Update TODO's.
Changeset: 10089c59d347 for MonetDB URL: https://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=10089c59d347 Modified Files: monetdb5/modules/mosaic/TODO_MOSAIC.txt Branch: mosaic Log Message: Update TODO's. diffs (34 lines): diff --git a/monetdb5/modules/mosaic/TODO_MOSAIC.txt b/monetdb5/modules/mosaic/TODO_MOSAIC.txt --- a/monetdb5/modules/mosaic/TODO_MOSAIC.txt +++ b/monetdb5/modules/mosaic/TODO_MOSAIC.txt @@ -1,5 +1,5 @@ technical depth -make sure candidate lists are working properly for select, theta_select and join (2) +make sure candidate lists are working properly (2) Be sure alignment is correct. Use asserts for this. Add test for float/double Add test for huge @@ -7,17 +7,19 @@ Check MOSanalysis and its test. Fix/check/test MOSlayout. make mitosis work with mosaic and see how this relates to the previous landmark stuff. Remove floating point support in prefix compression and use prefix-like-frame semantics -Apply compressed operations directly on keys in dictionary compression. - +Apply compressed operations more directly: on keys in dictionary compression (1) +Apply compressed operations more directly: on prefix|suffix in prefix compression (1) +Apply compressed operations more directly: on linear in prefix compression (1) consider packing mosaic_block_header's more efficiently. Add asserts on type degeneration for bit => bte daytime => lng timestmap => lng date => int. benchmark code issues -Update copyright. create dedicated type for bitvector internal type which is currently unsignned int. E.g. introduce #define BitVectorChunkType unsigned int. create a common header/source file for mosaic for common tasks e.g. to house a generic IS_NIL macro, IPtpe, etc. move mask, bits, framebits from MosaicHdr to MosaicBlkHeader_frame_t. -mosaic_.c: for type macro's use macro-wised inline functions. +mosaic_.c: for type macro's use macro-wised inline functions. (1) +Factor out more boilerplate code in various compression techniques. All compression follow similar patterns. (1*) Improve the administration of the various hard coded compression techniques, e.g. MOSskipit skip (2*) +Update copyright in 2020. ___ checkin-list mailing list checkin-list@monetdb.org https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: mosaic - Update TODO's.
Changeset: da485bd3bdc2 for MonetDB URL: https://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=da485bd3bdc2 Modified Files: monetdb5/modules/mosaic/TODO_MOSAIC.txt Branch: mosaic Log Message: Update TODO's. diffs (21 lines): diff --git a/monetdb5/modules/mosaic/TODO_MOSAIC.txt b/monetdb5/modules/mosaic/TODO_MOSAIC.txt --- a/monetdb5/modules/mosaic/TODO_MOSAIC.txt +++ b/monetdb5/modules/mosaic/TODO_MOSAIC.txt @@ -1,6 +1,7 @@ technical depth +FIX anti stuff in select and antiselect Finish null semantics plus tests -Add test for join nil_matches set for non-nill tests. +Add test for join nil_matches set for non-nill tests in combination with (1) make sure candidate lists are working properly for select, theta_select and join (2) Be sure alignment is correct. Use asserts for this. Add test for float/double @@ -17,6 +18,8 @@ Add asserts on type degeneration for bit benchmark code issues +FIX multi column sort in join tests(1) +rename test files like (.*)nills(.*) to $1nils$2 Update copyright. create dedicated type for bitvector internal type which is currently unsignned int. E.g. introduce #define BitVectorChunkType unsigned int. create a common header/source file for mosaic for common tasks e.g. to house a generic IS_NIL macro, IPtpe, etc. ___ checkin-list mailing list checkin-list@monetdb.org https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: mosaic - Update TODO's.
Changeset: 66deb446b4e4 for MonetDB URL: https://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=66deb446b4e4 Modified Files: monetdb5/modules/mosaic/TODO_MOSAIC.txt Branch: mosaic Log Message: Update TODO's. diffs (18 lines): diff --git a/monetdb5/modules/mosaic/TODO_MOSAIC.txt b/monetdb5/modules/mosaic/TODO_MOSAIC.txt --- a/monetdb5/modules/mosaic/TODO_MOSAIC.txt +++ b/monetdb5/modules/mosaic/TODO_MOSAIC.txt @@ -1,4 +1,5 @@ technical depth +FIX HAS_NIL optimization in join algorithms Use bit vectors in mosaic_delta --- then resolve test mosaic_delta_timestamp --- and possibly other compression techniques as well.(2) Finish null semantics plus tests Add test for join nil_matches set for non-nill tests. @@ -21,7 +22,7 @@ benchmark code issues Update copyright. create dedicated type for bitvector internal type which is currently unsignned int. E.g. introduce #define BitVectorChunkType unsigned int. -create a common header/source file for mosaic for common tasks e.g. to house a generic IS_NIL macro. +create a common header/source file for mosaic for common tasks e.g. to house a generic IS_NIL macro, IPtpe, etc. move mask, bits, framebits from MosaicHdr to MosaicBlkHeader_frame_t. mosaic_delta.c: improve the macro's: for instance get rid of hardcoded expression arguments in DELTAcompress. (2) mosaic_.c: for type macro's use macro-wised inline functions. ___ checkin-list mailing list checkin-list@monetdb.org https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: mosaic - Update TODO's.
Changeset: c5bf287ab2b4 for MonetDB URL: https://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=c5bf287ab2b4 Modified Files: monetdb5/modules/mosaic/TODO_MOSAIC.txt Branch: mosaic Log Message: Update TODO's. diffs (20 lines): diff --git a/monetdb5/modules/mosaic/TODO_MOSAIC.txt b/monetdb5/modules/mosaic/TODO_MOSAIC.txt --- a/monetdb5/modules/mosaic/TODO_MOSAIC.txt +++ b/monetdb5/modules/mosaic/TODO_MOSAIC.txt @@ -1,14 +1,14 @@ technical depth +Use bit vectors in mosaic_delta --- then resolve test mosaic_delta_timestamp --- and possibly other compression techniques as well.(2) Finish null semantics plus tests Add test for join nil_matches set for non-nill tests. make sure candidate lists are working properly for select, theta_select and join -make mitosis work with mosaic and see how this relates to the previous landmark stuff. Be sure alignment is correct. Use asserts for this. Add test for float/double Add test for huge Check MOSanalysis and its test. Fix/check/test MOSlayout. -Use bit vectors in mosaic_delta --- then resolve test mosaic_delta_timestamp --- and possibly other compression techniques as well.(2) +make mitosis work with mosaic and see how this relates to the previous landmark stuff. Remove floating point support in prefix compression and use prefix-like-frame semantics Apply compressed operations directly on keys in dictionary compression. ___ checkin-list mailing list checkin-list@monetdb.org https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: mosaic - Update TODO's.
Changeset: 282b8928cc17 for MonetDB URL: https://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=282b8928cc17 Modified Files: monetdb5/modules/mosaic/TODO_MOSAIC.txt Branch: mosaic Log Message: Update TODO's. diffs (39 lines): diff --git a/monetdb5/modules/mosaic/TODO_MOSAIC.txt b/monetdb5/modules/mosaic/TODO_MOSAIC.txt --- a/monetdb5/modules/mosaic/TODO_MOSAIC.txt +++ b/monetdb5/modules/mosaic/TODO_MOSAIC.txt @@ -1,16 +1,21 @@ technical depth -null semantics plus tests +Look into select cases * v v A* B* (false|false) they some to be wrong & not tested +Finish null semantics plus tests make sure candidate lists are working properly for select, theta_select and join +make mitosis work with mosaic and see how this relates to the previous landmark stuff. +Be sure alignment is correct. Use asserts for this. Add test for float/double Add test for huge -Use bit vectors more aggressively in mosaic_delta --- then resolve test mosaic_delta_timestamp --- and possibly other compression techniques as well. -Be sure alignment is correct. Use asserts for this. -consider packing mosaic_block_header's more efficiently. Check MOSanalysis and its test. Fix/check/test MOSlayout. -delta compression: check if the hardcoded compression expression are correct. See also (2) +Use bit vectors in mosaic_delta --- then resolve test mosaic_delta_timestamp --- and possibly other compression techniques as well.(2) +Remove floating point support in prefix compression and use prefix-like-frame semantics +Apply compressed operations directly on keys in dictionary compression. + +consider packing mosaic_block_header's more efficiently. +delta compression: check if the hardcoded compression expression are correct. (2*) Add asserts on type degeneration for bit => bte daytime => lng timestmap => lng date => int. -See if we can re-introduce landmark stuff somehow. + benchmark code issues @@ -20,6 +25,6 @@ create a common header/source file for m Introduce FrameTpe similar to PrefixTpe. share more code between capped and var. move mask, bits, framebits from MosaicHdr to MosaicBlkHeader_frame_t. -mosaic_delta.c: improve the macro's: for instance get rid of hardcoded expression arguments in DELTAcompress. (2*) +mosaic_delta.c: improve the macro's: for instance get rid of hardcoded expression arguments in DELTAcompress. (2) mosaic_.c: for type macro's use macro-wised inline functions. Improve the administration of the various hard coded compression techniques. (2*) ___ checkin-list mailing list checkin-list@monetdb.org https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: mosaic - Update TODO's.
Changeset: cb8f1e41d854 for MonetDB URL: https://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=cb8f1e41d854 Modified Files: monetdb5/modules/mosaic/TODO_MOSAIC.txt Branch: mosaic Log Message: Update TODO's. diffs (29 lines): diff --git a/monetdb5/modules/mosaic/TODO_MOSAIC.txt b/monetdb5/modules/mosaic/TODO_MOSAIC.txt --- a/monetdb5/modules/mosaic/TODO_MOSAIC.txt +++ b/monetdb5/modules/mosaic/TODO_MOSAIC.txt @@ -1,13 +1,13 @@ technical depth null semantics plus tests make sure candidate lists are working properly for select, theta_select and join +Add test for float/double +Add test for huge Use bit vectors more aggressively in mosaic_delta --- then resolve test mosaic_delta_timestamp --- and possibly other compression techniques as well. Be sure alignment is correct. Use asserts for this. consider packing mosaic_block_header's more efficiently. Check MOSanalysis and its test. Fix/check/test MOSlayout. -Add test for float/double -Add test for huge delta compression: check if the hardcoded compression expression are correct. See also (2) Add asserts on type degeneration for bit => bte daytime => lng timestmap => lng date => int. See if we can re-introduce landmark stuff somehow. @@ -16,6 +16,9 @@ benchmark code issues Update copyright. create dedicated type for bitvector internal type which is currently unsignned int. E.g. introduce #define BitVectorChunkType unsigned int. +create a common header/source file for mosaic for common tasks e.g. to house a generic IS_NIL macro. +Introduce FrameTpe similar to PrefixTpe. +share more code between capped and var. move mask, bits, framebits from MosaicHdr to MosaicBlkHeader_frame_t. mosaic_delta.c: improve the macro's: for instance get rid of hardcoded expression arguments in DELTAcompress. (2*) mosaic_.c: for type macro's use macro-wised inline functions. ___ checkin-list mailing list checkin-list@monetdb.org https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: mosaic - Update TODO's.
Changeset: b95e01e84897 for MonetDB URL: https://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=b95e01e84897 Modified Files: monetdb5/modules/mosaic/TODO_MOSAIC.txt Branch: mosaic Log Message: Update TODO's. diffs (27 lines): diff --git a/monetdb5/modules/mosaic/TODO_MOSAIC.txt b/monetdb5/modules/mosaic/TODO_MOSAIC.txt --- a/monetdb5/modules/mosaic/TODO_MOSAIC.txt +++ b/monetdb5/modules/mosaic/TODO_MOSAIC.txt @@ -1,11 +1,13 @@ technical depth -Test null semantics -mosaic.select (c,nil:any_1, nil:any_1,true,true,(true|false)) is not working correctly: should result in c == nil or c!=nil(anti). +null semantics plus tests +make sure candidate lists are working properly for select, theta_select and join Use bit vectors more aggressively in mosaic_delta --- then resolve test mosaic_delta_timestamp --- and possibly other compression techniques as well. Be sure alignment is correct. Use asserts for this. +consider packing mosaic_block_header's more efficiently. Check MOSanalysis and its test. Fix/check/test MOSlayout. Add test for float/double +Add test for huge delta compression: check if the hardcoded compression expression are correct. See also (2) Add asserts on type degeneration for bit => bte daytime => lng timestmap => lng date => int. See if we can re-introduce landmark stuff somehow. @@ -15,7 +17,6 @@ benchmark Update copyright. create dedicated type for bitvector internal type which is currently unsignned int. E.g. introduce #define BitVectorChunkType unsigned int. move mask, bits, framebits from MosaicHdr to MosaicBlkHeader_frame_t. -mosaic_prefix.c: prefix code does not use macro's mosaic_delta.c: improve the macro's: for instance get rid of hardcoded expression arguments in DELTAcompress. (2*) mosaic_.c: for type macro's use macro-wised inline functions. Improve the administration of the various hard coded compression techniques. (2*) ___ checkin-list mailing list checkin-list@monetdb.org https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: mosaic - Update TODO's.
Changeset: 3817007ac9b0 for MonetDB URL: https://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=3817007ac9b0 Modified Files: monetdb5/modules/mosaic/TODO_MOSAIC.txt Branch: mosaic Log Message: Update TODO's. diffs (25 lines): diff --git a/monetdb5/modules/mosaic/TODO_MOSAIC.txt b/monetdb5/modules/mosaic/TODO_MOSAIC.txt --- a/monetdb5/modules/mosaic/TODO_MOSAIC.txt +++ b/monetdb5/modules/mosaic/TODO_MOSAIC.txt @@ -1,21 +1,15 @@ technical depth -Check if compressors are always called in tests. Use MOSdestroy trick for that. -Make dictionary variable-sized so it can always compress. We are going to use vheap for this. -dictionary encoding should use the ordered property of its dictionary in the algebraic operators Test null semantics mosaic.select (c,nil:any_1, nil:any_1,true,true,(true|false)) is not working correctly: should result in c == nil or c!=nil(anti). Use bit vectors more aggressively in mosaic_delta --- then resolve test mosaic_delta_timestamp --- and possibly other compression techniques as well. -Somehow force mosaic to only use specified compressions even if inefficient. This poses a design problem for dictionary compression though. Be sure alignment is correct. Use asserts for this. Check MOSanalysis and its test. Fix/check/test MOSlayout. Add test for float/double delta compression: check if the hardcoded compression expression are correct. See also (2) Add asserts on type degeneration for bit => bte daytime => lng timestmap => lng date => int. -Fix the task-factor array. See if we can re-introduce landmark stuff somehow. benchmark -Do estimation before mosaic heap creation.That saves a potential heap extend. code issues Update copyright. ___ checkin-list mailing list checkin-list@monetdb.org https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: mosaic - Update TODO's.
Changeset: b50fa802d315 for MonetDB URL: https://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=b50fa802d315 Modified Files: monetdb5/modules/mosaic/TODO_MOSAIC.txt Branch: mosaic Log Message: Update TODO's. diffs (26 lines): diff --git a/monetdb5/modules/mosaic/TODO_MOSAIC.txt b/monetdb5/modules/mosaic/TODO_MOSAIC.txt --- a/monetdb5/modules/mosaic/TODO_MOSAIC.txt +++ b/monetdb5/modules/mosaic/TODO_MOSAIC.txt @@ -1,19 +1,19 @@ technical depth +Check if compressors are always called in tests. Use MOSdestroy trick for that. Make dictionary variable-sized so it can always compress. We are going to use vheap for this. dictionary encoding should use the ordered property of its dictionary in the algebraic operators -decouple estimation completely from compression: there should be two runs: one for estimating the blocks and one to actual compress according to the estimations. Test null semantics mosaic.select (c,nil:any_1, nil:any_1,true,true,(true|false)) is not working correctly: should result in c == nil or c!=nil(anti). Use bit vectors more aggressively in mosaic_delta --- then resolve test mosaic_delta_timestamp --- and possibly other compression techniques as well. Somehow force mosaic to only use specified compressions even if inefficient. This poses a design problem for dictionary compression though. Be sure alignment is correct. Use asserts for this. -fix layout call. +Check MOSanalysis and its test. +Fix/check/test MOSlayout. Add test for float/double delta compression: check if the hardcoded compression expression are correct. See also (2) Add asserts on type degeneration for bit => bte daytime => lng timestmap => lng date => int. Fix the task-factor array. See if we can re-introduce landmark stuff somehow. -Fix/check/test MOSlayout. benchmark Do estimation before mosaic heap creation.That saves a potential heap extend. ___ checkin-list mailing list checkin-list@monetdb.org https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: mosaic - Update TODO's.
Changeset: d5c0b11a9d94 for MonetDB URL: https://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=d5c0b11a9d94 Modified Files: monetdb5/modules/mosaic/TODO_MOSAIC.txt Branch: mosaic Log Message: Update TODO's. diffs (11 lines): diff --git a/monetdb5/modules/mosaic/TODO_MOSAIC.txt b/monetdb5/modules/mosaic/TODO_MOSAIC.txt --- a/monetdb5/modules/mosaic/TODO_MOSAIC.txt +++ b/monetdb5/modules/mosaic/TODO_MOSAIC.txt @@ -18,6 +18,7 @@ benchmark code issues Update copyright. +create dedicated type for bitvector internal type which is currently unsignned int. E.g. introduce #define BitVectorChunkType unsigned int. move mask, bits, framebits from MosaicHdr to MosaicBlkHeader_frame_t. mosaic_prefix.c: prefix code does not use macro's mosaic_delta.c: improve the macro's: for instance get rid of hardcoded expression arguments in DELTAcompress. (2*) ___ checkin-list mailing list checkin-list@monetdb.org https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: mosaic - Update TODO's.
Changeset: 557d77fd57c6 for MonetDB URL: https://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=557d77fd57c6 Modified Files: monetdb5/modules/mosaic/TODO_MOSAIC.txt Branch: mosaic Log Message: Update TODO's. diffs (11 lines): diff --git a/monetdb5/modules/mosaic/TODO_MOSAIC.txt b/monetdb5/modules/mosaic/TODO_MOSAIC.txt --- a/monetdb5/modules/mosaic/TODO_MOSAIC.txt +++ b/monetdb5/modules/mosaic/TODO_MOSAIC.txt @@ -6,6 +6,7 @@ Test null semantics mosaic.select (c,nil:any_1, nil:any_1,true,true,(true|false)) is not working correctly: should result in c == nil or c!=nil(anti). Use bit vectors more aggressively in mosaic_delta --- then resolve test mosaic_delta_timestamp --- and possibly other compression techniques as well. Somehow force mosaic to only use specified compressions even if inefficient. This poses a design problem for dictionary compression though. +Be sure alignment is correct. Use asserts for this. fix layout call. Add test for float/double delta compression: check if the hardcoded compression expression are correct. See also (2) ___ checkin-list mailing list checkin-list@monetdb.org https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: mosaic - Update TODO's.
Changeset: 6ac5952288da for MonetDB URL: https://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=6ac5952288da Modified Files: monetdb5/modules/mosaic/TODO_MOSAIC.txt Branch: mosaic Log Message: Update TODO's. diffs (11 lines): diff --git a/monetdb5/modules/mosaic/TODO_MOSAIC.txt b/monetdb5/modules/mosaic/TODO_MOSAIC.txt --- a/monetdb5/modules/mosaic/TODO_MOSAIC.txt +++ b/monetdb5/modules/mosaic/TODO_MOSAIC.txt @@ -16,6 +16,7 @@ Fix/check/test MOSlayout. benchmark code issues +Update copyright. move mask, bits, framebits from MosaicHdr to MosaicBlkHeader_frame_t. mosaic_prefix.c: prefix code does not use macro's mosaic_delta.c: improve the macro's: for instance get rid of hardcoded expression arguments in DELTAcompress. (2*) ___ checkin-list mailing list checkin-list@monetdb.org https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: mosaic - Update TODO's.
Changeset: 58a5d3dc8685 for MonetDB URL: https://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=58a5d3dc8685 Modified Files: monetdb5/modules/mosaic/TODO_MOSAIC.txt Branch: mosaic Log Message: Update TODO's. diffs (30 lines): diff --git a/monetdb5/modules/mosaic/TODO_MOSAIC.txt b/monetdb5/modules/mosaic/TODO_MOSAIC.txt --- a/monetdb5/modules/mosaic/TODO_MOSAIC.txt +++ b/monetdb5/modules/mosaic/TODO_MOSAIC.txt @@ -1,20 +1,22 @@ technical depth -Make dictionary variable-sized so it can always compress. -Add asserts for unknown compression strategies. (1) +Make dictionary variable-sized so it can always compress. We are going to use vheap for this. +dictionary encoding should use the ordered property of its dictionary in the algebraic operators +decouple estimation completely from compression: there should be two runs: one for estimating the blocks and one to actual compress according to the estimations. Test null semantics mosaic.select (c,nil:any_1, nil:any_1,true,true,(true|false)) is not working correctly: should result in c == nil or c!=nil(anti). -dictionary encoding should use the ordered property of its dictionary in the algebraic operators Use bit vectors more aggressively in mosaic_delta --- then resolve test mosaic_delta_timestamp --- and possibly other compression techniques as well. Somehow force mosaic to only use specified compressions even if inefficient. This poses a design problem for dictionary compression though. +fix layout call. Add test for float/double delta compression: check if the hardcoded compression expression are correct. See also (2) Add asserts on type degeneration for bit => bte daytime => lng timestmap => lng date => int. -Remove task-factor and range->factor or refactor them. +Fix the task-factor array. See if we can re-introduce landmark stuff somehow. Fix/check/test MOSlayout. benchmark code issues +move mask, bits, framebits from MosaicHdr to MosaicBlkHeader_frame_t. mosaic_prefix.c: prefix code does not use macro's mosaic_delta.c: improve the macro's: for instance get rid of hardcoded expression arguments in DELTAcompress. (2*) mosaic_.c: for type macro's use macro-wised inline functions. ___ checkin-list mailing list checkin-list@monetdb.org https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: mosaic - Update TODO's.
Changeset: 3bcbac3e0e4e for MonetDB URL: https://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=3bcbac3e0e4e Modified Files: monetdb5/modules/mosaic/TODO_MOSAIC.txt Branch: mosaic Log Message: Update TODO's. diffs (23 lines): diff --git a/monetdb5/modules/mosaic/TODO_MOSAIC.txt b/monetdb5/modules/mosaic/TODO_MOSAIC.txt --- a/monetdb5/modules/mosaic/TODO_MOSAIC.txt +++ b/monetdb5/modules/mosaic/TODO_MOSAIC.txt @@ -1,16 +1,16 @@ technical depth -Somehow log/print which compression is being used. Potentially use (3) +Add asserts for unknown compression strategies. Test null semantics mosaic.select (c,nil:any_1, nil:any_1,true,true,(true|false)) is not working correctly: should result in c == nil or c!=nil(anti). dictionary encoding should use the ordered property of its dictionary in the algebraic operators Use bit vectors more aggressively in mosaic_delta --- then resolve test mosaic_delta_timestamp --- and possibly other compression techniques as well. -Somehow force mosaic to only use specified compressions even if inefficient. +Somehow force mosaic to only use specified compressions even if inefficient. This poses a design problem for dictionary compression though. Add test for float/double delta compression: check if the hardcoded compression expression are correct. See also (2) Add asserts on type degeneration for bit => bte daytime => lng timestmap => lng date => int. Remove task-factor and range->factor or refactor them. See if we can re-introduce landmark stuff somehow. -Fix/check/test MOSlayout. (3*) +Fix/check/test MOSlayout. benchmark code issues ___ checkin-list mailing list checkin-list@monetdb.org https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: mosaic - Update TODO's.
Changeset: 4a06e3e7784a for MonetDB URL: https://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=4a06e3e7784a Modified Files: monetdb5/modules/mosaic/TODO_MOSAIC.txt Branch: mosaic Log Message: Update TODO's. diffs (25 lines): diff --git a/monetdb5/modules/mosaic/TODO_MOSAIC.txt b/monetdb5/modules/mosaic/TODO_MOSAIC.txt --- a/monetdb5/modules/mosaic/TODO_MOSAIC.txt +++ b/monetdb5/modules/mosaic/TODO_MOSAIC.txt @@ -2,17 +2,19 @@ Somehow log/print which compression is being used. Potentially use (3) Test null semantics mosaic.select (c,nil:any_1, nil:any_1,true,true,(true|false)) is not working correctly: should result in c == nil or c!=nil(anti). +dictionary encoding should use the ordered property of its dictionary in the algebraic operators Use bit vectors more aggressively in mosaic_delta --- then resolve test mosaic_delta_timestamp --- and possibly other compression techniques as well. +Somehow force mosaic to only use specified compressions even if inefficient. Add test for float/double -dictionary encoding should use the ordered property of its dictionary in the algebraic operators delta compression: check if the hardcoded compression expression are correct. See also (2) Add asserts on type degeneration for bit => bte daytime => lng timestmap => lng date => int. -Somehow force mosaic to only use specified compressions even if inefficient. Remove task-factor and range->factor or refactor them. See if we can re-introduce landmark stuff somehow. Fix/check/test MOSlayout. (3*) +benchmark code issues mosaic_prefix.c: prefix code does not use macro's mosaic_delta.c: improve the macro's: for instance get rid of hardcoded expression arguments in DELTAcompress. (2*) mosaic_.c: for type macro's use macro-wised inline functions. +Improve the administration of the various hard coded compression techniques. ___ checkin-list mailing list checkin-list@monetdb.org https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: mosaic - Update TODO's.
Changeset: 869562145dba for MonetDB URL: https://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=869562145dba Modified Files: monetdb5/modules/mosaic/TODO_MOSAIC.txt Branch: mosaic Log Message: Update TODO's. diffs (29 lines): diff --git a/monetdb5/modules/mosaic/TODO_MOSAIC.txt b/monetdb5/modules/mosaic/TODO_MOSAIC.txt --- a/monetdb5/modules/mosaic/TODO_MOSAIC.txt +++ b/monetdb5/modules/mosaic/TODO_MOSAIC.txt @@ -1,17 +1,18 @@ technical depth +Somehow log/print which compression is being used. Potentially use (3) Test null semantics mosaic.select (c,nil:any_1, nil:any_1,true,true,(true|false)) is not working correctly: should result in c == nil or c!=nil(anti). +Use bit vectors more aggressively in mosaic_delta --- then resolve test mosaic_delta_timestamp --- and possibly other compression techniques as well. Add test for float/double -Add test for bit -Remove task-factor and range->factor or refactor them. -Somehow log/print which compression is being used. +dictionary encoding should use the ordered property of its dictionary in the algebraic operators delta compression: check if the hardcoded compression expression are correct. See also (2) -Use bit vectors more aggressively in mosaic_delta --- then resolve test mosaic_delta_timestamp --- and possibly other compression techniques as well. -dictionary encoding should use the ordered property of its dictionary in the algebraic operators +Add asserts on type degeneration for bit => bte daytime => lng timestmap => lng date => int. +Somehow force mosaic to only use specified compressions even if inefficient. +Remove task-factor and range->factor or refactor them. See if we can re-introduce landmark stuff somehow. -Add asserts on type degeneration for bit => bte daytime => lng timestmap => lng date => int. +Fix/check/test MOSlayout. (3*) code issues mosaic_prefix.c: prefix code does not use macro's -mosaic_delta.c: improve the macro's: for instance get rid of hardcoded expression arguments in DELTAcompress. (2) +mosaic_delta.c: improve the macro's: for instance get rid of hardcoded expression arguments in DELTAcompress. (2*) mosaic_.c: for type macro's use macro-wised inline functions. ___ checkin-list mailing list checkin-list@monetdb.org https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: mosaic - Update TODO's.
Changeset: c1c8714d044a for MonetDB URL: https://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=c1c8714d044a Modified Files: monetdb5/modules/mosaic/TODO_MOSAIC.txt Branch: mosaic Log Message: Update TODO's. diffs (17 lines): diff --git a/monetdb5/modules/mosaic/TODO_MOSAIC.txt b/monetdb5/modules/mosaic/TODO_MOSAIC.txt --- a/monetdb5/modules/mosaic/TODO_MOSAIC.txt +++ b/monetdb5/modules/mosaic/TODO_MOSAIC.txt @@ -1,5 +1,4 @@ technical depth -Check which compressions can handle bit/boolean types. Test null semantics mosaic.select (c,nil:any_1, nil:any_1,true,true,(true|false)) is not working correctly: should result in c == nil or c!=nil(anti). Add test for float/double @@ -10,6 +9,7 @@ delta compression: check if the hardcode Use bit vectors more aggressively in mosaic_delta --- then resolve test mosaic_delta_timestamp --- and possibly other compression techniques as well. dictionary encoding should use the ordered property of its dictionary in the algebraic operators See if we can re-introduce landmark stuff somehow. +Add asserts on type degeneration for bit => bte daytime => lng timestmap => lng date => int. code issues mosaic_prefix.c: prefix code does not use macro's ___ checkin-list mailing list checkin-list@monetdb.org https://www.monetdb.org/mailman/listinfo/checkin-list
