Hi all,
Let's discuss whether it is necessary to support SORT_COLUMNS
modification.
*Background*
"SORT_COLUMNS" is a table level property, and we can't change it after
creating a table.
*Motivation*
When we want to optimize the query performance and found that it needs
to change SORT_COLUMNS, Carbon should support changing SORT_COLUMNS.
SORT_COLUMNS just like the main data index and impact the data layout. At
the same time, we can re-sort old segment data by new SORT_COLUMNS.
*Modification*
1. loading data use table level "SORT_COLUMNS" and store it as a
segment level property
2. query should use segment level property to read data files
3. only compacting segments with same "SORT_COLUMNS"
4. convert old segment one by one by new SORT_COLUMNS and refresh
DataMap if needed
5. show segment command output SORT_COLUMNS of each segment
Any suggestions and questions?
-----
Best Regards
David Cai
--
Sent from:
http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
