GitHub user xuchuanyin opened a pull request:
https://github.com/apache/carbondata/pull/2380
[CARBONDATA-2509][CARBONDATA-2510][CARBONDATA-2511][32K] Add validate for
long string columns
1. long string columns cannot be sort_columns
2. long string columns cannot be dictionary include
3. long string columns cannot be dictionary exclude
4. long string columns can only be string columns
Be sure to do all of the following checklist to help us incorporate
your contribution quickly and easily:
- [x] Any interfaces changed?
`NO`
- [ ] Any backward compatibility impacted?
`NO`
- [ ] Document update required?
`NO`
- [ ] Testing done
Please provide details on
- Whether new unit test cases have been added or why no new tests
are required?
`Added tests`
- How it is tested? Please attach test report.
`Tested in local`
- Is it a performance related change? Please attach the performance
test report.
`NO`
- Any additional information to help reviewers in testing this
change.
- [ ] For large changes, please consider breaking it into sub-tasks under
an umbrella JIRA.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/xuchuanyin/carbondata
0620_longstring_nodict_nosort
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/carbondata/pull/2380.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2380
----
commit b689d66493521452ff9938415e0d0aa66b56c2c5
Author: xuchuanyin <xuchuanyin@...>
Date: 2018-06-02T07:17:04Z
Support string longer than 32000 characters
Add a table property 'long_string_columns' in create table DDL that
indicate those columns will contain more than 32000 characters.
Internally in Carbondata,
1. add a new datatype called `text` to represent the long string column
2. add a new encoding called `DIRECT_COMPRESS_TEXT` to the text column
page meta
3. Use an integer (previously short) to store the length of bytes
content.
commit f145c6c60238c400b5db6a6bf2696246b698154a
Author: xuchuanyin <xuchuanyin@...>
Date: 2018-06-05T12:46:26Z
rename datatype name from text to varchar
commit 4180f8118d1ff90205b0f1567bef2cdfee3a1b62
Author: xuchuanyin <xuchuanyin@...>
Date: 2018-06-12T12:35:58Z
Add 2GB constraint for one column page
commit 710845b155ed5b7a611a900c70b0d766d80ae48d
Author: xuchuanyin <xuchuanyin@...>
Date: 2018-06-14T12:11:40Z
update tests
commit b994cee490b82d8ec0a1a4cefec6a3523785b016
Author: xuchuanyin <xuchuanyin@...>
Date: 2018-06-19T07:11:21Z
Add validate for long string columns
1. long string columns cannot be sort_columns
2. long string columns cannot be dictionary include
3. long string columns cannot be dictionary exclude
4. long string columns can only be string columns
----
---
