GitHub user foryou2030 opened a pull request:
https://github.com/apache/incubator-carbondata/pull/459
[CARBONDATA-558] Fix load performace when use_kettle=false
Why raise this pr?
When I load a data file, the measure column contains many empty strings, if
use_kettle=false, the load performance has a sharp decline
I checked the logs of executor, many warnnings printed like below:
16/12/22 07:03:12 WARN MeasureFieldConverterImpl: pool-22-thread-6 Cant not
convert : to Numeric type value. Value considered as null.
How to solve it?
When measureValue = "", we should set it as null directly, no need to do
datatype conversion
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/foryou2030/incubator-carbondata msr_null
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-carbondata/pull/459.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #459
----
commit eae356ba8e918f60c78c584ad31856c0f10403db
Author: foryou2030 <[email protected]>
Date: 2016-12-23T08:46:21Z
fix load performace
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
