[GitHub] carbondata pull request #2903: [CARBONDATA-3084]dataload failure fix when fl...

2018-11-13 Thread asfgit 
Github user asfgit closed the pull request at:

https://github.com/apache/carbondata/pull/2903


---

[GitHub] carbondata pull request #2903: [CARBONDATA-3084]dataload failure fix when fl...

2018-11-06 Thread kunal642 
Github user kunal642 commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2903#discussion_r231038773
  
--- Diff: 
core/src/main/java/org/apache/carbondata/core/datastore/page/encoding/DefaultEncodingFactory.java
 ---
@@ -325,32 +325,42 @@ static ColumnPageCodec 
selectCodecByAlgorithmForFloating(SimpleStatsResult stats
 //Here we should use the Max abs as max to getDatatype, let's say -1 
and -1000, -1 is max,
 //but we can't use -1 to getDatatype, we should use -1000.
 double absMaxValue = Math.max(Math.abs(maxValue), Math.abs(minValue));
-if (decimalCount == 0) {
+if (decimalCount == 0 && srcDataType == DataTypes.FLOAT) {
--- End diff --

no need to check for decimalCount == 0. If dataType is FLOAT call  
getColumnPageCodec(..)


---

[GitHub] carbondata pull request #2903: [CARBONDATA-3084]dataload failure fix when fl...

2018-11-05 Thread akashrn5 
GitHub user akashrn5 opened a pull request:

https://github.com/apache/carbondata/pull/2903

[CARBONDATA-3084]dataload failure fix when float value exceeds the limit

### Problem:
when the float value exceeds the range and we try to insert that data, data 
load fails.

### Analysis:
when the value exceeds the range, the max is set as `Infinity`, so the 
decimal count of that value will be 0, so when decimal count is zero we go for 
CodecByAlgorithmForIntegral, so it fails

### Solution:
when the value exceeds, and decimal count is zero , source datatype is 
float, then select DirectCompressCodec

### How this tested:
test cases are added to validate the load and data

Be sure to do all of the following checklist to help us incorporate 
your contribution quickly and easily:

 - [ ] Any interfaces changed?
 
 - [ ] Any backward compatibility impacted?
 
 - [ ] Document update required?

 - [ ] Testing done
Please provide details on 
- Whether new unit test cases have been added or why no new tests 
are required?
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance 
test report.
- Any additional information to help reviewers in testing this 
change.
   
 - [ ] For large changes, please consider breaking it into sub-tasks under 
an umbrella JIRA. 



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/akashrn5/incubator-carbondata float

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/2903.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2903


commit 798592441f77300dd89b1dec4dd51c15bc6b2c07
Author: akashrn5 
Date:   2018-11-06T07:06:24Z

dataload failure fix when float value exceeds the limit




---