[
https://issues.apache.org/jira/browse/CARBONDATA-315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15575376#comment-15575376
]
ASF GitHub Bot commented on CARBONDATA-315:
-------------------------------------------
Github user jackylk commented on a diff in the pull request:
https://github.com/apache/incubator-carbondata/pull/234#discussion_r83424392
--- Diff:
processing/src/main/java/org/apache/carbondata/processing/surrogatekeysgenerator/csvbased/CarbonCSVBasedSeqGenStep.java
---
@@ -994,29 +994,31 @@ private String getCarbonLocalBaseStoreLocation() {
}
} else {
try {
- out[memberMapping[dimLen + index] - meta.complexTypes.size()] =
- (isNull || msr == null || msr.length() == 0) ?
- null :
- DataTypeUtil
- .getMeasureValueBasedOnDataType(msr,
msrDataType[meta.msrMapping[msrCount]],
-
meta.carbonMeasures[meta.msrMapping[msrCount]]);
- } catch (NumberFormatException e) {
- try {
- msr = msr.replaceAll(",", "");
- out[memberMapping[dimLen + index] -
meta.complexTypes.size()] = DataTypeUtil
+ if (!isNull && null != msr && msr.length() > 0) {
+ Object measureValueBasedOnDataType = DataTypeUtil
--- End diff --
put `DataTypeUtil` in next line
> Data loading fails if parsing a double value returns infinity
> -------------------------------------------------------------
>
> Key: CARBONDATA-315
> URL: https://issues.apache.org/jira/browse/CARBONDATA-315
> Project: CarbonData
> Issue Type: Bug
> Affects Versions: 0.1.0-incubating, 0.2.0-incubating
> Reporter: Manish Gupta
> Assignee: Manish Gupta
> Priority: Minor
> Fix For: 0.2.0-incubating
>
>
> During data load, if a value specified is too big for a double DataType
> column then while parsing that value as double result is returned as
> "Infinity". Due to this while we calculate min and max value for measures in
> carbon data writer step it throws an exception.
> ERROR 13-10 15:27:56,968 - [t3: Graph - MDKeyGent3][partitionID:0]
> org.apache.carbondata.processing.store.writer.exception.CarbonDataWriterException
> java.util.concurrent.ExecutionException:
> org.apache.carbondata.processing.store.writer.exception.CarbonDataWriterException
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:188)
> at
> org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar.processWriteTaskSubmitList(CarbonFactDataHandlerColumnar.java:812)
> at
> org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar.finish(CarbonFactDataHandlerColumnar.java:779)
> at
> org.apache.carbondata.processing.mdkeygen.MDKeyGenStep.processRow(MDKeyGenStep.java:222)
> at org.pentaho.di.trans.step.RunThread.run(RunThread.java:50)
> at java.lang.Thread.run(Thread.java:745)
> Caused by:
> org.apache.carbondata.processing.store.writer.exception.CarbonDataWriterException
> at
> org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar$Producer.call(CarbonFactDataHandlerColumnar.java:1244)
> at
> org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar$Producer.call(CarbonFactDataHandlerColumnar.java:1215)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> ... 1 more
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
