[
https://issues.apache.org/jira/browse/KYLIN-689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14497461#comment-14497461
]
Julian Hyde commented on KYLIN-689:
-----------------------------------
A number like 35.800000000000004 probably comes from a floating point number
(java float or double) being expressed as a decimal. Many decimal numbers (e.g
35.8) cannot be expressed exactly as a floating point, and so when they are
expressed as a floating point they acquire "noise" in their least significant
digits.
The solution to this issue would be to allow a dictionary based upon floating
point numbers.
But of course users would be well advised to stick to integer or decimal values
instead.
> Too many digits for NumberDictionary
> ------------------------------------
>
> Key: KYLIN-689
> URL: https://issues.apache.org/jira/browse/KYLIN-689
> Project: Kylin
> Issue Type: Bug
> Components: Metadata
> Affects Versions: v0.6.5, v0.7.1
> Reporter: Xiaoyu Wang
> Assignee: Shaofeng SHI
>
> The double field value is 35.800000000000004
> Build cube got error:
> java.lang.IllegalArgumentException: Too many digits for NumberDictionary:
> 35.800000000000004. Expect 16 digits before decimal point at max.
> at
> com.kylinolap.dict.NumberDictionary$NumberBytesCodec.encodeNumber(NumberDictionary.java:76)
> at
> com.kylinolap.dict.NumberDictionaryBuilder.addValue(NumberDictionaryBuilder.java:35)
> at
> com.kylinolap.dict.TrieDictionaryBuilder.addValue(TrieDictionaryBuilder.java:83)
> at
> com.kylinolap.dict.DictionaryGenerator.buildNumberDict(DictionaryGenerator.java:164)
> at
> com.kylinolap.dict.DictionaryGenerator.buildDictionaryFromValueList(DictionaryGenerator.java:63)
> at
> com.kylinolap.dict.DictionaryGenerator.buildDictionary(DictionaryGenerator.java:114)
> at
> com.kylinolap.dict.DictionaryManager.buildDictionary(DictionaryManager.java:166)
> at
> com.kylinolap.cube.CubeManager.buildDictionary(CubeManager.java:171)
> at
> com.kylinolap.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:50)
> at
> com.kylinolap.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:39)
> at
> com.kylinolap.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:39)
> at
> com.kylinolap.job.hadoop.dict.CreateDictionaryJob.run(CreateDictionaryJob.java:51)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> at com.kylinolap.job.cmd.JavaHadoopCmd.execute(JavaHadoopCmd.java:54)
> at com.kylinolap.job.flow.JobFlowNode.execute(JobFlowNode.java:77)
> at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
> at
> org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)
> [QuartzScheduler_Worker-19]:[2015-04-15
> 10:29:39,381][ERROR][com.kylinolap.job.hadoop.dict.CreateDictionaryJob.run(CreateDictionaryJob.java:55)]
> - Too many digits for NumberDictionary: 35.800000000000004. Expect 16 digits
> before decimal point at max.
> java.lang.IllegalArgumentException: Too many digits for NumberDictionary:
> 35.800000000000004. Expect 16 digits before decimal point at max.
> at
> com.kylinolap.dict.NumberDictionary$NumberBytesCodec.encodeNumber(NumberDictionary.java:76)
> at
> com.kylinolap.dict.NumberDictionaryBuilder.addValue(NumberDictionaryBuilder.java:35)
> at
> com.kylinolap.dict.TrieDictionaryBuilder.addValue(TrieDictionaryBuilder.java:83)
> at
> com.kylinolap.dict.DictionaryGenerator.buildNumberDict(DictionaryGenerator.java:164)
> at
> com.kylinolap.dict.DictionaryGenerator.buildDictionaryFromValueList(DictionaryGenerator.java:63)
> at
> com.kylinolap.dict.DictionaryGenerator.buildDictionary(DictionaryGenerator.java:114)
> at
> com.kylinolap.dict.DictionaryManager.buildDictionary(DictionaryManager.java:166)
> at
> com.kylinolap.cube.CubeManager.buildDictionary(CubeManager.java:171)
> at
> com.kylinolap.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:50)
> at
> com.kylinolap.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:39)
> at
> com.kylinolap.job.hadoop.dict.CreateDictionaryJob.run(CreateDictionaryJob.java:51)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> at com.kylinolap.job.cmd.JavaHadoopCmd.execute(JavaHadoopCmd.java:54)
> at com.kylinolap.job.flow.JobFlowNode.execute(JobFlowNode.java:77)
> at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
> at
> org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
