[jira] [Commented] (KYLIN-1973) java.lang.NegativeArraySizeException when Build Dimension Dictionary

[Copperfield (JIRA)]

[ 
https://issues.apache.org/jira/browse/KYLIN-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15529126#comment-15529126
 ] 

Copperfield commented on KYLIN-1973:


Ok, I will try 1.5.4.1

On Wed, Sep 28, 2016 at 5:52 PM, Mario Copperfield 




-- 
Best regards,
Amuro Copperfield


> java.lang.NegativeArraySizeException when Build Dimension Dictionary
> 
>
> Key: KYLIN-1973
> URL: https://issues.apache.org/jira/browse/KYLIN-1973
> Project: Kylin
>  Issue Type: Bug
>Affects Versions: v1.5.3
>Reporter: zhengdong
>Assignee: liyang
> Fix For: v1.5.4
>
>
>  exception when Build Dimension Dictionary:
> java.lang.NegativeArraySizeException
>   at 
> org.apache.kylin.dict.TrieDictionary.getValueFromIdImpl(TrieDictionary.java:274)
>   at 
> org.apache.kylin.common.util.Dictionary.getValueFromId(Dictionary.java:130)
>   at 
> org.apache.kylin.dict.lookup.SnapshotTable$1.getRow(SnapshotTable.java:138)
>   at org.apache.kylin.dict.lookup.LookupTable.init(LookupTable.java:67)
>   at 
> org.apache.kylin.dict.lookup.LookupStringTable.init(LookupStringTable.java:79)
>   at org.apache.kylin.dict.lookup.LookupTable.(LookupTable.java:55)
>   at 
> org.apache.kylin.dict.lookup.LookupStringTable.(LookupStringTable.java:65)
>   at 
> org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:619)
>   at 
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:61)
>   at 
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:42)
>   at 
> org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:56)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>   at 
> org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
>   at 
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112)
>   at 
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57)
>   at 
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112)
>   at 
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:127)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>   at java.lang.Thread.run(Thread.java:745)
> result code:2



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KYLIN-1973) java.lang.NegativeArraySizeException when Build Dimension Dictionary

[Copperfield (JIRA)]

[ 
https://issues.apache.org/jira/browse/KYLIN-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15529122#comment-15529122
 ] 

Copperfield commented on KYLIN-1973:


I have tried it，but as Shaofeng say it also built in
lib/kylin-job-, so it still can't work well.





-- 
Best regards,
Amuro Copperfield


> java.lang.NegativeArraySizeException when Build Dimension Dictionary
> 
>
> Key: KYLIN-1973
> URL: https://issues.apache.org/jira/browse/KYLIN-1973
> Project: Kylin
>  Issue Type: Bug
>Affects Versions: v1.5.3
>Reporter: zhengdong
>Assignee: liyang
> Fix For: v1.5.4
>
>
>  exception when Build Dimension Dictionary:
> java.lang.NegativeArraySizeException
>   at 
> org.apache.kylin.dict.TrieDictionary.getValueFromIdImpl(TrieDictionary.java:274)
>   at 
> org.apache.kylin.common.util.Dictionary.getValueFromId(Dictionary.java:130)
>   at 
> org.apache.kylin.dict.lookup.SnapshotTable$1.getRow(SnapshotTable.java:138)
>   at org.apache.kylin.dict.lookup.LookupTable.init(LookupTable.java:67)
>   at 
> org.apache.kylin.dict.lookup.LookupStringTable.init(LookupStringTable.java:79)
>   at org.apache.kylin.dict.lookup.LookupTable.(LookupTable.java:55)
>   at 
> org.apache.kylin.dict.lookup.LookupStringTable.(LookupStringTable.java:65)
>   at 
> org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:619)
>   at 
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:61)
>   at 
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:42)
>   at 
> org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:56)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>   at 
> org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
>   at 
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112)
>   at 
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57)
>   at 
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112)
>   at 
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:127)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>   at java.lang.Thread.run(Thread.java:745)
> result code:2



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KYLIN-1973) java.lang.NegativeArraySizeException when Build Dimension Dictionary

[Shaofeng SHI (JIRA)]

[ 
https://issues.apache.org/jira/browse/KYLIN-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15529049#comment-15529049
 ] 

Shaofeng SHI commented on KYLIN-1973:
-

The change in kylin-core-dictionry will also be built into 
lib/kylin-job-.jar, so only update the kylin-core-dictionry.jar 
doesn't work. Suggest you apply the patch and then make a new binary package.

> java.lang.NegativeArraySizeException when Build Dimension Dictionary
> 
>
> Key: KYLIN-1973
> URL: https://issues.apache.org/jira/browse/KYLIN-1973
> Project: Kylin
>  Issue Type: Bug
>Affects Versions: v1.5.3
>Reporter: zhengdong
>Assignee: liyang
> Fix For: v1.5.4
>
>
>  exception when Build Dimension Dictionary:
> java.lang.NegativeArraySizeException
>   at 
> org.apache.kylin.dict.TrieDictionary.getValueFromIdImpl(TrieDictionary.java:274)
>   at 
> org.apache.kylin.common.util.Dictionary.getValueFromId(Dictionary.java:130)
>   at 
> org.apache.kylin.dict.lookup.SnapshotTable$1.getRow(SnapshotTable.java:138)
>   at org.apache.kylin.dict.lookup.LookupTable.init(LookupTable.java:67)
>   at 
> org.apache.kylin.dict.lookup.LookupStringTable.init(LookupStringTable.java:79)
>   at org.apache.kylin.dict.lookup.LookupTable.(LookupTable.java:55)
>   at 
> org.apache.kylin.dict.lookup.LookupStringTable.(LookupStringTable.java:65)
>   at 
> org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:619)
>   at 
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:61)
>   at 
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:42)
>   at 
> org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:56)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>   at 
> org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
>   at 
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112)
>   at 
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57)
>   at 
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112)
>   at 
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:127)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>   at java.lang.Thread.run(Thread.java:745)
> result code:2



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KYLIN-1973) java.lang.NegativeArraySizeException when Build Dimension Dictionary

[Copperfield (JIRA)]

[ 
https://issues.apache.org/jira/browse/KYLIN-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15528823#comment-15528823
 ] 

Copperfield commented on KYLIN-1973:


>From the step 1？
Do you mean that i should discard this job and rebuild the segment?

> java.lang.NegativeArraySizeException when Build Dimension Dictionary
> 
>
> Key: KYLIN-1973
> URL: https://issues.apache.org/jira/browse/KYLIN-1973
> Project: Kylin
>  Issue Type: Bug
>Affects Versions: v1.5.3
>Reporter: zhengdong
>Assignee: liyang
> Fix For: v1.5.4
>
>
>  exception when Build Dimension Dictionary:
> java.lang.NegativeArraySizeException
>   at 
> org.apache.kylin.dict.TrieDictionary.getValueFromIdImpl(TrieDictionary.java:274)
>   at 
> org.apache.kylin.common.util.Dictionary.getValueFromId(Dictionary.java:130)
>   at 
> org.apache.kylin.dict.lookup.SnapshotTable$1.getRow(SnapshotTable.java:138)
>   at org.apache.kylin.dict.lookup.LookupTable.init(LookupTable.java:67)
>   at 
> org.apache.kylin.dict.lookup.LookupStringTable.init(LookupStringTable.java:79)
>   at org.apache.kylin.dict.lookup.LookupTable.(LookupTable.java:55)
>   at 
> org.apache.kylin.dict.lookup.LookupStringTable.(LookupStringTable.java:65)
>   at 
> org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:619)
>   at 
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:61)
>   at 
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:42)
>   at 
> org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:56)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>   at 
> org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
>   at 
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112)
>   at 
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57)
>   at 
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112)
>   at 
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:127)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>   at java.lang.Thread.run(Thread.java:745)
> result code:2



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KYLIN-1973) java.lang.NegativeArraySizeException when Build Dimension Dictionary

[liyang (JIRA)]

[ 
https://issues.apache.org/jira/browse/KYLIN-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15528804#comment-15528804
 ] 

liyang commented on KYLIN-1973:
---

The error happens during the build of the dictionary. At minimal you have to 
rebuild the cube I'm afraid.

> java.lang.NegativeArraySizeException when Build Dimension Dictionary
> 
>
> Key: KYLIN-1973
> URL: https://issues.apache.org/jira/browse/KYLIN-1973
> Project: Kylin
>  Issue Type: Bug
>Affects Versions: v1.5.3
>Reporter: zhengdong
>Assignee: liyang
> Fix For: v1.5.4
>
>
>  exception when Build Dimension Dictionary:
> java.lang.NegativeArraySizeException
>   at 
> org.apache.kylin.dict.TrieDictionary.getValueFromIdImpl(TrieDictionary.java:274)
>   at 
> org.apache.kylin.common.util.Dictionary.getValueFromId(Dictionary.java:130)
>   at 
> org.apache.kylin.dict.lookup.SnapshotTable$1.getRow(SnapshotTable.java:138)
>   at org.apache.kylin.dict.lookup.LookupTable.init(LookupTable.java:67)
>   at 
> org.apache.kylin.dict.lookup.LookupStringTable.init(LookupStringTable.java:79)
>   at org.apache.kylin.dict.lookup.LookupTable.(LookupTable.java:55)
>   at 
> org.apache.kylin.dict.lookup.LookupStringTable.(LookupStringTable.java:65)
>   at 
> org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:619)
>   at 
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:61)
>   at 
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:42)
>   at 
> org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:56)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>   at 
> org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
>   at 
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112)
>   at 
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57)
>   at 
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112)
>   at 
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:127)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>   at java.lang.Thread.run(Thread.java:745)
> result code:2



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KYLIN-1973) java.lang.NegativeArraySizeException when Build Dimension Dictionary

[Copperfield (JIRA)]

[ 
https://issues.apache.org/jira/browse/KYLIN-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15528791#comment-15528791
 ] 

Copperfield commented on KYLIN-1973:


When I use kylin-core-dictionary.jar of 1.5.4 to as a patch for kylin 1.5.2.1
The error still happen
and my error stack is 
java.lang.RuntimeException: Failed to create dictionary on DEFAULT.X
at 
org.apache.kylin.dict.DictionaryManager.buildDictionary(DictionaryManager.java:329)
at 
org.apache.kylin.dict.DictionaryManager.buildDictionary(DictionaryManager.java:274)
at 
org.apache.kylin.cube.CubeManager.buildDictionary(CubeManager.java:165)
at 
org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:51)
at 
org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:42)
at 
org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:56)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at 
org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:60)
at 
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:114)
at 
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:50)
at 
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:114)
at 
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:124)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Caused by: java.lang.NegativeArraySizeException
at 
org.apache.kylin.dict.TrieDictionaryBuilder.buildTrieBytes(TrieDictionaryBuilder.java:443)
at 
org.apache.kylin.dict.TrieDictionaryBuilder.build(TrieDictionaryBuilder.java:408)
at 
org.apache.kylin.dict.DictionaryGenerator$StringDictBuilder.build(DictionaryGenerator.java:165)
at 
org.apache.kylin.dict.DictionaryGenerator.buildDictionary(DictionaryGenerator.java:81)
at 
org.apache.kylin.dict.DictionaryGenerator.buildDictionary(DictionaryGenerator.java:73)
at 
org.apache.kylin.dict.DictionaryManager.buildDictionary(DictionaryManager.java:325)
... 15 more


and I logged the wrong value:

logger.info("Length of mbpn_footprint is: " + stats.mbpn_footprint);
logger.info("Length of head.length is: " + head.length);
logger.info("Why there is a navigate number: " + stats.mbpn_footprint + 
head.length);

Length of mbpn_footprint is: -472279743
2016-09-28 15:54:50,460 INFO  [pool-2-thread-1] dict.TrieDictionaryBuilder:447 
: Length of head.length is: 64
2016-09-28 15:54:50,461 INFO  [pool-2-thread-1] dict.TrieDictionaryBuilder:448 
: Why there is a navigate number: -47227974364



> java.lang.NegativeArraySizeException when Build Dimension Dictionary
> 
>
> Key: KYLIN-1973
> URL: https://issues.apache.org/jira/browse/KYLIN-1973
> Project: Kylin
>  Issue Type: Bug
>Affects Versions: v1.5.3
>Reporter: zhengdong
>Assignee: liyang
> Fix For: v1.5.4
>
>
>  exception when Build Dimension Dictionary:
> java.lang.NegativeArraySizeException
>   at 
> org.apache.kylin.dict.TrieDictionary.getValueFromIdImpl(TrieDictionary.java:274)
>   at 
> org.apache.kylin.common.util.Dictionary.getValueFromId(Dictionary.java:130)
>   at 
> org.apache.kylin.dict.lookup.SnapshotTable$1.getRow(SnapshotTable.java:138)
>   at org.apache.kylin.dict.lookup.LookupTable.init(LookupTable.java:67)
>   at 
> org.apache.kylin.dict.lookup.LookupStringTable.init(LookupStringTable.java:79)
>   at org.apache.kylin.dict.lookup.LookupTable.(LookupTable.java:55)
>   at 
> org.apache.kylin.dict.lookup.LookupStringTable.(LookupStringTable.java:65)
>   at 
> org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:619)
>   at 
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:61)
>   at 
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:42)
>   at 
> org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:56)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>   at 
> org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
>   at 
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112)
>   at 
> 

[jira] [Commented] (KYLIN-1973) java.lang.NegativeArraySizeException when Build Dimension Dictionary

[Copperfield (JIRA)]

[ 
https://issues.apache.org/jira/browse/KYLIN-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15528704#comment-15528704
 ] 

Copperfield commented on KYLIN-1973:


It's my error stack:

java.lang.RuntimeException: Failed to create dictionary on
DEFAULT.HDP_LIKE_ADVERTISE_NGINX.ACTION_URL
at 
org.apache.kylin.dict.DictionaryManager.buildDictionary(DictionaryManager.java:329)
at 
org.apache.kylin.dict.DictionaryManager.buildDictionary(DictionaryManager.java:274)
at 
org.apache.kylin.cube.CubeManager.buildDictionary(CubeManager.java:165)
at 
org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:51)
at 
org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:42)
at 
org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:56)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at 
org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:60)
at 
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:114)
at 
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:50)
at 
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:114)
at 
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:124)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Caused by: java.lang.NegativeArraySizeException
at 
org.apache.kylin.dict.TrieDictionaryBuilder.buildTrieBytes(TrieDictionaryBuilder.java:443)
at 
org.apache.kylin.dict.TrieDictionaryBuilder.build(TrieDictionaryBuilder.java:408)
at 
org.apache.kylin.dict.DictionaryGenerator$StringDictBuilder.build(DictionaryGenerator.java:165)
at 
org.apache.kylin.dict.DictionaryGenerator.buildDictionary(DictionaryGenerator.java:81)
at 
org.apache.kylin.dict.DictionaryGenerator.buildDictionary(DictionaryGenerator.java:73)
at 
org.apache.kylin.dict.DictionaryManager.buildDictionary(DictionaryManager.java:325)
... 15 more



On Wed, Sep 28, 2016 at 3:18 PM, Mario Copperfield 




-- 
Best regards,
Amuro Copperfield


> java.lang.NegativeArraySizeException when Build Dimension Dictionary
> 
>
> Key: KYLIN-1973
> URL: https://issues.apache.org/jira/browse/KYLIN-1973
> Project: Kylin
>  Issue Type: Bug
>Affects Versions: v1.5.3
>Reporter: zhengdong
>Assignee: liyang
> Fix For: v1.5.4
>
>
>  exception when Build Dimension Dictionary:
> java.lang.NegativeArraySizeException
>   at 
> org.apache.kylin.dict.TrieDictionary.getValueFromIdImpl(TrieDictionary.java:274)
>   at 
> org.apache.kylin.common.util.Dictionary.getValueFromId(Dictionary.java:130)
>   at 
> org.apache.kylin.dict.lookup.SnapshotTable$1.getRow(SnapshotTable.java:138)
>   at org.apache.kylin.dict.lookup.LookupTable.init(LookupTable.java:67)
>   at 
> org.apache.kylin.dict.lookup.LookupStringTable.init(LookupStringTable.java:79)
>   at org.apache.kylin.dict.lookup.LookupTable.(LookupTable.java:55)
>   at 
> org.apache.kylin.dict.lookup.LookupStringTable.(LookupStringTable.java:65)
>   at 
> org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:619)
>   at 
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:61)
>   at 
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:42)
>   at 
> org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:56)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>   at 
> org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
>   at 
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112)
>   at 
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57)
>   at 
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112)
>   at 
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:127)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>   at 
> 

[jira] [Commented] (KYLIN-1973) java.lang.NegativeArraySizeException when Build Dimension Dictionary

[Copperfield (JIRA)]

[ 
https://issues.apache.org/jira/browse/KYLIN-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15528693#comment-15528693
 ] 

Copperfield commented on KYLIN-1973:


I use the jar: kylin-core-dictionry of 1.5.4 as a patch to 1.5.2.1
but the error still happen
what should i do?





-- 
Best regards,
Amuro Copperfield


> java.lang.NegativeArraySizeException when Build Dimension Dictionary
> 
>
> Key: KYLIN-1973
> URL: https://issues.apache.org/jira/browse/KYLIN-1973
> Project: Kylin
>  Issue Type: Bug
>Affects Versions: v1.5.3
>Reporter: zhengdong
>Assignee: liyang
> Fix For: v1.5.4
>
>
>  exception when Build Dimension Dictionary:
> java.lang.NegativeArraySizeException
>   at 
> org.apache.kylin.dict.TrieDictionary.getValueFromIdImpl(TrieDictionary.java:274)
>   at 
> org.apache.kylin.common.util.Dictionary.getValueFromId(Dictionary.java:130)
>   at 
> org.apache.kylin.dict.lookup.SnapshotTable$1.getRow(SnapshotTable.java:138)
>   at org.apache.kylin.dict.lookup.LookupTable.init(LookupTable.java:67)
>   at 
> org.apache.kylin.dict.lookup.LookupStringTable.init(LookupStringTable.java:79)
>   at org.apache.kylin.dict.lookup.LookupTable.(LookupTable.java:55)
>   at 
> org.apache.kylin.dict.lookup.LookupStringTable.(LookupStringTable.java:65)
>   at 
> org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:619)
>   at 
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:61)
>   at 
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:42)
>   at 
> org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:56)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>   at 
> org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
>   at 
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112)
>   at 
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57)
>   at 
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112)
>   at 
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:127)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>   at java.lang.Thread.run(Thread.java:745)
> result code:2



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KYLIN-1973) java.lang.NegativeArraySizeException when Build Dimension Dictionary

[Copperfield (JIRA)]

[ 
https://issues.apache.org/jira/browse/KYLIN-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15525799#comment-15525799
 ] 

Copperfield commented on KYLIN-1973:


OK，Thanks

> java.lang.NegativeArraySizeException when Build Dimension Dictionary
> 
>
> Key: KYLIN-1973
> URL: https://issues.apache.org/jira/browse/KYLIN-1973
> Project: Kylin
>  Issue Type: Bug
>Affects Versions: v1.5.3
>Reporter: zhengdong
>Assignee: liyang
> Fix For: v1.5.4
>
>
>  exception when Build Dimension Dictionary:
> java.lang.NegativeArraySizeException
>   at 
> org.apache.kylin.dict.TrieDictionary.getValueFromIdImpl(TrieDictionary.java:274)
>   at 
> org.apache.kylin.common.util.Dictionary.getValueFromId(Dictionary.java:130)
>   at 
> org.apache.kylin.dict.lookup.SnapshotTable$1.getRow(SnapshotTable.java:138)
>   at org.apache.kylin.dict.lookup.LookupTable.init(LookupTable.java:67)
>   at 
> org.apache.kylin.dict.lookup.LookupStringTable.init(LookupStringTable.java:79)
>   at org.apache.kylin.dict.lookup.LookupTable.(LookupTable.java:55)
>   at 
> org.apache.kylin.dict.lookup.LookupStringTable.(LookupStringTable.java:65)
>   at 
> org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:619)
>   at 
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:61)
>   at 
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:42)
>   at 
> org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:56)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>   at 
> org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
>   at 
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112)
>   at 
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57)
>   at 
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112)
>   at 
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:127)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>   at java.lang.Thread.run(Thread.java:745)
> result code:2



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KYLIN-1973) java.lang.NegativeArraySizeException when Build Dimension Dictionary

[zhengdong (JIRA)]

[ 
https://issues.apache.org/jira/browse/KYLIN-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15525654#comment-15525654
 ] 

zhengdong commented on KYLIN-1973:
--

Yes. You can use commits related with KYLIN-1834 to generate a patch file

> java.lang.NegativeArraySizeException when Build Dimension Dictionary
> 
>
> Key: KYLIN-1973
> URL: https://issues.apache.org/jira/browse/KYLIN-1973
> Project: Kylin
>  Issue Type: Bug
>Affects Versions: v1.5.3
>Reporter: zhengdong
>Assignee: liyang
> Fix For: v1.5.4
>
>
>  exception when Build Dimension Dictionary:
> java.lang.NegativeArraySizeException
>   at 
> org.apache.kylin.dict.TrieDictionary.getValueFromIdImpl(TrieDictionary.java:274)
>   at 
> org.apache.kylin.common.util.Dictionary.getValueFromId(Dictionary.java:130)
>   at 
> org.apache.kylin.dict.lookup.SnapshotTable$1.getRow(SnapshotTable.java:138)
>   at org.apache.kylin.dict.lookup.LookupTable.init(LookupTable.java:67)
>   at 
> org.apache.kylin.dict.lookup.LookupStringTable.init(LookupStringTable.java:79)
>   at org.apache.kylin.dict.lookup.LookupTable.(LookupTable.java:55)
>   at 
> org.apache.kylin.dict.lookup.LookupStringTable.(LookupStringTable.java:65)
>   at 
> org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:619)
>   at 
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:61)
>   at 
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:42)
>   at 
> org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:56)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>   at 
> org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
>   at 
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112)
>   at 
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57)
>   at 
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112)
>   at 
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:127)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>   at java.lang.Thread.run(Thread.java:745)
> result code:2



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KYLIN-1973) java.lang.NegativeArraySizeException when Build Dimension Dictionary

[Copperfield (JIRA)]

[ 
https://issues.apache.org/jira/browse/KYLIN-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15525624#comment-15525624
 ] 

Copperfield commented on KYLIN-1973:


This bug can use a patch to fix?
We now use a kylin in 1.5.2.1

> java.lang.NegativeArraySizeException when Build Dimension Dictionary
> 
>
> Key: KYLIN-1973
> URL: https://issues.apache.org/jira/browse/KYLIN-1973
> Project: Kylin
>  Issue Type: Bug
>Affects Versions: v1.5.3
>Reporter: zhengdong
>Assignee: liyang
> Fix For: v1.5.4
>
>
>  exception when Build Dimension Dictionary:
> java.lang.NegativeArraySizeException
>   at 
> org.apache.kylin.dict.TrieDictionary.getValueFromIdImpl(TrieDictionary.java:274)
>   at 
> org.apache.kylin.common.util.Dictionary.getValueFromId(Dictionary.java:130)
>   at 
> org.apache.kylin.dict.lookup.SnapshotTable$1.getRow(SnapshotTable.java:138)
>   at org.apache.kylin.dict.lookup.LookupTable.init(LookupTable.java:67)
>   at 
> org.apache.kylin.dict.lookup.LookupStringTable.init(LookupStringTable.java:79)
>   at org.apache.kylin.dict.lookup.LookupTable.(LookupTable.java:55)
>   at 
> org.apache.kylin.dict.lookup.LookupStringTable.(LookupStringTable.java:65)
>   at 
> org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:619)
>   at 
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:61)
>   at 
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:42)
>   at 
> org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:56)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>   at 
> org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
>   at 
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112)
>   at 
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57)
>   at 
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112)
>   at 
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:127)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>   at java.lang.Thread.run(Thread.java:745)
> result code:2



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KYLIN-1973) java.lang.NegativeArraySizeException when Build Dimension Dictionary

[liyang (JIRA)]

[ 
https://issues.apache.org/jira/browse/KYLIN-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15464641#comment-15464641
 ] 

liyang commented on KYLIN-1973:
---

The root cause is the dictionary bigger than 2GB and overflows the int range, 
as mentioned in KYLIN-1834

> java.lang.NegativeArraySizeException when Build Dimension Dictionary
> 
>
> Key: KYLIN-1973
> URL: https://issues.apache.org/jira/browse/KYLIN-1973
> Project: Kylin
>  Issue Type: Bug
>Affects Versions: v1.5.3
>Reporter: zhengdong
>Assignee: liyang
> Fix For: v1.5.4
>
>
>  exception when Build Dimension Dictionary:
> java.lang.NegativeArraySizeException
>   at 
> org.apache.kylin.dict.TrieDictionary.getValueFromIdImpl(TrieDictionary.java:274)
>   at 
> org.apache.kylin.common.util.Dictionary.getValueFromId(Dictionary.java:130)
>   at 
> org.apache.kylin.dict.lookup.SnapshotTable$1.getRow(SnapshotTable.java:138)
>   at org.apache.kylin.dict.lookup.LookupTable.init(LookupTable.java:67)
>   at 
> org.apache.kylin.dict.lookup.LookupStringTable.init(LookupStringTable.java:79)
>   at org.apache.kylin.dict.lookup.LookupTable.(LookupTable.java:55)
>   at 
> org.apache.kylin.dict.lookup.LookupStringTable.(LookupStringTable.java:65)
>   at 
> org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:619)
>   at 
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:61)
>   at 
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:42)
>   at 
> org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:56)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>   at 
> org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
>   at 
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112)
>   at 
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57)
>   at 
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112)
>   at 
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:127)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>   at java.lang.Thread.run(Thread.java:745)
> result code:2



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KYLIN-1973) java.lang.NegativeArraySizeException when Build Dimension Dictionary

[hongbin ma (JIRA)]

[ 
https://issues.apache.org/jira/browse/KYLIN-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15460558#comment-15460558
 ] 

hongbin ma commented on KYLIN-1973:
---

​test​





-- 
Regards,

*Bin Mahone | 马洪宾*


> java.lang.NegativeArraySizeException when Build Dimension Dictionary
> 
>
> Key: KYLIN-1973
> URL: https://issues.apache.org/jira/browse/KYLIN-1973
> Project: Kylin
>  Issue Type: Bug
>Affects Versions: v1.5.3
>Reporter: zhengdong
>Assignee: liyang
> Fix For: v1.5.4
>
>
>  exception when Build Dimension Dictionary:
> java.lang.NegativeArraySizeException
>   at 
> org.apache.kylin.dict.TrieDictionary.getValueFromIdImpl(TrieDictionary.java:274)
>   at 
> org.apache.kylin.common.util.Dictionary.getValueFromId(Dictionary.java:130)
>   at 
> org.apache.kylin.dict.lookup.SnapshotTable$1.getRow(SnapshotTable.java:138)
>   at org.apache.kylin.dict.lookup.LookupTable.init(LookupTable.java:67)
>   at 
> org.apache.kylin.dict.lookup.LookupStringTable.init(LookupStringTable.java:79)
>   at org.apache.kylin.dict.lookup.LookupTable.(LookupTable.java:55)
>   at 
> org.apache.kylin.dict.lookup.LookupStringTable.(LookupStringTable.java:65)
>   at 
> org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:619)
>   at 
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:61)
>   at 
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:42)
>   at 
> org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:56)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>   at 
> org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
>   at 
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112)
>   at 
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57)
>   at 
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112)
>   at 
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:127)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>   at java.lang.Thread.run(Thread.java:745)
> result code:2



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KYLIN-1973) java.lang.NegativeArraySizeException when Build Dimension Dictionary

[hongbin ma (JIRA)]

[ 
https://issues.apache.org/jira/browse/KYLIN-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15460523#comment-15460523
 ] 

hongbin ma commented on KYLIN-1973:
---

also, for better JIRA management we can link two JIRAs if they're highly related

> java.lang.NegativeArraySizeException when Build Dimension Dictionary
> 
>
> Key: KYLIN-1973
> URL: https://issues.apache.org/jira/browse/KYLIN-1973
> Project: Kylin
>  Issue Type: Bug
>Affects Versions: v1.5.3
>Reporter: zhengdong
>Assignee: liyang
> Fix For: v1.5.4
>
>
>  exception when Build Dimension Dictionary:
> java.lang.NegativeArraySizeException
>   at 
> org.apache.kylin.dict.TrieDictionary.getValueFromIdImpl(TrieDictionary.java:274)
>   at 
> org.apache.kylin.common.util.Dictionary.getValueFromId(Dictionary.java:130)
>   at 
> org.apache.kylin.dict.lookup.SnapshotTable$1.getRow(SnapshotTable.java:138)
>   at org.apache.kylin.dict.lookup.LookupTable.init(LookupTable.java:67)
>   at 
> org.apache.kylin.dict.lookup.LookupStringTable.init(LookupStringTable.java:79)
>   at org.apache.kylin.dict.lookup.LookupTable.(LookupTable.java:55)
>   at 
> org.apache.kylin.dict.lookup.LookupStringTable.(LookupStringTable.java:65)
>   at 
> org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:619)
>   at 
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:61)
>   at 
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:42)
>   at 
> org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:56)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>   at 
> org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
>   at 
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112)
>   at 
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57)
>   at 
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112)
>   at 
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:127)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>   at java.lang.Thread.run(Thread.java:745)
> result code:2



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KYLIN-1973) java.lang.NegativeArraySizeException when Build Dimension Dictionary

[hongbin ma (JIRA)]

[ 
https://issues.apache.org/jira/browse/KYLIN-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15460521#comment-15460521
 ] 

hongbin ma commented on KYLIN-1973:
---

what is the root cause? same as [~zhengd]'s guess?

> java.lang.NegativeArraySizeException when Build Dimension Dictionary
> 
>
> Key: KYLIN-1973
> URL: https://issues.apache.org/jira/browse/KYLIN-1973
> Project: Kylin
>  Issue Type: Bug
>Affects Versions: v1.5.3
>Reporter: zhengdong
>Assignee: liyang
> Fix For: v1.5.4
>
>
>  exception when Build Dimension Dictionary:
> java.lang.NegativeArraySizeException
>   at 
> org.apache.kylin.dict.TrieDictionary.getValueFromIdImpl(TrieDictionary.java:274)
>   at 
> org.apache.kylin.common.util.Dictionary.getValueFromId(Dictionary.java:130)
>   at 
> org.apache.kylin.dict.lookup.SnapshotTable$1.getRow(SnapshotTable.java:138)
>   at org.apache.kylin.dict.lookup.LookupTable.init(LookupTable.java:67)
>   at 
> org.apache.kylin.dict.lookup.LookupStringTable.init(LookupStringTable.java:79)
>   at org.apache.kylin.dict.lookup.LookupTable.(LookupTable.java:55)
>   at 
> org.apache.kylin.dict.lookup.LookupStringTable.(LookupStringTable.java:65)
>   at 
> org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:619)
>   at 
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:61)
>   at 
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:42)
>   at 
> org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:56)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>   at 
> org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
>   at 
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112)
>   at 
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57)
>   at 
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112)
>   at 
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:127)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>   at java.lang.Thread.run(Thread.java:745)
> result code:2



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KYLIN-1973) java.lang.NegativeArraySizeException when Build Dimension Dictionary

[zhengdong (JIRA)]

[ 
https://issues.apache.org/jira/browse/KYLIN-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15454107#comment-15454107
 ] 

zhengdong commented on KYLIN-1973:
--

I guess the filed 'transient private int maxValueLength' in TrieDictionary is 
overflowed since it's initialized by 'this.maxValueLength = 
headIn.readShort()',but not sure. Could you help to verify this?
After changed the init method from 'this.maxValueLength = headIn.readShort()' 
to 'this.maxValueLength = headIn.readUnsignedShort()', 
java.lang.NegativeArraySizeException disappeared. And then, we get 
java.lang.IllegalArgumentException  just the same as KYLIN-1834

> java.lang.NegativeArraySizeException when Build Dimension Dictionary
> 
>
> Key: KYLIN-1973
> URL: https://issues.apache.org/jira/browse/KYLIN-1973
> Project: Kylin
>  Issue Type: Bug
>Affects Versions: v1.5.3
>Reporter: zhengdong
>
>  exception when Build Dimension Dictionary:
> java.lang.NegativeArraySizeException
>   at 
> org.apache.kylin.dict.TrieDictionary.getValueFromIdImpl(TrieDictionary.java:274)
>   at 
> org.apache.kylin.common.util.Dictionary.getValueFromId(Dictionary.java:130)
>   at 
> org.apache.kylin.dict.lookup.SnapshotTable$1.getRow(SnapshotTable.java:138)
>   at org.apache.kylin.dict.lookup.LookupTable.init(LookupTable.java:67)
>   at 
> org.apache.kylin.dict.lookup.LookupStringTable.init(LookupStringTable.java:79)
>   at org.apache.kylin.dict.lookup.LookupTable.(LookupTable.java:55)
>   at 
> org.apache.kylin.dict.lookup.LookupStringTable.(LookupStringTable.java:65)
>   at 
> org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:619)
>   at 
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:61)
>   at 
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:42)
>   at 
> org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:56)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>   at 
> org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
>   at 
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112)
>   at 
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57)
>   at 
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112)
>   at 
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:127)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>   at java.lang.Thread.run(Thread.java:745)
> result code:2



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KYLIN-1973) java.lang.NegativeArraySizeException when Build Dimension Dictionary

[zhengdong (JIRA)]

[ 
https://issues.apache.org/jira/browse/KYLIN-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15454060#comment-15454060
 ] 

zhengdong commented on KYLIN-1973:
--

When take snapshot of a  large lookup table(1000K rows +) in the 4th step of 
building cube.


> java.lang.NegativeArraySizeException when Build Dimension Dictionary
> 
>
> Key: KYLIN-1973
> URL: https://issues.apache.org/jira/browse/KYLIN-1973
> Project: Kylin
>  Issue Type: Bug
>Affects Versions: v1.5.3
>Reporter: zhengdong
>
>  exception when Build Dimension Dictionary:
> java.lang.NegativeArraySizeException
>   at 
> org.apache.kylin.dict.TrieDictionary.getValueFromIdImpl(TrieDictionary.java:274)
>   at 
> org.apache.kylin.common.util.Dictionary.getValueFromId(Dictionary.java:130)
>   at 
> org.apache.kylin.dict.lookup.SnapshotTable$1.getRow(SnapshotTable.java:138)
>   at org.apache.kylin.dict.lookup.LookupTable.init(LookupTable.java:67)
>   at 
> org.apache.kylin.dict.lookup.LookupStringTable.init(LookupStringTable.java:79)
>   at org.apache.kylin.dict.lookup.LookupTable.(LookupTable.java:55)
>   at 
> org.apache.kylin.dict.lookup.LookupStringTable.(LookupStringTable.java:65)
>   at 
> org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:619)
>   at 
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:61)
>   at 
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:42)
>   at 
> org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:56)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>   at 
> org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
>   at 
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112)
>   at 
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57)
>   at 
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112)
>   at 
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:127)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>   at java.lang.Thread.run(Thread.java:745)
> result code:2



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KYLIN-1973) java.lang.NegativeArraySizeException when Build Dimension Dictionary

[zhengdong (JIRA)]

[ 
https://issues.apache.org/jira/browse/KYLIN-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15454049#comment-15454049
 ] 

zhengdong commented on KYLIN-1973:
--

Hi,shaofeng,i don't think this issue is related with KYLIN-1985. They are found 
when building different cube. 
Furthermore, KYLIN-1985 can be fixed by applying the given patch or ensuring 
hive tables are all synchronized 

> java.lang.NegativeArraySizeException when Build Dimension Dictionary
> 
>
> Key: KYLIN-1973
> URL: https://issues.apache.org/jira/browse/KYLIN-1973
> Project: Kylin
>  Issue Type: Bug
>Affects Versions: v1.5.3
>Reporter: zhengdong
>
>  exception when Build Dimension Dictionary:
> java.lang.NegativeArraySizeException
>   at 
> org.apache.kylin.dict.TrieDictionary.getValueFromIdImpl(TrieDictionary.java:274)
>   at 
> org.apache.kylin.common.util.Dictionary.getValueFromId(Dictionary.java:130)
>   at 
> org.apache.kylin.dict.lookup.SnapshotTable$1.getRow(SnapshotTable.java:138)
>   at org.apache.kylin.dict.lookup.LookupTable.init(LookupTable.java:67)
>   at 
> org.apache.kylin.dict.lookup.LookupStringTable.init(LookupStringTable.java:79)
>   at org.apache.kylin.dict.lookup.LookupTable.(LookupTable.java:55)
>   at 
> org.apache.kylin.dict.lookup.LookupStringTable.(LookupStringTable.java:65)
>   at 
> org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:619)
>   at 
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:61)
>   at 
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:42)
>   at 
> org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:56)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>   at 
> org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
>   at 
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112)
>   at 
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57)
>   at 
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112)
>   at 
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:127)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>   at java.lang.Thread.run(Thread.java:745)
> result code:2



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KYLIN-1973) java.lang.NegativeArraySizeException when Build Dimension Dictionary

[Shaofeng SHI (JIRA)]

[ 
https://issues.apache.org/jira/browse/KYLIN-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15438700#comment-15438700
 ] 

Shaofeng SHI commented on KYLIN-1973:
-

Hi dong, can you describe the steps to reproduce this error? thanks!

> java.lang.NegativeArraySizeException when Build Dimension Dictionary
> 
>
> Key: KYLIN-1973
> URL: https://issues.apache.org/jira/browse/KYLIN-1973
> Project: Kylin
>  Issue Type: Bug
>Affects Versions: v1.5.3
>Reporter: zhengdong
>
>  exception when Build Dimension Dictionary:
> java.lang.NegativeArraySizeException
>   at 
> org.apache.kylin.dict.TrieDictionary.getValueFromIdImpl(TrieDictionary.java:274)
>   at 
> org.apache.kylin.common.util.Dictionary.getValueFromId(Dictionary.java:130)
>   at 
> org.apache.kylin.dict.lookup.SnapshotTable$1.getRow(SnapshotTable.java:138)
>   at org.apache.kylin.dict.lookup.LookupTable.init(LookupTable.java:67)
>   at 
> org.apache.kylin.dict.lookup.LookupStringTable.init(LookupStringTable.java:79)
>   at org.apache.kylin.dict.lookup.LookupTable.(LookupTable.java:55)
>   at 
> org.apache.kylin.dict.lookup.LookupStringTable.(LookupStringTable.java:65)
>   at 
> org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:619)
>   at 
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:61)
>   at 
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:42)
>   at 
> org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:56)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>   at 
> org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
>   at 
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112)
>   at 
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57)
>   at 
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112)
>   at 
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:127)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>   at java.lang.Thread.run(Thread.java:745)
> result code:2



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)