[jira] [Created] (KYLIN-3324) NegativeArraySizeException in CreateDictionaryJob$2.getDictionary()
liyang created KYLIN-3324:
-
Summary: NegativeArraySizeException in
CreateDictionaryJob$2.getDictionary()
Key: KYLIN-3324
URL: https://issues.apache.org/jira/browse/KYLIN-3324
Project: Kylin
Issue Type: Bug
Reporter: liyang
Assignee: liyang
During cube build, got following exception:
{noformat}
2018-01-31 09:58:14,982 ERROR [Scheduler 311988476 Job
b931b081-3b44-462b-9bce-e15bf245f1b9-862] common.HadoopShellExecutable : error
execute HadoopShellExecutable{id=b931b081-3b44-462b-9bce-e15bf245f1b9-03,
name=Build Dimension Dictionary, state=RUNNING}
java.lang.NegativeArraySizeException
at
org.apache.hadoop.io.BytesWritable.setCapacity(BytesWritable.java:144)
at org.apache.hadoop.io.BytesWritable.setSize(BytesWritable.java:123)
at org.apache.hadoop.io.BytesWritable.readFields(BytesWritable.java:179)
at
org.apache.hadoop.io.SequenceFile$Reader.getCurrentValue(SequenceFile.java:2259)
at org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:2387)
at
org.apache.kylin.engine.mr.steps.CreateDictionaryJob$2.getDictionary(CreateDictionaryJob.java:87)
at
org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:62)
at
org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:49)
at
org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:66)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at
org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:62)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:125)
at
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:64)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:125)
at
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:156)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745){noformat}
Root cause seems to be
https://stackoverflow.com/questions/24127304/negativearraysizeexception-when-creating-a-sequencefile-with-large-1gb-bytesw
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3297) sql在解析>大值 and <小值的时候，kylin出现内存溢出。
[ https://issues.apache.org/jira/browse/KYLIN-3297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16417047#comment-16417047 ] 陈家宇 commented on KYLIN-3297: cube是正常创建的，就是执行select f.plant_code,f.report_business_area,f.orderby,sum(f.a)a, case when sum(f.a)=0 then 0 else sum(f.qty)/sum(f.a) end zt from ( select h.business_area_desc,h.plant_code, case when p.report_business_area like '%切大片%' then '切大片' else p.report_business_area end report_business_area, case when p.report_business_area like '%切大片%' then '00' else p.orderby end orderby, sum(p.quantity) qty,sum(a)a from pkn_kpi p inner join hcm_area h on p.plant_id=h.plant_id where 1=1 and h.business_area_desc='上海' and h.plant_code='S1' and p.calendar_day>='2018-03-19' and p.calendar_day<='2018-03-18' and trim(p.report_business_area) is not null group by h.business_area_desc,h.plant_code,p.report_business_area,p.orderby ) f group by f.plant_code,f.report_business_area,f.orderby order by orderby 时，kylin出现奔溃的情况，javadump分析的结果也指示这个query出现问题。 以上语句的问题是 and p.calendar_day>='2018-03-19' and p.calendar_day<='2018-03-18'值写反了。 > sql在解析>大值 and <小值的时候，kylin出现内存溢出。 > - > > Key: KYLIN-3297 > URL: https://issues.apache.org/jira/browse/KYLIN-3297 > Project: Kylin > Issue Type: Bug > Components: RDBMS Source >Affects Versions: v2.2.0 > Environment: oracle linux 6.8 > cup 8C > memory 48G > apache kylin v2.2.0 >Reporter: 陈家宇 >Priority: Major > > select f.plant_code,f.report_business_area,f.orderby,sum(f.a)a, > case when sum(f.a)=0 then 0 else sum(f.qty)/sum(f.a) end zt from ( > select h.business_area_desc,h.plant_code, > case when p.report_business_area like '%切大片%' then '切大片' > else p.report_business_area end report_business_area, > case when p.report_business_area like '%切大片%' then '00' > else p.orderby end orderby, > sum(p.quantity) qty,sum(a)a from pkn_kpi p > inner join hcm_area h on p.plant_id=h.plant_id > where 1=1 > and h.business_area_desc='上海' > and h.plant_code='S1' > and p.calendar_day>='2018-03-19' > and p.calendar_day<='2018-03-18' > and trim(p.report_business_area) is not null > group by h.business_area_desc,h.plant_code,p.report_business_area,p.orderby > ) f > group by f.plant_code,f.report_business_area,f.orderby > order by orderby > 以上的calendar_day条件，顺序输入反了，变成calendar_day>大值 and > calendar_day<小值，出现了kylin内存溢出，并且崩溃。正常这种情况是没有数据输出才对。 > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3323) Allow debugTomcat add more arguments
[ https://issues.apache.org/jira/browse/KYLIN-3323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16417012#comment-16417012 ] ASF GitHub Bot commented on KYLIN-3323: --- coveralls commented on issue #125: KYLIN-3323 allow debugTomcat add more arguments URL: https://github.com/apache/kylin/pull/125#issuecomment-376798999 ## Pull Request Test Coverage Report for [Build 3093](https://coveralls.io/builds/16218689) * **0** of **6** **(0.0%)** changed or added relevant lines in **1** file are covered. * **5** unchanged lines in **1** file lost coverage. * Overall coverage decreased (**-0.004%**) to **23.962%** --- | Changes Missing Coverage | Covered Lines | Changed/Added Lines | % | | :-|--||---: | | [server/src/main/java/org/apache/kylin/rest/DebugTomcat.java](https://coveralls.io/builds/16218689/source?filename=server%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2FDebugTomcat.java#L108) | 0 | 6 | 0.0% | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://coveralls.io/builds/16218689/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Finmemcubing%2FMemDiskStore.java#L439) | 5 | 77.81% | | Totals | [](https://coveralls.io/builds/16218689) | | :-- | --: | | Change from base [Build 3091](https://coveralls.io/builds/16197822): | -0.004% | | Covered Lines: | 14341 | | Relevant Lines: | 59848 | --- # - [Coveralls](https://coveralls.io) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Allow debugTomcat add more arguments > > > Key: KYLIN-3323 > URL: https://issues.apache.org/jira/browse/KYLIN-3323 > Project: Kylin > Issue Type: Improvement >Reporter: yongjie zhao >Assignee: yongjie zhao >Priority: Minor > Attachments: Screen Shot 2018-03-28 at 3.05.00 PM.png > > > On current, debugTomcat only accepts port argument. I must edit > kylin.property for execute remote command, such as > kylin.job.use-remote-cli=true. This issue can be passed arguments for > debugTomcat -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3323) Allow debugTomcat add more arguments
[ https://issues.apache.org/jira/browse/KYLIN-3323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16416928#comment-16416928 ] ASF GitHub Bot commented on KYLIN-3323: --- asfgit commented on issue #125: KYLIN-3323 allow debugTomcat add more arguments URL: https://github.com/apache/kylin/pull/125#issuecomment-376784840 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Allow debugTomcat add more arguments > > > Key: KYLIN-3323 > URL: https://issues.apache.org/jira/browse/KYLIN-3323 > Project: Kylin > Issue Type: Improvement >Reporter: yongjie zhao >Assignee: yongjie zhao >Priority: Minor > Attachments: Screen Shot 2018-03-28 at 3.05.00 PM.png > > > On current, debugTomcat only accepts port argument. I must edit > kylin.property for execute remote command, such as > kylin.job.use-remote-cli=true. This issue can be passed arguments for > debugTomcat -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3323) Allow debugTomcat add more arguments
[ https://issues.apache.org/jira/browse/KYLIN-3323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yongjie zhao updated KYLIN-3323: Attachment: Screen Shot 2018-03-28 at 3.05.00 PM.png > Allow debugTomcat add more arguments > > > Key: KYLIN-3323 > URL: https://issues.apache.org/jira/browse/KYLIN-3323 > Project: Kylin > Issue Type: Improvement >Reporter: yongjie zhao >Assignee: yongjie zhao >Priority: Minor > Attachments: Screen Shot 2018-03-28 at 3.05.00 PM.png > > > On current, debugTomcat only accepts port argument. I must edit > kylin.property for execute remote command, such as > kylin.job.use-remote-cli=true. This issue can be passed arguments for > debugTomcat -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3323) Allow debugTomcat add more arguments
yongjie zhao created KYLIN-3323: --- Summary: Allow debugTomcat add more arguments Key: KYLIN-3323 URL: https://issues.apache.org/jira/browse/KYLIN-3323 Project: Kylin Issue Type: Improvement Reporter: yongjie zhao Assignee: yongjie zhao On current, debugTomcat only accepts port argument. I must edit kylin.property for execute remote command, such as kylin.job.use-remote-cli=true. This issue can be passed arguments for debugTomcat -- This message was sent by Atlassian JIRA (v7.6.3#76005)
