[jira] [Updated] (KYLIN-3560) Should not depend on personal repository
[
https://issues.apache.org/jira/browse/KYLIN-3560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI updated KYLIN-3560:
Fix Version/s: v2.6.0
Thank you Ted; We will investigate it.
> Should not depend on personal repository
>
>
> Key: KYLIN-3560
> URL: https://issues.apache.org/jira/browse/KYLIN-3560
> Project: Kylin
> Issue Type: Task
>Reporter: Ted Yu
>Priority: Major
> Fix For: v2.6.0
>
>
> In core-common/pom.xml:
> {code}
> com.github.joshelser
> {code}
> We shouldn't depend on personal repository.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (KYLIN-3561) Upgrade to JUnit 5
Ted Yu created KYLIN-3561: - Summary: Upgrade to JUnit 5 Key: KYLIN-3561 URL: https://issues.apache.org/jira/browse/KYLIN-3561 Project: Kylin Issue Type: Task Reporter: Ted Yu JUnit 5 brings multiple useful features so tests are easier to read and write. We can bump up the dependency version and create new tests with JUnit 5 features. Relevant features of JUnit 5: dynamic test, nested tests, parameterized tests https://twitter.com/nipafx/status/1027095088059559936 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3171) Support hadoop 3 release
[
https://issues.apache.org/jira/browse/KYLIN-3171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16615047#comment-16615047
]
Ted Yu commented on KYLIN-3171:
---
How would master-hadoop3.1 branch be sync'ed with master branch ?
Currently it lags by few days' commits.
> Support hadoop 3 release
>
>
> Key: KYLIN-3171
> URL: https://issues.apache.org/jira/browse/KYLIN-3171
> Project: Kylin
> Issue Type: Improvement
>Reporter: Ted Yu
>Priority: Major
>
> When compiling against hadoop 3, I got:
> {code}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.5.1:compile
> (default-compile) on project kylin-engine-mr: Compilation failure:
> Compilation failure:
> [ERROR]
> /a/kylin/engine-mr/src/main/java/org/apache/kylin/engine/mr/common/DefaultSslProtocolSocketFactory.java:[29,36]
> error: package org.apache.commons.httpclient does not exist
> [ERROR]
> /a/kylin/engine-mr/src/main/java/org/apache/kylin/engine/mr/common/DefaultSslProtocolSocketFactory.java:[30,36]
> error: package org.apache.commons.httpclient does not exist
> [ERROR]
> /a/kylin/engine-mr/src/main/java/org/apache/kylin/engine/mr/common/DefaultSslProtocolSocketFactory.java:[31,43]
> error: package org.apache.commons.httpclient.params does not exist
> [ERROR]
> /a/kylin/engine-mr/src/main/java/org/apache/kylin/engine/mr/common/DefaultSslProtocolSocketFactory.java:[32,45]
> error: package org.apache.commons.httpclient.protocol does not exist
> [ERROR]
> /a/kylin/engine-mr/src/main/java/org/apache/kylin/engine/mr/common/DefaultSslProtocolSocketFactory.java:[33,45]
> error: package org.apache.commons.httpclient.protocol does not exist
> [ERROR]
> /a/kylin/engine-mr/src/main/java/org/apache/kylin/engine/mr/common/DefaultSslProtocolSocketFactory.java:[41,56]
> error: cannot find symbol
> [ERROR] symbol: class SecureProtocolSocketFactory
> [ERROR]
> /a/kylin/engine-mr/src/main/java/org/apache/kylin/engine/mr/common/DefaultSslProtocolSocketFactory.java:[94,125]
> error: cannot find symbol
> [ERROR] symbol: class HttpConnectionParams
> [ERROR] location: class DefaultSslProtocolSocketFactory
> [ERROR]
> /a/kylin/engine-mr/src/main/java/org/apache/kylin/engine/mr/common/DefaultSslProtocolSocketFactory.java:[94,196]
> error: cannot find symbol
> [ERROR] symbol: class ConnectTimeoutException
> [ERROR] location: class DefaultSslProtocolSocketFactory
> [ERROR]
> /a/kylin/engine-mr/src/main/java/org/apache/kylin/engine/mr/common/DefaultSslProtocolSocketFactory.java:[105,19]
> error: cannot find symbol
> [ERROR] symbol: variable ControllerThreadSocketFactory
> [ERROR] location: class DefaultSslProtocolSocketFactory
> {code}
> We should allow building against hadoop 3.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (KYLIN-3560) Should not depend on personal repository
Ted Yu created KYLIN-3560:
-
Summary: Should not depend on personal repository
Key: KYLIN-3560
URL: https://issues.apache.org/jira/browse/KYLIN-3560
Project: Kylin
Issue Type: Task
Reporter: Ted Yu
In core-common/pom.xml:
{code}
com.github.joshelser
{code}
We shouldn't depend on personal repository.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (KYLIN-3546) Kylin doesn't persist FK to intermediate table if FK is not a dimension
[ https://issues.apache.org/jira/browse/KYLIN-3546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16614630#comment-16614630 ] XiaoXiang Yu edited comment on KYLIN-3546 at 9/14/18 10:15 AM: --- Hi [~xiewenliang], could you please show me the JSON representation of your model and your cube? It is best to show us the complete detail about how to reproduce that error.:) was (Author: hit_lacus): Hi [~xiewenliang], could you please show me the JSON representation of your model and your cube? > Kylin doesn't persist FK to intermediate table if FK is not a dimension > --- > > Key: KYLIN-3546 > URL: https://issues.apache.org/jira/browse/KYLIN-3546 > Project: Kylin > Issue Type: Bug > Components: Streaming >Affects Versions: v2.4.0 >Reporter: wlxie >Assignee: XiaoXiang Yu >Priority: Major > Fix For: v2.4.2, v2.5.1 > > Attachments: kylin_kafka.txt > > > 各位老师好， > 在使用kafka的streaming > table作为事实表创建cube时候，如果维度和指标都不选事实表中的关联键的话，在跑cube作业的时候，创建保存kafka数据的临时表会没有这个关联键，导致后面创建关联表的临时表会报错，具体报错信息请查看附件，谢谢。 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3555) Garbage collection on HBase step fails with S3 selected as storage
[
https://issues.apache.org/jira/browse/KYLIN-3555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16614536#comment-16614536
]
Iñigo Martinez commented on KYLIN-3555:
---
Hi Shaofeng.
According to install documentation for S3 / EMR, we should use an absolute URI.
[http://kylin.apache.org/docs23/install/kylin_aws_emr.html]
By using a relative path, it fails into HDFS. This causes some troubles because
we use a different cluster for Hive jobs and only way to share files between
hive and our kylin build clusters is by using S3 storage.
> Garbage collection on HBase step fails with S3 selected as storage
> --
>
> Key: KYLIN-3555
> URL: https://issues.apache.org/jira/browse/KYLIN-3555
> Project: Kylin
> Issue Type: Bug
> Components: Job Engine
>Affects Versions: v2.4.1
>Reporter: Iñigo Martinez
>Priority: Major
> Labels: build
> Attachments: Screenshot from 2018-09-11 12-31-25.png
>
>
> When building a cube with S3 selected has storage, build process fails at
> latest step.
> Although s3 has been defined as storage, cleanup task tries to delete from
> HDFS and, of course, there is no file at HDFS.
>
> {code:java}
> 2018-09-11 12:27:56,311 DEBUG [Scheduler 1407846257 Job
> f8416975-eea6-4500-9cb7-4374f28451dc-237]
> steps.HDFSPathGarbageCollectionStep:78 : Drop HDFS path on FileSystem:
> s3://XXX-emr-kylin
> 2018-09-11 12:27:57,364 DEBUG [Scheduler 1407846257 Job
> f8416975-eea6-4500-9cb7-4374f28451dc-237]
> steps.HDFSPathGarbageCollectionStep:87 : HDFS path
> /kylin/kylin_metadata/kylin-f8416975-eea6-4500-9cb7-4374f28451dc/plataforma_transacciones_cubo_v1/fact_distinct_columns
> is dropped.
> 2018-09-11 12:27:58,104 DEBUG [Scheduler 1407846257 Job
> f8416975-eea6-4500-9cb7-4374f28451dc-237]
> steps.HDFSPathGarbageCollectionStep:87 : HDFS path
> /kylin/kylin_metadata/kylin-f8416975-eea6-4500-9cb7-4374f28451dc/plataforma_transacciones_cubo_v1/hfile
> is dropped.
> 2018-09-11 12:27:58,140 DEBUG [Scheduler 1407846257 Job
> f8416975-eea6-4500-9cb7-4374f28451dc-237]
> steps.HDFSPathGarbageCollectionStep:78 : Drop HDFS path on FileSystem:
> hdfs://ip-10-0-1-63.eu-west-1.compute.internal:8020
> 2018-09-11 12:27:58,142 DEBUG [Scheduler 1407846257 Job
> f8416975-eea6-4500-9cb7-4374f28451dc-237]
> steps.HDFSPathGarbageCollectionStep:90 : HDFS path
> /kylin/kylin_metadata/kylin-f8416975-eea6-4500-9cb7-4374f28451dc/plataforma_transacciones_cubo_v1/fact_distinct_columns
> not exists.
> 2018-09-11 12:27:58,147 ERROR [Scheduler 1407846257 Job
> f8416975-eea6-4500-9cb7-4374f28451dc-237]
> steps.HDFSPathGarbageCollectionStep:68 :
> job:f8416975-eea6-4500-9cb7-4374f28451dc-15 execute finished with exception
> java.io.FileNotFoundException: File
> /kylin/kylin_metadata/kylin-f8416975-eea6-4500-9cb7-4374f28451dc/plataforma_transacciones_cubo_v1
> does not exist.
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:904)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.access$600(DistributedFileSystem.java:114)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:964)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:961)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:971)
> at
> org.apache.kylin.storage.hbase.steps.HDFSPathGarbageCollectionStep.dropHdfsPathOnCluster(HDFSPathGarbageCollectionStep.java:95)
> at
> org.apache.kylin.storage.hbase.steps.HDFSPathGarbageCollectionStep.doWork(HDFSPathGarbageCollectionStep.java:65)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:162)
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:69)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:162)
> at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:113)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748){code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Closed] (KYLIN-2614) DefaultX509TrustManager has unimplemented methods
[ https://issues.apache.org/jira/browse/KYLIN-2614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI closed KYLIN-2614. --- > DefaultX509TrustManager has unimplemented methods > - > > Key: KYLIN-2614 > URL: https://issues.apache.org/jira/browse/KYLIN-2614 > Project: Kylin > Issue Type: Bug >Reporter: Ted Yu >Priority: Major > Labels: scope > > checkClientTrusted() and checkServerTrusted() are not implemented. > Please refer to > hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/ssl/ReloadingX509TrustManager.java > for how they can be implemented. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (KYLIN-2315) Kylin run mode query not update metadata
[ https://issues.apache.org/jira/browse/KYLIN-2315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI closed KYLIN-2315. --- > Kylin run mode query not update metadata > > > Key: KYLIN-2315 > URL: https://issues.apache.org/jira/browse/KYLIN-2315 > Project: Kylin > Issue Type: Bug >Affects Versions: v1.6.0 > Environment: CentOS >Reporter: hoangle >Priority: Major > > I'm running two kylin instance on two nodes in my cluster. > Two kylin point the same kylin.rest.servers=mycluster:7070 > Node1: I set kyin.server.mode=all ( for build cube and query) > Node2: I set kyin.server.mode=query ( for query only) > I face with problem when I finish build cube on Node 1, but then I move to > Node 2 I don't see any update about cube. > The cube data on Node 2 only update when I restart Kylin instance on Node 2 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (KYLIN-1044) Update the how to contribute page to allow Github pull requests
[ https://issues.apache.org/jira/browse/KYLIN-1044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI closed KYLIN-1044. --- > Update the how to contribute page to allow Github pull requests > --- > > Key: KYLIN-1044 > URL: https://issues.apache.org/jira/browse/KYLIN-1044 > Project: Kylin > Issue Type: Improvement > Components: Documentation >Reporter: Henry Saputra >Assignee: liyang >Priority: Minor > > With more and more people using Github as open source contributions we need > to let Apache Kylin to accept patches via Github PR to the mirror repo. > To do this we need to update the how to contribute doc page [1] to describe > how to accept contributions via Github PR. > [1] https://kylin.incubator.apache.org/development/howto_contribute.html -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (KYLIN-3273) Download page must not link to dist.apache.org
[ https://issues.apache.org/jira/browse/KYLIN-3273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI closed KYLIN-3273. --- > Download page must not link to dist.apache.org > -- > > Key: KYLIN-3273 > URL: https://issues.apache.org/jira/browse/KYLIN-3273 > Project: Kylin > Issue Type: Bug > Environment: https://kylin.apache.org/download/ >Reporter: Sebb >Assignee: Shaofeng SHI >Priority: Major > > The download page currently links to https://dist.apache.org/ for the hashes > and sigs. > However that host is only intended as a staging area for use by developers. > Please can you change the links to use the ASF webserver instead? > i.e.. change > https://dist.apache.org/repos/dist/release/... > to > https://www.apache.org/dist/... > wherever it appears. > Also, the KEYS link should use https (SSL). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (KYLIN-3274) Please delete old releases from mirroring system
[ https://issues.apache.org/jira/browse/KYLIN-3274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI closed KYLIN-3274. --- > Please delete old releases from mirroring system > > > Key: KYLIN-3274 > URL: https://issues.apache.org/jira/browse/KYLIN-3274 > Project: Kylin > Issue Type: Bug > Environment: https://dist.apache.org/repos/dist/release/kylin/ >Reporter: Sebb >Assignee: Shaofeng SHI >Priority: Major > > To reduce the load on the ASF mirrors, projects are required to delete old > releases [1] > Please can you remove all non-current releases? > [From the download page, it looks like all but the latest release (2.3.0) are > superseded.] > It's unfair to expect the 3rd party mirrors to carry old releases. > Note that older releases can still be linked from the download page, but such > links should use the archive server at: > https://archive.apache.org/dist/incubator/mxnet/ > Thanks! > [1] http://www.apache.org/dev/release.html#when-to-archive -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3321) Set MALLOC_ARENA_MAX in script
[ https://issues.apache.org/jira/browse/KYLIN-3321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-3321: Fix Version/s: v2.5.0 > Set MALLOC_ARENA_MAX in script > -- > > Key: KYLIN-3321 > URL: https://issues.apache.org/jira/browse/KYLIN-3321 > Project: Kylin > Issue Type: Improvement > Components: Environment >Reporter: Ted Yu >Assignee: jiatao.tao >Priority: Major > Fix For: v2.5.0 > > > conf/setenv.sh would be good place to set MALLOC_ARENA_MAX which prevents > native memory OOM. > See https://github.com/prestodb/presto/issues/8993 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3399) Leaked lookup table in DictionaryGeneratorCLI#processSegment
[
https://issues.apache.org/jira/browse/KYLIN-3399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI updated KYLIN-3399:
Fix Version/s: v2.5.0
> Leaked lookup table in DictionaryGeneratorCLI#processSegment
>
>
> Key: KYLIN-3399
> URL: https://issues.apache.org/jira/browse/KYLIN-3399
> Project: Kylin
> Issue Type: Bug
> Components: Client - CLI
>Reporter: Ted Yu
>Assignee: jiatao.tao
>Priority: Major
> Fix For: v2.5.0
>
>
> {code}
> for (TableRef lookup : toCheckLookup) {
> logger.info("Checking snapshot of " + lookup);
> JoinDesc join =
> cubeSeg.getModel().getJoinsTree().getJoinByPKSide(lookup);
> cubeMgr.getLookupTable(cubeSeg, join);
> {code}
> The lookup table from the last line doesn't seem to be used.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
