[jira] [Resolved] (KYLIN-4662) Migrate from third-party Spark to offical Apache Spark
[ https://issues.apache.org/jira/browse/KYLIN-4662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhichao Zhang resolved KYLIN-4662. --- Resolution: Fixed > Migrate from third-party Spark to offical Apache Spark > -- > > Key: KYLIN-4662 > URL: https://issues.apache.org/jira/browse/KYLIN-4662 > Project: Kylin > Issue Type: Sub-task >Reporter: Xiaoxiang Yu >Assignee: Zhichao Zhang >Priority: Major > Fix For: v4.0.0-beta > > Original Estimate: 336h > Remaining Estimate: 336h > > Currently, Parquet Storage depend on a third-party Spark binary, it will > discourage some user, because they may have to use their own specific > version of Spark. > > Let's return to Offical Spark. Myabe Spark 2.4.x . -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-4693) avoid NPE when HDFSPathGarbageCollectionStep run
[
https://issues.apache.org/jira/browse/KYLIN-4693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chuxiao closed KYLIN-4693.
--
Resolution: Invalid
> avoid NPE when HDFSPathGarbageCollectionStep run
>
>
> Key: KYLIN-4693
> URL: https://issues.apache.org/jira/browse/KYLIN-4693
> Project: Kylin
> Issue Type: Bug
>Reporter: chuxiao
>Priority: Major
>
> {code:java}
> java.lang.NullPointerException
> at
> org.apache.kylin.storage.hbase.steps.HDFSPathGarbageCollectionStep.doWork(HDFSPathGarbageCollectionStep.java:97)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:179)
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:179)
> at
> org.apache.kylin.job.impl.threadpool.DistributedScheduler$JobRunner.run(DistributedScheduler.java:110)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (KYLIN-4693) avoid NPE when HDFSPathGarbageCollectionStep run
chuxiao created KYLIN-4693:
--
Summary: avoid NPE when HDFSPathGarbageCollectionStep run
Key: KYLIN-4693
URL: https://issues.apache.org/jira/browse/KYLIN-4693
Project: Kylin
Issue Type: Bug
Reporter: chuxiao
{code:java}
java.lang.NullPointerException
at
org.apache.kylin.storage.hbase.steps.HDFSPathGarbageCollectionStep.doWork(HDFSPathGarbageCollectionStep.java:97)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:179)
at
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:179)
at
org.apache.kylin.job.impl.threadpool.DistributedScheduler$JobRunner.run(DistributedScheduler.java:110)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KYLIN-4516) Support System Cube
[ https://issues.apache.org/jira/browse/KYLIN-4516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17176358#comment-17176358 ] ASF subversion and git services commented on KYLIN-4516: Commit 8e6c90accf7e852c90158f7439a6d3bc48c5732b in kylin's branch refs/heads/kylin-on-parquet-v2 from Zhichao Zhang [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=8e6c90a ] KYLIN-4516 Fix 'next on empty iterator' error When 'scanRows' is a empty list, it will throw 'next on empty interator' error. > Support System Cube > --- > > Key: KYLIN-4516 > URL: https://issues.apache.org/jira/browse/KYLIN-4516 > Project: Kylin > Issue Type: Sub-task >Reporter: wangrupeng >Assignee: wangrupeng >Priority: Major > Fix For: v4.0.0-beta > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4662) Migrate from third-party Spark to offical Apache Spark
[ https://issues.apache.org/jira/browse/KYLIN-4662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17176356#comment-17176356 ] ASF GitHub Bot commented on KYLIN-4662: --- hit-lacus merged pull request #1354: URL: https://github.com/apache/kylin/pull/1354 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Migrate from third-party Spark to offical Apache Spark > -- > > Key: KYLIN-4662 > URL: https://issues.apache.org/jira/browse/KYLIN-4662 > Project: Kylin > Issue Type: Sub-task >Reporter: Xiaoxiang Yu >Assignee: Zhichao Zhang >Priority: Major > Fix For: v4.0.0-beta > > Original Estimate: 336h > Remaining Estimate: 336h > > Currently, Parquet Storage depend on a third-party Spark binary, it will > discourage some user, because they may have to use their own specific > version of Spark. > > Let's return to Offical Spark. Myabe Spark 2.4.x . -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4680) Avoid annoying log messages of unit test and integration test
[ https://issues.apache.org/jira/browse/KYLIN-4680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17176359#comment-17176359 ] ASF subversion and git services commented on KYLIN-4680: Commit 47dd4a23d7b83b68a6e75c2aed0552efe41f893d in kylin's branch refs/heads/kylin-on-parquet-v2 from Zhichao Zhang [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=47dd4a2 ] KYLIN-4680 Avoid annoying log messages of query module and unit test > Avoid annoying log messages of unit test and integration test > - > > Key: KYLIN-4680 > URL: https://issues.apache.org/jira/browse/KYLIN-4680 > Project: Kylin > Issue Type: Improvement > Components: Integration, Tools, Build and Test >Affects Versions: v4.0.0-beta >Reporter: wangrupeng >Assignee: wangrupeng >Priority: Major > Fix For: v4.0.0-beta > > > When running a unit test case, it will output too much annoying log messages. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4662) Migrate from third-party Spark to offical Apache Spark
[ https://issues.apache.org/jira/browse/KYLIN-4662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17176355#comment-17176355 ] ASF GitHub Bot commented on KYLIN-4662: --- hit-lacus commented on pull request #1354: URL: https://github.com/apache/kylin/pull/1354#issuecomment-672890438 In general, this look good to me. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Migrate from third-party Spark to offical Apache Spark > -- > > Key: KYLIN-4662 > URL: https://issues.apache.org/jira/browse/KYLIN-4662 > Project: Kylin > Issue Type: Sub-task >Reporter: Xiaoxiang Yu >Assignee: Zhichao Zhang >Priority: Major > Fix For: v4.0.0-beta > > Original Estimate: 336h > Remaining Estimate: 336h > > Currently, Parquet Storage depend on a third-party Spark binary, it will > discourage some user, because they may have to use their own specific > version of Spark. > > Let's return to Offical Spark. Myabe Spark 2.4.x . -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4662) Migrate from third-party Spark to offical Apache Spark
[ https://issues.apache.org/jira/browse/KYLIN-4662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17176357#comment-17176357 ] ASF subversion and git services commented on KYLIN-4662: Commit 15305eaf86fb8925f559e3b3d0efda05f4a022f8 in kylin's branch refs/heads/kylin-on-parquet-v2 from Zhichao Zhang [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=15305ea ] KYLIN-4662 Migrate from third-party Spark to offical Apache Spark Use Apache Spark 2.4.6 as default spark version > Migrate from third-party Spark to offical Apache Spark > -- > > Key: KYLIN-4662 > URL: https://issues.apache.org/jira/browse/KYLIN-4662 > Project: Kylin > Issue Type: Sub-task >Reporter: Xiaoxiang Yu >Assignee: Zhichao Zhang >Priority: Major > Fix For: v4.0.0-beta > > Original Estimate: 336h > Remaining Estimate: 336h > > Currently, Parquet Storage depend on a third-party Spark binary, it will > discourage some user, because they may have to use their own specific > version of Spark. > > Let's return to Offical Spark. Myabe Spark 2.4.x . -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] hit-lacus merged pull request #1354: KYLIN-4662 Migrate from third-party Spark to offical Apache Spark
hit-lacus merged pull request #1354: URL: https://github.com/apache/kylin/pull/1354 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kylin] hit-lacus commented on pull request #1354: KYLIN-4662 Migrate from third-party Spark to offical Apache Spark
hit-lacus commented on pull request #1354: URL: https://github.com/apache/kylin/pull/1354#issuecomment-672890438 In general, this look good to me. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (KYLIN-4662) Migrate from third-party Spark to offical Apache Spark
[ https://issues.apache.org/jira/browse/KYLIN-4662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17176350#comment-17176350 ] ASF GitHub Bot commented on KYLIN-4662: --- hit-lacus commented on pull request #1354: URL: https://github.com/apache/kylin/pull/1354#issuecomment-672888369 This patch has some unpredictable performance lose impact on both build engine and query engine. But I think we can merge it and do a benchmark to analyse the difference . So let's keep tuned. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Migrate from third-party Spark to offical Apache Spark > -- > > Key: KYLIN-4662 > URL: https://issues.apache.org/jira/browse/KYLIN-4662 > Project: Kylin > Issue Type: Sub-task >Reporter: Xiaoxiang Yu >Assignee: Zhichao Zhang >Priority: Major > Fix For: v4.0.0-beta > > Original Estimate: 336h > Remaining Estimate: 336h > > Currently, Parquet Storage depend on a third-party Spark binary, it will > discourage some user, because they may have to use their own specific > version of Spark. > > Let's return to Offical Spark. Myabe Spark 2.4.x . -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] hit-lacus commented on pull request #1354: KYLIN-4662 Migrate from third-party Spark to offical Apache Spark
hit-lacus commented on pull request #1354: URL: https://github.com/apache/kylin/pull/1354#issuecomment-672888369 This patch has some unpredictable performance lose impact on both build engine and query engine. But I think we can merge it and do a benchmark to analyse the difference . So let's keep tuned. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kylin] coveralls commented on pull request #1356: KYLIN-4687 addendum - add apache license header
coveralls commented on pull request #1356: URL: https://github.com/apache/kylin/pull/1356#issuecomment-672743442 ## Pull Request Test Coverage Report for [Build 6245](https://coveralls.io/builds/32698479) * **0** of **0** changed or added relevant lines in **0** files are covered. * **9** unchanged lines in **3** files lost coverage. * Overall coverage decreased (**-0.004%**) to **28.029%** --- | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [core-job/src/main/java/org/apache/kylin/job/impl/threadpool/DefaultScheduler.java](https://coveralls.io/builds/32698479/source?filename=core-job%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fjob%2Fimpl%2Fthreadpool%2FDefaultScheduler.java#L194) | 2 | 80.23% | | [stream-core/src/main/java/org/apache/kylin/stream/core/storage/CheckPointStore.java](https://coveralls.io/builds/32698479/source?filename=stream-core%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fstream%2Fcore%2Fstorage%2FCheckPointStore.java#L98) | 2 | 73.74% | | [core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://coveralls.io/builds/32698479/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Finmemcubing%2FMemDiskStore.java#L439) | 5 | 77.81% | | Totals | [](https://coveralls.io/builds/32698479) | | :-- | --: | | Change from base [Build 6240](https://coveralls.io/builds/32679886): | -0.004% | | Covered Lines: | 26239 | | Relevant Lines: | 93613 | --- # - [Coveralls](https://coveralls.io) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (KYLIN-4687) add unify clean sh to excute some clean shells
[
https://issues.apache.org/jira/browse/KYLIN-4687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17176175#comment-17176175
]
ASF GitHub Bot commented on KYLIN-4687:
---
coveralls commented on pull request #1356:
URL: https://github.com/apache/kylin/pull/1356#issuecomment-672743442
## Pull Request Test Coverage Report for [Build
6245](https://coveralls.io/builds/32698479)
* **0** of **0** changed or added relevant lines in **0** files are
covered.
* **9** unchanged lines in **3** files lost coverage.
* Overall coverage decreased (**-0.004%**) to **28.029%**
---
| Files with Coverage Reduction | New Missed Lines | % |
| :-|--|--: |
|
[core-job/src/main/java/org/apache/kylin/job/impl/threadpool/DefaultScheduler.java](https://coveralls.io/builds/32698479/source?filename=core-job%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fjob%2Fimpl%2Fthreadpool%2FDefaultScheduler.java#L194)
| 2 | 80.23% |
|
[stream-core/src/main/java/org/apache/kylin/stream/core/storage/CheckPointStore.java](https://coveralls.io/builds/32698479/source?filename=stream-core%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fstream%2Fcore%2Fstorage%2FCheckPointStore.java#L98)
| 2 | 73.74% |
|
[core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://coveralls.io/builds/32698479/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Finmemcubing%2FMemDiskStore.java#L439)
| 5 | 77.81% |
| Totals | [](https://coveralls.io/builds/32698479)
|
| :-- | --: |
| Change from base [Build 6240](https://coveralls.io/builds/32679886): |
-0.004% |
| Covered Lines: | 26239 |
| Relevant Lines: | 93613 |
---
# - [Coveralls](https://coveralls.io)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> add unify clean sh to excute some clean shells
> --
>
> Key: KYLIN-4687
> URL: https://issues.apache.org/jira/browse/KYLIN-4687
> Project: Kylin
> Issue Type: Improvement
>Reporter: chuxiao
>Priority: Major
>
> you can use as "${KYLIN_HOME}/bin/clean-kylin-dirty-data.sh" every day by
> crontab
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (KYLIN-4671) Avoid FetchRunner printing too many logs due to NPE
[ https://issues.apache.org/jira/browse/KYLIN-4671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chuxiao updated KYLIN-4671: --- Description: See pcitcure. When there are some dirty metadatas, NPE exceptions will continue to be log. Abnormal skip, dirty data can be cleaned up when cleaning up historical job information. So adjust the log level to debug。 was:too many log. Ignore , until remove job info when clean metadata. > Avoid FetchRunner printing too many logs due to NPE > --- > > Key: KYLIN-4671 > URL: https://issues.apache.org/jira/browse/KYLIN-4671 > Project: Kylin > Issue Type: Improvement >Reporter: chuxiao >Priority: Minor > Attachments: D-Chat_20200729164446.png > > > See pcitcure. > When there are some dirty metadatas, NPE exceptions will continue to be log. > Abnormal skip, dirty data can be cleaned up when cleaning up historical job > information. So adjust the log level to debug。 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4671) Avoid FetchRunner printing too many logs due to NPE
[ https://issues.apache.org/jira/browse/KYLIN-4671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chuxiao updated KYLIN-4671: --- Summary: Avoid FetchRunner printing too many logs due to NPE (was: modify FetchRunner catch getOutput NPE log level to debug for ignore too many logs) > Avoid FetchRunner printing too many logs due to NPE > --- > > Key: KYLIN-4671 > URL: https://issues.apache.org/jira/browse/KYLIN-4671 > Project: Kylin > Issue Type: Improvement >Reporter: chuxiao >Priority: Minor > Attachments: D-Chat_20200729164446.png > > > too many log. Ignore , until remove job info when clean metadata. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4687) add unify clean sh to excute some clean shells
[
https://issues.apache.org/jira/browse/KYLIN-4687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17176144#comment-17176144
]
ASF subversion and git services commented on KYLIN-4687:
Commit 28c21ef6127a72d960a66ac44b46ea57df62dbb8 in kylin's branch
refs/heads/master from Guangxu Cheng
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=28c21ef ]
KYLIN-4687 addendum - add apache license header
> add unify clean sh to excute some clean shells
> --
>
> Key: KYLIN-4687
> URL: https://issues.apache.org/jira/browse/KYLIN-4687
> Project: Kylin
> Issue Type: Improvement
>Reporter: chuxiao
>Priority: Major
>
> you can use as "${KYLIN_HOME}/bin/clean-kylin-dirty-data.sh" every day by
> crontab
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KYLIN-4687) add unify clean sh to excute some clean shells
[
https://issues.apache.org/jira/browse/KYLIN-4687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17176141#comment-17176141
]
ASF GitHub Bot commented on KYLIN-4687:
---
hit-lacus commented on pull request #1356:
URL: https://github.com/apache/kylin/pull/1356#issuecomment-672712162
LGTM
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> add unify clean sh to excute some clean shells
> --
>
> Key: KYLIN-4687
> URL: https://issues.apache.org/jira/browse/KYLIN-4687
> Project: Kylin
> Issue Type: Improvement
>Reporter: chuxiao
>Priority: Major
>
> you can use as "${KYLIN_HOME}/bin/clean-kylin-dirty-data.sh" every day by
> crontab
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KYLIN-4687) add unify clean sh to excute some clean shells
[
https://issues.apache.org/jira/browse/KYLIN-4687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17176137#comment-17176137
]
ASF GitHub Bot commented on KYLIN-4687:
---
hit-lacus commented on pull request #1356:
URL: https://github.com/apache/kylin/pull/1356#issuecomment-672712042
LGTM
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> add unify clean sh to excute some clean shells
> --
>
> Key: KYLIN-4687
> URL: https://issues.apache.org/jira/browse/KYLIN-4687
> Project: Kylin
> Issue Type: Improvement
>Reporter: chuxiao
>Priority: Major
>
> you can use as "${KYLIN_HOME}/bin/clean-kylin-dirty-data.sh" every day by
> crontab
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KYLIN-4687) add unify clean sh to excute some clean shells
[
https://issues.apache.org/jira/browse/KYLIN-4687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17176142#comment-17176142
]
ASF GitHub Bot commented on KYLIN-4687:
---
hit-lacus removed a comment on pull request #1356:
URL: https://github.com/apache/kylin/pull/1356#issuecomment-672712162
LGTM
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> add unify clean sh to excute some clean shells
> --
>
> Key: KYLIN-4687
> URL: https://issues.apache.org/jira/browse/KYLIN-4687
> Project: Kylin
> Issue Type: Improvement
>Reporter: chuxiao
>Priority: Major
>
> you can use as "${KYLIN_HOME}/bin/clean-kylin-dirty-data.sh" every day by
> crontab
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KYLIN-4687) add unify clean sh to excute some clean shells
[
https://issues.apache.org/jira/browse/KYLIN-4687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17176143#comment-17176143
]
ASF GitHub Bot commented on KYLIN-4687:
---
hit-lacus merged pull request #1356:
URL: https://github.com/apache/kylin/pull/1356
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> add unify clean sh to excute some clean shells
> --
>
> Key: KYLIN-4687
> URL: https://issues.apache.org/jira/browse/KYLIN-4687
> Project: Kylin
> Issue Type: Improvement
>Reporter: chuxiao
>Priority: Major
>
> you can use as "${KYLIN_HOME}/bin/clean-kylin-dirty-data.sh" every day by
> crontab
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [kylin] hit-lacus removed a comment on pull request #1356: KYLIN-4687 addendum - add apache license header
hit-lacus removed a comment on pull request #1356: URL: https://github.com/apache/kylin/pull/1356#issuecomment-672712162 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kylin] hit-lacus merged pull request #1356: KYLIN-4687 addendum - add apache license header
hit-lacus merged pull request #1356: URL: https://github.com/apache/kylin/pull/1356 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kylin] hit-lacus commented on pull request #1356: KYLIN-4687 addendum - add apache license header
hit-lacus commented on pull request #1356: URL: https://github.com/apache/kylin/pull/1356#issuecomment-672712042 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kylin] hit-lacus commented on pull request #1356: KYLIN-4687 addendum - add apache license header
hit-lacus commented on pull request #1356: URL: https://github.com/apache/kylin/pull/1356#issuecomment-672712162 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kylin] hit-lacus commented on pull request #1254: KYLIN-4556 improve job service
hit-lacus commented on pull request #1254: URL: https://github.com/apache/kylin/pull/1254#issuecomment-672711464 Oh, my mistake. I apprecaite it, I forget to check the license header. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (KYLIN-4556) improve job service
[ https://issues.apache.org/jira/browse/KYLIN-4556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17176136#comment-17176136 ] ASF GitHub Bot commented on KYLIN-4556: --- hit-lacus commented on pull request #1254: URL: https://github.com/apache/kylin/pull/1254#issuecomment-672711464 Oh, my mistake. I apprecaite it, I forget to check the license header. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > improve job service > --- > > Key: KYLIN-4556 > URL: https://issues.apache.org/jira/browse/KYLIN-4556 > Project: Kylin > Issue Type: Task >Reporter: Guangxu Cheng >Assignee: Guangxu Cheng >Priority: Major > > At present, only cubing and checkpoint jobs can be obtained through job > API,for cardinality and lookup snapshot job, we can't get them which is very > inconvenient. > when the cardinality task fails, we can only find the reason why the task > failed from the system log, and we cannot rerun or delete these failed tasks > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4556) improve job service
[ https://issues.apache.org/jira/browse/KYLIN-4556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17176110#comment-17176110 ] ASF GitHub Bot commented on KYLIN-4556: --- guangxuCheng commented on pull request #1254: URL: https://github.com/apache/kylin/pull/1254#issuecomment-672690026 The failed check is caused by #1349, I have open a new pr #1356 to fix it This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > improve job service > --- > > Key: KYLIN-4556 > URL: https://issues.apache.org/jira/browse/KYLIN-4556 > Project: Kylin > Issue Type: Task >Reporter: Guangxu Cheng >Assignee: Guangxu Cheng >Priority: Major > > At present, only cubing and checkpoint jobs can be obtained through job > API,for cardinality and lookup snapshot job, we can't get them which is very > inconvenient. > when the cardinality task fails, we can only find the reason why the task > failed from the system log, and we cannot rerun or delete these failed tasks > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] guangxuCheng commented on pull request #1254: KYLIN-4556 improve job service
guangxuCheng commented on pull request #1254: URL: https://github.com/apache/kylin/pull/1254#issuecomment-672690026 The failed check is caused by #1349, I have open a new pr #1356 to fix it This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (KYLIN-4687) add unify clean sh to excute some clean shells
[
https://issues.apache.org/jira/browse/KYLIN-4687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17176107#comment-17176107
]
ASF GitHub Bot commented on KYLIN-4687:
---
guangxuCheng commented on pull request #1349:
URL: https://github.com/apache/kylin/pull/1349#issuecomment-672688877
@hit-lacus The new file is missing the apache license, which causes the rat
check to fail. I create a new pr to fix it.
https://github.com/apache/kylin/pull/1356
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> add unify clean sh to excute some clean shells
> --
>
> Key: KYLIN-4687
> URL: https://issues.apache.org/jira/browse/KYLIN-4687
> Project: Kylin
> Issue Type: Improvement
>Reporter: chuxiao
>Priority: Major
>
> you can use as "${KYLIN_HOME}/bin/clean-kylin-dirty-data.sh" every day by
> crontab
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [kylin] guangxuCheng commented on pull request #1349: KYLIN-4687 add unify clean sh to excute some clean shells
guangxuCheng commented on pull request #1349: URL: https://github.com/apache/kylin/pull/1349#issuecomment-672688877 @hit-lacus The new file is missing the apache license, which causes the rat check to fail. I create a new pr to fix it. https://github.com/apache/kylin/pull/1356 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (KYLIN-4687) add unify clean sh to excute some clean shells
[
https://issues.apache.org/jira/browse/KYLIN-4687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17176104#comment-17176104
]
ASF GitHub Bot commented on KYLIN-4687:
---
guangxuCheng opened a new pull request #1356:
URL: https://github.com/apache/kylin/pull/1356
## Proposed changes
Describe the big picture of your changes here to communicate to the
maintainers why we should accept this pull request. If it fixes a bug or
resolves a feature request, be sure to link to that issue.
## Types of changes
What types of changes does your code introduce to Kylin?
_Put an `x` in the boxes that apply_
- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to not work as expected)
- [ ] Documentation Update (if none of the other choices apply)
## Checklist
_Put an `x` in the boxes that apply. You can also fill these out after
creating the PR. If you're unsure about any of them, don't hesitate to ask.
We're here to help! This is simply a reminder of what we are going to look for
before merging your code._
- [x] I have create an issue on [Kylin's
jira](https://issues.apache.org/jira/browse/KYLIN), and have described the
bug/feature there in detail
- [x] Commit messages in my PR start with the related jira ID, like
"KYLIN- Make Kylin project open-source"
- [x] Compiling and unit tests pass locally with my changes
- [ ] I have added tests that prove my fix is effective or that my feature
works
- [ ] If this change need a document change, I will prepare another pr
against the `document` branch
- [ ] Any dependent changes have been merged
## Further comments
If this is a relatively large or complex change, kick off the discussion at
user@kylin or dev@kylin by explaining why you chose the solution you did and
what alternatives you considered, etc...
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> add unify clean sh to excute some clean shells
> --
>
> Key: KYLIN-4687
> URL: https://issues.apache.org/jira/browse/KYLIN-4687
> Project: Kylin
> Issue Type: Improvement
>Reporter: chuxiao
>Priority: Major
>
> you can use as "${KYLIN_HOME}/bin/clean-kylin-dirty-data.sh" every day by
> crontab
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [kylin] guangxuCheng opened a new pull request #1356: KYLIN-4687 addendum - add apache license header
guangxuCheng opened a new pull request #1356: URL: https://github.com/apache/kylin/pull/1356 ## Proposed changes Describe the big picture of your changes here to communicate to the maintainers why we should accept this pull request. If it fixes a bug or resolves a feature request, be sure to link to that issue. ## Types of changes What types of changes does your code introduce to Kylin? _Put an `x` in the boxes that apply_ - [x] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [x] I have create an issue on [Kylin's jira](https://issues.apache.org/jira/browse/KYLIN), and have described the bug/feature there in detail - [x] Commit messages in my PR start with the related jira ID, like "KYLIN- Make Kylin project open-source" - [x] Compiling and unit tests pass locally with my changes - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] If this change need a document change, I will prepare another pr against the `document` branch - [ ] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at user@kylin or dev@kylin by explaining why you chose the solution you did and what alternatives you considered, etc... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kylin] coveralls commented on pull request #1355: Update README.md
coveralls commented on pull request #1355: URL: https://github.com/apache/kylin/pull/1355#issuecomment-672677612 ## Pull Request Test Coverage Report for [Build 6244](https://coveralls.io/builds/32696874) * **0** of **0** changed or added relevant lines in **0** files are covered. * **5** unchanged lines in **3** files lost coverage. * Overall coverage increased (+**0.001%**) to **28.035%** --- | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [tool/src/main/java/org/apache/kylin/tool/query/ProbabilityGenerator.java](https://coveralls.io/builds/32696874/source?filename=tool%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Ftool%2Fquery%2FProbabilityGenerator.java#L44) | 1 | 83.33% | | [core-job/src/main/java/org/apache/kylin/job/impl/threadpool/DefaultScheduler.java](https://coveralls.io/builds/32696874/source?filename=core-job%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fjob%2Fimpl%2Fthreadpool%2FDefaultScheduler.java#L194) | 2 | 80.23% | | [stream-core/src/main/java/org/apache/kylin/stream/core/storage/CheckPointStore.java](https://coveralls.io/builds/32696874/source?filename=stream-core%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fstream%2Fcore%2Fstorage%2FCheckPointStore.java#L98) | 2 | 73.74% | | Totals | [](https://coveralls.io/builds/32696874) | | :-- | --: | | Change from base [Build 6240](https://coveralls.io/builds/32679886): | 0.001% | | Covered Lines: | 26244 | | Relevant Lines: | 93613 | --- # - [Coveralls](https://coveralls.io) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (KYLIN-4691) global dictionary merge failed when using hive/mr
[ https://issues.apache.org/jira/browse/KYLIN-4691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17176089#comment-17176089 ] Xiaoxiang Yu commented on KYLIN-4691: - i am not totally sure, but I guess this is fixed by [https://github.com/apache/kylin/pull/1304,] https://issues.apache.org/jira/browse/KYLIN-4616 . > global dictionary merge failed when using hive/mr > - > > Key: KYLIN-4691 > URL: https://issues.apache.org/jira/browse/KYLIN-4691 > Project: Kylin > Issue Type: Bug >Affects Versions: v3.1.0 >Reporter: hejian >Priority: Major > Attachments: image-2020-08-11-17-37-00-681.png, > image-2020-08-11-17-37-13-357.png, image-2020-08-11-17-37-48-225.png, > image-2020-08-11-17-37-49-205.png > > > !image-2020-08-11-17-37-00-681.png!!image-2020-08-11-17-37-13-357.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4691) global dictionary merge failed when using hive/mr
[ https://issues.apache.org/jira/browse/KYLIN-4691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hejian updated KYLIN-4691: -- Affects Version/s: v3.1.0 > global dictionary merge failed when using hive/mr > - > > Key: KYLIN-4691 > URL: https://issues.apache.org/jira/browse/KYLIN-4691 > Project: Kylin > Issue Type: Bug >Affects Versions: v3.1.0 >Reporter: hejian >Priority: Major > Attachments: image-2020-08-11-17-37-00-681.png, > image-2020-08-11-17-37-13-357.png, image-2020-08-11-17-37-48-225.png, > image-2020-08-11-17-37-49-205.png > > > !image-2020-08-11-17-37-00-681.png!!image-2020-08-11-17-37-13-357.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] Lujiusi commented on pull request #1355: Update README.md
Lujiusi commented on pull request #1355: URL: https://github.com/apache/kylin/pull/1355#issuecomment-672634708 dsfasdfasdfasdfaf This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (KYLIN-4626) add set kylin home sh
[ https://issues.apache.org/jira/browse/KYLIN-4626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17176046#comment-17176046 ] wangrupeng commented on KYLIN-4626: --- That's great! > add set kylin home sh > - > > Key: KYLIN-4626 > URL: https://issues.apache.org/jira/browse/KYLIN-4626 > Project: Kylin > Issue Type: Improvement >Reporter: chuxiao >Assignee: chuxiao >Priority: Major > > KYLIN_HOME 是重要的，几乎每个脚本都离不开它。但随便设置环境变量并不是一个最佳行为，比如安装了多套实例。增加set-kylin-home.sh, > kylin实例可以设置自己的环境变量。 > 这个主要是面向一台服务器部署多套kylin服务用的。 > 另外我们运维规范现在要求环境变量都放到服务自己的文件里，避免冲突。 > 而其他环境变量都可以放到setenv.sh里，但kylin_home在setenv.sh加载前就需要。所以默认是取系统环境变量，也可以根据需要修改 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] Lujiusi opened a new pull request #1355: Update README.md
Lujiusi opened a new pull request #1355: URL: https://github.com/apache/kylin/pull/1355 asdfasfasf ## Proposed changes Describe the big picture of your changes here to communicate to the maintainers why we should accept this pull request. If it fixes a bug or resolves a feature request, be sure to link to that issue. ## Types of changes What types of changes does your code introduce to Kylin? _Put an `x` in the boxes that apply_ - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [ ] I have create an issue on [Kylin's jira](https://issues.apache.org/jira/browse/KYLIN), and have described the bug/feature there in detail - [ ] Commit messages in my PR start with the related jira ID, like "KYLIN- Make Kylin project open-source" - [ ] Compiling and unit tests pass locally with my changes - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] If this change need a document change, I will prepare another pr against the `document` branch - [ ] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at user@kylin or dev@kylin by explaining why you chose the solution you did and what alternatives you considered, etc... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
