[jira] [Updated] (KYLIN-4672) Using Real-time Lambda to refresh the data lead to result not incorrect

2020-07-30 Thread zzshine (Jira)


 [ 
https://issues.apache.org/jira/browse/KYLIN-4672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

zzshine updated KYLIN-4672:
---
Fix Version/s: (was: v3.1.1)
   (was: v3.0.3)

> Using Real-time Lambda to refresh the data lead to result not incorrect
> ---
>
> Key: KYLIN-4672
> URL: https://issues.apache.org/jira/browse/KYLIN-4672
> Project: Kylin
>  Issue Type: Bug
>  Components: Real-time Streaming
>Affects Versions: v3.1.0, v3.0.2
> Environment: CDH 6.0.1
>Reporter: zzshine
>Priority: Major
>  Labels: kylin, lambda, realtime
> Attachments: Kylin Lambda 验证.docx
>
>
> Using Kylin Real-time Lambda model to refresh the data lead to result not 
> incorrect.
> Please,See the attachment(Kylin Lambda 验证) for operation steps.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (KYLIN-4672) Using Real-time Lambda to refresh the data lead to result not incorrect

2020-07-30 Thread zzshine (Jira)
zzshine created KYLIN-4672:
--

 Summary: Using Real-time Lambda to refresh the data lead to result 
not incorrect
 Key: KYLIN-4672
 URL: https://issues.apache.org/jira/browse/KYLIN-4672
 Project: Kylin
  Issue Type: Bug
  Components: Real-time Streaming
Affects Versions: v3.0.2, v3.1.0
 Environment: CDH 6.0.1
Reporter: zzshine
 Fix For: v3.0.3, v3.1.1
 Attachments: Kylin Lambda 验证.docx

Using Kylin Real-time Lambda model to refresh the data lead to result not 
incorrect.

Please,See the attachment(Kylin Lambda 验证) for operation steps.

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [kylin] coveralls commented on pull request #1337: Fix travis ci job log exceeded maximum log length

2020-07-30 Thread GitBox


coveralls commented on pull request #1337:
URL: https://github.com/apache/kylin/pull/1337#issuecomment-666900382


   ## Pull Request Test Coverage Report for [Build 
6210](https://coveralls.io/builds/32446757)
   
   * **0** of **0**   changed or added relevant lines in **0** files are 
covered.
   * **22** unchanged lines in **6** files lost coverage.
   * Overall coverage decreased (**-0.02%**) to **28.03%**
   
   ---
   
   
   |  Files with Coverage Reduction | New Missed Lines | % |
   | :-|--|--: |
   | 
[core-common/src/main/java/org/apache/kylin/common/persistence/FileResourceStore.java](https://coveralls.io/builds/32446757/source?filename=core-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcommon%2Fpersistence%2FFileResourceStore.java#L237)
 | 1 | 77.78% |
   | 
[core-cube/src/main/java/org/apache/kylin/cube/CubeInstance.java](https://coveralls.io/builds/32446757/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2FCubeInstance.java#L251)
 | 1 | 41.29% |
   | 
[stream-core/src/main/java/org/apache/kylin/stream/core/storage/StreamingCubeSegment.java](https://coveralls.io/builds/32446757/source?filename=stream-core%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fstream%2Fcore%2Fstorage%2FStreamingCubeSegment.java#L175)
 | 1 | 44.83% |
   | 
[stream-core/src/main/java/org/apache/kylin/stream/core/storage/StreamingSegmentManager.java](https://coveralls.io/builds/32446757/source?filename=stream-core%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fstream%2Fcore%2Fstorage%2FStreamingSegmentManager.java#L189)
 | 1 | 26.7% |
   | 
[core-metrics/src/main/java/org/apache/kylin/metrics/lib/impl/MetricsSystem.java](https://coveralls.io/builds/32446757/source?filename=core-metrics%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fmetrics%2Flib%2Fimpl%2FMetricsSystem.java#L46)
 | 5 | 66.67% |
   | 
[core-dictionary/src/main/java/org/apache/kylin/dict/global/AppendDictSlice.java](https://coveralls.io/builds/32446757/source?filename=core-dictionary%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fdict%2Fglobal%2FAppendDictSlice.java#L102)
 | 13 | 58.59% |
   
   
   |  Totals | [![Coverage 
Status](https://coveralls.io/builds/32446757/badge)](https://coveralls.io/builds/32446757)
 |
   | :-- | --: |
   | Change from base [Build 6203](https://coveralls.io/builds/32402099): |  
-0.02% |
   | Covered Lines: | 26227 |
   | Relevant Lines: | 93568 |
   
   ---
   #   - [Coveralls](https://coveralls.io)
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Commented] (KYLIN-4653) Make the capacity for the LinkedBlockingQueue of BlockingReservoir configurable

2020-07-30 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/KYLIN-4653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168370#comment-17168370
 ] 

ASF subversion and git services commented on KYLIN-4653:


Commit 159a0fffe0aff2babd7d6f97bab7de6c7dc2be35 in kylin's branch 
refs/heads/master from XiaoxiangYu
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=159a0ff ]

KYLIN-4653 Make the capacity for the LinkedBlockingQueue of BlockingReservoir 
configurable


> Make the capacity for the LinkedBlockingQueue of BlockingReservoir 
> configurable
> ---
>
> Key: KYLIN-4653
> URL: https://issues.apache.org/jira/browse/KYLIN-4653
> Project: Kylin
>  Issue Type: Improvement
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Fix For: v3.1.1
>
>
> Currently the blocking queue capacity is 50K. It is too small, which easily 
> goes into being blocked and slows down queries.
> It's better to make this value configurable and enlarge the default value.
> For real usage, I strongly recommend to use unbounded LinkedBlockingQueue, 
> which is supposed not to block queries due to report query metrics.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (KYLIN-4653) Make the capacity for the LinkedBlockingQueue of BlockingReservoir configurable

2020-07-30 Thread ASF GitHub Bot (Jira)


[ 
https://issues.apache.org/jira/browse/KYLIN-4653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168369#comment-17168369
 ] 

ASF GitHub Bot commented on KYLIN-4653:
---

hit-lacus commented on pull request #1334:
URL: https://github.com/apache/kylin/pull/1334#issuecomment-666896102


   I will merge this patch.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Make the capacity for the LinkedBlockingQueue of BlockingReservoir 
> configurable
> ---
>
> Key: KYLIN-4653
> URL: https://issues.apache.org/jira/browse/KYLIN-4653
> Project: Kylin
>  Issue Type: Improvement
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Fix For: v3.1.1
>
>
> Currently the blocking queue capacity is 50K. It is too small, which easily 
> goes into being blocked and slows down queries.
> It's better to make this value configurable and enlarge the default value.
> For real usage, I strongly recommend to use unbounded LinkedBlockingQueue, 
> which is supposed not to block queries due to report query metrics.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (KYLIN-4653) Make the capacity for the LinkedBlockingQueue of BlockingReservoir configurable

2020-07-30 Thread ASF GitHub Bot (Jira)


[ 
https://issues.apache.org/jira/browse/KYLIN-4653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168371#comment-17168371
 ] 

ASF GitHub Bot commented on KYLIN-4653:
---

hit-lacus merged pull request #1334:
URL: https://github.com/apache/kylin/pull/1334


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Make the capacity for the LinkedBlockingQueue of BlockingReservoir 
> configurable
> ---
>
> Key: KYLIN-4653
> URL: https://issues.apache.org/jira/browse/KYLIN-4653
> Project: Kylin
>  Issue Type: Improvement
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Fix For: v3.1.1
>
>
> Currently the blocking queue capacity is 50K. It is too small, which easily 
> goes into being blocked and slows down queries.
> It's better to make this value configurable and enlarge the default value.
> For real usage, I strongly recommend to use unbounded LinkedBlockingQueue, 
> which is supposed not to block queries due to report query metrics.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [kylin] hit-lacus merged pull request #1334: KYLIN-4653 Make the capacity for the LinkedBlockingQueue

2020-07-30 Thread GitBox


hit-lacus merged pull request #1334:
URL: https://github.com/apache/kylin/pull/1334


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [kylin] hit-lacus commented on pull request #1334: KYLIN-4653 Make the capacity for the LinkedBlockingQueue

2020-07-30 Thread GitBox


hit-lacus commented on pull request #1334:
URL: https://github.com/apache/kylin/pull/1334#issuecomment-666896102


   I will merge this patch.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Commented] (KYLIN-4660) Remove duplicated/misleading code or comment

2020-07-30 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/KYLIN-4660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168363#comment-17168363
 ] 

ASF subversion and git services commented on KYLIN-4660:


Commit dc1a80f2c1ad11fbd821014ae7975bb11c6db1d1 in kylin's branch 
refs/heads/kylin-on-parquet-v2 from XiaoxiangYu
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=dc1a80f ]

KYLIN-4660 fix package.sh


> Remove duplicated/misleading code or comment
> 
>
> Key: KYLIN-4660
> URL: https://issues.apache.org/jira/browse/KYLIN-4660
> Project: Kylin
>  Issue Type: Sub-task
>  Components: Storage - Parquet
>Reporter: Xiaoxiang Yu
>Assignee: wangrupeng
>Priority: Major
> Fix For: v4.0.0-beta
>
>   Original Estimate: 672h
>  Remaining Estimate: 672h
>
> Source code is located at 
> [https://github.com/apache/kylin/tree/kylin-on-parquet-v2] , we are still 
> test and verify it.
>  
>  
> ||Feature to be removed||commit id||Jira ID for the feature||
> |Advanced Snapshot|todo|todo|
> |JDBC Source|todo| |
> |Kafka Source (NRT)|todo| |
> |Kafka Source (Real-time OLAP)|todo| |
> | Build Engine - MR|todo| |
> | Build Engine - Spark|todo| |
> | Build Engine - Flink|todo| |
> | Storgae Engine - HBase|todo| |



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (KYLIN-4660) Remove duplicated/misleading code or comment

2020-07-30 Thread ASF GitHub Bot (Jira)


[ 
https://issues.apache.org/jira/browse/KYLIN-4660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168364#comment-17168364
 ] 

ASF GitHub Bot commented on KYLIN-4660:
---

hit-lacus merged pull request #1336:
URL: https://github.com/apache/kylin/pull/1336


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Remove duplicated/misleading code or comment
> 
>
> Key: KYLIN-4660
> URL: https://issues.apache.org/jira/browse/KYLIN-4660
> Project: Kylin
>  Issue Type: Sub-task
>  Components: Storage - Parquet
>Reporter: Xiaoxiang Yu
>Assignee: wangrupeng
>Priority: Major
> Fix For: v4.0.0-beta
>
>   Original Estimate: 672h
>  Remaining Estimate: 672h
>
> Source code is located at 
> [https://github.com/apache/kylin/tree/kylin-on-parquet-v2] , we are still 
> test and verify it.
>  
>  
> ||Feature to be removed||commit id||Jira ID for the feature||
> |Advanced Snapshot|todo|todo|
> |JDBC Source|todo| |
> |Kafka Source (NRT)|todo| |
> |Kafka Source (Real-time OLAP)|todo| |
> | Build Engine - MR|todo| |
> | Build Engine - Spark|todo| |
> | Build Engine - Flink|todo| |
> | Storgae Engine - HBase|todo| |



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (KYLIN-4660) Remove duplicated/misleading code or comment

2020-07-30 Thread ASF GitHub Bot (Jira)


[ 
https://issues.apache.org/jira/browse/KYLIN-4660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168362#comment-17168362
 ] 

ASF GitHub Bot commented on KYLIN-4660:
---

hit-lacus commented on pull request #1336:
URL: https://github.com/apache/kylin/pull/1336#issuecomment-666892996


   LGTM



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Remove duplicated/misleading code or comment
> 
>
> Key: KYLIN-4660
> URL: https://issues.apache.org/jira/browse/KYLIN-4660
> Project: Kylin
>  Issue Type: Sub-task
>  Components: Storage - Parquet
>Reporter: Xiaoxiang Yu
>Assignee: wangrupeng
>Priority: Major
> Fix For: v4.0.0-beta
>
>   Original Estimate: 672h
>  Remaining Estimate: 672h
>
> Source code is located at 
> [https://github.com/apache/kylin/tree/kylin-on-parquet-v2] , we are still 
> test and verify it.
>  
>  
> ||Feature to be removed||commit id||Jira ID for the feature||
> |Advanced Snapshot|todo|todo|
> |JDBC Source|todo| |
> |Kafka Source (NRT)|todo| |
> |Kafka Source (Real-time OLAP)|todo| |
> | Build Engine - MR|todo| |
> | Build Engine - Spark|todo| |
> | Build Engine - Flink|todo| |
> | Storgae Engine - HBase|todo| |



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [kylin] hit-lacus merged pull request #1336: KYLIN-4660 fix package.sh

2020-07-30 Thread GitBox


hit-lacus merged pull request #1336:
URL: https://github.com/apache/kylin/pull/1336


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [kylin] hit-lacus commented on pull request #1336: KYLIN-4660 fix package.sh

2020-07-30 Thread GitBox


hit-lacus commented on pull request #1336:
URL: https://github.com/apache/kylin/pull/1336#issuecomment-666892996


   LGTM



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [kylin] zhangayqian opened a new pull request #1337: Fix travis ci job log exceeded maximum log length

2020-07-30 Thread GitBox


zhangayqian opened a new pull request #1337:
URL: https://github.com/apache/kylin/pull/1337


   ## Proposed changes
   
   Describe the big picture of your changes here to communicate to the 
maintainers why we should accept this pull request. If it fixes a bug or 
resolves a feature request, be sure to link to that issue.
   
   ## Types of changes
   
   What types of changes does your code introduce to Kylin?
   _Put an `x` in the boxes that apply_
   
   - [ ] Bugfix (non-breaking change which fixes an issue)
   - [ ] New feature (non-breaking change which adds functionality)
   - [ ] Breaking change (fix or feature that would cause existing 
functionality to not work as expected)
   - [ ] Documentation Update (if none of the other choices apply)
   
   ## Checklist
   
   _Put an `x` in the boxes that apply. You can also fill these out after 
creating the PR. If you're unsure about any of them, don't hesitate to ask. 
We're here to help! This is simply a reminder of what we are going to look for 
before merging your code._
   
   - [ ] I have create an issue on [Kylin's 
jira](https://issues.apache.org/jira/browse/KYLIN), and have described the 
bug/feature there in detail
   - [ ] Commit messages in my PR start with the related jira ID, like 
"KYLIN- Make Kylin project open-source"
   - [ ] Compiling and unit tests pass locally with my changes
   - [ ] I have added tests that prove my fix is effective or that my feature 
works
   - [ ] If this change need a document change, I will prepare another pr 
against the `document` branch
   - [ ] Any dependent changes have been merged
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
user@kylin or dev@kylin by explaining why you chose the solution you did and 
what alternatives you considered, etc...
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Commented] (KYLIN-4653) Make the capacity for the LinkedBlockingQueue of BlockingReservoir configurable

2020-07-30 Thread ASF GitHub Bot (Jira)


[ 
https://issues.apache.org/jira/browse/KYLIN-4653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168349#comment-17168349
 ] 

ASF GitHub Bot commented on KYLIN-4653:
---

coveralls commented on pull request #1334:
URL: https://github.com/apache/kylin/pull/1334#issuecomment-666884589


   ## Pull Request Test Coverage Report for [Build 
6209](https://coveralls.io/builds/32445968)
   
   * **3** of **3**   **(100.0%)**  changed or added relevant lines in **1** 
file are covered.
   * **2** unchanged lines in **1** file lost coverage.
   * Overall coverage decreased (**-0.001%**) to **28.049%**
   
   ---
   
   
   |  Files with Coverage Reduction | New Missed Lines | % |
   | :-|--|--: |
   | 
[core-cube/src/main/java/org/apache/kylin/cube/cuboid/TreeCuboidScheduler.java](https://coveralls.io/builds/32445968/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Fcuboid%2FTreeCuboidScheduler.java#L124)
 | 2 | 68.46% |
   
   
   |  Totals | [![Coverage 
Status](https://coveralls.io/builds/32445968/badge)](https://coveralls.io/builds/32445968)
 |
   | :-- | --: |
   | Change from base [Build 6203](https://coveralls.io/builds/32402099): |  
-0.001% |
   | Covered Lines: | 26245 |
   | Relevant Lines: | 93568 |
   
   ---
   #   - [Coveralls](https://coveralls.io)
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Make the capacity for the LinkedBlockingQueue of BlockingReservoir 
> configurable
> ---
>
> Key: KYLIN-4653
> URL: https://issues.apache.org/jira/browse/KYLIN-4653
> Project: Kylin
>  Issue Type: Improvement
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Fix For: v3.1.1
>
>
> Currently the blocking queue capacity is 50K. It is too small, which easily 
> goes into being blocked and slows down queries.
> It's better to make this value configurable and enlarge the default value.
> For real usage, I strongly recommend to use unbounded LinkedBlockingQueue, 
> which is supposed not to block queries due to report query metrics.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [kylin] coveralls commented on pull request #1334: KYLIN-4653 Make the capacity for the LinkedBlockingQueue

2020-07-30 Thread GitBox


coveralls commented on pull request #1334:
URL: https://github.com/apache/kylin/pull/1334#issuecomment-666884589


   ## Pull Request Test Coverage Report for [Build 
6209](https://coveralls.io/builds/32445968)
   
   * **3** of **3**   **(100.0%)**  changed or added relevant lines in **1** 
file are covered.
   * **2** unchanged lines in **1** file lost coverage.
   * Overall coverage decreased (**-0.001%**) to **28.049%**
   
   ---
   
   
   |  Files with Coverage Reduction | New Missed Lines | % |
   | :-|--|--: |
   | 
[core-cube/src/main/java/org/apache/kylin/cube/cuboid/TreeCuboidScheduler.java](https://coveralls.io/builds/32445968/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Fcuboid%2FTreeCuboidScheduler.java#L124)
 | 2 | 68.46% |
   
   
   |  Totals | [![Coverage 
Status](https://coveralls.io/builds/32445968/badge)](https://coveralls.io/builds/32445968)
 |
   | :-- | --: |
   | Change from base [Build 6203](https://coveralls.io/builds/32402099): |  
-0.001% |
   | Covered Lines: | 26245 |
   | Relevant Lines: | 93568 |
   
   ---
   #   - [Coveralls](https://coveralls.io)
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Updated] (KYLIN-4670) Improve query performance by reusing LookupStringTable and using multi-threads

2020-07-30 Thread Zhong Yanghong (Jira)


 [ 
https://issues.apache.org/jira/browse/KYLIN-4670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhong Yanghong updated KYLIN-4670:
--
Attachment: (was: Single-Thread-First-Time.png)

> Improve query performance by reusing LookupStringTable and using multi-threads
> --
>
> Key: KYLIN-4670
> URL: https://issues.apache.org/jira/browse/KYLIN-4670
> Project: Kylin
>  Issue Type: Improvement
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Attachments: Five-Threads-First-Time.png, 
> Five-Threads-Second-Time.png, Single-Thread-First-Time.png, 
> Single-Thread-Second-Time.png
>
>
> For a cube with 37 segments and related snapshots are all different from each 
> other. Do a query with lookup table derived column. Here's the performance 
> comparison between single thread & five threads.
>  * For the first time query without snapshot table cache:
>  ** Single Thread
> !Single-Thread-First-Time.png|width=600,height=200!
>  ** Five Threads
> !Five-Threads-First-Time.png|width=600,height=200!
>  * For the second time query with snapshot table cache:
>  ** Single Thread
> !Single-Thread-Second-Time.png|width=600,height=200!
>  ** Five Threads
> !Five-Threads-Second-Time.png|width=600,height=200!
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (KYLIN-4670) Improve query performance by reusing LookupStringTable and using multi-threads

2020-07-30 Thread Zhong Yanghong (Jira)


 [ 
https://issues.apache.org/jira/browse/KYLIN-4670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhong Yanghong updated KYLIN-4670:
--
Attachment: Single-Thread-First-Time.png
Five-Threads-First-Time.png
Five-Threads-Second-Time.png
Single-Thread-Second-Time.png

> Improve query performance by reusing LookupStringTable and using multi-threads
> --
>
> Key: KYLIN-4670
> URL: https://issues.apache.org/jira/browse/KYLIN-4670
> Project: Kylin
>  Issue Type: Improvement
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Attachments: Five-Threads-First-Time.png, 
> Five-Threads-Second-Time.png, Single-Thread-First-Time.png, 
> Single-Thread-Second-Time.png
>
>
> For a cube with 37 segments and related snapshots are all different from each 
> other. Do a query with lookup table derived column. Here's the performance 
> comparison between single thread & five threads.
>  * For the first time query without snapshot table cache:
>  ** Single Thread
> !Single-Thread-First-Time.png|width=600,height=200!
>  ** Five Threads
> !Five-Threads-First-Time.png|width=600,height=200!
>  * For the second time query with snapshot table cache:
>  ** Single Thread
> !Single-Thread-Second-Time.png|width=600,height=200!
>  ** Five Threads
> !Five-Threads-Second-Time.png|width=600,height=200!
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (KYLIN-4670) Improve query performance by reusing LookupStringTable and using multi-threads

2020-07-30 Thread Zhong Yanghong (Jira)


 [ 
https://issues.apache.org/jira/browse/KYLIN-4670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhong Yanghong updated KYLIN-4670:
--
Attachment: (was: Single-Thread-Second-Time.png)

> Improve query performance by reusing LookupStringTable and using multi-threads
> --
>
> Key: KYLIN-4670
> URL: https://issues.apache.org/jira/browse/KYLIN-4670
> Project: Kylin
>  Issue Type: Improvement
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Attachments: Five-Threads-First-Time.png, 
> Five-Threads-Second-Time.png, Single-Thread-First-Time.png, 
> Single-Thread-Second-Time.png
>
>
> For a cube with 37 segments and related snapshots are all different from each 
> other. Do a query with lookup table derived column. Here's the performance 
> comparison between single thread & five threads.
>  * For the first time query without snapshot table cache:
>  ** Single Thread
> !Single-Thread-First-Time.png|width=600,height=200!
>  ** Five Threads
> !Five-Threads-First-Time.png|width=600,height=200!
>  * For the second time query with snapshot table cache:
>  ** Single Thread
> !Single-Thread-Second-Time.png|width=600,height=200!
>  ** Five Threads
> !Five-Threads-Second-Time.png|width=600,height=200!
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (KYLIN-4670) Improve query performance by reusing LookupStringTable and using multi-threads

2020-07-30 Thread Zhong Yanghong (Jira)


 [ 
https://issues.apache.org/jira/browse/KYLIN-4670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhong Yanghong updated KYLIN-4670:
--
Attachment: (was: Five-Threads-Second-Time.png)

> Improve query performance by reusing LookupStringTable and using multi-threads
> --
>
> Key: KYLIN-4670
> URL: https://issues.apache.org/jira/browse/KYLIN-4670
> Project: Kylin
>  Issue Type: Improvement
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Attachments: Single-Thread-First-Time.png, 
> Single-Thread-Second-Time.png
>
>
> For a cube with 37 segments and related snapshots are all different from each 
> other. Do a query with lookup table derived column. Here's the performance 
> comparison between single thread & five threads.
>  * For the first time query without snapshot table cache:
>  ** Single Thread
> !Single-Thread-First-Time.png|width=600,height=200!
>  ** Five Threads
> !Five-Threads-First-Time.png|width=600,height=200!
>  * For the second time query with snapshot table cache:
>  ** Single Thread
> !Single-Thread-Second-Time.png|width=600,height=200!
>  ** Five Threads
> !Five-Threads-Second-Time.png|width=600,height=200!
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (KYLIN-4670) Improve query performance by reusing LookupStringTable and using multi-threads

2020-07-30 Thread Zhong Yanghong (Jira)


 [ 
https://issues.apache.org/jira/browse/KYLIN-4670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhong Yanghong updated KYLIN-4670:
--
Attachment: (was: Five-Threads-First-Time.png)

> Improve query performance by reusing LookupStringTable and using multi-threads
> --
>
> Key: KYLIN-4670
> URL: https://issues.apache.org/jira/browse/KYLIN-4670
> Project: Kylin
>  Issue Type: Improvement
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Attachments: Single-Thread-First-Time.png, 
> Single-Thread-Second-Time.png
>
>
> For a cube with 37 segments and related snapshots are all different from each 
> other. Do a query with lookup table derived column. Here's the performance 
> comparison between single thread & five threads.
>  * For the first time query without snapshot table cache:
>  ** Single Thread
> !Single-Thread-First-Time.png|width=600,height=200!
>  ** Five Threads
> !Five-Threads-First-Time.png|width=600,height=200!
>  * For the second time query with snapshot table cache:
>  ** Single Thread
> !Single-Thread-Second-Time.png|width=600,height=200!
>  ** Five Threads
> !Five-Threads-Second-Time.png|width=600,height=200!
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (KYLIN-4670) Improve query performance by reusing LookupStringTable and using multi-threads

2020-07-30 Thread Zhong Yanghong (Jira)


 [ 
https://issues.apache.org/jira/browse/KYLIN-4670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhong Yanghong updated KYLIN-4670:
--
Description: 
For a cube with 37 segments and related snapshots are all different from each 
other. Do a query with lookup table derived column. Here's the performance 
comparison between single thread & five threads.
 * For the first time query without snapshot table cache:
 ** Single Thread
!Single-Thread-First-Time.png|width=600,height=200!
 ** Five Threads
!Five-Threads-First-Time.png|width=600,height=200!

 * For the second time query with snapshot table cache:
 ** Single Thread
!Single-Thread-Second-Time.png|width=600,height=200!
 ** Five Threads
!Five-Threads-Second-Time.png|width=600,height=200!
 

  was:For a cube with 37 segments and related snapshots are all different from 
each other.


> Improve query performance by reusing LookupStringTable and using multi-threads
> --
>
> Key: KYLIN-4670
> URL: https://issues.apache.org/jira/browse/KYLIN-4670
> Project: Kylin
>  Issue Type: Improvement
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Attachments: Five-Threads-First-Time.png, 
> Five-Threads-Second-Time.png, Single-Thread-First-Time.png, 
> Single-Thread-Second-Time.png
>
>
> For a cube with 37 segments and related snapshots are all different from each 
> other. Do a query with lookup table derived column. Here's the performance 
> comparison between single thread & five threads.
>  * For the first time query without snapshot table cache:
>  ** Single Thread
> !Single-Thread-First-Time.png|width=600,height=200!
>  ** Five Threads
> !Five-Threads-First-Time.png|width=600,height=200!
>  * For the second time query with snapshot table cache:
>  ** Single Thread
> !Single-Thread-Second-Time.png|width=600,height=200!
>  ** Five Threads
> !Five-Threads-Second-Time.png|width=600,height=200!
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (KYLIN-4670) Improve query performance by reusing LookupStringTable and using multi-threads

2020-07-30 Thread Zhong Yanghong (Jira)


 [ 
https://issues.apache.org/jira/browse/KYLIN-4670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhong Yanghong updated KYLIN-4670:
--
Attachment: Single-Thread-First-Time.png
Single-Thread-Second-Time.png
Five-Threads-Second-Time.png
Five-Threads-First-Time.png

> Improve query performance by reusing LookupStringTable and using multi-threads
> --
>
> Key: KYLIN-4670
> URL: https://issues.apache.org/jira/browse/KYLIN-4670
> Project: Kylin
>  Issue Type: Improvement
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Attachments: Five-Threads-First-Time.png, 
> Five-Threads-Second-Time.png, Single-Thread-First-Time.png, 
> Single-Thread-Second-Time.png
>
>
> For a cube with 37 segments and related snapshots are all different from each 
> other.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (KYLIN-4653) Make the capacity for the LinkedBlockingQueue of BlockingReservoir configurable

2020-07-30 Thread ASF GitHub Bot (Jira)


[ 
https://issues.apache.org/jira/browse/KYLIN-4653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17167852#comment-17167852
 ] 

ASF GitHub Bot commented on KYLIN-4653:
---

hit-lacus commented on a change in pull request #1334:
URL: https://github.com/apache/kylin/pull/1334#discussion_r462945077



##
File path: 
core-metrics/src/main/java/org/apache/kylin/metrics/lib/impl/BlockingReservoir.java
##
@@ -60,14 +60,15 @@ public BlockingReservoir(int minReportSize, int 
maxReportSize) {
 }
 
 public BlockingReservoir(int minReportSize, int maxReportSize, int 
maxReportTime) {
-this(minReportSize, maxReportSize, maxReportSize, MAX_QUEUE_SIZE);
+this(minReportSize, maxReportSize, maxReportTime, MAX_QUEUE_SIZE);
 }
 
 public BlockingReservoir(int minReportSize, int maxReportSize, int 
maxReportTime, int maxQueueSize) {
 Preconditions.checkArgument(minReportSize > 0, "minReportSize should 
be larger than 0");
 Preconditions.checkArgument(maxReportSize >= minReportSize,
 "maxReportSize should not be less than minBatchSize");
 Preconditions.checkArgument(maxReportTime > 0, "maxReportTime should 
be larger than 0");
+Preconditions.checkArgument(maxQueueSize > 0, "maxQueueSize should be 
larger than 0");

Review comment:
   Good advice!





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Make the capacity for the LinkedBlockingQueue of BlockingReservoir 
> configurable
> ---
>
> Key: KYLIN-4653
> URL: https://issues.apache.org/jira/browse/KYLIN-4653
> Project: Kylin
>  Issue Type: Improvement
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Fix For: v3.1.1
>
>
> Currently the blocking queue capacity is 50K. It is too small, which easily 
> goes into being blocked and slows down queries.
> It's better to make this value configurable and enlarge the default value.
> For real usage, I strongly recommend to use unbounded LinkedBlockingQueue, 
> which is supposed not to block queries due to report query metrics.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [kylin] hit-lacus commented on a change in pull request #1334: KYLIN-4653 Make the capacity for the LinkedBlockingQueue

2020-07-30 Thread GitBox


hit-lacus commented on a change in pull request #1334:
URL: https://github.com/apache/kylin/pull/1334#discussion_r462945077



##
File path: 
core-metrics/src/main/java/org/apache/kylin/metrics/lib/impl/BlockingReservoir.java
##
@@ -60,14 +60,15 @@ public BlockingReservoir(int minReportSize, int 
maxReportSize) {
 }
 
 public BlockingReservoir(int minReportSize, int maxReportSize, int 
maxReportTime) {
-this(minReportSize, maxReportSize, maxReportSize, MAX_QUEUE_SIZE);
+this(minReportSize, maxReportSize, maxReportTime, MAX_QUEUE_SIZE);
 }
 
 public BlockingReservoir(int minReportSize, int maxReportSize, int 
maxReportTime, int maxQueueSize) {
 Preconditions.checkArgument(minReportSize > 0, "minReportSize should 
be larger than 0");
 Preconditions.checkArgument(maxReportSize >= minReportSize,
 "maxReportSize should not be less than minBatchSize");
 Preconditions.checkArgument(maxReportTime > 0, "maxReportTime should 
be larger than 0");
+Preconditions.checkArgument(maxQueueSize > 0, "maxQueueSize should be 
larger than 0");

Review comment:
   Good advice!





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Updated] (KYLIN-4670) Improve query performance by reusing LookupStringTable and using multi-threads

2020-07-30 Thread Zhong Yanghong (Jira)


 [ 
https://issues.apache.org/jira/browse/KYLIN-4670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhong Yanghong updated KYLIN-4670:
--
Description: For a cube with 37 segments and related snapshots are all 
different from each other.

> Improve query performance by reusing LookupStringTable and using multi-threads
> --
>
> Key: KYLIN-4670
> URL: https://issues.apache.org/jira/browse/KYLIN-4670
> Project: Kylin
>  Issue Type: Improvement
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
>
> For a cube with 37 segments and related snapshots are all different from each 
> other.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (KYLIN-4670) Improve query performance by reusing LookupStringTable and using multi-threads

2020-07-30 Thread Zhong Yanghong (Jira)


 [ 
https://issues.apache.org/jira/browse/KYLIN-4670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhong Yanghong updated KYLIN-4670:
--
Summary: Improve query performance by reusing LookupStringTable and using 
multi-threads  (was: Improve query performance by reusing LookupStringTable)

> Improve query performance by reusing LookupStringTable and using multi-threads
> --
>
> Key: KYLIN-4670
> URL: https://issues.apache.org/jira/browse/KYLIN-4670
> Project: Kylin
>  Issue Type: Improvement
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (KYLIN-4660) Remove duplicated/misleading code or comment

2020-07-30 Thread ASF GitHub Bot (Jira)


[ 
https://issues.apache.org/jira/browse/KYLIN-4660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17167710#comment-17167710
 ] 

ASF GitHub Bot commented on KYLIN-4660:
---

hit-lacus opened a new pull request #1336:
URL: https://github.com/apache/kylin/pull/1336


   ## Proposed changes
   
   Describe the big picture of your changes here to communicate to the 
maintainers why we should accept this pull request. If it fixes a bug or 
resolves a feature request, be sure to link to that issue.
   
   ## Types of changes
   
   What types of changes does your code introduce to Kylin?
   _Put an `x` in the boxes that apply_
   
   - [ ] Bugfix (non-breaking change which fixes an issue)
   - [ ] New feature (non-breaking change which adds functionality)
   - [ ] Breaking change (fix or feature that would cause existing 
functionality to not work as expected)
   - [ ] Documentation Update (if none of the other choices apply)
   
   ## Checklist
   
   _Put an `x` in the boxes that apply. You can also fill these out after 
creating the PR. If you're unsure about any of them, don't hesitate to ask. 
We're here to help! This is simply a reminder of what we are going to look for 
before merging your code._
   
   - [ ] I have create an issue on [Kylin's 
jira](https://issues.apache.org/jira/browse/KYLIN), and have described the 
bug/feature there in detail
   - [ ] Commit messages in my PR start with the related jira ID, like 
"KYLIN- Make Kylin project open-source"
   - [ ] Compiling and unit tests pass locally with my changes
   - [ ] I have added tests that prove my fix is effective or that my feature 
works
   - [ ] If this change need a document change, I will prepare another pr 
against the `document` branch
   - [ ] Any dependent changes have been merged
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
user@kylin or dev@kylin by explaining why you chose the solution you did and 
what alternatives you considered, etc...
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Remove duplicated/misleading code or comment
> 
>
> Key: KYLIN-4660
> URL: https://issues.apache.org/jira/browse/KYLIN-4660
> Project: Kylin
>  Issue Type: Sub-task
>  Components: Storage - Parquet
>Reporter: Xiaoxiang Yu
>Assignee: wangrupeng
>Priority: Major
> Fix For: v4.0.0-beta
>
>   Original Estimate: 672h
>  Remaining Estimate: 672h
>
> Source code is located at 
> [https://github.com/apache/kylin/tree/kylin-on-parquet-v2] , we are still 
> test and verify it.
>  
>  
> ||Feature to be removed||commit id||Jira ID for the feature||
> |Advanced Snapshot|todo|todo|
> |JDBC Source|todo| |
> |Kafka Source (NRT)|todo| |
> |Kafka Source (Real-time OLAP)|todo| |
> | Build Engine - MR|todo| |
> | Build Engine - Spark|todo| |
> | Build Engine - Flink|todo| |
> | Storgae Engine - HBase|todo| |



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [kylin] hit-lacus opened a new pull request #1336: KYLIN-4660 fix package.sh

2020-07-30 Thread GitBox


hit-lacus opened a new pull request #1336:
URL: https://github.com/apache/kylin/pull/1336


   ## Proposed changes
   
   Describe the big picture of your changes here to communicate to the 
maintainers why we should accept this pull request. If it fixes a bug or 
resolves a feature request, be sure to link to that issue.
   
   ## Types of changes
   
   What types of changes does your code introduce to Kylin?
   _Put an `x` in the boxes that apply_
   
   - [ ] Bugfix (non-breaking change which fixes an issue)
   - [ ] New feature (non-breaking change which adds functionality)
   - [ ] Breaking change (fix or feature that would cause existing 
functionality to not work as expected)
   - [ ] Documentation Update (if none of the other choices apply)
   
   ## Checklist
   
   _Put an `x` in the boxes that apply. You can also fill these out after 
creating the PR. If you're unsure about any of them, don't hesitate to ask. 
We're here to help! This is simply a reminder of what we are going to look for 
before merging your code._
   
   - [ ] I have create an issue on [Kylin's 
jira](https://issues.apache.org/jira/browse/KYLIN), and have described the 
bug/feature there in detail
   - [ ] Commit messages in my PR start with the related jira ID, like 
"KYLIN- Make Kylin project open-source"
   - [ ] Compiling and unit tests pass locally with my changes
   - [ ] I have added tests that prove my fix is effective or that my feature 
works
   - [ ] If this change need a document change, I will prepare another pr 
against the `document` branch
   - [ ] Any dependent changes have been merged
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
user@kylin or dev@kylin by explaining why you chose the solution you did and 
what alternatives you considered, etc...
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Commented] (KYLIN-4671) modify FetchRunner catch getOutput NPE log level to debug for ignore too many logs

2020-07-30 Thread ASF GitHub Bot (Jira)


[ 
https://issues.apache.org/jira/browse/KYLIN-4671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17167705#comment-17167705
 ] 

ASF GitHub Bot commented on KYLIN-4671:
---

bigxiaochu opened a new pull request #1335:
URL: https://github.com/apache/kylin/pull/1335


   ## Proposed changes
   
   Describe the big picture of your changes here to communicate to the 
maintainers why we should accept this pull request. If it fixes a bug or 
resolves a feature request, be sure to link to that issue.
   
   ## Types of changes
   
   What types of changes does your code introduce to Kylin?
   _Put an `x` in the boxes that apply_
   
   - [ ] Bugfix (non-breaking change which fixes an issue)
   - [ ] New feature (non-breaking change which adds functionality)
   - [ ] Breaking change (fix or feature that would cause existing 
functionality to not work as expected)
   - [ ] Documentation Update (if none of the other choices apply)
   
   ## Checklist
   
   _Put an `x` in the boxes that apply. You can also fill these out after 
creating the PR. If you're unsure about any of them, don't hesitate to ask. 
We're here to help! This is simply a reminder of what we are going to look for 
before merging your code._
   
   - [ ] I have create an issue on [Kylin's 
jira](https://issues.apache.org/jira/browse/KYLIN), and have described the 
bug/feature there in detail
   - [ ] Commit messages in my PR start with the related jira ID, like 
"KYLIN- Make Kylin project open-source"
   - [ ] Compiling and unit tests pass locally with my changes
   - [ ] I have added tests that prove my fix is effective or that my feature 
works
   - [ ] If this change need a document change, I will prepare another pr 
against the `document` branch
   - [ ] Any dependent changes have been merged
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
user@kylin or dev@kylin by explaining why you chose the solution you did and 
what alternatives you considered, etc...
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> modify FetchRunner catch getOutput NPE log level to debug for ignore too many 
> logs
> --
>
> Key: KYLIN-4671
> URL: https://issues.apache.org/jira/browse/KYLIN-4671
> Project: Kylin
>  Issue Type: Improvement
>Reporter: chuxiao
>Priority: Minor
> Attachments: D-Chat_20200729164446.png
>
>
> too many log. Ignore , until remove  job info when clean metadata.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [kylin] bigxiaochu opened a new pull request #1335: KYLIN-4671 update log level for ignore too many logs

2020-07-30 Thread GitBox


bigxiaochu opened a new pull request #1335:
URL: https://github.com/apache/kylin/pull/1335


   ## Proposed changes
   
   Describe the big picture of your changes here to communicate to the 
maintainers why we should accept this pull request. If it fixes a bug or 
resolves a feature request, be sure to link to that issue.
   
   ## Types of changes
   
   What types of changes does your code introduce to Kylin?
   _Put an `x` in the boxes that apply_
   
   - [ ] Bugfix (non-breaking change which fixes an issue)
   - [ ] New feature (non-breaking change which adds functionality)
   - [ ] Breaking change (fix or feature that would cause existing 
functionality to not work as expected)
   - [ ] Documentation Update (if none of the other choices apply)
   
   ## Checklist
   
   _Put an `x` in the boxes that apply. You can also fill these out after 
creating the PR. If you're unsure about any of them, don't hesitate to ask. 
We're here to help! This is simply a reminder of what we are going to look for 
before merging your code._
   
   - [ ] I have create an issue on [Kylin's 
jira](https://issues.apache.org/jira/browse/KYLIN), and have described the 
bug/feature there in detail
   - [ ] Commit messages in my PR start with the related jira ID, like 
"KYLIN- Make Kylin project open-source"
   - [ ] Compiling and unit tests pass locally with my changes
   - [ ] I have added tests that prove my fix is effective or that my feature 
works
   - [ ] If this change need a document change, I will prepare another pr 
against the `document` branch
   - [ ] Any dependent changes have been merged
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
user@kylin or dev@kylin by explaining why you chose the solution you did and 
what alternatives you considered, etc...
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [kylin] coveralls commented on pull request #1335: KYLIN-4671 update log level for ignore too many logs

2020-07-30 Thread GitBox


coveralls commented on pull request #1335:
URL: https://github.com/apache/kylin/pull/1335#issuecomment-666073715


   ## Pull Request Test Coverage Report for [Build 
6204](https://coveralls.io/builds/32415585)
   
   * **0** of **2**   **(0.0%)**  changed or added relevant lines in **1** file 
are covered.
   * **7** unchanged lines in **2** files lost coverage.
   * Overall coverage decreased (**-0.001%**) to **28.049%**
   
   ---
   
   |  Changes Missing Coverage | Covered Lines | Changed/Added Lines | % |
   | :-|--||---: |
   | 
[core-job/src/main/java/org/apache/kylin/job/impl/threadpool/DefaultFetcherRunner.java](https://coveralls.io/builds/32415585/source?filename=core-job%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fjob%2Fimpl%2Fthreadpool%2FDefaultFetcherRunner.java#L77)
 | 0 | 2 | 0.0%
   
   
   |  Files with Coverage Reduction | New Missed Lines | % |
   | :-|--|--: |
   | 
[core-cube/src/main/java/org/apache/kylin/cube/cuboid/TreeCuboidScheduler.java](https://coveralls.io/builds/32415585/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Fcuboid%2FTreeCuboidScheduler.java#L124)
 | 2 | 68.46% |
   | 
[core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://coveralls.io/builds/32415585/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Finmemcubing%2FMemDiskStore.java#L439)
 | 5 | 77.81% |
   
   
   |  Totals | [![Coverage 
Status](https://coveralls.io/builds/32415585/badge)](https://coveralls.io/builds/32415585)
 |
   | :-- | --: |
   | Change from base [Build 6203](https://coveralls.io/builds/32402099): |  
-0.001% |
   | Covered Lines: | 26245 |
   | Relevant Lines: | 93569 |
   
   ---
   #   - [Coveralls](https://coveralls.io)
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Commented] (KYLIN-4671) modify FetchRunner catch getOutput NPE log level to debug for ignore too many logs

2020-07-30 Thread ASF GitHub Bot (Jira)


[ 
https://issues.apache.org/jira/browse/KYLIN-4671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17167699#comment-17167699
 ] 

ASF GitHub Bot commented on KYLIN-4671:
---

coveralls commented on pull request #1335:
URL: https://github.com/apache/kylin/pull/1335#issuecomment-666073715


   ## Pull Request Test Coverage Report for [Build 
6204](https://coveralls.io/builds/32415585)
   
   * **0** of **2**   **(0.0%)**  changed or added relevant lines in **1** file 
are covered.
   * **7** unchanged lines in **2** files lost coverage.
   * Overall coverage decreased (**-0.001%**) to **28.049%**
   
   ---
   
   |  Changes Missing Coverage | Covered Lines | Changed/Added Lines | % |
   | :-|--||---: |
   | 
[core-job/src/main/java/org/apache/kylin/job/impl/threadpool/DefaultFetcherRunner.java](https://coveralls.io/builds/32415585/source?filename=core-job%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fjob%2Fimpl%2Fthreadpool%2FDefaultFetcherRunner.java#L77)
 | 0 | 2 | 0.0%
   
   
   |  Files with Coverage Reduction | New Missed Lines | % |
   | :-|--|--: |
   | 
[core-cube/src/main/java/org/apache/kylin/cube/cuboid/TreeCuboidScheduler.java](https://coveralls.io/builds/32415585/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Fcuboid%2FTreeCuboidScheduler.java#L124)
 | 2 | 68.46% |
   | 
[core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://coveralls.io/builds/32415585/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Finmemcubing%2FMemDiskStore.java#L439)
 | 5 | 77.81% |
   
   
   |  Totals | [![Coverage 
Status](https://coveralls.io/builds/32415585/badge)](https://coveralls.io/builds/32415585)
 |
   | :-- | --: |
   | Change from base [Build 6203](https://coveralls.io/builds/32402099): |  
-0.001% |
   | Covered Lines: | 26245 |
   | Relevant Lines: | 93569 |
   
   ---
   #   - [Coveralls](https://coveralls.io)
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> modify FetchRunner catch getOutput NPE log level to debug for ignore too many 
> logs
> --
>
> Key: KYLIN-4671
> URL: https://issues.apache.org/jira/browse/KYLIN-4671
> Project: Kylin
>  Issue Type: Improvement
>Reporter: chuxiao
>Priority: Minor
> Attachments: D-Chat_20200729164446.png
>
>
> too many log. Ignore , until remove  job info when clean metadata.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)