date:20180320

[jira] [Updated] (KYLIN-3306) Fix the rarely happened unit test exception of generic algorithm

2018-03-20 Thread Zhong Yanghong (JIRA)


 [ 
https://issues.apache.org/jira/browse/KYLIN-3306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhong Yanghong updated KYLIN-3306:
--
Attachment: APACHE-KYLIN-3306.patch

> Fix the rarely happened unit test exception of generic algorithm
> 
>
> Key: KYLIN-3306
> URL: https://issues.apache.org/jira/browse/KYLIN-3306
> Project: Kylin
>  Issue Type: Bug
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Fix For: v2.3.1
>
> Attachments: APACHE-KYLIN-3306.patch, APACHE-KYLIN-3306.patch
>
>
> There's a bug in the following code:
> {code}
> private Chromosome rouletteWheel(final List chromosomes, final 
> double totalFitness) {
> float rnd = (float) 
> (GeneticAlgorithm.getRandomGenerator().nextDouble() * totalFitness);
> float runningScore = 0;
> for (Chromosome o : chromosomes) {
> if (rnd >= runningScore && rnd <= runningScore + o.getFitness()) {
> return o;
> }
> runningScore += o.getFitness();
> }
> return null;
> }
> {code}
> We should use double instead of float.
> For example, (float) (0.88948809 * 40.5109996) = 40.511 > 
> 40.5109996



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KYLIN-3306) Fix the rarely happened unit test exception of generic algorithm

2018-03-20 Thread Shaofeng SHI (JIRA)


[ 
https://issues.apache.org/jira/browse/KYLIN-3306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407495#comment-16407495
 ] 

Shaofeng SHI commented on KYLIN-3306:
-

kylin-it/src/test/java/org/apache/kylin/cube/cuboid/algorithm/ITGeneticAlgorithmTest.java:24:8:
 无用导入 - org.junit.Ignore 。 [UnusedImports]

> Fix the rarely happened unit test exception of generic algorithm
> 
>
> Key: KYLIN-3306
> URL: https://issues.apache.org/jira/browse/KYLIN-3306
> Project: Kylin
>  Issue Type: Bug
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Fix For: v2.3.1
>
> Attachments: APACHE-KYLIN-3306.patch, APACHE-KYLIN-3306.patch
>
>
> There's a bug in the following code:
> {code}
> private Chromosome rouletteWheel(final List chromosomes, final 
> double totalFitness) {
> float rnd = (float) 
> (GeneticAlgorithm.getRandomGenerator().nextDouble() * totalFitness);
> float runningScore = 0;
> for (Chromosome o : chromosomes) {
> if (rnd >= runningScore && rnd <= runningScore + o.getFitness()) {
> return o;
> }
> runningScore += o.getFitness();
> }
> return null;
> }
> {code}
> We should use double instead of float.
> For example, (float) (0.88948809 * 40.5109996) = 40.511 > 
> 40.5109996



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KYLIN-3306) Fix the rarely happened unit test exception of generic algorithm

2018-03-20 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/KYLIN-3306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407492#comment-16407492
 ] 

ASF subversion and git services commented on KYLIN-3306:


Commit aa4cf286f440273bad04ae30a3842d9143ef918d in kylin's branch 
refs/heads/2.3.x from Zhong
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=aa4cf28 ]

APACHE-KYLIN-3306: Fix the rarely happened unit test exception of generic 
algorithm

Signed-off-by: shaofengshi 


> Fix the rarely happened unit test exception of generic algorithm
> 
>
> Key: KYLIN-3306
> URL: https://issues.apache.org/jira/browse/KYLIN-3306
> Project: Kylin
>  Issue Type: Bug
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Fix For: v2.3.1
>
> Attachments: APACHE-KYLIN-3306.patch
>
>
> There's a bug in the following code:
> {code}
> private Chromosome rouletteWheel(final List chromosomes, final 
> double totalFitness) {
> float rnd = (float) 
> (GeneticAlgorithm.getRandomGenerator().nextDouble() * totalFitness);
> float runningScore = 0;
> for (Chromosome o : chromosomes) {
> if (rnd >= runningScore && rnd <= runningScore + o.getFitness()) {
> return o;
> }
> runningScore += o.getFitness();
> }
> return null;
> }
> {code}
> We should use double instead of float.
> For example, (float) (0.88948809 * 40.5109996) = 40.511 > 
> 40.5109996



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KYLIN-3306) Fix the rarely happened unit test exception of generic algorithm

2018-03-20 Thread Shaofeng SHI (JIRA)


[ 
https://issues.apache.org/jira/browse/KYLIN-3306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407491#comment-16407491
 ] 

Shaofeng SHI commented on KYLIN-3306:
-

+1 LGTM

> Fix the rarely happened unit test exception of generic algorithm
> 
>
> Key: KYLIN-3306
> URL: https://issues.apache.org/jira/browse/KYLIN-3306
> Project: Kylin
>  Issue Type: Bug
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Fix For: v2.3.1
>
> Attachments: APACHE-KYLIN-3306.patch
>
>
> There's a bug in the following code:
> {code}
> private Chromosome rouletteWheel(final List chromosomes, final 
> double totalFitness) {
> float rnd = (float) 
> (GeneticAlgorithm.getRandomGenerator().nextDouble() * totalFitness);
> float runningScore = 0;
> for (Chromosome o : chromosomes) {
> if (rnd >= runningScore && rnd <= runningScore + o.getFitness()) {
> return o;
> }
> runningScore += o.getFitness();
> }
> return null;
> }
> {code}
> We should use double instead of float.
> For example, (float) (0.88948809 * 40.5109996) = 40.511 > 
> 40.5109996



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (KYLIN-3306) Fix the rarely happened unit test exception of generic algorithm

2018-03-20 Thread Zhong Yanghong (JIRA)


 [ 
https://issues.apache.org/jira/browse/KYLIN-3306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhong Yanghong updated KYLIN-3306:
--
Attachment: APACHE-KYLIN-3306.patch

> Fix the rarely happened unit test exception of generic algorithm
> 
>
> Key: KYLIN-3306
> URL: https://issues.apache.org/jira/browse/KYLIN-3306
> Project: Kylin
>  Issue Type: Bug
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Fix For: v2.3.1
>
> Attachments: APACHE-KYLIN-3306.patch
>
>
> There's a bug in the following code:
> {code}
> private Chromosome rouletteWheel(final List chromosomes, final 
> double totalFitness) {
> float rnd = (float) 
> (GeneticAlgorithm.getRandomGenerator().nextDouble() * totalFitness);
> float runningScore = 0;
> for (Chromosome o : chromosomes) {
> if (rnd >= runningScore && rnd <= runningScore + o.getFitness()) {
> return o;
> }
> runningScore += o.getFitness();
> }
> return null;
> }
> {code}
> We should use double instead of float.
> For example, (float) (0.88948809 * 40.5109996) = 40.511 > 
> 40.5109996



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (KYLIN-3306) Fix the rarely happened unit test exception of generic algorithm

2018-03-20 Thread Zhong Yanghong (JIRA)


 [ 
https://issues.apache.org/jira/browse/KYLIN-3306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhong Yanghong updated KYLIN-3306:
--
Fix Version/s: v2.3.1

> Fix the rarely happened unit test exception of generic algorithm
> 
>
> Key: KYLIN-3306
> URL: https://issues.apache.org/jira/browse/KYLIN-3306
> Project: Kylin
>  Issue Type: Bug
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Fix For: v2.3.1
>
>
> There's a bug in the following code:
> {code}
> private Chromosome rouletteWheel(final List chromosomes, final 
> double totalFitness) {
> float rnd = (float) 
> (GeneticAlgorithm.getRandomGenerator().nextDouble() * totalFitness);
> float runningScore = 0;
> for (Chromosome o : chromosomes) {
> if (rnd >= runningScore && rnd <= runningScore + o.getFitness()) {
> return o;
> }
> runningScore += o.getFitness();
> }
> return null;
> }
> {code}
> We should use double instead of float.
> For example, (float) (0.88948809 * 40.5109996) = 40.511 > 
> 40.5109996



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (KYLIN-3306) Fix the rarely happened unit test exception of generic algorithm

2018-03-20 Thread Zhong Yanghong (JIRA)

Zhong Yanghong created KYLIN-3306:
-

 Summary: Fix the rarely happened unit test exception of generic 
algorithm
 Key: KYLIN-3306
 URL: https://issues.apache.org/jira/browse/KYLIN-3306
 Project: Kylin
  Issue Type: Bug
Reporter: Zhong Yanghong
Assignee: Zhong Yanghong


There's a bug in the following code:
{code}
private Chromosome rouletteWheel(final List chromosomes, final 
double totalFitness) {
float rnd = (float) (GeneticAlgorithm.getRandomGenerator().nextDouble() 
* totalFitness);
float runningScore = 0;
for (Chromosome o : chromosomes) {
if (rnd >= runningScore && rnd <= runningScore + o.getFitness()) {
return o;
}
runningScore += o.getFitness();
}
return null;
}
{code}

We should use double instead of float.

For example, (float) (0.88948809 * 40.5109996) = 40.511 > 
40.5109996



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KYLIN-3305) Fix typos 'kylin.job.run.as.remote.cmd' to 'kylin.job.use-remote-cli' for documents

2018-03-20 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/KYLIN-3305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407477#comment-16407477
 ] 

ASF GitHub Bot commented on KYLIN-3305:
---

zhaoyongjie opened a new pull request #121: KYLIN-3305 Fix typos 
`kylin.job.run.as.remote.cmd`
URL: https://github.com/apache/kylin/pull/121
 
 
   KYLIN-3305 Fix typos `kylin.job.run.as.remote.cmd`


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Fix typos 'kylin.job.run.as.remote.cmd' to 'kylin.job.use-remote-cli' for 
> documents
> ---
>
> Key: KYLIN-3305
> URL: https://issues.apache.org/jira/browse/KYLIN-3305
> Project: Kylin
>  Issue Type: Improvement
>Reporter: yongjie zhao
>Priority: Minor
>
> Fix typos 'kylin.job.run.as.remote.cmd' to 'kylin.job.use-remote-cli'



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (KYLIN-3188) Improve the Kylin document

2018-03-20 Thread yongjie zhao (JIRA)


 [ 
https://issues.apache.org/jira/browse/KYLIN-3188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

yongjie zhao closed KYLIN-3188.
---
Resolution: Fixed

> Improve the Kylin document
> --
>
> Key: KYLIN-3188
> URL: https://issues.apache.org/jira/browse/KYLIN-3188
> Project: Kylin
>  Issue Type: Wish
>Reporter: yongjie zhao
>Priority: Major
>
> As you may noticed, Apach Kylin documents still have room for improvements. 
> Thus I would like to ask if anyone may providing any suitable generation 
> tools for us to select while the re-ordering of the documents could be done.
> such as:
>  - Sphinx [http://www.sphinx-doc.org/en/stable/]
>  - Hexo [https://hexo.io/]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KYLIN-3270) Add integration test for Optimize Job

2018-03-20 Thread Zhong Yanghong (JIRA)


[ 
https://issues.apache.org/jira/browse/KYLIN-3270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407472#comment-16407472
 ] 

Zhong Yanghong commented on KYLIN-3270:
---

This patch is conflicted with commit a040c9a7f419fa1599d35f3becea3f4d67cfcef3. 
I'll refine the patch for master.

> Add integration test for Optimize Job
> -
>
> Key: KYLIN-3270
> URL: https://issues.apache.org/jira/browse/KYLIN-3270
> Project: Kylin
>  Issue Type: Sub-task
>  Components: Tools, Build and Test
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Critical
> Fix For: v2.3.1
>
> Attachments: APACHE-KYLIN-3270.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KYLIN-3270) Add integration test for Optimize Job

2018-03-20 Thread Dong Li (JIRA)


[ 
https://issues.apache.org/jira/browse/KYLIN-3270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407467#comment-16407467
 ] 

Dong Li commented on KYLIN-3270:


I did a "git am" on master branch, but not tried on 2.3.x branch.

> Add integration test for Optimize Job
> -
>
> Key: KYLIN-3270
> URL: https://issues.apache.org/jira/browse/KYLIN-3270
> Project: Kylin
>  Issue Type: Sub-task
>  Components: Tools, Build and Test
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Critical
> Fix For: v2.3.1
>
> Attachments: APACHE-KYLIN-3270.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KYLIN-3270) Add integration test for Optimize Job

2018-03-20 Thread Shaofeng SHI (JIRA)


[ 
https://issues.apache.org/jira/browse/KYLIN-3270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407417#comment-16407417
 ] 

Shaofeng SHI commented on KYLIN-3270:
-

The patch has been successfully applied in the 2.3.x branch. But on master, it 
has conflict. [~yaho] could you please provide a new patch against the latest 
master branch? Thanks!

> Add integration test for Optimize Job
> -
>
> Key: KYLIN-3270
> URL: https://issues.apache.org/jira/browse/KYLIN-3270
> Project: Kylin
>  Issue Type: Sub-task
>  Components: Tools, Build and Test
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Critical
> Fix For: v2.3.1
>
> Attachments: APACHE-KYLIN-3270.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KYLIN-3270) Add integration test for Optimize Job

2018-03-20 Thread Shaofeng SHI (JIRA)


[ 
https://issues.apache.org/jira/browse/KYLIN-3270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407412#comment-16407412
 ] 

Shaofeng SHI commented on KYLIN-3270:
-

I can apply the patch without any issue. Dong, what commands you're using to 
merge the patch?

 

> Add integration test for Optimize Job
> -
>
> Key: KYLIN-3270
> URL: https://issues.apache.org/jira/browse/KYLIN-3270
> Project: Kylin
>  Issue Type: Sub-task
>  Components: Tools, Build and Test
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Critical
> Fix For: v2.3.1
>
> Attachments: APACHE-KYLIN-3270.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KYLIN-3270) Add integration test for Optimize Job

2018-03-20 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/KYLIN-3270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407413#comment-16407413
 ] 

ASF subversion and git services commented on KYLIN-3270:


Commit 97e572c48338678f13f6da3b560dce0328570f06 in kylin's branch 
refs/heads/2.3.x from Zhong
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=97e572c ]

KYLIN-3270: add integration test for optimize job

Signed-off-by: shaofengshi 


> Add integration test for Optimize Job
> -
>
> Key: KYLIN-3270
> URL: https://issues.apache.org/jira/browse/KYLIN-3270
> Project: Kylin
>  Issue Type: Sub-task
>  Components: Tools, Build and Test
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Critical
> Fix For: v2.3.1
>
> Attachments: APACHE-KYLIN-3270.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KYLIN-3304) Unstable UT CacheServiceTest

2018-03-20 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/KYLIN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407404#comment-16407404
 ] 

ASF subversion and git services commented on KYLIN-3304:


Commit 6cc59cfb3c766f5dda77517be8a2bcf21c065c30 in kylin's branch 
refs/heads/master from shaofengshi
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=6cc59cf ]

KYLIN-3304 temporarily disable CacheServiceTest


> Unstable UT CacheServiceTest
> 
>
> Key: KYLIN-3304
> URL: https://issues.apache.org/jira/browse/KYLIN-3304
> Project: Kylin
>  Issue Type: Test
>  Components: Tools, Build and Test
>Reporter: Shaofeng SHI
>Priority: Major
>
> Kylin integration test is unstable, sometimes failed at the CacheServiceTest:
>  
> *00:48:11* Failed tests: *00:48:11*   
> CacheServiceTest.testCubeCRUD:237*00:48:11* Tests in error: *00:48:11*   
> CacheServiceTest.testMetaCRUD:316->waitForCounterAndClear:156 Runtime 
> timeout*00:48:11* *00:48:11* Tests run: 93, Failures: 1, Errors: 1, Skipped: 1



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (KYLIN-3304) Unstable UT CacheServiceTest

2018-03-20 Thread Shaofeng SHI (JIRA)


 [ 
https://issues.apache.org/jira/browse/KYLIN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shaofeng SHI updated KYLIN-3304:

Description: 
Kylin integration test is unstable, sometimes failed at the CacheServiceTest:

 
*00:48:11* Failed tests: *00:48:11*   
CacheServiceTest.testCubeCRUD:237*00:48:11* Tests in error: *00:48:11*   
CacheServiceTest.testMetaCRUD:316->waitForCounterAndClear:156 Runtime 
timeout*00:48:11* *00:48:11* Tests run: 93, Failures: 1, Errors: 1, Skipped: 1

  was:
Kylin integration test is unstable, sometimes failed at the CacheServiceTest:
*00:48:11* Failed tests: *00:48:11*   
CacheServiceTest.testCubeCRUD:237*00:48:11* Tests in error: *00:48:11*   
CacheServiceTest.testMetaCRUD:316->waitForCounterAndClear:156 Runtime timeout

Summary: Unstable UT CacheServiceTest  (was: Instable UT 
CacheServiceTest)

> Unstable UT CacheServiceTest
> 
>
> Key: KYLIN-3304
> URL: https://issues.apache.org/jira/browse/KYLIN-3304
> Project: Kylin
>  Issue Type: Test
>  Components: Tools, Build and Test
>Reporter: Shaofeng SHI
>Priority: Major
>
> Kylin integration test is unstable, sometimes failed at the CacheServiceTest:
>  
> *00:48:11* Failed tests: *00:48:11*   
> CacheServiceTest.testCubeCRUD:237*00:48:11* Tests in error: *00:48:11*   
> CacheServiceTest.testMetaCRUD:316->waitForCounterAndClear:156 Runtime 
> timeout*00:48:11* *00:48:11* Tests run: 93, Failures: 1, Errors: 1, Skipped: 1



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (KYLIN-3304) Instable UT CacheServiceTest

2018-03-20 Thread Shaofeng SHI (JIRA)

Shaofeng SHI created KYLIN-3304:
---

 Summary: Instable UT CacheServiceTest
 Key: KYLIN-3304
 URL: https://issues.apache.org/jira/browse/KYLIN-3304
 Project: Kylin
  Issue Type: Test
  Components: Tools, Build and Test
Reporter: Shaofeng SHI


Kylin integration test is unstable, sometimes failed at the CacheServiceTest:
*00:48:11* Failed tests: *00:48:11*   
CacheServiceTest.testCubeCRUD:237*00:48:11* Tests in error: *00:48:11*   
CacheServiceTest.testMetaCRUD:316->waitForCounterAndClear:156 Runtime timeout



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KYLIN-3278) Kylin should not distribute hive table by random at Step1

2018-03-20 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/KYLIN-3278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407401#comment-16407401
 ] 

ASF subversion and git services commented on KYLIN-3278:


Commit 0029d8a77a4588e613f766f3218595cf33e1fb2e in kylin's branch 
refs/heads/master from shaofengshi
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=0029d8a ]

KYLIN-3278 Revert "KYLIN-3084, fix issue by using distribute by"

This reverts commit 849faec7d63b59af58eadc331f00c96343f18cc8.

Remove unnecessary UT


> Kylin should not distribute hive table by random at Step1
> -
>
> Key: KYLIN-3278
> URL: https://issues.apache.org/jira/browse/KYLIN-3278
> Project: Kylin
>  Issue Type: Improvement
>  Components: Job Engine
>Affects Versions: v2.3.0
>Reporter: Chuqian Yu
>Assignee: Wang Cheng
>Priority: Major
> Fix For: v2.3.1
>
>
> Hi,Kylin team.
> When I upgrade from 2.2.0 to 2.3.0 ,I found that the step.1 "Create 
> Intermediate Flat Hive Table" of cube build was much slower than before, 
> because Kylin distribute hive table by random when created it in version 
> 2.3.0 . I wander is it necessary? Because 
> kylin.source.hive.redistribute-flat-table was set to true by default, so we 
> will do the distribution job at Step.2 "Redistribute Flat Hive Table".
> Best regards, 
>  Chuqian Yu



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (KYLIN-3278) Kylin should not distribute hive table by random at Step1

2018-03-20 Thread Shaofeng SHI (JIRA)


 [ 
https://issues.apache.org/jira/browse/KYLIN-3278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shaofeng SHI resolved KYLIN-3278.
-
Resolution: Fixed

> Kylin should not distribute hive table by random at Step1
> -
>
> Key: KYLIN-3278
> URL: https://issues.apache.org/jira/browse/KYLIN-3278
> Project: Kylin
>  Issue Type: Improvement
>  Components: Job Engine
>Affects Versions: v2.3.0
>Reporter: Chuqian Yu
>Assignee: Wang Cheng
>Priority: Major
> Fix For: v2.3.1
>
>
> Hi,Kylin team.
> When I upgrade from 2.2.0 to 2.3.0 ,I found that the step.1 "Create 
> Intermediate Flat Hive Table" of cube build was much slower than before, 
> because Kylin distribute hive table by random when created it in version 
> 2.3.0 . I wander is it necessary? Because 
> kylin.source.hive.redistribute-flat-table was set to true by default, so we 
> will do the distribution job at Step.2 "Redistribute Flat Hive Table".
> Best regards, 
>  Chuqian Yu



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KYLIN-3278) Kylin should not distribute hive table by random at Step1

2018-03-20 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/KYLIN-3278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407390#comment-16407390
 ] 

ASF subversion and git services commented on KYLIN-3278:


Commit aa2f8e796f387c1227c8fa216b9af69d0ee9de42 in kylin's branch 
refs/heads/2.3.x from shaofengshi
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=aa2f8e7 ]

KYLIN-3278 Revert "KYLIN-3084, fix issue by using distribute by"

This reverts commit 849faec7d63b59af58eadc331f00c96343f18cc8.

Remove unnecessary UT


> Kylin should not distribute hive table by random at Step1
> -
>
> Key: KYLIN-3278
> URL: https://issues.apache.org/jira/browse/KYLIN-3278
> Project: Kylin
>  Issue Type: Improvement
>  Components: Job Engine
>Affects Versions: v2.3.0
>Reporter: Chuqian Yu
>Assignee: Wang Cheng
>Priority: Major
> Fix For: v2.3.1
>
>
> Hi,Kylin team.
> When I upgrade from 2.2.0 to 2.3.0 ,I found that the step.1 "Create 
> Intermediate Flat Hive Table" of cube build was much slower than before, 
> because Kylin distribute hive table by random when created it in version 
> 2.3.0 . I wander is it necessary? Because 
> kylin.source.hive.redistribute-flat-table was set to true by default, so we 
> will do the distribution job at Step.2 "Redistribute Flat Hive Table".
> Best regards, 
>  Chuqian Yu



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KYLIN-3084) File not found Exception when processing union-all in TEZ mode

2018-03-20 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/KYLIN-3084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407391#comment-16407391
 ] 

ASF subversion and git services commented on KYLIN-3084:


Commit aa2f8e796f387c1227c8fa216b9af69d0ee9de42 in kylin's branch 
refs/heads/2.3.x from shaofengshi
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=aa2f8e7 ]

KYLIN-3278 Revert "KYLIN-3084, fix issue by using distribute by"

This reverts commit 849faec7d63b59af58eadc331f00c96343f18cc8.

Remove unnecessary UT


> File not found Exception when processing union-all in TEZ mode
> --
>
> Key: KYLIN-3084
> URL: https://issues.apache.org/jira/browse/KYLIN-3084
> Project: Kylin
>  Issue Type: Bug
>Reporter: Wang Cheng
>Assignee: Wang Cheng
>Priority: Minor
> Fix For: v2.3.0
>
>
> If hive.execution.engine=TEZ and hql contains union all, it causes exception 
> like: file not found when materializing the view or redistributing flat hive 
> table.
> Here is the reason:
> http://grokbase.com/t/hive/user/162r80a2g9/anyway-to-avoid-creating-subdirectories-by-insert-with-union
> i.e. "The Tez execution of UNION is entirely parallel &
> the task-ids overlaps - so the files created have to have unique names.
> But the total counts for "Map 1" and "Map 2" are only available as the job
> runs, so they write to different dirs."
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KYLIN-3084) File not found Exception when processing union-all in TEZ mode

2018-03-20 Thread Shaofeng SHI (JIRA)


[ 
https://issues.apache.org/jira/browse/KYLIN-3084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407372#comment-16407372
 ] 

Shaofeng SHI commented on KYLIN-3084:
-

Hi Le Anh Vu, this change will be reverted, and a fixpack 2.3.1 will be 
released soon.

> File not found Exception when processing union-all in TEZ mode
> --
>
> Key: KYLIN-3084
> URL: https://issues.apache.org/jira/browse/KYLIN-3084
> Project: Kylin
>  Issue Type: Bug
>Reporter: Wang Cheng
>Assignee: Wang Cheng
>Priority: Minor
> Fix For: v2.3.0
>
>
> If hive.execution.engine=TEZ and hql contains union all, it causes exception 
> like: file not found when materializing the view or redistributing flat hive 
> table.
> Here is the reason:
> http://grokbase.com/t/hive/user/162r80a2g9/anyway-to-avoid-creating-subdirectories-by-insert-with-union
> i.e. "The Tez execution of UNION is entirely parallel &
> the task-ids overlaps - so the files created have to have unique names.
> But the total counts for "Map 1" and "Map 2" are only available as the job
> runs, so they write to different dirs."
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (KYLIN-3301) Upgrade opensaml

2018-03-20 Thread Shaofeng SHI (JIRA)


 [ 
https://issues.apache.org/jira/browse/KYLIN-3301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shaofeng SHI resolved KYLIN-3301.
-
Resolution: Fixed

> Upgrade opensaml
> 
>
> Key: KYLIN-3301
> URL: https://issues.apache.org/jira/browse/KYLIN-3301
> Project: Kylin
>  Issue Type: Improvement
>  Components: Integration
>Affects Versions: v2.2.0, v2.3.0
>Reporter: Shaofeng SHI
>Assignee: Shaofeng SHI
>Priority: Major
> Fix For: v2.3.1
>
> Attachments: KYLIN-3301.master.002.patch
>
>
> opensaml 2.6.4 is reported with security issue (CVE-2015-1796), need upgrade.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (KYLIN-3300) Upgrade jackson-databind

2018-03-20 Thread Shaofeng SHI (JIRA)


 [ 
https://issues.apache.org/jira/browse/KYLIN-3300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shaofeng SHI resolved KYLIN-3300.
-
Resolution: Fixed

> Upgrade jackson-databind
> 
>
> Key: KYLIN-3300
> URL: https://issues.apache.org/jira/browse/KYLIN-3300
> Project: Kylin
>  Issue Type: Improvement
>  Components: Integration
>Affects Versions: v2.2.0, v2.3.0
>Reporter: Shaofeng SHI
>Assignee: Shaofeng SHI
>Priority: Major
> Fix For: v2.3.1
>
> Attachments: KYLIN-3300.master.001.patch
>
>
> jackson-databind 2.6.3 and 2.6.5 are reported with security issue 
> (CVE-2017-7525), need ugprade



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (KYLIN-3301) Upgrade opensaml

2018-03-20 Thread Shaofeng SHI (JIRA)


 [ 
https://issues.apache.org/jira/browse/KYLIN-3301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shaofeng SHI updated KYLIN-3301:

Fix Version/s: v2.3.1

> Upgrade opensaml
> 
>
> Key: KYLIN-3301
> URL: https://issues.apache.org/jira/browse/KYLIN-3301
> Project: Kylin
>  Issue Type: Improvement
>  Components: Integration
>Affects Versions: v2.2.0, v2.3.0
>Reporter: Shaofeng SHI
>Assignee: Shaofeng SHI
>Priority: Major
> Fix For: v2.3.1
>
> Attachments: KYLIN-3301.master.002.patch
>
>
> opensaml 2.6.4 is reported with security issue (CVE-2015-1796), need upgrade.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (KYLIN-3300) Upgrade jackson-databind

2018-03-20 Thread Shaofeng SHI (JIRA)


 [ 
https://issues.apache.org/jira/browse/KYLIN-3300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shaofeng SHI updated KYLIN-3300:

Fix Version/s: v2.3.1

> Upgrade jackson-databind
> 
>
> Key: KYLIN-3300
> URL: https://issues.apache.org/jira/browse/KYLIN-3300
> Project: Kylin
>  Issue Type: Improvement
>  Components: Integration
>Affects Versions: v2.2.0, v2.3.0
>Reporter: Shaofeng SHI
>Assignee: Shaofeng SHI
>Priority: Major
> Fix For: v2.3.1
>
> Attachments: KYLIN-3300.master.001.patch
>
>
> jackson-databind 2.6.3 and 2.6.5 are reported with security issue 
> (CVE-2017-7525), need ugprade



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KYLIN-3300) Upgrade jackson-databind

2018-03-20 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/KYLIN-3300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407359#comment-16407359
 ] 

ASF subversion and git services commented on KYLIN-3300:


Commit c649509c997d15dcaa283381c3f18eaef6095cbb in kylin's branch 
refs/heads/2.3.x from shaofengshi
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=c649509 ]

KYLIN-3300 Upgrade jackson-databind

Signed-off-by: shaofengshi 


> Upgrade jackson-databind
> 
>
> Key: KYLIN-3300
> URL: https://issues.apache.org/jira/browse/KYLIN-3300
> Project: Kylin
>  Issue Type: Improvement
>  Components: Integration
>Affects Versions: v2.2.0, v2.3.0
>Reporter: Shaofeng SHI
>Assignee: Shaofeng SHI
>Priority: Major
> Attachments: KYLIN-3300.master.001.patch
>
>
> jackson-databind 2.6.3 and 2.6.5 are reported with security issue 
> (CVE-2017-7525), need ugprade



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KYLIN-3301) Upgrade opensaml

2018-03-20 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/KYLIN-3301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407360#comment-16407360
 ] 

ASF subversion and git services commented on KYLIN-3301:


Commit c407ce6ab09a279f8aea925674b4befe98d5e99b in kylin's branch 
refs/heads/2.3.x from shaofengshi
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=c407ce6 ]

KYLIN-3301 Upgrade opensaml

Signed-off-by: shaofengshi 


> Upgrade opensaml
> 
>
> Key: KYLIN-3301
> URL: https://issues.apache.org/jira/browse/KYLIN-3301
> Project: Kylin
>  Issue Type: Improvement
>  Components: Integration
>Affects Versions: v2.2.0, v2.3.0
>Reporter: Shaofeng SHI
>Assignee: Shaofeng SHI
>Priority: Major
> Attachments: KYLIN-3301.master.002.patch
>
>
> opensaml 2.6.4 is reported with security issue (CVE-2015-1796), need upgrade.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KYLIN-3301) Upgrade opensaml

2018-03-20 Thread Shaofeng SHI (JIRA)


[ 
https://issues.apache.org/jira/browse/KYLIN-3301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407361#comment-16407361
 ] 

Shaofeng SHI commented on KYLIN-3301:
-

Patch merged in master and 2.3.x branch.

> Upgrade opensaml
> 
>
> Key: KYLIN-3301
> URL: https://issues.apache.org/jira/browse/KYLIN-3301
> Project: Kylin
>  Issue Type: Improvement
>  Components: Integration
>Affects Versions: v2.2.0, v2.3.0
>Reporter: Shaofeng SHI
>Assignee: Shaofeng SHI
>Priority: Major
> Attachments: KYLIN-3301.master.002.patch
>
>
> opensaml 2.6.4 is reported with security issue (CVE-2015-1796), need upgrade.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (KYLIN-3301) Upgrade opensaml

2018-03-20 Thread Shaofeng SHI (JIRA)


 [ 
https://issues.apache.org/jira/browse/KYLIN-3301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shaofeng SHI updated KYLIN-3301:

Attachment: (was: KYLIN-3301.master.001.patch)

> Upgrade opensaml
> 
>
> Key: KYLIN-3301
> URL: https://issues.apache.org/jira/browse/KYLIN-3301
> Project: Kylin
>  Issue Type: Improvement
>  Components: Integration
>Affects Versions: v2.2.0, v2.3.0
>Reporter: Shaofeng SHI
>Assignee: Shaofeng SHI
>Priority: Major
> Attachments: KYLIN-3301.master.002.patch
>
>
> opensaml 2.6.4 is reported with security issue (CVE-2015-1796), need upgrade.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KYLIN-3301) Upgrade opensaml

2018-03-20 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/KYLIN-3301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407357#comment-16407357
 ] 

ASF subversion and git services commented on KYLIN-3301:


Commit 3282fafb77c43729a241f99a50f010350db0f95d in kylin's branch 
refs/heads/master from shaofengshi
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=3282faf ]

KYLIN-3301 Upgrade opensaml

Signed-off-by: shaofengshi 


> Upgrade opensaml
> 
>
> Key: KYLIN-3301
> URL: https://issues.apache.org/jira/browse/KYLIN-3301
> Project: Kylin
>  Issue Type: Improvement
>  Components: Integration
>Affects Versions: v2.2.0, v2.3.0
>Reporter: Shaofeng SHI
>Assignee: Shaofeng SHI
>Priority: Major
> Attachments: KYLIN-3301.master.002.patch
>
>
> opensaml 2.6.4 is reported with security issue (CVE-2015-1796), need upgrade.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (KYLIN-3301) Upgrade opensaml

2018-03-20 Thread Shaofeng SHI (JIRA)


 [ 
https://issues.apache.org/jira/browse/KYLIN-3301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shaofeng SHI updated KYLIN-3301:

Attachment: KYLIN-3301.master.002.patch

> Upgrade opensaml
> 
>
> Key: KYLIN-3301
> URL: https://issues.apache.org/jira/browse/KYLIN-3301
> Project: Kylin
>  Issue Type: Improvement
>  Components: Integration
>Affects Versions: v2.2.0, v2.3.0
>Reporter: Shaofeng SHI
>Assignee: Shaofeng SHI
>Priority: Major
> Attachments: KYLIN-3301.master.001.patch, KYLIN-3301.master.002.patch
>
>
> opensaml 2.6.4 is reported with security issue (CVE-2015-1796), need upgrade.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (KYLIN-3303) 全局字典没有增加cube区分

2018-03-20 Thread RenZhiMin (JIRA)

RenZhiMin created KYLIN-3303:


 Summary: 全局字典没有增加cube区分
 Key: KYLIN-3303
 URL: https://issues.apache.org/jira/browse/KYLIN-3303
 Project: Kylin
  Issue Type: Bug
  Components: Job Engine
Affects Versions: v2.0.0
Reporter: RenZhiMin


cube设计中，某个COUNT_DISTINCT指标使用全局字典。当构建cube时会在hdfs生成字典文件，对应的目录是：{kylin.env.hdfs-working-dir}/resources/GlobalDict/dict/database.tableName/column/，同时会在hbase元数据中生成元数据信息，hbase中元数据rowkey信息：/dict/database.tableName/column/。value信息指定对应的hdfs路径。问题是：当删除cube时没有删除对应的hdfs数据和hbase数据。另外hdfs数据路径和hbase元数据rowkey信息都没有指定具体的cube，如果某一个表的列在多个cube中都使用到了全局字典，会导致多个cube共用一份字典数据。



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KYLIN-3300) Upgrade jackson-databind

2018-03-20 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/KYLIN-3300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407335#comment-16407335
 ] 

ASF subversion and git services commented on KYLIN-3300:


Commit ec8186e52a152b033c00ed6befa7fa9edaae31a5 in kylin's branch 
refs/heads/master from shaofengshi
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=ec8186e ]

KYLIN-3300 Upgrade jackson-databind

Signed-off-by: shaofengshi 


> Upgrade jackson-databind
> 
>
> Key: KYLIN-3300
> URL: https://issues.apache.org/jira/browse/KYLIN-3300
> Project: Kylin
>  Issue Type: Improvement
>  Components: Integration
>Affects Versions: v2.2.0, v2.3.0
>Reporter: Shaofeng SHI
>Assignee: Shaofeng SHI
>Priority: Major
> Attachments: KYLIN-3300.master.001.patch
>
>
> jackson-databind 2.6.3 and 2.6.5 are reported with security issue 
> (CVE-2017-7525), need ugprade



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KYLIN-3301) Upgrade opensaml

2018-03-20 Thread Billy Liu (JIRA)


[ 
https://issues.apache.org/jira/browse/KYLIN-3301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407330#comment-16407330
 ] 

Billy Liu commented on KYLIN-3301:
--

+1

> Upgrade opensaml
> 
>
> Key: KYLIN-3301
> URL: https://issues.apache.org/jira/browse/KYLIN-3301
> Project: Kylin
>  Issue Type: Improvement
>  Components: Integration
>Affects Versions: v2.2.0, v2.3.0
>Reporter: Shaofeng SHI
>Assignee: Shaofeng SHI
>Priority: Major
> Attachments: KYLIN-3301.master.001.patch
>
>
> opensaml 2.6.4 is reported with security issue (CVE-2015-1796), need upgrade.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KYLIN-3300) Upgrade jackson-databind

2018-03-20 Thread Billy Liu (JIRA)


[ 
https://issues.apache.org/jira/browse/KYLIN-3300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407326#comment-16407326
 ] 

Billy Liu commented on KYLIN-3300:
--

+1

> Upgrade jackson-databind
> 
>
> Key: KYLIN-3300
> URL: https://issues.apache.org/jira/browse/KYLIN-3300
> Project: Kylin
>  Issue Type: Improvement
>  Components: Integration
>Affects Versions: v2.2.0, v2.3.0
>Reporter: Shaofeng SHI
>Assignee: Shaofeng SHI
>Priority: Major
> Attachments: KYLIN-3300.master.001.patch
>
>
> jackson-databind 2.6.3 and 2.6.5 are reported with security issue 
> (CVE-2017-7525), need ugprade



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (KYLIN-3301) Upgrade opensaml

2018-03-20 Thread Shaofeng SHI (JIRA)


 [ 
https://issues.apache.org/jira/browse/KYLIN-3301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shaofeng SHI reassigned KYLIN-3301:
---

Assignee: Shaofeng SHI

> Upgrade opensaml
> 
>
> Key: KYLIN-3301
> URL: https://issues.apache.org/jira/browse/KYLIN-3301
> Project: Kylin
>  Issue Type: Improvement
>  Components: Integration
>Affects Versions: v2.2.0, v2.3.0
>Reporter: Shaofeng SHI
>Assignee: Shaofeng SHI
>Priority: Major
> Attachments: KYLIN-3301.master.001.patch
>
>
> opensaml 2.6.4 is reported with security issue (CVE-2015-1796), need upgrade.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (KYLIN-3302) Upgrade commons-email

2018-03-20 Thread Shaofeng SHI (JIRA)


 [ 
https://issues.apache.org/jira/browse/KYLIN-3302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shaofeng SHI closed KYLIN-3302.
---
Resolution: Not A Problem

Kylin already uses commons-email 1.5, which is not affected by this CVE.

> Upgrade commons-email
> -
>
> Key: KYLIN-3302
> URL: https://issues.apache.org/jira/browse/KYLIN-3302
> Project: Kylin
>  Issue Type: Improvement
>  Components: Integration
>Affects Versions: v2.2.0, v2.3.0
>Reporter: Shaofeng SHI
>Assignee: Shaofeng SHI
>Priority: Major
>
> Check [CVE-2017-9801|https://nvd.nist.gov/vuln/detail/CVE-2017-9801]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (KYLIN-3302) Upgrade commons-email

2018-03-20 Thread Shaofeng SHI (JIRA)


 [ 
https://issues.apache.org/jira/browse/KYLIN-3302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shaofeng SHI reassigned KYLIN-3302:
---

Assignee: Shaofeng SHI

> Upgrade commons-email
> -
>
> Key: KYLIN-3302
> URL: https://issues.apache.org/jira/browse/KYLIN-3302
> Project: Kylin
>  Issue Type: Improvement
>  Components: Integration
>Affects Versions: v2.2.0, v2.3.0
>Reporter: Shaofeng SHI
>Assignee: Shaofeng SHI
>Priority: Major
>
> Check [CVE-2017-9801|https://nvd.nist.gov/vuln/detail/CVE-2017-9801]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (KYLIN-3301) Upgrade opensaml

2018-03-20 Thread Shaofeng SHI (JIRA)


[ 
https://issues.apache.org/jira/browse/KYLIN-3301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16406454#comment-16406454
 ] 

Shaofeng SHI edited comment on KYLIN-3301 at 3/21/18 1:28 AM:
--

According to the CVE 
([https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2015-1796),] need upgrade 
to 2.6.5 or above, now upgrade it to 2.6.6.


was (Author: shaofengshi):
There is no newer version for opensaml. Seems that project is not well 
maintained.

> Upgrade opensaml
> 
>
> Key: KYLIN-3301
> URL: https://issues.apache.org/jira/browse/KYLIN-3301
> Project: Kylin
>  Issue Type: Improvement
>  Components: Integration
>Affects Versions: v2.2.0, v2.3.0
>Reporter: Shaofeng SHI
>Priority: Major
>
> opensaml 2.6.4 is reported with security issue (CVE-2015-1796), need upgrade.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KYLIN-3300) Upgrade jackson-databind

2018-03-20 Thread Shaofeng SHI (JIRA)


[ 
https://issues.apache.org/jira/browse/KYLIN-3300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407297#comment-16407297
 ] 

Shaofeng SHI commented on KYLIN-3300:
-

According to [this 
post|https://github.com/FasterXML/jackson-databind/issues/1723], the 
vnlerability is fixed in 2.6.7.1, so upgrade Kylin's dependency to this version

> Upgrade jackson-databind
> 
>
> Key: KYLIN-3300
> URL: https://issues.apache.org/jira/browse/KYLIN-3300
> Project: Kylin
>  Issue Type: Improvement
>  Components: Integration
>Affects Versions: v2.2.0, v2.3.0
>Reporter: Shaofeng SHI
>Assignee: Shaofeng SHI
>Priority: Major
> Attachments: KYLIN-3300.master.001.patch
>
>
> jackson-databind 2.6.3 and 2.6.5 are reported with security issue 
> (CVE-2017-7525), need ugprade



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (KYLIN-3300) Upgrade jackson-databind

2018-03-20 Thread Shaofeng SHI (JIRA)


 [ 
https://issues.apache.org/jira/browse/KYLIN-3300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shaofeng SHI updated KYLIN-3300:

Attachment: KYLIN-3300.master.001.patch

> Upgrade jackson-databind
> 
>
> Key: KYLIN-3300
> URL: https://issues.apache.org/jira/browse/KYLIN-3300
> Project: Kylin
>  Issue Type: Improvement
>  Components: Integration
>Affects Versions: v2.2.0, v2.3.0
>Reporter: Shaofeng SHI
>Priority: Major
> Attachments: KYLIN-3300.master.001.patch
>
>
> jackson-databind 2.6.3 and 2.6.5 are reported with security issue 
> (CVE-2017-7525), need ugprade



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (KYLIN-3300) Upgrade jackson-databind

2018-03-20 Thread Shaofeng SHI (JIRA)


 [ 
https://issues.apache.org/jira/browse/KYLIN-3300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shaofeng SHI reassigned KYLIN-3300:
---

Assignee: Shaofeng SHI

> Upgrade jackson-databind
> 
>
> Key: KYLIN-3300
> URL: https://issues.apache.org/jira/browse/KYLIN-3300
> Project: Kylin
>  Issue Type: Improvement
>  Components: Integration
>Affects Versions: v2.2.0, v2.3.0
>Reporter: Shaofeng SHI
>Assignee: Shaofeng SHI
>Priority: Major
> Attachments: KYLIN-3300.master.001.patch
>
>
> jackson-databind 2.6.3 and 2.6.5 are reported with security issue 
> (CVE-2017-7525), need ugprade



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (KYLIN-2565) Upgrade Kylin to Hadoop3.0

2018-03-20 Thread Shaofeng SHI (JIRA)


 [ 
https://issues.apache.org/jira/browse/KYLIN-2565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shaofeng SHI reassigned KYLIN-2565:
---

Assignee: Shaofeng SHI

> Upgrade Kylin to Hadoop3.0
> --
>
> Key: KYLIN-2565
> URL: https://issues.apache.org/jira/browse/KYLIN-2565
> Project: Kylin
>  Issue Type: New Feature
>Reporter: Wang Cheng
>Assignee: Shaofeng SHI
>Priority: Major
>
> Hadoop3.0-alpha is released, Kylin should also keep compatible with it. Below 
> is the Hadoop3.0 components requirements:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3.0.0+release 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KYLIN-3301) Upgrade opensaml

2018-03-20 Thread Shaofeng SHI (JIRA)


[ 
https://issues.apache.org/jira/browse/KYLIN-3301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16406454#comment-16406454
 ] 

Shaofeng SHI commented on KYLIN-3301:
-

There is no newer version for opensaml. Seems that project is not well 
maintained.

> Upgrade opensaml
> 
>
> Key: KYLIN-3301
> URL: https://issues.apache.org/jira/browse/KYLIN-3301
> Project: Kylin
>  Issue Type: Improvement
>  Components: Integration
>Affects Versions: v2.2.0, v2.3.0
>Reporter: Shaofeng SHI
>Priority: Major
>
> opensaml 2.6.4 is reported with security issue (CVE-2015-1796), need upgrade.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KYLIN-2565) Upgrade Kylin to Hadoop3.0

2018-03-20 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/KYLIN-2565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16406449#comment-16406449
 ] 

ASF subversion and git services commented on KYLIN-2565:


Commit 0099b61e9d330b36227cd5afef1165b23d78162e in kylin's branch 
refs/heads/master-hadoop3.0-3 from [~wormholer]
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=0099b61 ]

KYLIN-2565, upgrade to Hadoop3.0


> Upgrade Kylin to Hadoop3.0
> --
>
> Key: KYLIN-2565
> URL: https://issues.apache.org/jira/browse/KYLIN-2565
> Project: Kylin
>  Issue Type: New Feature
>Reporter: Wang Cheng
>Priority: Major
>
> Hadoop3.0-alpha is released, Kylin should also keep compatible with it. Below 
> is the Hadoop3.0 components requirements:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3.0.0+release 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KYLIN-2565) Upgrade Kylin to Hadoop3.0

2018-03-20 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/KYLIN-2565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16406450#comment-16406450
 ] 

ASF subversion and git services commented on KYLIN-2565:


Commit 8cecbf09ce671b304180afda1bc1a015745f4a64 in kylin's branch 
refs/heads/master-hadoop3.0-3 from shaofengshi
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=8cecbf0 ]

KYLIN-2565 upgrade to hadoop 3.0 hbase 2.0, pass UT


> Upgrade Kylin to Hadoop3.0
> --
>
> Key: KYLIN-2565
> URL: https://issues.apache.org/jira/browse/KYLIN-2565
> Project: Kylin
>  Issue Type: New Feature
>Reporter: Wang Cheng
>Priority: Major
>
> Hadoop3.0-alpha is released, Kylin should also keep compatible with it. Below 
> is the Hadoop3.0 components requirements:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3.0.0+release 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (KYLIN-3301) Upgrade opensaml

2018-03-20 Thread Shaofeng SHI (JIRA)

Shaofeng SHI created KYLIN-3301:
---

 Summary: Upgrade opensaml
 Key: KYLIN-3301
 URL: https://issues.apache.org/jira/browse/KYLIN-3301
 Project: Kylin
  Issue Type: Improvement
  Components: Integration
Affects Versions: v2.3.0
Reporter: Shaofeng SHI


opensaml 2.6.4 is reported with security issue (CVE-2015-1796), need upgrade.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (KYLIN-3300) Upgrade jackson-databind

2018-03-20 Thread Shaofeng SHI (JIRA)


 [ 
https://issues.apache.org/jira/browse/KYLIN-3300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shaofeng SHI updated KYLIN-3300:

Affects Version/s: v2.2.0
   v2.3.0

> Upgrade jackson-databind
> 
>
> Key: KYLIN-3300
> URL: https://issues.apache.org/jira/browse/KYLIN-3300
> Project: Kylin
>  Issue Type: Improvement
>  Components: Integration
>Affects Versions: v2.2.0, v2.3.0
>Reporter: Shaofeng SHI
>Priority: Major
>
> jackson-databind 2.6.3 and 2.6.5 are reported with security issue 
> (CVE-2017-7525), need ugprade



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (KYLIN-3301) Upgrade opensaml

2018-03-20 Thread Shaofeng SHI (JIRA)


 [ 
https://issues.apache.org/jira/browse/KYLIN-3301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shaofeng SHI updated KYLIN-3301:

Affects Version/s: v2.2.0

> Upgrade opensaml
> 
>
> Key: KYLIN-3301
> URL: https://issues.apache.org/jira/browse/KYLIN-3301
> Project: Kylin
>  Issue Type: Improvement
>  Components: Integration
>Affects Versions: v2.2.0, v2.3.0
>Reporter: Shaofeng SHI
>Priority: Major
>
> opensaml 2.6.4 is reported with security issue (CVE-2015-1796), need upgrade.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (KYLIN-3300) Upgrade jackson-databind

2018-03-20 Thread Shaofeng SHI (JIRA)

Shaofeng SHI created KYLIN-3300:
---

 Summary: Upgrade jackson-databind
 Key: KYLIN-3300
 URL: https://issues.apache.org/jira/browse/KYLIN-3300
 Project: Kylin
  Issue Type: Improvement
  Components: Integration
Reporter: Shaofeng SHI


jackson-databind 2.6.3 and 2.6.5 are reported with security issue 
(CVE-2017-7525), need ugprade



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (KYLIN-3299) When a column in int data type and encoding as date may will lead build wrong value

2018-03-20 Thread jiatao.tao (JIRA)


 [ 
https://issues.apache.org/jira/browse/KYLIN-3299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

jiatao.tao updated KYLIN-3299:
--
Description: When the year is less than 1000, build and query will result 
1970.

> When a column in int data type and encoding as date may will lead build wrong 
> value
> ---
>
> Key: KYLIN-3299
> URL: https://issues.apache.org/jira/browse/KYLIN-3299
> Project: Kylin
>  Issue Type: Bug
>Reporter: jiatao.tao
>Assignee: jiatao.tao
>Priority: Major
>
> When the year is less than 1000, build and query will result 1970.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (KYLIN-3299) When a column in int data type and encoding as date may will lead build wrong value

2018-03-20 Thread jiatao.tao (JIRA)


 [ 
https://issues.apache.org/jira/browse/KYLIN-3299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

jiatao.tao updated KYLIN-3299:
--
Summary: When a column in int data type and encoding as date may will lead 
build wrong value  (was: When a column in int data type and encoding as date 
and the year less than 1000 will lead build error)

> When a column in int data type and encoding as date may will lead build wrong 
> value
> ---
>
> Key: KYLIN-3299
> URL: https://issues.apache.org/jira/browse/KYLIN-3299
> Project: Kylin
>  Issue Type: Bug
>Reporter: jiatao.tao
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (KYLIN-3299) When a column in int data type and encoding as date may will lead build wrong value

2018-03-20 Thread jiatao.tao (JIRA)


 [ 
https://issues.apache.org/jira/browse/KYLIN-3299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

jiatao.tao reassigned KYLIN-3299:
-

Assignee: jiatao.tao

> When a column in int data type and encoding as date may will lead build wrong 
> value
> ---
>
> Key: KYLIN-3299
> URL: https://issues.apache.org/jira/browse/KYLIN-3299
> Project: Kylin
>  Issue Type: Bug
>Reporter: jiatao.tao
>Assignee: jiatao.tao
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (KYLIN-3299) When a column in int data type and encoding as date and the year less than 1000 will lead build error

2018-03-20 Thread jiatao.tao (JIRA)

jiatao.tao created KYLIN-3299:
-

 Summary: When a column in int data type and encoding as date and 
the year less than 1000 will lead build error
 Key: KYLIN-3299
 URL: https://issues.apache.org/jira/browse/KYLIN-3299
 Project: Kylin
  Issue Type: Bug
Reporter: jiatao.tao






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (KYLIN-3298) Kylin 2.3 start error(启动故障火狐浏览器白屏，谷歌浏览器显示404错误)

2018-03-20 Thread neo0820 (JIRA)


 [ 
https://issues.apache.org/jira/browse/KYLIN-3298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

neo0820 updated KYLIN-3298:
---
Summary: Kylin 2.3 start error(启动故障 火狐浏览器白屏，谷歌浏览器显示404错误)  (was: Kylin 2.3 
启动故障 火狐浏览器白屏，谷歌浏览器显示404错误)

> Kylin 2.3 start error(启动故障 火狐浏览器白屏，谷歌浏览器显示404错误)
> 
>
> Key: KYLIN-3298
> URL: https://issues.apache.org/jira/browse/KYLIN-3298
> Project: Kylin
>  Issue Type: Bug
>Affects Versions: v2.3.0
>Reporter: neo0820
>Priority: Blocker
> Attachments: 启动Kylin失败命令行提示.txt, 启动Kylin失败日志提示.txt, 捕获环境.JPG, 
> 捕获谷歌浏览器.JPG
>
>
> 环境说明：Linux 16.04  Hadoop:2.7.5   HBase:1.4.2  Hive:2.3.2   
> Kylin:2.3.0    for HBase 1.x (includes HDP 2.3+, AWS EMR 5.0+, Azure 
> HDInsight 3.4 - 3.6)
>  
> Hadoop、Hbase、Kylin都是部署在同一台Virtual BOX虚拟机上面。Hadoop、HBase用的是伪集群部署。
>  
> Kylin之前能启动，一次重启过后，就发现下述问题，在终端命令行显示启动成功
> {code:java}
> //代码占位符 ... ...
> A new Kylin instance is started by hadoop. To stop it, run 'kylin.sh stop'
> Check the log at /usr/local/kylin/logs/kylin.log
> Web UI is at http://:7070/kylin{code}
> 但是在浏览器输入网址登录不能进去。火狐浏览器显示白屏，谷歌浏览器显示404错误。
> {code:java}
> //日志提示如下：
> OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=512M; support 
> was removed in 8.0
> usage: java org.apache.catalina.startup.Catalina [ -config {pathname} ] [ 
> -nonaming ] { -help | start | stop }
> 三月 19, 2018 10:43:52 上午 org.apache.catalina.core.AprLifecycleListener 
> lifecycleEvent
> 信息: The APR based Apache Tomcat Native library which allows optimal 
> performance in production environments was not found on the 
> java.library.path: /usr/local/hadoop/lib/native
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in 
> [jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in 
> [jar:file:/usr/local/hbase/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in 
> [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an 
> explanation.
> SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
> 三月 19, 2018 10:43:54 上午 org.apache.coyote.AbstractProtocol init
> 信息: Initializing ProtocolHandler ["http-bio-7070"]
> 三月 19, 2018 10:43:55 上午 org.apache.coyote.AbstractProtocol init
> 信息: Initializing ProtocolHandler ["ajp-bio-9009"]
> 三月 19, 2018 10:43:55 上午 org.apache.catalina.startup.Catalina load
> 信息: Initialization processed in 4359 ms
> 三月 19, 2018 10:43:55 上午 org.apache.catalina.core.StandardService startInternal
> 信息: Starting service Catalina
> 三月 19, 2018 10:43:55 上午 org.apache.catalina.core.StandardEngine startInternal
> 信息: Starting Servlet Engine: Apache Tomcat/7.0.85
> 三月 19, 2018 10:43:55 上午 org.apache.catalina.startup.HostConfig deployWAR
> 信息: Deploying web application archive 
> /usr/local/kylin/tomcat/webapps/kylin.war
> 三月 19, 2018 10:43:57 上午 org.apache.catalina.core.ApplicationContext log
> 信息: No Spring WebApplicationInitializer types detected on classpath
> 2018-03-19T10:43:58,247 INFO [localhost-startStop-1] 
> org.apache.kylin.common.KylinConfig - Loading kylin-defaults.properties from 
> file:/usr/local/kylin/tomcat/webapps/kylin/WEB-INF/lib/kylin-core-common-2.3.0.jar!/kylin-defaults.properties
> 2018-03-19T10:43:58,381 WARN [localhost-startStop-1] 
> org.apache.kylin.common.BackwardCompatibilityConfig - Config 
> 'kylin.job.hive.database.for.intermediatetable' is deprecated, use 
> 'kylin.source.hive.database-for-flat-table' instead
> 2018-03-19T10:43:58,387 INFO [localhost-startStop-1] 
> org.apache.kylin.common.KylinConfig - Initialized a new KylinConfig from 
> getInstanceFromEnv : 979544209
> 三月 19, 2018 10:43:58 上午 org.apache.catalina.core.ApplicationContext log
> 信息: Set web app root system property: 'webapp.root' = 
> [/usr/local/kylin/bin/../tomcat/webapps/kylin/]
> 三月 19, 2018 10:43:58 上午 org.apache.catalina.core.ApplicationContext log
> 信息: Initializing log4j from [classpath:kylin-server-log4j.properties]
> 三月 19, 2018 10:43:58 上午 org.apache.catalina.core.ApplicationContext log
> 信息: Initializing Spring root WebApplicationContext
> 2018-03-19T10:43:58,639 INFO [localhost-startStop-1] 
> org.springframework.web.context.ContextLoader - Root WebApplicationContext: 
> initialization started
> 2018-03-19T10:43:59,209 INFO [localhost-startStop-1] 
> org.springframework.web.context.support.XmlWebApplicationContext - Refreshing 
> Root WebApplicationContext: startup date [Mon Mar 19 10:43:59 EDT 2018]; root 
> of context hierarchy
> 2018-03-19T10:43:59,607 INFO [localhost-startStop-1] 
> org.springframework.beans.factory.xml.XmlBeanDefinitionReader - Loading XML 
> bean definitions from class path resource

[jira] [Commented] (KYLIN-2565) Upgrade Kylin to Hadoop3.0

2018-03-20 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/KYLIN-2565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405894#comment-16405894
 ] 

ASF subversion and git services commented on KYLIN-2565:


Commit 8c00c8e66fcbe883fc33eeaaa0d78b718b01f53c in kylin's branch 
refs/heads/master-hadoop3.0-2 from shaofengshi
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=8c00c8e ]

KYLIN-2565 upgrade to hadoop 3.0 hbase 2.0, pass UT


> Upgrade Kylin to Hadoop3.0
> --
>
> Key: KYLIN-2565
> URL: https://issues.apache.org/jira/browse/KYLIN-2565
> Project: Kylin
>  Issue Type: New Feature
>Reporter: Wang Cheng
>Priority: Major
>
> Hadoop3.0-alpha is released, Kylin should also keep compatible with it. Below 
> is the Hadoop3.0 components requirements:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3.0.0+release 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KYLIN-2565) Upgrade Kylin to Hadoop3.0

2018-03-20 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/KYLIN-2565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405893#comment-16405893
 ] 

ASF subversion and git services commented on KYLIN-2565:


Commit b5b422802322ad08156bc8598fb93cfdcaf98751 in kylin's branch 
refs/heads/master-hadoop3.0-2 from [~wormholer]
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=b5b4228 ]

KYLIN-2565, upgrade to Hadoop3.0


> Upgrade Kylin to Hadoop3.0
> --
>
> Key: KYLIN-2565
> URL: https://issues.apache.org/jira/browse/KYLIN-2565
> Project: Kylin
>  Issue Type: New Feature
>Reporter: Wang Cheng
>Priority: Major
>
> Hadoop3.0-alpha is released, Kylin should also keep compatible with it. Below 
> is the Hadoop3.0 components requirements:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3.0.0+release 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

58 matches

Mail list logo