[jira] [Reopened] (KYLIN-3264) kylin.job.retry=1 cause build error
[ https://issues.apache.org/jira/browse/KYLIN-3264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] readme_kylin reopened KYLIN-3264: - i pull the request :[GitHub Pull Request #104|https://github.com/apache/kylin/pull/104] and build bin,then deploy. but the error still exists > kylin.job.retry=1 cause build error > --- > > Key: KYLIN-3264 > URL: https://issues.apache.org/jira/browse/KYLIN-3264 > Project: Kylin > Issue Type: Bug > Components: Job Engine >Affects Versions: v2.3.0 >Reporter: readme_kylin >Priority: Major > Fix For: v2.3.0 > > > when i set kylin.job.retry to 1,it cause build error at 6 step :Create HTable > . > here is the error info. > java.lang.RuntimeException: HBase table KYLIN230_CFJ854DYZ4 exists! > at > org.apache.kylin.storage.hbase.steps.CubeHTableUtil.createHTable(CubeHTableUtil.java:100) > at > org.apache.kylin.storage.hbase.steps.CreateHTableJob.run(CreateHTableJob.java:111) > at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:97) > at > org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:162) > at > org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:67) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:162) > at > org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:300) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-1864) keep volatile time range out of auto merging
[ https://issues.apache.org/jira/browse/KYLIN-1864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16378002#comment-16378002 ] lidaxue commented on KYLIN-1864: hi, my kylin version is v2.2 ,can the problem of KYLIN-1864 be set in Web GUI ? can you tell me how to use the volatile time range out of auto merging ? > keep volatile time range out of auto merging > > > Key: KYLIN-1864 > URL: https://issues.apache.org/jira/browse/KYLIN-1864 > Project: Kylin > Issue Type: Improvement >Reporter: hongbin ma >Assignee: hongbin ma >Priority: Major > > Current segment auto merging logic will try to merge all segments, including > all the recent segments. In some scenarios user don't want recent segments > being merged because they may still undergo data updates. A "volatile time > range" may be introduced to ease such cases. A "volatile time range" specify > a recent time range, segments in this range will be immune from being auto > merged. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3119) A few bugs in the function 'massageSql' of 'QueryUtil.java'
[
https://issues.apache.org/jira/browse/KYLIN-3119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16377987#comment-16377987
]
peng.jianhua commented on KYLIN-3119:
-
it's ok now,https://github.com/apache/kylin/pull/107
> A few bugs in the function 'massageSql' of 'QueryUtil.java'
> ---
>
> Key: KYLIN-3119
> URL: https://issues.apache.org/jira/browse/KYLIN-3119
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Reporter: peng.jianhua
>Assignee: peng.jianhua
>Priority: Major
> Attachments:
> 0001-KYLIN-3119-A-few-bugs-in-the-function-massageSql-of-.patch,
> 0001-KYLIN-3119-add-test-cases.patch,
> 0001-kylin-3119-add-more-test-cases-with-enter-and-space.patch,
> 0001-kylin-3119.patch, 01.PNG, 02.PNG
>
>
> in the function 'massageSql' of 'QueryUtil.java',there are a few bugs:
> 01:
> {code:java}
> while (sql.endsWith(";"))
> sql = sql.substring(0, sql.length() - 1);
> {code}
> if the sql end with ';' and after the ';' still has comments,will be
> error.Because the sql will add ("\nLIMIT " + limit) at the end.
> 02:
> {code:java}
> if (limit > 0 && !sql.toLowerCase().contains("limit")) {
> sql += ("\nLIMIT " + limit);
> }
> if (offset > 0 && !sql.toLowerCase().contains("offset")) {
> sql += ("\nOFFSET " + offset);
> }
> {code}
> if the sql already has word 'limit' in it,such as Alias,Subquery,Comments,the
> ‘limit’ in the Input box will be invalid.
> for example,the sql has subquery,and there is 'limit' in subquery.
> {code:java}
> select KYLIN_SALES.PART_DT, count(KYLIN_SALES.PRICE)
> from KYLIN_SALES
> inner join (select ACCOUNT_ID, ACCOUNT_BUYER_LEVEL from KYLIN_ACCOUNT where
> ACCOUNT_COUNTRY = 'US' limit 1) as TT
> on KYLIN_SALES.BUYER_ID = TT.ACCOUNT_ID
> group by KYLIN_SALES.PART_DT
> {code}
> the ‘limit’ in the Input box will be invalid.please refer to 01.png and
> 02.png.
> 03:
> {code:java}
> // https://issues.apache.org/jira/browse/KYLIN-2649
> if (kylinConfig.getForceLimit() > 0 &&
> !sql.toLowerCase().contains("limit")
> && sql.toLowerCase().contains("*")) {
> sql += ("\nLIMIT " + kylinConfig.getForceLimit());
> }
> {code}
> Because KYLIN-2649 is still unresolved,so I didn't change the code,but it
> has same 'limit' word bugs like above.
> And there are some situations that the sql contains \*,such as
> {code:java}count(*),kylin_sales.*{code}
> Maybe the \* in subquery too.
> it seems improperly to deal with the sql just use {code:java}
> sql.toLowerCase().contains("*")
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3002) Use Spark as default engine in web
[ https://issues.apache.org/jira/browse/KYLIN-3002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16377967#comment-16377967 ] liyang commented on KYLIN-3002: --- Recently verified that the measures are already supported by Spark engine. So the remaining gap is KYLIN-2484 mostly. > Use Spark as default engine in web > -- > > Key: KYLIN-3002 > URL: https://issues.apache.org/jira/browse/KYLIN-3002 > Project: Kylin > Issue Type: Improvement > Components: Spark Engine, Web >Reporter: kangkaisen >Assignee: kangkaisen >Priority: Major > > After KYLIN-2997, like KYLIN-2963, we could use Spark as default engine for > none-global-dict cube. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3119) A few bugs in the function 'massageSql' of 'QueryUtil.java'
[
https://issues.apache.org/jira/browse/KYLIN-3119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16377959#comment-16377959
]
Roger Shi commented on KYLIN-3119:
--
[~peng.jianhua], it's a commit, not a PR. On how to create a PR, please refer
to https://help.github.com/articles/about-pull-requests/
> A few bugs in the function 'massageSql' of 'QueryUtil.java'
> ---
>
> Key: KYLIN-3119
> URL: https://issues.apache.org/jira/browse/KYLIN-3119
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Reporter: peng.jianhua
>Assignee: peng.jianhua
>Priority: Major
> Attachments:
> 0001-KYLIN-3119-A-few-bugs-in-the-function-massageSql-of-.patch,
> 0001-KYLIN-3119-add-test-cases.patch,
> 0001-kylin-3119-add-more-test-cases-with-enter-and-space.patch,
> 0001-kylin-3119.patch, 01.PNG, 02.PNG
>
>
> in the function 'massageSql' of 'QueryUtil.java',there are a few bugs:
> 01:
> {code:java}
> while (sql.endsWith(";"))
> sql = sql.substring(0, sql.length() - 1);
> {code}
> if the sql end with ';' and after the ';' still has comments,will be
> error.Because the sql will add ("\nLIMIT " + limit) at the end.
> 02:
> {code:java}
> if (limit > 0 && !sql.toLowerCase().contains("limit")) {
> sql += ("\nLIMIT " + limit);
> }
> if (offset > 0 && !sql.toLowerCase().contains("offset")) {
> sql += ("\nOFFSET " + offset);
> }
> {code}
> if the sql already has word 'limit' in it,such as Alias,Subquery,Comments,the
> ‘limit’ in the Input box will be invalid.
> for example,the sql has subquery,and there is 'limit' in subquery.
> {code:java}
> select KYLIN_SALES.PART_DT, count(KYLIN_SALES.PRICE)
> from KYLIN_SALES
> inner join (select ACCOUNT_ID, ACCOUNT_BUYER_LEVEL from KYLIN_ACCOUNT where
> ACCOUNT_COUNTRY = 'US' limit 1) as TT
> on KYLIN_SALES.BUYER_ID = TT.ACCOUNT_ID
> group by KYLIN_SALES.PART_DT
> {code}
> the ‘limit’ in the Input box will be invalid.please refer to 01.png and
> 02.png.
> 03:
> {code:java}
> // https://issues.apache.org/jira/browse/KYLIN-2649
> if (kylinConfig.getForceLimit() > 0 &&
> !sql.toLowerCase().contains("limit")
> && sql.toLowerCase().contains("*")) {
> sql += ("\nLIMIT " + kylinConfig.getForceLimit());
> }
> {code}
> Because KYLIN-2649 is still unresolved,so I didn't change the code,but it
> has same 'limit' word bugs like above.
> And there are some situations that the sql contains \*,such as
> {code:java}count(*),kylin_sales.*{code}
> Maybe the \* in subquery too.
> it seems improperly to deal with the sql just use {code:java}
> sql.toLowerCase().contains("*")
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3267) add override MR config at project/cube level only for mem-hungry build steps like "Build columnar page index"
[
https://issues.apache.org/jira/browse/KYLIN-3267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yifei Wu updated KYLIN-3267:
Description:
Currently, users can override MR config at project/cube level via kylin
properties {{kylin.engine.mr.config-override.}}
However, this applies to all MR steps. It is more reasonable to add
override-configs only for certain memory-hungry steps like in-mem cubing and
"Build columnar page index"
was:
Currently user can override MR config at project/cube level via kylin
properties {{kylin.engine.mr.config-override.}}
However this applies to all MR steps.
Add override-configs that enable similar project/cube level config, but only
for certain memory-hungry steps like in-mem cubing and "Build columnar page
index"
> add override MR config at project/cube level only for mem-hungry build steps
> like "Build columnar page index"
> -
>
> Key: KYLIN-3267
> URL: https://issues.apache.org/jira/browse/KYLIN-3267
> Project: Kylin
> Issue Type: Bug
>Reporter: Yifei Wu
>Assignee: Yifei Wu
>Priority: Major
>
> Currently, users can override MR config at project/cube level via kylin
> properties {{kylin.engine.mr.config-override.}}
> However, this applies to all MR steps. It is more reasonable to add
> override-configs only for certain memory-hungry steps like in-mem cubing and
> "Build columnar page index"
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3267) add override MR config at project/cube level only for mem-hungry build steps like "Build columnar page index"
[
https://issues.apache.org/jira/browse/KYLIN-3267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yifei Wu updated KYLIN-3267:
Description:
Currently user can override MR config at project/cube level via kylin
properties {{kylin.engine.mr.config-override.}}
However this applies to all MR steps.
Add override-configs that enable similar project/cube level config, but only
for certain memory-hungry steps like in-mem cubing and "Build columnar page
index"
> add override MR config at project/cube level only for mem-hungry build steps
> like "Build columnar page index"
> -
>
> Key: KYLIN-3267
> URL: https://issues.apache.org/jira/browse/KYLIN-3267
> Project: Kylin
> Issue Type: Bug
>Reporter: Yifei Wu
>Assignee: Yifei Wu
>Priority: Major
>
> Currently user can override MR config at project/cube level via kylin
> properties {{kylin.engine.mr.config-override.}}
> However this applies to all MR steps.
> Add override-configs that enable similar project/cube level config, but only
> for certain memory-hungry steps like in-mem cubing and "Build columnar page
> index"
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (KYLIN-3267) add override MR config at project/cube level only for mem-hungry build steps like "Build columnar page index"
Yifei Wu created KYLIN-3267: --- Summary: add override MR config at project/cube level only for mem-hungry build steps like "Build columnar page index" Key: KYLIN-3267 URL: https://issues.apache.org/jira/browse/KYLIN-3267 Project: Kylin Issue Type: Bug Reporter: Yifei Wu Assignee: Yifei Wu -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3119) A few bugs in the function 'massageSql' of 'QueryUtil.java'
[
https://issues.apache.org/jira/browse/KYLIN-3119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16377904#comment-16377904
]
peng.jianhua commented on KYLIN-3119:
-
hi,[~R0ger],please review this,thanks.
https://github.com/pengjianhua/kylin/commit/beba49a608ce45eeed1118114cef7ab3ffefa816
> A few bugs in the function 'massageSql' of 'QueryUtil.java'
> ---
>
> Key: KYLIN-3119
> URL: https://issues.apache.org/jira/browse/KYLIN-3119
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Reporter: peng.jianhua
>Assignee: peng.jianhua
>Priority: Major
> Attachments:
> 0001-KYLIN-3119-A-few-bugs-in-the-function-massageSql-of-.patch,
> 0001-KYLIN-3119-add-test-cases.patch,
> 0001-kylin-3119-add-more-test-cases-with-enter-and-space.patch,
> 0001-kylin-3119.patch, 01.PNG, 02.PNG
>
>
> in the function 'massageSql' of 'QueryUtil.java',there are a few bugs:
> 01:
> {code:java}
> while (sql.endsWith(";"))
> sql = sql.substring(0, sql.length() - 1);
> {code}
> if the sql end with ';' and after the ';' still has comments,will be
> error.Because the sql will add ("\nLIMIT " + limit) at the end.
> 02:
> {code:java}
> if (limit > 0 && !sql.toLowerCase().contains("limit")) {
> sql += ("\nLIMIT " + limit);
> }
> if (offset > 0 && !sql.toLowerCase().contains("offset")) {
> sql += ("\nOFFSET " + offset);
> }
> {code}
> if the sql already has word 'limit' in it,such as Alias,Subquery,Comments,the
> ‘limit’ in the Input box will be invalid.
> for example,the sql has subquery,and there is 'limit' in subquery.
> {code:java}
> select KYLIN_SALES.PART_DT, count(KYLIN_SALES.PRICE)
> from KYLIN_SALES
> inner join (select ACCOUNT_ID, ACCOUNT_BUYER_LEVEL from KYLIN_ACCOUNT where
> ACCOUNT_COUNTRY = 'US' limit 1) as TT
> on KYLIN_SALES.BUYER_ID = TT.ACCOUNT_ID
> group by KYLIN_SALES.PART_DT
> {code}
> the ‘limit’ in the Input box will be invalid.please refer to 01.png and
> 02.png.
> 03:
> {code:java}
> // https://issues.apache.org/jira/browse/KYLIN-2649
> if (kylinConfig.getForceLimit() > 0 &&
> !sql.toLowerCase().contains("limit")
> && sql.toLowerCase().contains("*")) {
> sql += ("\nLIMIT " + kylinConfig.getForceLimit());
> }
> {code}
> Because KYLIN-2649 is still unresolved,so I didn't change the code,but it
> has same 'limit' word bugs like above.
> And there are some situations that the sql contains \*,such as
> {code:java}count(*),kylin_sales.*{code}
> Maybe the \* in subquery too.
> it seems improperly to deal with the sql just use {code:java}
> sql.toLowerCase().contains("*")
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3187) JDK APIs using the default locale, time zone or character set should be avoided
[ https://issues.apache.org/jira/browse/KYLIN-3187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated KYLIN-3187: -- Component/s: Storage - HBase > JDK APIs using the default locale, time zone or character set should be > avoided > --- > > Key: KYLIN-3187 > URL: https://issues.apache.org/jira/browse/KYLIN-3187 > Project: Kylin > Issue Type: Bug > Components: Storage - HBase >Reporter: Ted Yu >Priority: Major > Labels: usability > > Here are a few examples: > server-base/src/main/java/org/apache/kylin/rest/service/JobService.java: > Calendar calendar = Calendar.getInstance(); > storage-hbase/src/main/java/org/apache/kylin/storage/hbase/util/HbaseStreamingInput.java: > Calendar cal = Calendar.getInstance(); > Locale should be specified. > See CALCITE-1667 for related information. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3250) Upgrade jetty version to 9.2.24.v20180105
[ https://issues.apache.org/jira/browse/KYLIN-3250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated KYLIN-3250: -- Component/s: Tools, Build and Test > Upgrade jetty version to 9.2.24.v20180105 > - > > Key: KYLIN-3250 > URL: https://issues.apache.org/jira/browse/KYLIN-3250 > Project: Kylin > Issue Type: Improvement > Components: Tools, Build and Test >Reporter: Ted Yu >Priority: Minor > > Currently Kylin depends on 9.2.20.v20161216 which is quite old. > Memory leak was fixed in Jetty 9.4 and later backported however the version > of Jetty 9.2.20 came before this fix. See > [https://github.com/eclipse/jetty.project/issues/1804] and > [https://github.com/apache/hbase/blob/master/pom.xml#L1416.] > This issue is to upgrade to > [9.3.22.v20171030|https://mvnrepository.com/artifact/org.eclipse.jetty/jetty-server/9.3.22.v20171030] > which is the latest version of 9.3 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
