[jira] [Commented] (KYLIN-3393) Building streaming cube do not need to set “KAFKA_HOME” environment variable
[ https://issues.apache.org/jira/browse/KYLIN-3393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16498889#comment-16498889 ] Billy Liu commented on KYLIN-3393: -- [~wuyingjun], Kylin does not depend on the Kafka Server, but the Kafka Client. So you do not need to install Kafka into kylin node, but copy the necessary client library. That's the same thing we are doing with Hadoop/Hive/HBase as Shaofeng said. Kylin keeps the dependency libraries out of the package to make it more compatible with the environment. If Kylin ships the fixed-version client libraries in binary, you have to recompile everything if the client library has conflicts with the server side. > Building streaming cube do not need to set “KAFKA_HOME” environment variable > - > > Key: KYLIN-3393 > URL: https://issues.apache.org/jira/browse/KYLIN-3393 > Project: Kylin > Issue Type: Improvement >Reporter: wuyingjun >Assignee: wuyingjun >Priority: Major > Attachments: 1.png, 2.png, KYLIN-3393.patch > > > You need hava KAFKA environment and set “KAFKA_HOME” environment variable > when building streaming cube, or else you can not submit a streaming cube > building job, like pic 1； > After we change the kafka dependency scope mode described in the > KYLIN-3393.patch , we can build streaming cube without a kafka environment in > the kylin service node successfully， like pic 2； -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3395) 无法删除model的问题。
[ https://issues.apache.org/jira/browse/KYLIN-3395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16498883#comment-16498883 ] Billy Liu commented on KYLIN-3395: -- [~chenfeng_0829] First, please do not ask questions in JIRA. The Kylin user mailer is the right place to ask for help. JIRA is for confirmed bug or feature discussion. Second, if you think this is the repeatable bug, please repeat it against the sample project, and list all necessary steps and each step result. Then others will help. > 无法删除model的问题。 > - > > Key: KYLIN-3395 > URL: https://issues.apache.org/jira/browse/KYLIN-3395 > Project: Kylin > Issue Type: Bug > Components: REST Service >Affects Versions: v2.3.0 >Reporter: chenchen >Priority: Major > > org.apache.kylin.rest.exception.BadRequestException: Model is referenced by > Cube '***analysis_trd_detail_d_kq_1' , could not dropped > at > org.apache.kylin.rest.service.ModelService.dropModel(ModelService.java:154) > at > org.apache.kylin.rest.controller.ModelController.deleteModel(ModelController.java:173) > at sun.reflect.GeneratedMethodAccessor254.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) > at > org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133) > at > org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97) > at > org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827) > at > org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738) > at > org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85) > at > org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967) > at > org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901) > at > org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970) > at > org.springframework.web.servlet.FrameworkServlet.doDelete(FrameworkServlet.java:894) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:656) > at > org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:731) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) > cube已经被删除了，但是在删除model的时候还是提示关联的cube存在？ > 我的model表是一个视图。不知道是不是这个原因？ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3355) Improve the HTTP return code of Rest API
[ https://issues.apache.org/jira/browse/KYLIN-3355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-3355: Fix Version/s: v2.4.0 > Improve the HTTP return code of Rest API > > > Key: KYLIN-3355 > URL: https://issues.apache.org/jira/browse/KYLIN-3355 > Project: Kylin > Issue Type: Improvement > Components: REST Service >Affects Versions: v2.1.0, v2.2.0, v2.3.0 >Reporter: Shaofeng SHI >Assignee: Shaofeng SHI >Priority: Major > Fix For: v2.4.0 > > > Kylin returns 500 error for some invalid input; for example an invalid cube > name, when the job exceeds the maximum number, etc. > We need to refine the behavior; For invalid user input, return 4XX http code. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3358) Support more sqls
[ https://issues.apache.org/jira/browse/KYLIN-3358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-3358: Fix Version/s: v2.4.0 Component/s: Query Engine > Support more sqls > - > > Key: KYLIN-3358 > URL: https://issues.apache.org/jira/browse/KYLIN-3358 > Project: Kylin > Issue Type: Improvement > Components: Query Engine >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong >Priority: Major > Fix For: v2.4.0 > > > Because of not storing raw data, kylin is not able to support all of the ANSI > SQLs. For other cases, kylin should support as much as possible. For example, > sum(case when...), sum(2*price+1), count(column), etc. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3359) Support sum(expression) if possible
[
https://issues.apache.org/jira/browse/KYLIN-3359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI updated KYLIN-3359:
Fix Version/s: v2.4.0
Component/s: Query Engine
> Support sum(expression) if possible
> ---
>
> Key: KYLIN-3359
> URL: https://issues.apache.org/jira/browse/KYLIN-3359
> Project: Kylin
> Issue Type: Sub-task
> Components: Query Engine
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Fix For: v2.4.0
>
> Attachments: KYLIN-3359-Hive-query.png, KYLIN-3359-Kylin-query.png
>
>
> The expression can be as follows:
> # a ~1~*col ~1~ + a ~2~*col ~2~ + ... + a ~n~*col ~n~ + b, if sum(col
> ~1~),sum(col ~2~),...sum(col ~n~) are defined
> # case when {{filter}} ~1~ then expr ~1~
> when {{filter}} ~2~ then expr ~2~
> ...
> else expr ~N~
> end, if {{filter}} ~1~,{{filter}} ~2~, ... {{filter}} ~N-1~, and expr
> ~1~,expr ~2~,...expr ~N~ are supported
> There's a constraint for the filter. That is it's able to push down the
> related filters in case when.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3364) Make the behavior of BigDecimalSumAggregator consistent with hive
[ https://issues.apache.org/jira/browse/KYLIN-3364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-3364: Fix Version/s: v2.4.0 > Make the behavior of BigDecimalSumAggregator consistent with hive > - > > Key: KYLIN-3364 > URL: https://issues.apache.org/jira/browse/KYLIN-3364 > Project: Kylin > Issue Type: Sub-task >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong >Priority: Major > Fix For: v2.4.0 > > Attachments: KYLIN-3364.patch > > > When aggregating on a sequence of BigDecimal values, if all values are null, > the result should be null. Otherwise, null should be regarded as 0. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (KYLIN-3366) Configure automatic enabling of cubes after a build process
[ https://issues.apache.org/jira/browse/KYLIN-3366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI resolved KYLIN-3366. - Resolution: Fixed > Configure automatic enabling of cubes after a build process > --- > > Key: KYLIN-3366 > URL: https://issues.apache.org/jira/browse/KYLIN-3366 > Project: Kylin > Issue Type: New Feature >Affects Versions: v2.3.1 > Environment: Kylin 2.3.1 and Hadoop EMR 5.7 >Reporter: Roberto Tardío Olmos >Assignee: Pan, Julian >Priority: Minor > Labels: features > Fix For: v2.4.0 > > Attachments: KYLIN-3366.patch > > > Kylin automatically enables the disabled cubes after a construction process. > This behavior forces us to constantly disable a new cube that is under > development to replace an existing and enabled cube. If we do not disable it, > we could have problems with the routing of the queries. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3370) Enhance segment pruning
[ https://issues.apache.org/jira/browse/KYLIN-3370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-3370: Priority: Major (was: Minor) Component/s: Query Engine Metadata > Enhance segment pruning > --- > > Key: KYLIN-3370 > URL: https://issues.apache.org/jira/browse/KYLIN-3370 > Project: Kylin > Issue Type: Improvement > Components: Metadata, Query Engine >Reporter: Shaoxiong Zhan >Assignee: Shaoxiong Zhan >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (KYLIN-3372) Upgrade jackson-databind version due to security concerns
[ https://issues.apache.org/jira/browse/KYLIN-3372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI resolved KYLIN-3372. - Resolution: Fixed > Upgrade jackson-databind version due to security concerns > - > > Key: KYLIN-3372 > URL: https://issues.apache.org/jira/browse/KYLIN-3372 > Project: Kylin > Issue Type: Task >Reporter: Ted Yu >Assignee: Chao Long >Priority: Major > Fix For: v2.4.0 > > > * https://nvd.nist.gov/vuln/detail/CVE-2018-5968 > * https://nvd.nist.gov/vuln/detail/CVE-2018-7489 > * https://nvd.nist.gov/vuln/detail/CVE-2017-7525 > * https://nvd.nist.gov/vuln/detail/CVE-2017-17485 > * https://nvd.nist.gov/vuln/detail/CVE-2017-15095 > We should either remove the dependency or upgrade to version 2.8.11.1 or the > latest, if possible. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3373) Some improvements for lookup table - UI part change
[ https://issues.apache.org/jira/browse/KYLIN-3373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-3373: Fix Version/s: v2.4.0 > Some improvements for lookup table - UI part change > --- > > Key: KYLIN-3373 > URL: https://issues.apache.org/jira/browse/KYLIN-3373 > Project: Kylin > Issue Type: Sub-task >Reporter: Ma Gang >Assignee: Pan, Julian >Priority: Major > Fix For: v2.4.0 > > > UI part change -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3377) Some improvements for lookup table - snapshot management
[ https://issues.apache.org/jira/browse/KYLIN-3377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-3377: Fix Version/s: v2.4.0 > Some improvements for lookup table - snapshot management > > > Key: KYLIN-3377 > URL: https://issues.apache.org/jira/browse/KYLIN-3377 > Project: Kylin > Issue Type: Sub-task > Components: Job Engine, REST Service >Reporter: Ma Gang >Assignee: Ma Gang >Priority: Major > Fix For: v2.4.0 > > > including build snapshot independently, view snapshot information, etc -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3374) Some improvements for lookup table - metadata change
[ https://issues.apache.org/jira/browse/KYLIN-3374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-3374: Fix Version/s: v2.4.0 > Some improvements for lookup table - metadata change > > > Key: KYLIN-3374 > URL: https://issues.apache.org/jira/browse/KYLIN-3374 > Project: Kylin > Issue Type: Sub-task > Components: Metadata >Reporter: Ma Gang >Assignee: Ma Gang >Priority: Major > Fix For: v2.4.0 > > > Some improvements for lookup table - metadata change -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3376) Some improvements for lookup table - query change
[ https://issues.apache.org/jira/browse/KYLIN-3376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-3376: Fix Version/s: v2.4.0 > Some improvements for lookup table - query change > - > > Key: KYLIN-3376 > URL: https://issues.apache.org/jira/browse/KYLIN-3376 > Project: Kylin > Issue Type: Sub-task > Components: Query Engine >Reporter: Ma Gang >Assignee: Ma Gang >Priority: Major > Fix For: v2.4.0 > > > query part change -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3375) Some improvements for lookup table - build change
[ https://issues.apache.org/jira/browse/KYLIN-3375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-3375: Fix Version/s: v2.4.0 > Some improvements for lookup table - build change > - > > Key: KYLIN-3375 > URL: https://issues.apache.org/jira/browse/KYLIN-3375 > Project: Kylin > Issue Type: Sub-task > Components: Job Engine >Reporter: Ma Gang >Assignee: Ma Gang >Priority: Major > Fix For: v2.4.0 > > > build change for new lookup table -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3299) When a column in int data type and encoding as date may will lead build wrong value
[ https://issues.apache.org/jira/browse/KYLIN-3299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16497664#comment-16497664 ] ASF GitHub Bot commented on KYLIN-3299: --- shaofengshi commented on issue #146: KYLIN-3299, fix when year less than 1000 in int data type and date encoding URL: https://github.com/apache/kylin/pull/146#issuecomment-393789208 Hi Jiatao, under what condition, there will be a number less than 1000 to represent a date? Can you give an example? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > When a column in int data type and encoding as date may will lead build wrong > value > --- > > Key: KYLIN-3299 > URL: https://issues.apache.org/jira/browse/KYLIN-3299 > Project: Kylin > Issue Type: Bug > Components: Job Engine >Reporter: jiatao.tao >Assignee: jiatao.tao >Priority: Major > Fix For: v2.4.0 > > > When the year is less than 1000, build and query will result 1970. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3299) When a column in int data type and encoding as date may will lead build wrong value
[ https://issues.apache.org/jira/browse/KYLIN-3299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-3299: Fix Version/s: v2.4.0 Component/s: Job Engine > When a column in int data type and encoding as date may will lead build wrong > value > --- > > Key: KYLIN-3299 > URL: https://issues.apache.org/jira/browse/KYLIN-3299 > Project: Kylin > Issue Type: Bug > Components: Job Engine >Reporter: jiatao.tao >Assignee: jiatao.tao >Priority: Major > Fix For: v2.4.0 > > > When the year is less than 1000, build and query will result 1970. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (KYLIN-3137) Spark cubing without hive-site.xml
[
https://issues.apache.org/jira/browse/KYLIN-3137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI resolved KYLIN-3137.
-
Resolution: Fixed
Fix Version/s: v2.4.0
> Spark cubing without hive-site.xml
> --
>
> Key: KYLIN-3137
> URL: https://issues.apache.org/jira/browse/KYLIN-3137
> Project: Kylin
> Issue Type: Improvement
> Components: Job Engine, Others, Spark Engine
>Affects Versions: v2.2.0
>Reporter: Ruslan Dautkhanov
>Assignee: Shaofeng SHI
>Priority: Major
> Labels: cdh, cloudera, configuration, hive
> Fix For: v2.4.0
>
>
> Getting following exception while trying to build a cube
> {noformat}
> java.lang.RuntimeException: Cannot find hive-site.xml in
> kylin_hadoop_conf_dir: /etc/hadoop/conf. In order to enable spark cubing, you
> must set kylin.env.hadoop-conf-dir to a dir which contains at least
> core-site.xml, hdfs-site.xml, hive-site.xml, mapred-site.xml, yarn-site.xml
> at
> org.apache.kylin.engine.spark.SparkExecutable.doWork(SparkExecutable.java:117)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:125)
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:64)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:125)
> at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:144)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> I am using Kylin binaries for CDH downloaded from kylin.apache.org.
> Yes, indeed hive-site.xml is not in /etc/hadoop/conf in Cloudera's
> distribution for Hadoop.
> hive-site.xml is in /etc/hive/conf, not in /etc/hadoop/conf
> The other four files:
> core-site.xml, hdfs-site.xml, mapred-site.xml, yarn-site.xml
> can be found in /etc/hadoop/conf but, again, not hive-site.xml which is in
> /etc/hive/conf .
> Would be great to have this adjusted for CDH.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3137) Spark cubing without hive-site.xml
[
https://issues.apache.org/jira/browse/KYLIN-3137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16497656#comment-16497656
]
Shaofeng SHI commented on KYLIN-3137:
-
I updated the code to directly parse the sequence files into RDD for cubing,
then without hive context.
I have tested it can work. This will make the deploy much easier.
> Spark cubing without hive-site.xml
> --
>
> Key: KYLIN-3137
> URL: https://issues.apache.org/jira/browse/KYLIN-3137
> Project: Kylin
> Issue Type: Improvement
> Components: Job Engine, Others, Spark Engine
>Affects Versions: v2.2.0
>Reporter: Ruslan Dautkhanov
>Assignee: Shaofeng SHI
>Priority: Major
> Labels: cdh, cloudera, configuration, hive
> Fix For: v2.4.0
>
>
> Getting following exception while trying to build a cube
> {noformat}
> java.lang.RuntimeException: Cannot find hive-site.xml in
> kylin_hadoop_conf_dir: /etc/hadoop/conf. In order to enable spark cubing, you
> must set kylin.env.hadoop-conf-dir to a dir which contains at least
> core-site.xml, hdfs-site.xml, hive-site.xml, mapred-site.xml, yarn-site.xml
> at
> org.apache.kylin.engine.spark.SparkExecutable.doWork(SparkExecutable.java:117)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:125)
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:64)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:125)
> at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:144)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> I am using Kylin binaries for CDH downloaded from kylin.apache.org.
> Yes, indeed hive-site.xml is not in /etc/hadoop/conf in Cloudera's
> distribution for Hadoop.
> hive-site.xml is in /etc/hive/conf, not in /etc/hadoop/conf
> The other four files:
> core-site.xml, hdfs-site.xml, mapred-site.xml, yarn-site.xml
> can be found in /etc/hadoop/conf but, again, not hive-site.xml which is in
> /etc/hive/conf .
> Would be great to have this adjusted for CDH.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3137) Spark cubing without hive-site.xml
[
https://issues.apache.org/jira/browse/KYLIN-3137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI updated KYLIN-3137:
Summary: Spark cubing without hive-site.xml (was: Update hive-site.xml
locating logic for CDH distro)
> Spark cubing without hive-site.xml
> --
>
> Key: KYLIN-3137
> URL: https://issues.apache.org/jira/browse/KYLIN-3137
> Project: Kylin
> Issue Type: Improvement
> Components: Job Engine, Others, Spark Engine
>Affects Versions: v2.2.0
>Reporter: Ruslan Dautkhanov
>Assignee: Shaofeng SHI
>Priority: Major
> Labels: cdh, cloudera, configuration, hive
>
> Getting following exception while trying to build a cube
> {noformat}
> java.lang.RuntimeException: Cannot find hive-site.xml in
> kylin_hadoop_conf_dir: /etc/hadoop/conf. In order to enable spark cubing, you
> must set kylin.env.hadoop-conf-dir to a dir which contains at least
> core-site.xml, hdfs-site.xml, hive-site.xml, mapred-site.xml, yarn-site.xml
> at
> org.apache.kylin.engine.spark.SparkExecutable.doWork(SparkExecutable.java:117)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:125)
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:64)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:125)
> at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:144)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> I am using Kylin binaries for CDH downloaded from kylin.apache.org.
> Yes, indeed hive-site.xml is not in /etc/hadoop/conf in Cloudera's
> distribution for Hadoop.
> hive-site.xml is in /etc/hive/conf, not in /etc/hadoop/conf
> The other four files:
> core-site.xml, hdfs-site.xml, mapred-site.xml, yarn-site.xml
> can be found in /etc/hadoop/conf but, again, not hive-site.xml which is in
> /etc/hive/conf .
> Would be great to have this adjusted for CDH.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3137) Update hive-site.xml locating logic for CDH distro
[
https://issues.apache.org/jira/browse/KYLIN-3137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16497653#comment-16497653
]
ASF subversion and git services commented on KYLIN-3137:
Commit e0f29e29164d44581c133314ac01c419e4277c35 in kylin's branch
refs/heads/master from shaofengshi
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=e0f29e2 ]
KYLIN-3137 Spark cubing refine
> Update hive-site.xml locating logic for CDH distro
> --
>
> Key: KYLIN-3137
> URL: https://issues.apache.org/jira/browse/KYLIN-3137
> Project: Kylin
> Issue Type: Improvement
> Components: Job Engine, Others, Spark Engine
>Affects Versions: v2.2.0
>Reporter: Ruslan Dautkhanov
>Assignee: Shaofeng SHI
>Priority: Major
> Labels: cdh, cloudera, configuration, hive
>
> Getting following exception while trying to build a cube
> {noformat}
> java.lang.RuntimeException: Cannot find hive-site.xml in
> kylin_hadoop_conf_dir: /etc/hadoop/conf. In order to enable spark cubing, you
> must set kylin.env.hadoop-conf-dir to a dir which contains at least
> core-site.xml, hdfs-site.xml, hive-site.xml, mapred-site.xml, yarn-site.xml
> at
> org.apache.kylin.engine.spark.SparkExecutable.doWork(SparkExecutable.java:117)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:125)
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:64)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:125)
> at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:144)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> I am using Kylin binaries for CDH downloaded from kylin.apache.org.
> Yes, indeed hive-site.xml is not in /etc/hadoop/conf in Cloudera's
> distribution for Hadoop.
> hive-site.xml is in /etc/hive/conf, not in /etc/hadoop/conf
> The other four files:
> core-site.xml, hdfs-site.xml, mapred-site.xml, yarn-site.xml
> can be found in /etc/hadoop/conf but, again, not hive-site.xml which is in
> /etc/hive/conf .
> Would be great to have this adjusted for CDH.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
