Hello Ted,

It is OK now, it wxas just oa problem of rights on user hive on /kylin/metadata 
on HDFS.
I succeeded in created my first cube.
Unfortunately, I cannot query it. I selected the project, and the cube then 
moved to ‘Insight tab’ and typed : select count(*) from kylin_sales. I got the 
following message :
null while executing SQL: "select count(*) from kylin_sales LIMIT 50000"

the log says : ‘unknown source’

2018-02-13 09:21:16,854 ERROR [http-bio-7070-exec-6] 
controller.BasicController:57 :
org.apache.kylin.rest.exception.InternalErrorException: null
while executing SQL: "select count(*) from kylin_sales LIMIT 50000"
        at 
org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:479)
        at 
org.apache.kylin.rest.controller.QueryController.query(QueryController.java:78)
        at sun.reflect.GeneratedMethodAccessor205.invoke(Unknown Source)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
        at 
org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133)
        at 
org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)
        at 
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827)
        at 
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738)

How could I solve this ? By rebuilding the model and then the cube ?
If yes, which commands/process shoud I use ?

Thank you for your help.

Best regards,
Jean-Luc.

De : BELLIER Jean-luc
Envoyé : lundi 12 février 2018 15:20
À : 'user@kylin.apache.org' 
<user@kylin.apache.org<mailto:user@kylin.apache.org>>
Objet : RE: Cannot rebuild cubes

Hello again Ted,

I did the following :

·         launched the command ‘beeline’

·         launched the command (under beeline) : !connect 
jdbc:hive2://<host>:<port>/<db>;principal=<Server_Principal_of_HiveServer2>
And I got the following message :

Driver: Hive JDBC (version 1.1.0-cdh5.14.0)
Transaction isolation: TRANSACTION_REPEATABLE_READ

Could this be the source of the problem, and, if yes, how to correct it ?

Thank you for your help.

Best regards,
Jean-Luc.

De : BELLIER Jean-luc
Envoyé : lundi 12 février 2018 14:39
À : user@kylin.apache.org<mailto:user@kylin.apache.org>
Objet : RE: Cannot rebuild cubes

Hello again,

My test is not conclusive, because the time slice I chose was before 2012. 
There is no data before this year. So it is normal.
I did then another test, to get the data between 2010-01-01 and 2014-02-02, but 
I still get the same result : no data available on step 1. I attach my 
kylin.log file.

In my kylin.log file, I can find the following error message while trying to 
create the temp Hive tables :
2018-02-12 14:30:01,574 INFO  [Scheduler 481383589 Job 
8a97abb9-69c0-4484-b2e0-ed801dcb845a-161] hive.CreateFlatHiveTableStep:38 : 
Connecting to jdbc:hive2://cloudera-manager.hades.rte-france.com:10000/
2018-02-12 14:30:01,754 INFO  [Scheduler 481383589 Job 
8a97abb9-69c0-4484-b2e0-ed801dcb845a-161] hive.CreateFlatHiveTableStep:38 : 
Unknown HS2 problem when communicating with Thrift server.
2018-02-12 14:30:01,754 INFO  [Scheduler 481383589 Job 
8a97abb9-69c0-4484-b2e0-ed801dcb845a-161] hive.CreateFlatHiveTableStep:38 : 
Error: Could not open client transport with JDBC Uri: 
jdbc:hive2://cloudera-manager.hades.rte-france.com:10000/: Peer indicated 
failure: Unsupported mechanism type PLAIN (state=08S01,code=0)
2018-02-12 14:30:01,754 INFO  [Scheduler 481383589 Job 
8a97abb9-69c0-4484-b2e0-ed801dcb845a-161] hive.CreateFlatHiveTableStep:38 : 
Beeline version 1.1.0-cdh5.14.0 by Apache Hive
2018-02-12 14:30:11,729 DEBUG 
[localhost-startStop-1-SendThread(node4.hades.rte-france.com:2181)] 
zookeeper.ClientCnxn:717 : Got ping response for sessionid: 0x261895d4a1c0140 
after 0ms

Could you please give me any clue ?

NB : my kylin.properties file remains the same as before (see parameters below).

Thank you for your help.
Have a good day.

Best regards,
Jean-Luc.

De : BELLIER Jean-luc [mailto:jean-luc.bell...@rte-france.com]
Envoyé : lundi 12 février 2018 12:29
À : user@kylin.apache.org<mailto:user@kylin.apache.org>
Objet : RE: Cannot rebuild cubes

Hello Ted

Thank you for your reply. What is the command to check what you are suggesting ?
In the meanwhile, I changed the way accessing to hive to ‘beeline’, and faced 
to write rights problems to HDFS.
This is now solved.

Now I try to build the cube ‘kylin_sales_cube’. All the tables are present in 
Hive, and they contain data. The metadata are also in HBase in ‘kylin_metadata’ 
table.
What I do not understand is that the job seems to run with no data. In the step 
#1, the log tells me ‘No Data Available’

[cid:image001.gif@01D3A4A6.AA66DA30][cid:image002.gif@01D3A4A6.AA66DA30]

Could it be a problem of rights ?

Here is my kylin.properties

## Hive client, valid value [cli, beeline]
kylin.source.hive.client=beeline
#
## Parameters for beeline client, only necessary if hive client is beeline
kylin.source.hive.beeline-params=-n root --hiveconf 
hive.security.authorization.sqlstd.confwhitelist.append='mapreduce.job.*|dfs.*' 
-u 
jdbc:hive2://cloudera-manager.hades.rte-france.com:10000/;principal=hive/cloudera-manager.hades.rte-france....@hades.rte-france.com
#
#kylin.source.hive.keep-flat-table=false
#
## Hive database name for putting the intermediate flat tables
kylin.source.hive.database-for-flat-table=default

The full log files have been attached in this mail (using ‘Diagnosis’ option) 
on the job in Web interface.

PS  : I noticed the presence of file ‘health-check.sh’ in /tmp/kylin on HDFS. 
Is is normal ?

Thank you in advance for your help.

Best regards,
Jean-Luc.

De : Ted Yu [mailto:yuzhih...@gmail.com]
Envoyé : vendredi 9 février 2018 22:28
À : user@kylin.apache.org<mailto:user@kylin.apache.org>
Objet : Re: Cannot rebuild cubes

In the log, I saw:

2018-02-09 17:39:57,634 ERROR [http-bio-7070-exec-8] 
controller.CubeController:337 : Segments overlap: 
kylin_sales_cube[20100101000000_20140102000000] and          
kylin_sales_cube[20120101000000_20140102000000]
java.lang.IllegalStateException: Segments overlap: 
kylin_sales_cube[20100101000000_20140102000000] and 
kylin_sales_cube[20120101000000_20140102000000]
  at org.apache.kylin.metadata.model.Segments.validate(Segments.java:404)
  at org.apache.kylin.cube.CubeManager.updateCubeWithRetry(CubeManager.java:393)
  at org.apache.kylin.cube.CubeManager.updateCube(CubeManager.java:349)
  at org.apache.kylin.cube.CubeManager.appendSegment(CubeManager.java:485)
  at org.apache.kylin.cube.CubeManager.appendSegment(CubeManager.java:455)
  at 
org.apache.kylin.rest.service.JobService.submitJobInternal(JobService.java:239)

It seems there was some wrong segment in metadata.
Have you tried dumping out metadata and find the invalid segment ?

Cheers

On Fri, Feb 9, 2018 at 9:18 AM, BELLIER Jean-luc 
<jean-luc.bell...@rte-france.com<mailto:jean-luc.bell...@rte-france.com>> wrote:
Hello,

I am unable to rebuild cubes. The logs give me the following error :

Exception in thread "Thread-13" java.lang.RuntimeException: 
java.lang.IllegalStateException: Cannot start job scheduler due to lack of job 
lock
        at org.apache.kylin.rest.service.JobService$1.run(JobService.java:131)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: Cannot start job scheduler due to 
lack of job lock
        at 
org.apache.kylin.job.impl.threadpool.DefaultScheduler.init(DefaultScheduler.java:211)
        at org.apache.kylin.rest.service.JobService$1.run(JobService.java:126)
        ... 1 more

As far as I can see on this topic, I understand that there should be a process 
that is still running or was not remove correctly.
I tried several ‘kylin.sh stop’ / ‘kylin.sh start’ with no effect.

I attach my kylin.out and kylin.log files.

Thank you in advance for your help.

Best regards,
Jean-Luc.


"Ce message est destiné exclusivement aux personnes ou entités auxquelles il 
est adressé et peut contenir des informations privilégiées ou confidentielles. 
Si vous avez reçu ce document par erreur, merci de nous l'indiquer par retour, 
de ne pas le transmettre et de procéder à sa destruction.

This message is solely intended for the use of the individual or entity to 
which it is addressed and may contain information that is privileged or 
confidential. If you have received this communication by error, please notify 
us immediately by electronic mail, do not disclose it and delete the original 
message."



"Ce message est destiné exclusivement aux personnes ou entités auxquelles il 
est adressé et peut contenir des informations privilégiées ou confidentielles. 
Si vous avez reçu ce document par erreur, merci de nous l'indiquer par retour, 
de ne pas le transmettre et de procéder à sa destruction.

This message is solely intended for the use of the individual or entity to 
which it is addressed and may contain information that is privileged or 
confidential. If you have received this communication by error, please notify 
us immediately by electronic mail, do not disclose it and delete the original 
message."


"Ce message est destiné exclusivement aux personnes ou entités auxquelles il 
est adressé et peut contenir des informations privilégiées ou confidentielles. 
Si vous avez reçu ce document par erreur, merci de nous l'indiquer par retour, 
de ne pas le transmettre et de procéder à sa destruction.

This message is solely intended for the use of the individual or entity to 
which it is addressed and may contain information that is privileged or 
confidential. If you have received this communication by error, please notify 
us immediately by electronic mail, do not disclose it and delete the original 
message."

Reply via email to