Hello again Ted, I did the following :
· launched the command ‘beeline’ · launched the command (under beeline) : !connect jdbc:hive2://<host>:<port>/<db>;principal=<Server_Principal_of_HiveServer2> And I got the following message : Driver: Hive JDBC (version 1.1.0-cdh5.14.0) Transaction isolation: TRANSACTION_REPEATABLE_READ Could this be the source of the problem, and, if yes, how to correct it ? Thank you for your help. Best regards, Jean-Luc. De : BELLIER Jean-luc Envoyé : lundi 12 février 2018 14:39 À : user@kylin.apache.org Objet : RE: Cannot rebuild cubes Hello again, My test is not conclusive, because the time slice I chose was before 2012. There is no data before this year. So it is normal. I did then another test, to get the data between 2010-01-01 and 2014-02-02, but I still get the same result : no data available on step 1. I attach my kylin.log file. In my kylin.log file, I can find the following error message while trying to create the temp Hive tables : 2018-02-12 14:30:01,574 INFO [Scheduler 481383589 Job 8a97abb9-69c0-4484-b2e0-ed801dcb845a-161] hive.CreateFlatHiveTableStep:38 : Connecting to jdbc:hive2://cloudera-manager.hades.rte-france.com:10000/ 2018-02-12 14:30:01,754 INFO [Scheduler 481383589 Job 8a97abb9-69c0-4484-b2e0-ed801dcb845a-161] hive.CreateFlatHiveTableStep:38 : Unknown HS2 problem when communicating with Thrift server. 2018-02-12 14:30:01,754 INFO [Scheduler 481383589 Job 8a97abb9-69c0-4484-b2e0-ed801dcb845a-161] hive.CreateFlatHiveTableStep:38 : Error: Could not open client transport with JDBC Uri: jdbc:hive2://cloudera-manager.hades.rte-france.com:10000/: Peer indicated failure: Unsupported mechanism type PLAIN (state=08S01,code=0) 2018-02-12 14:30:01,754 INFO [Scheduler 481383589 Job 8a97abb9-69c0-4484-b2e0-ed801dcb845a-161] hive.CreateFlatHiveTableStep:38 : Beeline version 1.1.0-cdh5.14.0 by Apache Hive 2018-02-12 14:30:11,729 DEBUG [localhost-startStop-1-SendThread(node4.hades.rte-france.com:2181)] zookeeper.ClientCnxn:717 : Got ping response for sessionid: 0x261895d4a1c0140 after 0ms Could you please give me any clue ? NB : my kylin.properties file remains the same as before (see parameters below). Thank you for your help. Have a good day. Best regards, Jean-Luc. De : BELLIER Jean-luc [mailto:jean-luc.bell...@rte-france.com] Envoyé : lundi 12 février 2018 12:29 À : user@kylin.apache.org<mailto:user@kylin.apache.org> Objet : RE: Cannot rebuild cubes Hello Ted Thank you for your reply. What is the command to check what you are suggesting ? In the meanwhile, I changed the way accessing to hive to ‘beeline’, and faced to write rights problems to HDFS. This is now solved. Now I try to build the cube ‘kylin_sales_cube’. All the tables are present in Hive, and they contain data. The metadata are also in HBase in ‘kylin_metadata’ table. What I do not understand is that the job seems to run with no data. In the step #1, the log tells me ‘No Data Available’ [cid:image001.gif@01D3A415.024BD8F0][cid:image002.gif@01D3A415.024BD8F0] Could it be a problem of rights ? Here is my kylin.properties ## Hive client, valid value [cli, beeline] kylin.source.hive.client=beeline # ## Parameters for beeline client, only necessary if hive client is beeline kylin.source.hive.beeline-params=-n root --hiveconf hive.security.authorization.sqlstd.confwhitelist.append='mapreduce.job.*|dfs.*' -u jdbc:hive2://cloudera-manager.hades.rte-france.com:10000/;principal=hive/cloudera-manager.hades.rte-france....@hades.rte-france.com # #kylin.source.hive.keep-flat-table=false # ## Hive database name for putting the intermediate flat tables kylin.source.hive.database-for-flat-table=default The full log files have been attached in this mail (using ‘Diagnosis’ option) on the job in Web interface. PS : I noticed the presence of file ‘health-check.sh’ in /tmp/kylin on HDFS. Is is normal ? Thank you in advance for your help. Best regards, Jean-Luc. De : Ted Yu [mailto:yuzhih...@gmail.com] Envoyé : vendredi 9 février 2018 22:28 À : user@kylin.apache.org<mailto:user@kylin.apache.org> Objet : Re: Cannot rebuild cubes In the log, I saw: 2018-02-09 17:39:57,634 ERROR [http-bio-7070-exec-8] controller.CubeController:337 : Segments overlap: kylin_sales_cube[20100101000000_20140102000000] and kylin_sales_cube[20120101000000_20140102000000] java.lang.IllegalStateException: Segments overlap: kylin_sales_cube[20100101000000_20140102000000] and kylin_sales_cube[20120101000000_20140102000000] at org.apache.kylin.metadata.model.Segments.validate(Segments.java:404) at org.apache.kylin.cube.CubeManager.updateCubeWithRetry(CubeManager.java:393) at org.apache.kylin.cube.CubeManager.updateCube(CubeManager.java:349) at org.apache.kylin.cube.CubeManager.appendSegment(CubeManager.java:485) at org.apache.kylin.cube.CubeManager.appendSegment(CubeManager.java:455) at org.apache.kylin.rest.service.JobService.submitJobInternal(JobService.java:239) It seems there was some wrong segment in metadata. Have you tried dumping out metadata and find the invalid segment ? Cheers On Fri, Feb 9, 2018 at 9:18 AM, BELLIER Jean-luc <jean-luc.bell...@rte-france.com<mailto:jean-luc.bell...@rte-france.com>> wrote: Hello, I am unable to rebuild cubes. The logs give me the following error : Exception in thread "Thread-13" java.lang.RuntimeException: java.lang.IllegalStateException: Cannot start job scheduler due to lack of job lock at org.apache.kylin.rest.service.JobService$1.run(JobService.java:131) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.IllegalStateException: Cannot start job scheduler due to lack of job lock at org.apache.kylin.job.impl.threadpool.DefaultScheduler.init(DefaultScheduler.java:211) at org.apache.kylin.rest.service.JobService$1.run(JobService.java:126) ... 1 more As far as I can see on this topic, I understand that there should be a process that is still running or was not remove correctly. I tried several ‘kylin.sh stop’ / ‘kylin.sh start’ with no effect. I attach my kylin.out and kylin.log files. Thank you in advance for your help. Best regards, Jean-Luc. "Ce message est destiné exclusivement aux personnes ou entités auxquelles il est adressé et peut contenir des informations privilégiées ou confidentielles. Si vous avez reçu ce document par erreur, merci de nous l'indiquer par retour, de ne pas le transmettre et de procéder à sa destruction. This message is solely intended for the use of the individual or entity to which it is addressed and may contain information that is privileged or confidential. If you have received this communication by error, please notify us immediately by electronic mail, do not disclose it and delete the original message." "Ce message est destiné exclusivement aux personnes ou entités auxquelles il est adressé et peut contenir des informations privilégiées ou confidentielles. Si vous avez reçu ce document par erreur, merci de nous l'indiquer par retour, de ne pas le transmettre et de procéder à sa destruction. This message is solely intended for the use of the individual or entity to which it is addressed and may contain information that is privileged or confidential. If you have received this communication by error, please notify us immediately by electronic mail, do not disclose it and delete the original message." "Ce message est destiné exclusivement aux personnes ou entités auxquelles il est adressé et peut contenir des informations privilégiées ou confidentielles. Si vous avez reçu ce document par erreur, merci de nous l'indiquer par retour, de ne pas le transmettre et de procéder à sa destruction. This message is solely intended for the use of the individual or entity to which it is addressed and may contain information that is privileged or confidential. If you have received this communication by error, please notify us immediately by electronic mail, do not disclose it and delete the original message."