Hello ,
1. I did the following (in this order) : · Stop the kylin process (kylin.sh stop) · Rename the kylin.log and kylin.out files in $KYLIN_HOME/logs · Checked the environment (check-env.sh) · Start the kylin process (kylin.sh start) So I get new kylin.out and kylin.log files. Apparentlty they seem to be OK. The version is the http://apache.mindstudios.com/kylin/apache-kylin-2.2.0/apache-kylin-2.2.0-bin-cdh57.tar.gz. 2. I did another test : I cleaned everything, changed my version of Kylin (http://mirror.ibcp.fr/pub/apache/kylin/apache-kylin-2.2.0/apache-kylin-2.2.0-bin-hbase1x.tar.gz instead of http://apache.mindstudios.com/kylin/apache-kylin-2.2.0/apache-kylin-2.2.0-bin-cdh57.tar.gz), put it in a separate directory and changed the value of $KYLIN_HOME. I got strange messages like this one : 2018-02-06 15:24:49,857 INFO [main] Configuration.deprecation:1175 : hadoop.native.lib is deprecated. Instead, use io.native.lib.available Exception in thread "main" java.lang.IllegalArgumentException: Failed to find metadata store by url: kylin_metadata@hbase at org.apache.kylin.common.persistence.ResourceStore.createResourceStore(ResourceStore.java:89) at org.apache.kylin.common.persistence.ResourceStore.getStore(ResourceStore.java:101) at org.apache.kylin.common.persistence.ResourceTool.copy(ResourceTool.java:165) at org.apache.kylin.common.persistence.ResourceTool.copy(ResourceTool.java:196) at org.apache.kylin.common.persistence.ResourceTool.copy(ResourceTool.java:190) at org.apache.kylin.common.persistence.ResourceTool.main(ResourceTool.java:84) Caused by: java.lang.NoSuchMethodError: org.apache.hadoop.hbase.client.Get.setCheckExistenceOnly(Z)Lorg/apache/hadoop/hbase/client/Get; at org.apache.kylin.storage.hbase.HBaseResourceStore.internalGetFromHTable(HBaseResourceStore.java:377) at org.apache.kylin.storage.hbase.HBaseResourceStore.getFromHTable(HBaseResourceStore.java:363) at org.apache.kylin.storage.hbase.HBaseResourceStore.existsImpl(HBaseResourceStore.java:116) at org.apache.kylin.common.persistence.ResourceStore.exists(ResourceStore.java:144) at org.apache.kylin.common.persistence.ResourceStore.createResourceStore(ResourceStore.java:84) ... 5 more 2018-02-06 15:24:50,039 INFO [Thread-1] client.ConnectionManager$HConnectionImplementation:2259 : Closing master protocol: MasterService 2018-02-06 15:24:50,046 INFO [Thread-1] client.ConnectionManager$HConnectionImplementation:1830 : Closing zookeeper sessionid=0x36165799fcb08ad 2018-02-06 15:24:50,049 INFO [Thread-1] zookeeper.ZooKeeper:684 : Session: 0x36165799fcb08ad closed 2018-02-06 15:24:50,049 INFO [main-EventThread] zookeeper.ClientCnxn:512 : EventThread shut down However, I can create tables in HBase, so it is not a problem of rights. So I came back to my first version of Kylin. All the folders and bases are back to their state. In the Web interface, the models are still loading indefinitely, but I could not also see any Hive table, and cannot load them. Could there be the cause of the problem ? Thank you in advance for your help. Best regards, Jean-Luc. De : ShaoFeng Shi [mailto:[email protected]] Envoyé : mardi 6 février 2018 13:30 À : user <[email protected]> Objet : Re: Problem starting Kylin Web interface All looks good; Can't believe that it can not create the folder on HDFS. Please check more. 2018-02-06 18:58 GMT+08:00 BELLIER Jean-luc <[email protected]<mailto:[email protected]>>: Hello, In the /etc/hadoop/conf folder, I can see under the fs.defaultFS property the following value : hdfs://<cloudera-manager_name>:8020. The file indicates that the content was automatically generated by Cloudera Manager. Are there other points to check in this file ? According to the trace of $KYLIN_HOME/bin/find-hadoop-conf-dir.sh –v below : kylin_hadoop_conf_dir is /etc/hadoop/conf So I presume it is correct. The file called ‘yarn-site.xml’ is also in this folder. I hope this helps. Best regards, Jean-Luc. De : ShaoFeng Shi [mailto:[email protected]<mailto:[email protected]>] Envoyé : mardi 6 février 2018 11:39 À : user <[email protected]<mailto:[email protected]>> Objet : Re: Problem starting Kylin Web interface Kylin has located the Hadoop config folder as "/etc/hadoop/conf", is this the right location? Please check whether the core-site.xml under this folder has the proper configuration, for example, 'fs.defaultFS' points to the HDFS. 2018-02-06 16:58 GMT+08:00 BELLIER Jean-luc <[email protected]<mailto:[email protected]>>: Hello, I have cleaned all then environments (HBase, Hive) and restarted the sample.sh script again. In the output messages, I can see several times the following : 2018-02-06 09:43:21,034 INFO [main] util.ZookeeperDistributedLock:154 : [email protected]<mailto:[email protected]> acquired lock at /kylin/kylin_metadata/create_htable/kylin_metadata/lock 2018-02-06 09:43:21,050 DEBUG [main] hbase.HBaseConnection:309 : Creating HTable 'kylin_metadata' 2018-02-06 09:43:21,083 INFO [main] Configuration.deprecation:1175 : hadoop.native.lib is deprecated. Instead, use io.native.lib.available However, my HBase table (‘kylin_metadata’) is created and contains 20 rows. I suppose it is correct. I have a question : in the KYLIN Interface, where are the models loaded from ? I could help to understand the problem. Thank you again for your help. Have a good ay. Best regards, Jean-Luc. De : BELLIER Jean-luc Envoyé : mardi 6 février 2018 09:12 À : user <[email protected]<mailto:[email protected]>> Objet : RE: Problem starting Kylin Web interface Hello, I thank you for your feedback and the information about the kylin.properties file, which gives lots of information. To answer your question, here is the result of the ./bin/find-hadoop-conf-dir.sh –v command bellierjlu@node2:~$ $KYLIN_HOME/bin/find-hadoop-conf-dir.sh -v Turn on verbose mode. Retrieving hadoop conf dir... /opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hbase/bin/../conf is not valid hadoop dir conf because yarn-site.xml is missing /opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hbase/bin/.. is not valid hadoop dir conf because yarn-site.xml is missing kylin_hadoop_conf_dir is /etc/hadoop/conf bellierjlu@node2:~$ Could this be the cause of the problem ? Thank you in advance for your help. Best regards, Jean-Luc. De : ShaoFeng Shi [mailto:[email protected]] Envoyé : mardi 6 février 2018 01:45 À : user <[email protected]<mailto:[email protected]>> Objet : Re: Problem starting Kylin Web interface Hi Jean-Luc, Kylin's metadata is persisted in HBase, the metadata table is "kylin_metadata" by default, which is also configurable in conf/kylin.properties with key kylin.metadata.url; The HDFS path '/kylin/kylin_metadata' is for the cube and other files; Before you build the first cube, the folder is empty I think. If sample.sh couldn't create the /tmp/kylin folder on HDFS; If the folder wasn't create, Kylin might not find the Hadoop config properly. You can run: ./bin/find-hadoop-conf-dir.sh -v to see what it prints. 2018-02-05 23:24 GMT+08:00 BELLIER Jean-luc <[email protected]<mailto:[email protected]>>: Hello, I am sorry to come back again to you, but I am really stuck. I restarted all the services and agents on my Cloudera cluster, so it is OK now. I noticed that some HDFS directories were empty, especially /tmp/kylin. When I have a look on the code of sample.sh, I can see that this script should create sthe subfolder /tmp/kylin/sample_cube/data. These data are used to create the Hive tables and are removed after the creation of these tables. Let call the node N1 I have another machine on which Kylin is installed and I succeeded in launching sample.sh. On this machine, I can see a HDFS folder called /kylin/kylin_metadata with, I presume, the metadata of the cube stored in it. I cannot succeed in creating this folder on N1. I removed the /tmp/kylin folder, the hbase table and the Hive tables. I really do not know what to do to restart the creation of the project from scratch. I also checked the HDFS space using the command ‘hdfs dfs –df’ and it tells me a usage of 2%. Could you please guide me in how to achieve it. There may be something I did wrong, but I really do not what. Is my way of doing correct ? Are there other files to remove (temporary files, for example), and in this case, where are they located ? Any help would be greatly appreciated. Thank you in advance for your help. Have a good day. Best regards, Jean-Luc. De : BELLIER Jean-luc Envoyé : vendredi 2 février 2018 18:54 À : user <[email protected]<mailto:[email protected]>> Objet : RE: Problem starting Kylin Web interface Hello, Thanks for the suggestion. I tried on another machine with Firefox and Chrome, but I get the same result as before. I also rebuilt the project ‘sample’ from scratch, with no change. My cluster seems to have problems on its nodes, especially memory which is nearly full on most nodes. I suspect this could be one of the cause of the problem : no available memory and swap space also nearly full. What do you think of this assumption ? I will see that with my Cloudera manager next week, as I am not a specialist at all. Have a good week-end. Best regards, Jean-Luc. De : ShaoFeng Shi [mailto:[email protected]] Envoyé : vendredi 2 février 2018 14:59 À : user <[email protected]<mailto:[email protected]>> Objet : Re: Problem starting Kylin Web interface Please try Chrome; Most developers are using Chrome; And if there is an error, you can press F12 to check the HTTP request/response, there should be more information. 2018-02-02 21:54 GMT+08:00 BELLIER Jean-luc <[email protected]<mailto:[email protected]>>: Hello again, I forgot to mention that I use IE or Secured Firefox as Web browsers. I have the same issue with both. Could there be the cause of the problem ? Best regards Jean-Luc. De : BELLIER Jean-luc Envoyé : vendredi 2 février 2018 14:48 À : user <[email protected]<mailto:[email protected]>> Objet : RE: Problem starting Kylin Web interface Hello Thank you for your answer. I did the following : • rename my current kylin.log and kylin.out files to keep a trace • restarted the commands ‘kylin.sh start’ and ‘sample.sh’ I cannot see anything special in the new .log or .out files. The Hive tables exist, and the kylin_metadata also. I can scan it through Hbase shell command. It gives me 44 rows. However, the problem is still the same with the Kylin Interface : Models loading indefinitely, no table displayed, and nothing in the ‘System’ section. When I try to refresh the server config and the server environement, I still have the message ‘Oops ! Failed to take action. I also have this message as soon as I click to the ‘System’ button. Should I rebuild the project from scratch to be sure that everything is correctly created, or should I investigate somewhere else (services, node states, …) ? I send in attachement the now kylin.log and kylin.out files Thank you in advance for your help. Have a good day. Best regards, Jean-Luc. De : ShaoFeng Shi [mailto:[email protected]] Envoyé : vendredi 2 février 2018 12:06 À : user <[email protected]<mailto:[email protected]>> Objet : Re: Problem starting Kylin Web interface Hi Jean-luc, The kylin.log seems okay, the service is started and the job scheduler is running up. The kylin.out has some error about missing ".keystore", but by default if you accessing kylin via http, it shouldn't need to load keystore. Please try visit http://host:7070/kylin again to see whether there is any issue in the web console. 2018-02-01 23:17 GMT+08:00 BELLIER Jean-luc <[email protected]<mailto:[email protected]>>: Hello, I am new in Kylin. I would like to test it for a POC for my company. I opened a ticket yesterday in the issues mail box through Jira, but I am not sure it is the best place. Here is the reference of the ticket : https://issues.apache.org/jira/browse/KYLIN-3218 I am using the version apache-kylin-2.2.0-bin-cdh57, with Linux. I installed the software on a client node. I successfully started the process and created the cube given in the example project. Apparently, no mistake appears when starting the kylin.sh shell or generating the cube using sample.sh. When I launch the web interface using <nodename>:7070/kylin, I cannot load the models (the cursor is turning indefinitely). By looking at the kylin.out log, I could see that the process could not find the $KYLIN_HOME/tomcat/conf/.keystore file. So I created it manually, and relaunched the cube generation and the ‘kylin.sh start’ command. Now I face to other mistakes : • Exception in thread "Thread-14" java.lang.RuntimeException: Error while peeking at /kylin/kylin_metadata/job_engine/global_job_engine_lock • SEVERE: The web application [/kylin] appears to have started a thread named [TGT Renewer for [email protected]<mailto:[email protected]>] but has failed to stop it. This is very likely to create a memory leak. This one appears several times • SEVERE: The web application [/kylin] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@10cb8259]) and a value of type [org.apache.kylin.rest.msg.Message] (value [org.apache.kylin.rest.msg.Message@267ae210]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak. This one appears several times Apparently, something else is missing, or I did something wrong, but I do not know what. I would also like to know what will be the impact of the two ‘SEVERE’ issues above and how I could correct it. I attach the last version of kylin.out file. Any help would be kindly appreciated. Thank you in advance for your help. Have a good day. Best regards. Jean-Luc. "Ce message est destiné exclusivement aux personnes ou entités auxquelles il est adressé et peut contenir des informations privilégiées ou confidentielles. Si vous avez reçu ce document par erreur, merci de nous l'indiquer par retour, de ne pas le transmettre et de procéder à sa destruction. This message is solely intended for the use of the individual or entity to which it is addressed and may contain information that is privileged or confidential. If you have received this communication by error, please notify us immediately by electronic mail, do not disclose it and delete the original message." -- Best regards, Shaofeng Shi 史少锋 "Ce message est destiné exclusivement aux personnes ou entités auxquelles il est adressé et peut contenir des informations privilégiées ou confidentielles. Si vous avez reçu ce document par erreur, merci de nous l'indiquer par retour, de ne pas le transmettre et de procéder à sa destruction. This message is solely intended for the use of the individual or entity to which it is addressed and may contain information that is privileged or confidential. If you have received this communication by error, please notify us immediately by electronic mail, do not disclose it and delete the original message." -- Best regards, Shaofeng Shi 史少锋 "Ce message est destiné exclusivement aux personnes ou entités auxquelles il est adressé et peut contenir des informations privilégiées ou confidentielles. Si vous avez reçu ce document par erreur, merci de nous l'indiquer par retour, de ne pas le transmettre et de procéder à sa destruction. This message is solely intended for the use of the individual or entity to which it is addressed and may contain information that is privileged or confidential. If you have received this communication by error, please notify us immediately by electronic mail, do not disclose it and delete the original message." -- Best regards, Shaofeng Shi 史少锋 "Ce message est destiné exclusivement aux personnes ou entités auxquelles il est adressé et peut contenir des informations privilégiées ou confidentielles. Si vous avez reçu ce document par erreur, merci de nous l'indiquer par retour, de ne pas le transmettre et de procéder à sa destruction. This message is solely intended for the use of the individual or entity to which it is addressed and may contain information that is privileged or confidential. If you have received this communication by error, please notify us immediately by electronic mail, do not disclose it and delete the original message." -- Best regards, Shaofeng Shi 史少锋 "Ce message est destiné exclusivement aux personnes ou entités auxquelles il est adressé et peut contenir des informations privilégiées ou confidentielles. Si vous avez reçu ce document par erreur, merci de nous l'indiquer par retour, de ne pas le transmettre et de procéder à sa destruction. This message is solely intended for the use of the individual or entity to which it is addressed and may contain information that is privileged or confidential. If you have received this communication by error, please notify us immediately by electronic mail, do not disclose it and delete the original message." -- Best regards, Shaofeng Shi 史少锋 "Ce message est destiné exclusivement aux personnes ou entités auxquelles il est adressé et peut contenir des informations privilégiées ou confidentielles. Si vous avez reçu ce document par erreur, merci de nous l'indiquer par retour, de ne pas le transmettre et de procéder à sa destruction. This message is solely intended for the use of the individual or entity to which it is addressed and may contain information that is privileged or confidential. If you have received this communication by error, please notify us immediately by electronic mail, do not disclose it and delete the original message."
