Hi Jean-Luc, thanks for the update, and hope you can enjoy Kylin! 2018-02-10 1:11 GMT+08:00 BELLIER Jean-luc <jean-luc.bell...@rte-france.com> :
> Hello Shaofeng, > > > > At this time, I can now see both models and cubes in Kylin Web interface. > In fact, there were mistakes that I discovered using the –v option when > launching the kylin.sh start command. > > In particular, there were some conflicts with functions that were > identical in jar files, but with different signatures. > > I also discovered that some files were not looked in the rights place, so > were identified as missing. > > > > What I did is the following : > > · I downloaded the configuration files from HBase, Hive and > Hadoop and put them all in a specific folder (called F). > > · I defined the env variable HADOOP_CONF_DIR and HIVE_CONF to > target to this folder F > > · I changed in my kylin.properties files the kylin.source.hive.client > parameter to ‘beeline’ (that is what we use in fact), and adjusted the > kylin.source.hive.beeline-params parameter to fit to my environment > > · I launched setenv.sh then kylin.sh start > > > > Best regards, > > Jean-Luc. > > > > *De :* ShaoFeng Shi [mailto:shaofeng...@apache.org] > *Envoyé :* vendredi 9 février 2018 04:04 > *À :* user <user@kylin.apache.org> > *Objet :* Re: Problem starting Kylin Web interface > > > > Today another user reports that: > > > > "deploy kylin on aliyun， > web console net::ERR_CONTENT_DECODING_FAILED > and when tomcat server.xml compress=on. > so I disable compress ,then it's ok to use" > > > > Please manually edit $KYLIN_HOME/tomcat/conf/server.xml, find the > "compress=on", change it to off and try again. > > > > > > 2018-02-08 22:26 GMT+08:00 ShaoFeng Shi <shaofeng...@apache.org>: > > It looks like the Chrome couldn't load some config data from Kylin > backend; the "*failed to load kylin.properties" is an error from > kylin's javascript. But the root cause wasn't printed.* > > > > *https://github.com/apache/kylin/blob/v2.2.0-release/webapp/app/js/services/kylinProperties.js#L29 > <https://github.com/apache/kylin/blob/v2.2.0-release/webapp/app/js/services/kylinProperties.js#L29>* > > > > *I'm not sure whether it is a bug of Kylin 2.2 for CDH; Or don't mind, > could you please download a previous version like Kylin 2.1 for CDH? Can be > found in:* > > *https://archive.apache.org/dist/kylin/apache-kylin-2.1.0/ > <https://archive.apache.org/dist/kylin/apache-kylin-2.1.0/>* > > > > 2018-02-08 18:51 GMT+08:00 BELLIER Jean-luc <jean-luc.bellier@rte-france. > com>: > > Hello Shaofeng, > > > > > > I checked again on Chrome but I can only do it via a remote desktop > connection, as I am not administrator of my machine. I discovered this > message : > > > > *Failed to load resource: net::ERR_CONTENT_DECODING_FAILED* > > *scripts.min.20171030211457.js:10159 failed to load > kylin.properties[object Object]* > > *(anonymous) @ scripts.min.20171030211457.js:10159* > > *login:1 This page includes a password or credit card input in a > non-secure context. A warning has been added to the URL bar. For more > information, see https://goo.gl/zmWq3m <https://goo.gl/zmWq3m>.* > > *scripts.min.20171030211457.js:51857 num root branches = 0* > > *api/models Failed to load resource: net::ERR_CONTENT_DECODING_FAILED* > > > > May be this could give a clue on the problem. > > > > Could you please guide me in how to solve it ? > > > > Thank you in advance for your help. > > > > Best regards, > > Jean-Luc. > > > > *De :* BELLIER Jean-luc > *Envoyé :* mercredi 7 février 2018 19:00 > *À :* 'user@kylin.apache.org' <user@kylin.apache.org> > *Objet :* RE: Problem starting Kylin Web interface > > > > Hello, > > > > I notice that I did not receive any answer for the following error that I > get in the kylin.out file, and that is mentioned at the bottom of this mail > : > > > > *SEVERE: The web application [/kylin] appears to have started a thread > named [TGT Renewer for bellier...@hades.rte-france.com > <bellier...@hades.rte-france.com>] but has failed to stop it. This is very > likely to create a memory leak.* This one appears several times > > · *SEVERE: The web application [/kylin] created a ThreadLocal > with key of type [java.lang.ThreadLocal] (value > [java.lang.ThreadLocal@10cb8259]) and a value of type > [org.apache.kylin.rest.msg.Message] (value > [org.apache.kylin.rest.msg.Message@267ae210]) but failed to remove it when > the web application was stopped. Threads are going to be renewed over time > to try and avoid a probable memory leak. *This one appears several times > > > > Is there a way to solve this ? Could it be the cause of the problem ? You > will find the result of the ‘jmap tomcat histo’ command in attachment. > > > > Thank your for your help. > > > > Best regards, > > Jean-Luc > > > > *De :* BELLIER Jean-luc > *Envoyé :* mardi 6 février 2018 17:12 > *À :* 'user@kylin.apache.org' <user@kylin.apache.org> > *Objet :* RE: Problem starting Kylin Web interface > > > > May be the fact that I cannot load Hive tables is due to the indefinite > load of the models. Is there a workaround to see if I can see the tables > without loading the models ? > > > > Thank you again for your help. > > > > Best regards, > > Jean-Luc. > > > > *De :* BELLIER Jean-luc > *Envoyé :* mardi 6 février 2018 16:34 > *À :* user <user@kylin.apache.org> > *Objet :* RE: Problem starting Kylin Web interface > > > > > > Hello , > > > > 1. I did the following (in this order) : > > > > · Stop the kylin process (kylin.sh stop) > > · Rename the kylin.log and kylin.out files in $KYLIN_HOME/logs > > · Checked the environment (check-env.sh) > > · Start the kylin process (kylin.sh start) > > > > So I get new kylin.out and kylin.log files. Apparentlty they seem to be > OK. The version is the http://apache.mindstudios.com/ > kylin/apache-kylin-2.2.0/apache-kylin-2.2.0-bin-cdh57.tar.gz. > > > > 2. I did another test : > > I cleaned everything, changed my version of Kylin ( > http://mirror.ibcp.fr/pub/apache/kylin/apache-kylin-2.2. > 0/apache-kylin-2.2.0-bin-hbase1x.tar.gz instead of > http://apache.mindstudios.com/kylin/apache-kylin-2.2.0/ > apache-kylin-2.2.0-bin-cdh57.tar.gz), put it in a separate directory and > changed the value of $KYLIN_HOME. > > > > I got strange messages like this one : > > 2018-02-06 15:24:49,857 INFO [main] Configuration.deprecation:1175 : > hadoop.native.lib is deprecated. Instead, use io.native.lib.available > > Exception in thread "main" java.lang.IllegalArgumentException: Failed to > find metadata store by url: kylin_metadata@hbase > > at org.apache.kylin.common.persistence.ResourceStore. > createResourceStore(ResourceStore.java:89) > > at org.apache.kylin.common.persistence.ResourceStore. > getStore(ResourceStore.java:101) > > at org.apache.kylin.common.persistence.ResourceTool.copy( > ResourceTool.java:165) > > at org.apache.kylin.common.persistence.ResourceTool.copy( > ResourceTool.java:196) > > at org.apache.kylin.common.persistence.ResourceTool.copy( > ResourceTool.java:190) > > at org.apache.kylin.common.persistence.ResourceTool.main( > ResourceTool.java:84) > > Caused by: java.lang.NoSuchMethodError: org.apache.hadoop.hbase. > client.Get.setCheckExistenceOnly(Z)Lorg/apache/hadoop/hbase/client/Get; > > at org.apache.kylin.storage.hbase.HBaseResourceStore. > internalGetFromHTable(HBaseResourceStore.java:377) > > at org.apache.kylin.storage.hbase.HBaseResourceStore. > getFromHTable(HBaseResourceStore.java:363) > > at org.apache.kylin.storage.hbase.HBaseResourceStore. > existsImpl(HBaseResourceStore.java:116) > > at org.apache.kylin.common.persistence.ResourceStore. > exists(ResourceStore.java:144) > > at org.apache.kylin.common.persistence.ResourceStore. > createResourceStore(ResourceStore.java:84) > > ... 5 more > > 2018-02-06 15:24:50,039 INFO [Thread-1] > client.ConnectionManager$HConnectionImplementation:2259 > : Closing master protocol: MasterService > > 2018-02-06 15:24:50,046 INFO [Thread-1] > client.ConnectionManager$HConnectionImplementation:1830 > : Closing zookeeper sessionid=0x36165799fcb08ad > > 2018-02-06 15:24:50,049 INFO [Thread-1] zookeeper.ZooKeeper:684 : > Session: 0x36165799fcb08ad closed > > 2018-02-06 15:24:50,049 INFO [main-EventThread] zookeeper.ClientCnxn:512 > : EventThread shut down > > > > However, I can create tables in HBase, so it is not a problem of rights. > > > > So I came back to my first version of Kylin. All the folders and bases are > back to their state. In the Web interface, the models are still loading > indefinitely, but I could not also see any Hive table, and cannot load > them. > > Could there be the cause of the problem ? > > > > Thank you in advance for your help. > > > > Best regards, > > Jean-Luc. > > > > > > *De :* ShaoFeng Shi [mailto:shaofeng...@apache.org > <shaofeng...@apache.org>] > *Envoyé :* mardi 6 février 2018 13:30 > > > *À :* user <user@kylin.apache.org> > *Objet :* Re: Problem starting Kylin Web interface > > > > All looks good; Can't believe that it can not create the folder on HDFS. > Please check more. > > > > 2018-02-06 18:58 GMT+08:00 BELLIER Jean-luc <jean-luc.bellier@rte-france. > com>: > > Hello, > > > > In the /etc/hadoop/conf folder, I can see under the fs.defaultFS property > the following value : hdfs://<cloudera-manager_name>:8020. The file > indicates that the content was automatically generated by Cloudera Manager. > Are there other points to check in this file ? > > > > According to the trace of $KYLIN_HOME/bin/find-hadoop-conf-dir.sh –v > below : > > *kylin_hadoop_conf_dir is /etc/hadoop/conf* > > So I presume it is correct. The file called ‘yarn-site.xml’ is also in > this folder. > > > > I hope this helps. > > > > Best regards, > > Jean-Luc. > > > > *De :* ShaoFeng Shi [mailto:shaofeng...@apache.org] > *Envoyé :* mardi 6 février 2018 11:39 > > > *À :* user <user@kylin.apache.org> > *Objet :* Re: Problem starting Kylin Web interface > > > > Kylin has located the Hadoop config folder as "/etc/hadoop/conf", is this > the right location? Please check whether the core-site.xml under this > folder has the proper configuration, for example, 'fs.defaultFS' points to > the HDFS. > > > > 2018-02-06 16:58 GMT+08:00 BELLIER Jean-luc <jean-luc.bellier@rte-france. > com>: > > Hello, > > > > I have cleaned all then environments (HBase, Hive) and restarted the > sample.sh script again. In the output messages, I can see several times the > following : > > > > 2018-02-06 09:43:21,034 INFO [main] util.ZookeeperDistributedLock:154 *: > 4...@node2.hades.rte-france.com <4...@node2.hades.rte-france.com> acquired > lock at /kylin/kylin_metadata/create_htable/kylin_metadata/lock* > > 2018-02-06 09:43:21,050 DEBUG [main] hbase.HBaseConnection:309 : Creating > HTable 'kylin_metadata' > > 2018-02-06 09:43:21,083 INFO [main] Configuration.deprecation:1175 : > hadoop.native.lib is deprecated. Instead, use io.native.lib.available > > > > However, my HBase table (‘kylin_metadata’) is created and contains 20 > rows. I suppose it is correct. > > > > I have a question : in the KYLIN Interface, where are the models loaded > from ? I could help to understand the problem. > > > > Thank you again for your help. Have a good ay. > > > > Best regards, > > Jean-Luc. > > > > > > *De :* BELLIER Jean-luc > *Envoyé :* mardi 6 février 2018 09:12 > *À :* user <user@kylin.apache.org> > *Objet :* RE: Problem starting Kylin Web interface > > > > Hello, > > > > I thank you for your feedback and the information about the > kylin.properties file, which gives lots of information. > > > > To answer your question, here is the result of the > ./bin/find-hadoop-conf-dir.sh –v command > > > > bellierjlu@node2:~$ $KYLIN_HOME/bin/find-hadoop-conf-dir.sh -v > > Turn on verbose mode. > > Retrieving hadoop conf dir... > > */opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hbase/bin/../conf > is not valid hadoop dir conf because yarn-site.xml is missing* > > */opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hbase/bin/.. is > not valid hadoop dir conf because yarn-site.xml is missing* > > *kylin_hadoop_conf_dir is /etc/hadoop/conf* > > bellierjlu@node2:~$ > > > > Could this be the cause of the problem ? > > > > Thank you in advance for your help. > > > > Best regards, > > Jean-Luc. > > > > *De :* ShaoFeng Shi [mailto:shaofeng...@apache.org > <shaofeng...@apache.org>] > *Envoyé :* mardi 6 février 2018 01:45 > > *À :* user <user@kylin.apache.org> > *Objet :* Re: Problem starting Kylin Web interface > > > > Hi Jean-Luc, > > > > Kylin's metadata is persisted in HBase, the metadata table is > "kylin_metadata" by default, which is also configurable in > conf/kylin.properties with key *kylin.metadata.url*; > > The HDFS path '/kylin/kylin_metadata' is for the cube and other files; > Before you build the first cube, the folder is empty I think. > > > > If sample.sh couldn't create the /tmp/kylin folder on HDFS; If the folder > wasn't create, Kylin might not find the Hadoop config properly. You can run: > > > > ./bin/find-hadoop-conf-dir.sh -v > > > > to see what it prints. > > > > > > > > > > 2018-02-05 23:24 GMT+08:00 BELLIER Jean-luc <jean-luc.bellier@rte-france. > com>: > > Hello, > > > > I am sorry to come back again to you, but I am really stuck. I restarted > all the services and agents on my Cloudera cluster, so it is OK now. > > I noticed that some HDFS directories were empty, especially /tmp/kylin. > When I have a look on the code of sample.sh, I can see that this script > should create sthe subfolder /tmp/kylin/sample_cube/data. These data are > used to create the Hive tables and are removed after the creation of these > tables. Let call the node N1 > > > > I have another machine on which Kylin is installed and I succeeded in > launching sample.sh. On this machine, I can see a HDFS folder called > /kylin/kylin_metadata with, I presume, the metadata of the cube stored in > it. > > > > I cannot succeed in creating this folder on N1. I removed the /tmp/kylin > folder, the hbase table and the Hive tables. > > I really do not know what to do to restart the creation of the project > from scratch. > > I also checked the HDFS space using the command ‘hdfs dfs –df’ and it > tells me a usage of 2%. > > > > Could you please guide me in how to achieve it. There may be something I > did wrong, but I really do not what. Is my way of doing correct ? Are > there other files to remove (temporary files, for example), and in this > case, where are they located ? > > > > Any help would be greatly appreciated. > > > > Thank you in advance for your help. Have a good day. > > > > Best regards, > > Jean-Luc. > > > > *De :* BELLIER Jean-luc > *Envoyé :* vendredi 2 février 2018 18:54 > *À :* user <user@kylin.apache.org> > *Objet :* RE: Problem starting Kylin Web interface > > > > Hello, > > > > Thanks for the suggestion. I tried on another machine with Firefox and > Chrome, but I get the same result as before. I also rebuilt the project > ‘sample’ from scratch, with no change. > > My cluster seems to have problems on its nodes, especially memory which is > nearly full on most nodes. > > I suspect this could be one of the cause of the problem : no available > memory and swap space also nearly full. > > > > What do you think of this assumption ? > > > > I will see that with my Cloudera manager next week, as I am not a > specialist at all. > > > > Have a good week-end. > > > > Best regards, > > Jean-Luc. > > > > > > *De :* ShaoFeng Shi [mailto:shaofeng...@apache.org > <shaofeng...@apache.org>] > *Envoyé :* vendredi 2 février 2018 14:59 > > > *À :* user <user@kylin.apache.org> > *Objet :* Re: Problem starting Kylin Web interface > > > > Please try Chrome; Most developers are using Chrome; And if there is an > error, you can press F12 to check the HTTP request/response, there should > be more information. > > > > 2018-02-02 21:54 GMT+08:00 BELLIER Jean-luc <jean-luc.bellier@rte-france. > com>: > > Hello again, > > > > I forgot to mention that I use IE or Secured Firefox as Web browsers. I > have the same issue with both. Could there be the cause of the problem ? > > > > Best regards > > Jean-Luc. > > > > *De :* BELLIER Jean-luc > *Envoyé :* vendredi 2 février 2018 14:48 > *À :* user <user@kylin.apache.org> > *Objet :* RE: Problem starting Kylin Web interface > > > > Hello > > > > Thank you for your answer. > > I did the following : > > · rename my current kylin.log and kylin.out files to keep a trace > > · restarted the commands ‘kylin.sh start’ and ‘sample.sh’ > > I cannot see anything special in the new .log or .out files. The Hive > tables exist, and the kylin_metadata also. I can scan it through Hbase > shell command. It gives me 44 rows. > > > > However, the problem is still the same with the Kylin Interface : Models > loading indefinitely, no table displayed, and nothing in the ‘System’ > section. When I try to refresh the server config and the server > environement, I still have the message ‘Oops ! Failed to take action. I > also have this message as soon as I click to the ‘System’ button. > > > > Should I rebuild the project from scratch to be sure that everything is > correctly created, or should I investigate somewhere else (services, node > states, …) ? > > > > I send in attachement the now kylin.log and kylin.out files > > Thank you in advance for your help. Have a good day. > > > > Best regards, > > Jean-Luc. > > > > *De :* ShaoFeng Shi [mailto:shaofeng...@apache.org > <shaofeng...@apache.org>] > *Envoyé :* vendredi 2 février 2018 12:06 > *À :* user <user@kylin.apache.org> > *Objet :* Re: Problem starting Kylin Web interface > > > > Hi Jean-luc, > > > > The kylin.log seems okay, the service is started and the job scheduler is > running up. > > > > The kylin.out has some error about missing ".keystore", but by default if > you accessing kylin via http, it shouldn't need to load keystore. Please > try visit http://host:7070/kylin again to see whether there is any issue > in the web console. > > > > 2018-02-01 23:17 GMT+08:00 BELLIER Jean-luc <jean-luc.bellier@rte-france. > com>: > > Hello, > > > > I am new in Kylin. I would like to test it for a POC for my company. I > opened a ticket yesterday in the issues mail box through Jira, but I am not > sure it is the best place. > > Here is the reference of the ticket : https://issues.apache.org/ > jira/browse/KYLIN-3218 > > > > I am using the version apache-kylin-2.2.0-bin-cdh57, with Linux. I > installed the software on a client node. I successfully started the process > and created the cube given in the example project. > > Apparently, no mistake appears when starting the kylin.sh shell or > generating the cube using sample.sh. > > > > When I launch the web interface using <nodename>:7070/kylin, I cannot load > the models (the cursor is turning indefinitely). > > By looking at the kylin.out log, I could see that the process could not > find the $KYLIN_HOME/tomcat/conf/.keystore file. So I created it > manually, and relaunched the cube generation and the ‘kylin.sh start’ > command. > > Now I face to other mistakes : > > > > · *Exception in thread "Thread-14" java.lang.RuntimeException: > Error while peeking at > /kylin/kylin_metadata/job_engine/global_job_engine_lock* > > · *SEVERE: The web application [/kylin] appears to have started a > thread named [TGT Renewer for bellier...@hades.rte-france.com > <bellier...@hades.rte-france.com>] but has failed to stop it. This is very > likely to create a memory leak.* This one appears several times > > · *SEVERE: The web application [/kylin] created a ThreadLocal > with key of type [java.lang.ThreadLocal] (value > [java.lang.ThreadLocal@10cb8259]) and a value of type > [org.apache.kylin.rest.msg.Message] (value > [org.apache.kylin.rest.msg.Message@267ae210]) but failed to remove it when > the web application was stopped. Threads are going to be renewed over time > to try and avoid a probable memory leak. *This one appears several times > > > > Apparently, something else is missing, or I did something wrong, but I do > not know what. I would also like to know what will be the impact of the two > ‘SEVERE’ issues above and how I could correct it. > > > > I attach the last version of kylin.out file. Any help would be kindly > appreciated. > > > > Thank you in advance for your help. Have a good day. > > > > Best regards. > > Jean-Luc. > > > > > > "Ce message est destiné exclusivement aux personnes ou entités auxquelles > il est adressé et peut contenir des informations privilégiées ou > confidentielles. Si vous avez reçu ce document par erreur, merci de nous > l'indiquer par retour, de ne pas le transmettre et de procéder à sa > destruction. > > This message is solely intended for the use of the individual or entity to > which it is addressed and may contain information that is privileged or > confidential. If you have received this communication by error, please > notify us immediately by electronic mail, do not disclose it and delete the > original message." > > > > > > -- > > Best regards, > > > > Shaofeng Shi 史少锋 > > > > > > "Ce message est destiné exclusivement aux personnes ou entités auxquelles > il est adressé et peut contenir des informations privilégiées ou > confidentielles. Si vous avez reçu ce document par erreur, merci de nous > l'indiquer par retour, de ne pas le transmettre et de procéder à sa > destruction. > > This message is solely intended for the use of the individual or entity to > which it is addressed and may contain information that is privileged or > confidential. If you have received this communication by error, please > notify us immediately by electronic mail, do not disclose it and delete the > original message." > > > > > > -- > > Best regards, > > > > Shaofeng Shi 史少锋 > > > > > > "Ce message est destiné exclusivement aux personnes ou entités auxquelles > il est adressé et peut contenir des informations privilégiées ou > confidentielles. Si vous avez reçu ce document par erreur, merci de nous > l'indiquer par retour, de ne pas le transmettre et de procéder à sa > destruction. > > This message is solely intended for the use of the individual or entity to > which it is addressed and may contain information that is privileged or > confidential. If you have received this communication by error, please > notify us immediately by electronic mail, do not disclose it and delete the > original message." > > > > > > -- > > Best regards, > > > > Shaofeng Shi 史少锋 > > > > > > "Ce message est destiné exclusivement aux personnes ou entités auxquelles > il est adressé et peut contenir des informations privilégiées ou > confidentielles. Si vous avez reçu ce document par erreur, merci de nous > l'indiquer par retour, de ne pas le transmettre et de procéder à sa > destruction. > > This message is solely intended for the use of the individual or entity to > which it is addressed and may contain information that is privileged or > confidential. If you have received this communication by error, please > notify us immediately by electronic mail, do not disclose it and delete the > original message." > > > > > > -- > > Best regards, > > > > Shaofeng Shi 史少锋 > > > > > > "Ce message est destiné exclusivement aux personnes ou entités auxquelles > il est adressé et peut contenir des informations privilégiées ou > confidentielles. Si vous avez reçu ce document par erreur, merci de nous > l'indiquer par retour, de ne pas le transmettre et de procéder à sa > destruction. > > This message is solely intended for the use of the individual or entity to > which it is addressed and may contain information that is privileged or > confidential. If you have received this communication by error, please > notify us immediately by electronic mail, do not disclose it and delete the > original message." > > > > > > -- > > Best regards, > > > > Shaofeng Shi 史少锋 > > > > > > "Ce message est destiné exclusivement aux personnes ou entités auxquelles > il est adressé et peut contenir des informations privilégiées ou > confidentielles. Si vous avez reçu ce document par erreur, merci de nous > l'indiquer par retour, de ne pas le transmettre et de procéder à sa > destruction. > > This message is solely intended for the use of the individual or entity to > which it is addressed and may contain information that is privileged or > confidential. If you have received this communication by error, please > notify us immediately by electronic mail, do not disclose it and delete the > original message." > > > > > > -- > > Best regards, > > > > Shaofeng Shi 史少锋 > > > > > > > > -- > > Best regards, > > > > Shaofeng Shi 史少锋 > > > > > "Ce message est destiné exclusivement aux personnes ou entités auxquelles > il est adressé et peut contenir des informations privilégiées ou > confidentielles. Si vous avez reçu ce document par erreur, merci de nous > l'indiquer par retour, de ne pas le transmettre et de procéder à sa > destruction. > > This message is solely intended for the use of the individual or entity to > which it is addressed and may contain information that is privileged or > confidential. If you have received this communication by error, please > notify us immediately by electronic mail, do not disclose it and delete the > original message." > -- Best regards, Shaofeng Shi 史少锋
