Hive mode does not matters. What to check is if the right hive conf is on classpath.
Run `bin/find-hive-dependency.sh` and check if the printed classpath contains the expected hive config dir. On Mon, Dec 28, 2015 at 10:48 PM, zz <[email protected]> wrote: > when I execute : hive -e "describe kylin_sales" , it work well . Also I > can execute sql for metastore table from mysql client successfully ! > I guess metastore is OK . I config hive as remote model, but Kylin can't > use it . From Kylin's log , seems "DataNucleus" regard it as embedded > model then use DERBY DB. Actually, I use mysql to store metadata . > > below is my full log, who know what's wrong with it ... > > ESTER=ADMIN;REQ_TIME=GMT-08:00 2015-12-28 > 06:25:40;URI=/kylin/api/tables;METHOD=GET;QUERY_STRING=ext=true&project=learn_kylin;PAYLOAD=;RESP_STATUS=200; > 2015-12-28 14:26:03,159 INFO [http-bio-7070-exec-1] > Configuration.deprecation: mapred.reduce.tasks is deprecated. Instead, use > mapreduce.job.reduces > 2015-12-28 14:26:03,159 INFO [http-bio-7070-exec-1] > Configuration.deprecation: mapred.min.split.size is deprecated. Instead, > use mapreduce.input.fileinputformat.split.minsize > 2015-12-28 14:26:03,159 INFO [http-bio-7070-exec-1] > Configuration.deprecation: mapred.reduce.tasks.speculative.execution is > deprecated. Instead, use mapreduce.reduce.speculative > 2015-12-28 14:26:03,159 INFO [http-bio-7070-exec-1] > Configuration.deprecation: mapred.min.split.size.per.node is deprecated. > Instead, use mapreduce.input.fileinputformat.split.minsize.per.node > 2015-12-28 14:26:03,159 INFO [http-bio-7070-exec-1] > Configuration.deprecation: mapred.input.dir.recursive is deprecated. > Instead, use mapreduce.input.fileinputformat.input.dir.recursive > 2015-12-28 14:26:03,159 INFO [http-bio-7070-exec-1] > Configuration.deprecation: mapred.min.split.size.per.rack is deprecated. > Instead, use mapreduce.input.fileinputformat.split.minsize.per.rack > 2015-12-28 14:26:03,159 INFO [http-bio-7070-exec-1] > Configuration.deprecation: mapred.max.split.size is deprecated. Instead, > use mapreduce.input.fileinputformat.split.maxsize > 2015-12-28 14:26:03,160 INFO [http-bio-7070-exec-1] > Configuration.deprecation: mapred.committer.job.setup.cleanup.needed is > deprecated. Instead, use mapreduce.job.committer.setup.cleanup.needed > 2015-12-28 14:26:03,446 INFO [http-bio-7070-exec-1] > metastore.HiveMetaStore: 0: Opening raw store with implemenation > class:org.apache.hadoop.hive.metastore.ObjectStore > 2015-12-28 14:26:03,481 INFO [http-bio-7070-exec-1] > metastore.ObjectStore: ObjectStore, initialize called > 2015-12-28 14:26:03,734 INFO [http-bio-7070-exec-1] > DataNucleus.Persistence: Property datanucleus.cache.level2 unknown - will > be ignored > 2015-12-28 14:26:03,735 INFO [http-bio-7070-exec-1] > DataNucleus.Persistence: Property hive.metastore.integral.jdo.pushdown > unknown - will be ignored > 2015-12-28 14:26:06,771 INFO [http-bio-7070-exec-1] > metastore.ObjectStore: Setting MetaStore object pin classes with > hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order" > 2015-12-28 14:26:09,744 INFO [http-bio-7070-exec-1] > DataNucleus.Datastore: The class > "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as > "embedded-only" so does not have its own datastore table. > 2015-12-28 14:26:09,744 INFO [http-bio-7070-exec-1] > DataNucleus.Datastore: The class > "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as > "embedded-only" so does not have its own datastore table. > 2015-12-28 14:26:10,175 INFO [http-bio-7070-exec-1] > DataNucleus.Datastore: The class > "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as > "embedded-only" so does not have its own datastore table. > 2015-12-28 14:26:10,175 INFO [http-bio-7070-exec-1] > DataNucleus.Datastore: The class > "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as > "embedded-only" so does not have its own datastore table. > 2015-12-28 14:26:10,325 INFO [http-bio-7070-exec-1] DataNucleus.Query: > Reading in results for query "org.datanucleus.store.rdbms.query.SQLQuery@0" > since the connection used is closing > 2015-12-28 14:26:10,327 INFO [http-bio-7070-exec-1] > metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY > 2015-12-28 14:26:10,329 INFO [http-bio-7070-exec-1] > metastore.ObjectStore: Initialized ObjectStore > 2015-12-28 14:26:10,665 INFO [http-bio-7070-exec-1] > metastore.HiveMetaStore: Added admin role in metastore > 2015-12-28 14:26:10,667 INFO [http-bio-7070-exec-1] > metastore.HiveMetaStore: Added public role in metastore > 2015-12-28 14:26:10,772 INFO [http-bio-7070-exec-1] > metastore.HiveMetaStore: No user is added in admin role, since config is > empty > 2015-12-28 14:26:10,922 INFO [http-bio-7070-exec-1] > metastore.HiveMetaStore: 0: get_table : db=default tbl=kylin_sales > 2015-12-28 14:26:10,923 INFO [http-bio-7070-exec-1] HiveMetaStore.audit: > ugi=ocp ip=unknown-ip-addr cmd=get_table : db=default > tbl=kylin_sales > NoSuchObjectException(message:default.kylin_sales table not found) > at > org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table_core(HiveMetaStore.java:1808) > > > > ------------------ 原始邮件 ------------------ > *发件人:* "ShaoFeng Shi";<[email protected]>; > *发送时间:* 2015年12月25日(星期五) 晚上8:53 > *收件人:* "user"<[email protected]>; > *主题:* Re: how to load table from hive > > what's the output of executing this command?: > > hive -e "describe <your_table_name>" > > 2015-12-25 11:19 GMT+08:00 zz <[email protected]>: > >> Hi Yang & all, >> I checked hive config by "hive -e set" in the server which kylin >> deployed, seems nothing wrong. >> Then I tried to run sample.sh in kylin/bin , it created tables: >> "kylin_cal_dt" "kylin_category_groupings" "kylin_sales" successfully and >> I can find these in metastore DB (mysql) . >> Need I do some config in kylin.properties to make it load hive table >> metadata by hive remote model? I guess some property make it use hive >> default Derby . >> Below is my hive-site.xml: >> >> <configuration> >> <property> >> <name>javax.jdo.option.ConnectionURL</name> >> >> <value>jdbc:mysql://localhost:3306/metastore?createDatabaseIfNotExist=true</value> >> <description>metadata is stored in a MySQL server</description> >> </property> >> <property> >> <name>javax.jdo.option.ConnectionDriverName</name> >> <value>com.mysql.jdbc.Driver</value> >> <description>MySQL JDBC driver class</description> >> </property> >> <property> >> <name>javax.jdo.option.ConnectionUserName</name> >> <value>hive</value> >> <description>user name for connecting to mysql server</description> >> </property> >> <property> >> <name>javax.jdo.option.ConnectionPassword</name> >> <value>password</value> >> <description>password for connecting to mysql server</description> >> </property> >> <property> >> <name>hive.metastore.uris</name> >> <value>thrift://10.224.243.127:9083</value> >> </property> >> <property> >> <name>hive.metastore.schema.verification</name> >> <value>false</value> >> </property> >> <property> >> <name>hive.metastore.warehouse.dir</name> >> <value>/user/hive/warehouse</value> >> </property> >> <!--<property> >> <name>hive.server.thrift.port</name> >> <value>10000</value> >> </property> --> >> </configuration> >> >> >> >> >> ------------------ 原始邮件 ------------------ >> *发件人:* "Li Yang";<[email protected]>; >> *发送时间:* 2015年12月25日(星期五) 上午10:10 >> *收件人:* "user"<[email protected]>; >> *主题:* Re: how to load table from hive >> >> What matters is the hive-site.xml Kylin picks up. Kylin detects hive >> config by command `hive -e set`. You can try hive command line on the node >> where Kylin is deployed and make sure it connects to the desired metastore. >> >> Try diagnose `bin/find-hive-dependency.sh` if you still got problem. >> >> On Fri, Dec 25, 2015 at 9:49 AM, zz <[email protected]> wrote: >> >>> Hi Kylin developers, >>> >>> I'm a newbie for Kylin. Glad to see the excellent OLAP tool. As a >>> Chinese acmen , I like the name. >>> >>> After I download and run it , I can access the website( >>> http://IP:7070/kylin) and create project successfully . However, when >>> I try to load Hive table, "NoSuchObjectException" alert (Oops...) ! >>> >>> From kylin.log in tomcat , I found some sensitive tracks: >>> 2015-12-24 15:01:16,613 INFO [http-bio-7070-exec-6] >>> metastore.ObjectStore: ObjectStore, initialize called >>> 2015-12-24 15:01:16,926 INFO [http-bio-7070-exec-6] >>> DataNucleus.Persistence: Property datanucleus.cache.level2 unknown - will >>> be ignored >>> 2015-12-24 15:01:16,927 INFO [http-bio-7070-exec-6] >>> DataNucleus.Persistence: Property hive.metastore.integral.jdo.pushdown >>> unknown - will be ignored >>> ... >>> 2015-12-24 15:01:24,350 INFO [http-bio-7070-exec-6] >>> metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY >>> >>> I'm not sure these ignored property is OK or not. >>> I config Hive as remote mode(use mysql to store metadata, config thrift >>> uris) but kylin try to access DERBY which I do not use ! >>> >>> Could you tell me how to config it ? >>> >> >> > > > -- > Best regards, > > Shaofeng Shi > >
