when I execute : hive -e "describe kylin_sales" , it work well . Also I can
execute sql for metastore table from mysql client successfully !
I guess metastore is OK . I config hive as remote model, but Kylin can't use it
. From Kylin's log , seems "DataNucleus" regard it as embedded model then use
DERBY DB. Actually, I use mysql to store metadata .
below is my full log, who know what's wrong with it ...
ESTER=ADMIN;REQ_TIME=GMT-08:00 2015-12-28
06:25:40;URI=/kylin/api/tables;METHOD=GET;QUERY_STRING=ext=true&project=learn_kylin;PAYLOAD=;RESP_STATUS=200;
2015-12-28 14:26:03,159 INFO [http-bio-7070-exec-1] Configuration.deprecation:
mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
2015-12-28 14:26:03,159 INFO [http-bio-7070-exec-1] Configuration.deprecation:
mapred.min.split.size is deprecated. Instead, use
mapreduce.input.fileinputformat.split.minsize
2015-12-28 14:26:03,159 INFO [http-bio-7070-exec-1] Configuration.deprecation:
mapred.reduce.tasks.speculative.execution is deprecated. Instead, use
mapreduce.reduce.speculative
2015-12-28 14:26:03,159 INFO [http-bio-7070-exec-1] Configuration.deprecation:
mapred.min.split.size.per.node is deprecated. Instead, use
mapreduce.input.fileinputformat.split.minsize.per.node
2015-12-28 14:26:03,159 INFO [http-bio-7070-exec-1] Configuration.deprecation:
mapred.input.dir.recursive is deprecated. Instead, use
mapreduce.input.fileinputformat.input.dir.recursive
2015-12-28 14:26:03,159 INFO [http-bio-7070-exec-1] Configuration.deprecation:
mapred.min.split.size.per.rack is deprecated. Instead, use
mapreduce.input.fileinputformat.split.minsize.per.rack
2015-12-28 14:26:03,159 INFO [http-bio-7070-exec-1] Configuration.deprecation:
mapred.max.split.size is deprecated. Instead, use
mapreduce.input.fileinputformat.split.maxsize
2015-12-28 14:26:03,160 INFO [http-bio-7070-exec-1] Configuration.deprecation:
mapred.committer.job.setup.cleanup.needed is deprecated. Instead, use
mapreduce.job.committer.setup.cleanup.needed
2015-12-28 14:26:03,446 INFO [http-bio-7070-exec-1] metastore.HiveMetaStore:
0: Opening raw store with implemenation
class:org.apache.hadoop.hive.metastore.ObjectStore
2015-12-28 14:26:03,481 INFO [http-bio-7070-exec-1] metastore.ObjectStore:
ObjectStore, initialize called
2015-12-28 14:26:03,734 INFO [http-bio-7070-exec-1] DataNucleus.Persistence:
Property datanucleus.cache.level2 unknown - will be ignored
2015-12-28 14:26:03,735 INFO [http-bio-7070-exec-1] DataNucleus.Persistence:
Property hive.metastore.integral.jdo.pushdown unknown - will be ignored
2015-12-28 14:26:06,771 INFO [http-bio-7070-exec-1] metastore.ObjectStore:
Setting MetaStore object pin classes with
hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order"
2015-12-28 14:26:09,744 INFO [http-bio-7070-exec-1] DataNucleus.Datastore: The
class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as
"embedded-only" so does not have its own datastore table.
2015-12-28 14:26:09,744 INFO [http-bio-7070-exec-1] DataNucleus.Datastore: The
class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as
"embedded-only" so does not have its own datastore table.
2015-12-28 14:26:10,175 INFO [http-bio-7070-exec-1] DataNucleus.Datastore: The
class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as
"embedded-only" so does not have its own datastore table.
2015-12-28 14:26:10,175 INFO [http-bio-7070-exec-1] DataNucleus.Datastore: The
class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as
"embedded-only" so does not have its own datastore table.
2015-12-28 14:26:10,325 INFO [http-bio-7070-exec-1] DataNucleus.Query: Reading
in results for query "org.datanucleus.store.rdbms.query.SQLQuery@0" since the
connection used is closing
2015-12-28 14:26:10,327 INFO [http-bio-7070-exec-1]
metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY
2015-12-28 14:26:10,329 INFO [http-bio-7070-exec-1] metastore.ObjectStore:
Initialized ObjectStore
2015-12-28 14:26:10,665 INFO [http-bio-7070-exec-1] metastore.HiveMetaStore:
Added admin role in metastore
2015-12-28 14:26:10,667 INFO [http-bio-7070-exec-1] metastore.HiveMetaStore:
Added public role in metastore
2015-12-28 14:26:10,772 INFO [http-bio-7070-exec-1] metastore.HiveMetaStore:
No user is added in admin role, since config is empty
2015-12-28 14:26:10,922 INFO [http-bio-7070-exec-1] metastore.HiveMetaStore:
0: get_table : db=default tbl=kylin_sales
2015-12-28 14:26:10,923 INFO [http-bio-7070-exec-1] HiveMetaStore.audit:
ugi=ocp ip=unknown-ip-addr cmd=get_table : db=default tbl=kylin_sales
NoSuchObjectException(message:default.kylin_sales table not found)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table_core(HiveMetaStore.java:1808)
------------------ ???????? ------------------
??????: "ShaoFeng Shi";<[email protected]>;
????????: 2015??12??25??(??????) ????8:53
??????: "user"<[email protected]>;
????: Re: how to load table from hive
what's the output of executing this command?:
hive -e "describe <your_table_name>"
2015-12-25 11:19 GMT+08:00 zz <[email protected]>:
Hi Yang & all,
I checked hive config by "hive -e set" in the server which kylin deployed,
seems nothing wrong.
Then I tried to run sample.sh in kylin/bin , it created tables: "kylin_cal_dt"
"kylin_category_groupings" "kylin_sales" successfully and I can find these in
metastore DB (mysql) .
Need I do some config in kylin.properties to make it load hive table metadata
by hive remote model? I guess some property make it use hive default Derby .
Below is my hive-site.xml:
<configuration>
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://localhost:3306/metastore?createDatabaseIfNotExist=true</value>
<description>metadata is stored in a MySQL server</description>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
<description>MySQL JDBC driver class</description>
</property>
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>hive</value>
<description>user name for connecting to mysql server</description>
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>password</value>
<description>password for connecting to mysql server</description>
</property>
<property>
<name>hive.metastore.uris</name>
<value>thrift://10.224.243.127:9083</value>
</property>
<property>
<name>hive.metastore.schema.verification</name>
<value>false</value>
</property>
<property>
<name>hive.metastore.warehouse.dir</name>
<value>/user/hive/warehouse</value>
</property>
<!--<property>
<name>hive.server.thrift.port</name>
<value>10000</value>
</property> -->
</configuration>
------------------ ???????? ------------------
??????: "Li Yang";<[email protected]>;
????????: 2015??12??25??(??????) ????10:10
??????: "user"<[email protected]>;
????: Re: how to load table from hive
What matters is the hive-site.xml Kylin picks up. Kylin detects hive config by
command `hive -e set`. You can try hive command line on the node where Kylin is
deployed and make sure it connects to the desired metastore.
Try diagnose `bin/find-hive-dependency.sh` if you still got problem.
On Fri, Dec 25, 2015 at 9:49 AM, zz <[email protected]> wrote:
Hi Kylin developers,
I'm a newbie for Kylin. Glad to see the excellent OLAP tool. As a Chinese acmen
, I like the name.
After I download and run it , I can access the website(http://IP:7070/kylin)
and create project successfully . However, when I try to load Hive table,
"NoSuchObjectException" alert (Oops...) !
From kylin.log in tomcat , I found some sensitive tracks:
2015-12-24 15:01:16,613 INFO [http-bio-7070-exec-6] metastore.ObjectStore:
ObjectStore, initialize called
2015-12-24 15:01:16,926 INFO [http-bio-7070-exec-6] DataNucleus.Persistence:
Property datanucleus.cache.level2 unknown - will be ignored
2015-12-24 15:01:16,927 INFO [http-bio-7070-exec-6] DataNucleus.Persistence:
Property hive.metastore.integral.jdo.pushdown unknown - will be ignored
...
2015-12-24 15:01:24,350 INFO [http-bio-7070-exec-6]
metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY
I'm not sure these ignored property is OK or not.
I config Hive as remote mode(use mysql to store metadata, config thrift uris)
but kylin try to access DERBY which I do not use !
Could you tell me how to config it ?
--
Best regards,
Shaofeng Shi
