Hello,I am writing a application which needs the metastore in hive. I am 
preparing to use webhcat to get the information about tables and process them. 
But a simple request takes over eight seconds to respond on localhost. Why is 
this so slow, and how can I fix it?
$ time curl -s 
'http://localhost:50111/templeton/v1/ddl/database/default/table/haha?user.name=ctdean'
{"columns": 
  [{"name":"id","type":"int"}],
  "database":"default",
  "table":"haha"}

real    0m8.400s
user    0m0.053s
sys     0m0.019s
The webhcat.log is very short, it seems to run a hcat.py, I have looked up the 
log, and I can't figure out what's going on here. Here is the info when I 
simply run hcat. It seems take all the time. Thank you very much for your kind 
reply.


$hcat.py -e "use default; desc haha; "
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in 
[jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in 
[jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
18/04/21 16:38:13 INFO conf.HiveConf: Found configuration file 
file:/usr/local/hive/conf/hive-site.xml
18/04/21 16:38:15 WARN util.NativeCodeLoader: Unable to load native-hadoop 
library for your platform... using builtin-java classes where applicable
18/04/21 16:38:16 INFO session.SessionState: Created HDFS directory: 
/tmp/hive/kousouda/05096382-f9b6-4dae-aee2-dfa6750c0668
18/04/21 16:38:16 INFO session.SessionState: Created local directory: 
/tmp/hive/java/kousouda/05096382-f9b6-4dae-aee2-dfa6750c0668
18/04/21 16:38:16 INFO session.SessionState: Created HDFS directory: 
/tmp/hive/kousouda/05096382-f9b6-4dae-aee2-dfa6750c0668/_tmp_space.db
18/04/21 16:38:16 INFO ql.Driver: Compiling 
command(queryId=kousouda_20180421163816_58c38a44-25e3-4665-8bb5-a9b17fdf2d62): 
use default
18/04/21 16:38:17 INFO metastore.HiveMetaStore: 0: Opening raw store with 
implementation class:org.apache.hadoop.hive.metastore.ObjectStore
18/04/21 16:38:17 INFO metastore.ObjectStore: ObjectStore, initialize called
18/04/21 16:38:18 INFO DataNucleus.Persistence: Property 
hive.metastore.integral.jdo.pushdown unknown - will be ignored
18/04/21 16:38:18 INFO DataNucleus.Persistence: Property 
datanucleus.cache.level2 unknown - will be ignored
18/04/21 16:38:18 INFO metastore.ObjectStore: Setting MetaStore object pin 
classes with 
hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order"
18/04/21 16:38:20 INFO metastore.MetaStoreDirectSql: Using direct SQL, 
underlying DB is MYSQL
18/04/21 16:38:20 INFO metastore.ObjectStore: Initialized ObjectStore
18/04/21 16:38:20 INFO metastore.HiveMetaStore: Added admin role in metastore
18/04/21 16:38:20 INFO metastore.HiveMetaStore: Added public role in metastore
18/04/21 16:38:20 INFO metastore.HiveMetaStore: No user is added in admin role, 
since config is empty
18/04/21 16:38:20 INFO metastore.HiveMetaStore: 0: get_all_functions
18/04/21 16:38:20 INFO HiveMetaStore.audit: ugi=kousouda        
ip=unknown-ip-addr      cmd=get_all_functions
18/04/21 16:38:20 INFO metastore.HiveMetaStore: 0: get_database: default
18/04/21 16:38:20 INFO HiveMetaStore.audit: ugi=kousouda        
ip=unknown-ip-addr      cmd=get_database: default
18/04/21 16:38:20 INFO ql.Driver: Semantic Analysis Completed
18/04/21 16:38:20 INFO ql.Driver: Returning Hive schema: 
Schema(fieldSchemas:null, properties:null)
18/04/21 16:38:20 INFO ql.Driver: Completed compiling 
command(queryId=kousouda_20180421163816_58c38a44-25e3-4665-8bb5-a9b17fdf2d62); 
Time taken: 3.936 seconds
18/04/21 16:38:20 INFO ql.Driver: Concurrency mode is disabled, not creating a 
lock manager
18/04/21 16:38:20 INFO ql.Driver: Executing 
command(queryId=kousouda_20180421163816_58c38a44-25e3-4665-8bb5-a9b17fdf2d62): 
use default
18/04/21 16:38:20 INFO sqlstd.SQLStdHiveAccessController: Created 
SQLStdHiveAccessController for session context : HiveAuthzSessionContext 
[sessionString=05096382-f9b6-4dae-aee2-dfa6750c0668, clientType=HIVECLI]
18/04/21 16:38:20 WARN session.SessionState: METASTORE_FILTER_HOOK will be 
ignored, since hive.security.authorization.manager is set to instance of 
HiveAuthorizerFactory.
18/04/21 16:38:20 INFO hive.metastore: Mestastore configuration 
hive.metastore.filter.hook changed from 
org.apache.hadoop.hive.metastore.DefaultMetaStoreFilterHookImpl to 
org.apache.hadoop.hive.ql.security.authorization.plugin.AuthorizationMetaStoreFilterHook
18/04/21 16:38:20 INFO metastore.HiveMetaStore: 0: Cleaning up thread local 
RawStore...
18/04/21 16:38:20 INFO HiveMetaStore.audit: ugi=kousouda        
ip=unknown-ip-addr      cmd=Cleaning up thread local RawStore...
18/04/21 16:38:20 INFO metastore.HiveMetaStore: 0: Done cleaning up thread 
local RawStore
18/04/21 16:38:20 INFO HiveMetaStore.audit: ugi=kousouda        
ip=unknown-ip-addr      cmd=Done cleaning up thread local RawStore
18/04/21 16:38:20 INFO ql.Driver: Starting task [Stage-0:DDL] in serial mode
18/04/21 16:38:20 INFO metastore.HiveMetaStore: 0: get_database: default
18/04/21 16:38:20 INFO HiveMetaStore.audit: ugi=kousouda        
ip=unknown-ip-addr      cmd=get_database: default
18/04/21 16:38:20 INFO metastore.HiveMetaStore: 0: Opening raw store with 
implementation class:org.apache.hadoop.hive.metastore.ObjectStore
18/04/21 16:38:20 INFO metastore.ObjectStore: ObjectStore, initialize called
18/04/21 16:38:20 INFO metastore.MetaStoreDirectSql: Using direct SQL, 
underlying DB is MYSQL
18/04/21 16:38:20 INFO metastore.ObjectStore: Initialized ObjectStore
18/04/21 16:38:20 INFO metastore.HiveMetaStore: 0: get_database: default
18/04/21 16:38:20 INFO HiveMetaStore.audit: ugi=kousouda        
ip=unknown-ip-addr      cmd=get_database: default
18/04/21 16:38:20 INFO ql.Driver: Completed executing 
command(queryId=kousouda_20180421163816_58c38a44-25e3-4665-8bb5-a9b17fdf2d62); 
Time taken: 0.202 seconds
OK
18/04/21 16:38:20 INFO ql.Driver: OK
Time taken: 4.16 seconds
18/04/21 16:38:20 INFO ql.Driver: Compiling 
command(queryId=kousouda_20180421163820_c67f5029-7496-422d-b812-fe72661cf999): 
desc haha
18/04/21 16:38:20 INFO metastore.HiveMetaStore: 0: get_table : db=default 
tbl=haha
18/04/21 16:38:20 INFO HiveMetaStore.audit: ugi=kousouda        
ip=unknown-ip-addr      cmd=get_table : db=default tbl=haha
18/04/21 16:38:20 INFO parse.DDLSemanticAnalyzer: analyzeDescribeTable done
18/04/21 16:38:20 INFO ql.Driver: Semantic Analysis Completed
18/04/21 16:38:20 INFO ql.Driver: Returning Hive schema: 
Schema(fieldSchemas:[FieldSchema(name:col_name, type:string, comment:from 
deserializer), FieldSchema(name:data_type, type:string, comment:from 
deserializer), FieldSchema(name:comment, type:string, comment:from 
deserializer)], properties:null)
18/04/21 16:38:20 INFO exec.ListSinkOperator: Initializing operator LIST_SINK[0]
18/04/21 16:38:20 INFO ql.Driver: Completed compiling 
command(queryId=kousouda_20180421163820_c67f5029-7496-422d-b812-fe72661cf999); 
Time taken: 0.237 seconds
18/04/21 16:38:20 INFO ql.Driver: Concurrency mode is disabled, not creating a 
lock manager
18/04/21 16:38:20 INFO ql.Driver: Executing 
command(queryId=kousouda_20180421163820_c67f5029-7496-422d-b812-fe72661cf999): 
desc haha
18/04/21 16:38:20 INFO ql.Driver: Starting task [Stage-0:DDL] in serial mode
18/04/21 16:38:20 INFO metastore.HiveMetaStore: 0: get_table : db=default 
tbl=haha
18/04/21 16:38:20 INFO HiveMetaStore.audit: ugi=kousouda        
ip=unknown-ip-addr      cmd=get_table : db=default tbl=haha
18/04/21 16:38:20 INFO ql.Driver: Completed executing 
command(queryId=kousouda_20180421163820_c67f5029-7496-422d-b812-fe72661cf999); 
Time taken: 0.081 seconds
OK
18/04/21 16:38:20 INFO ql.Driver: OK
18/04/21 16:38:20 INFO Configuration.deprecation: mapred.input.dir is 
deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
18/04/21 16:38:20 INFO mapred.FileInputFormat: Total input paths to process : 1
18/04/21 16:38:20 INFO exec.ListSinkOperator: Closing operator LIST_SINK[0]
id                      int
Time taken: 0.377 seconds
18/04/21 16:38:20 INFO session.SessionState: Deleted directory: 
/tmp/hive/kousouda/05096382-f9b6-4dae-aee2-dfa6750c0668 on fs with scheme hdfs
18/04/21 16:38:20 INFO session.SessionState: Deleted directory: 
/tmp/hive/java/kousouda/05096382-f9b6-4dae-aee2-dfa6750c0668 on fs with scheme 
file
18/04/21 16:38:20 INFO metastore.HiveMetaStore: 0: Cleaning up thread local 
RawStore...
18/04/21 16:38:20 INFO HiveMetaStore.audit: ugi=kousouda        
ip=unknown-ip-addr      cmd=Cleaning up thread local RawStore...
18/04/21 16:38:20 INFO metastore.HiveMetaStore: 0: Done cleaning up thread 
local RawStore
18/04/21 16:38:20 INFO HiveMetaStore.audit: ugi=kousouda        
ip=unknown-ip-addr      cmd=Done cleaning up thread local RawStore

Reply via email to