Re: how to extract metadata of hive tables in speed

Thanks for the info too! :) > On Apr 25, 2018, at 11:12 AM, 侯宗田 wrote: > > Hi, > > Thank you, I have looked up the source code of Hcatalog, it seems every time > when I run hcat -e “query”, it called hcatCli, then it make configuration, > create and start a session,

Re: how to extract metadata of hive tables in speed

Hi, Thank you, I have looked up the source code of Hcatalog, it seems every time when I run hcat -e “query”, it called hcatCli, then it make configuration, create and start a session, then dump it after being used. It can’t keep a session or connection and don’t have a Cli. The initialization

Re: how to extract metadata of hive tables in speed

Hi Hou, Kudu uses the Thrift HMS interface, and written in C. An example could be found here: https://github.com/apache/kudu/tree/master/src/kudu/hms As for parametrizing Hcatalog I have only found this:

Re: how to extract metadata of hive tables in speed

Hi, Peter: I have started a standalone metastore server and it indeed short that part of time, it does connection instead of initialization. But I still have some questions, First, I believe the Hcatalog must be quick because it is a mature product and I have not seen others complaining about

Re: how to extract metadata of hive tables in speed

Hi, This really helps me a lot, and I will try it soon, Thank you very much! Hou > 在 2018年4月23日，下午3:57，Peter Vary 写道： > > Hi, > > Disclaimer: I am not too familiar with the webhcat yet. > From the logs, I see, that: > - the first 3 seconds spent on starting a new session,

Re: how to extract metadata of hive tables in speed

Hi, Disclaimer: I am not too familiar with the webhcat yet. From the logs, I see, that: - the first 3 seconds spent on starting a new session, and maybe a driver - this can be reduced, if the session is already there, and the HiveServer2 is started (but do not know if webhcat could use HS2, or

Re: how to extract metadata of hive tables in speed

Thank you very much for your reply, I am wondering whether I use the webhcat rightly, I don’t think it is normal to create all the directories and objects to get a table describ and take 8 seconds. The webhcat should not be so slow, Or it is because I forget to start some server which can

Re: how to extract metadata of hive tables in speed

Hi, Alexander Kolbasov has a project which might interest you (keeping in mind, that this is not production ready - more like a proof of concept): https://github.com/akolb1/gometastore/blob/master/hmstool/doc/hmstool.md Also you can use HMS thrift API directly to access the MetaStore, or if you

Re: how to extract metadata of hive tables in speed

Can anyone give me some suggestions? I have been stuck in this problem for several days. Need help!! > 在 2018年4月22日，下午9:38，侯宗田 写道： > > > Hi, > > I am writing a application which needs the metastore about hive tables. I > have used webhcat to get the information about

how to extract metadata of hive tables in speed

Hi, I am writing a application which needs the metastore about hive tables. I have used webhcat to get the information about tables and process them. But a simple request takes over eight seconds to respond on localhost. Why is this so slow, and how can I fix it or is there other way I can