Best, Xiao
On Tue, Apr 8, 2014 at 8:07 PM, Xiao Meng <[email protected]> wrote: > Hi, > We finally get the hive storage engine up (the previous problem is mainly due > to some environmental variable settings) > However, it has problem when we try to query a specific hive table. > For example, we have the following table in the INFROMATION_SCHEMA.TABLES: > | DRILL | hivestg | store_sales_fact_denorm | TABLE | > When we query: >> select * from hivestg.`store_sales_fact_denorm` limit 1; > It shows the following exception: > Query failed: org.apache.drill.exec.rpc.RpcException: Remote failure while > running query.[error_id: "8bcf902e-a0c6-461c-94db-916d2188d1b2" > endpoint { > address: <my host name> > user_port: 31016 > control_port: 31017 > data_port: 31018 > } > error_type: 0 > message: "Failure while parsing sql. < OptimizerException:[ Failure while > attempting to retrieve storage engine. ] < IOException:[ > org.apache.drill.common.exceptions.ExecutionSetupException: > org.apache.hadoop.ipc.RemoteException: Server IPC version 9 cannot > communicate with client version 4 ] < ExecutionSetupException:[ > org.apache.hadoop.ipc.RemoteException: Server IPC version 9 cannot > communicate with client version 4 ] < RemoteException:[ Server IPC version 9 > cannot communicate with client version 4 ]" > ] > It seems a version problem. Any suggestion on this? > BTW, we use the following storage-engines settings: > { > ... > hivestg : { > type:"hive", > config : > { > "hive.metastore.uris" : "thrift://<hostname>:9083", > "hive.metastore.sasl.enabled" : "false" > } > } > } > } > Thanks, > Xiao > -----Original Message----- > From: George Chow [mailto:[email protected]] > Sent: March-27-14 5:12 PM > To: [email protected] > Subject: Re: Setting up Hive... > Hi Venki, > Thanks for the tip. I think I may be confused about my setup. > I have a remote metastore running alongside my HiveServer1. Thus, my > HiveServer1's hive-site.xml has the line > ConnectionURL=jdbc:mysql://localhost:3306/metastore. > I have tried configuring my Drillbit by pointing hive.metastore.uris to this > same URL (substituting the IP for localhost). But I'm not seeing anything. > Looking thru the various logs doesn't turn up anything. > I have already tried starting up the metastore in addition to the > HiveServer1 to no avail. > What am I doing wrong? > George > On Tue, Mar 25, 2014 at 7:07 PM, Venki Korukanti > <[email protected]>wrote: >> On Tue, Mar 25, 2014 at 6:56 PM, George Chow <[email protected]> wrote: >> >> > Hi Venki, >> > >> > I made a mistake initially and thought I was embedded. But I'm >> > actually >> in >> > scenario 1 (remote metastore). >> >> >> > I am setting hive.metastore.uris to my HiveServer's address and >> > port. Is that correct? >> > >> hive.metastore.uris should point to metastore server which is >> different from HiveServer1/2. Drill uses Metastore thrift interface to >> fetch the table/db info. From your first email, it looks like you are >> embedding the metastore within HiveServer1. You can copy the following >> settings from you current hiveserver's hive-site.xml to >> storage-engines.json to embed metastore in Drillbit or start the >> metastore as separate service using 'hive --service metastore' and point >> hive.metastore.uris to the address. >> >> "javax.jdo.option.ConnectionURL" : >> "jdbc:mysql://hostname/hivedrill?createDatabaseIfNotExist=true", >> >> "javax.jdo.option.ConnectionDriverName" : "com.mysql.jdbc.Driver", >> >> >> "javax.jdo.option.ConnectionUserName" : "user", >> >> "javax.jdo.option.ConnectionPassword" : "passwd" >> >> >> Thanks >> >> Venki >> >> >> >> >> > George >> > >> > >> > On Tue, Mar 25, 2014 at 6:48 PM, Venki Korukanti >> > <[email protected]>wrote: >> > >> > > Hi George, >> > > >> > > Are you embedding the metastore in drillbit >> > > (hive.metastore.uris="") or drillbit is connecting to remote >> > > metastore (hive.metastore.uris=valid address)? >> > > >> > > If you are embedding the metastore, make sure you have the >> > > appropriate driver (if MySql backend then mysql jdbc driver) in >> > > classpath or >> > drill/lib >> > > folder. >> > > >> > > Thanks >> > > Venki >> > > >> > > >> > > On Tue, Mar 25, 2014 at 6:30 PM, George Chow <[email protected]> >> > wrote: >> > > >> > > > Hi, >> > > > >> > > > Thanks Tim for the tip. It helps... so I copied the settings >> > > > from my hive-site.xml over to storage-engines.json. >> > > > >> > > > I restart my drillbit but I still don't see the contents from my >> > > > Hive metastore. The log file >> > > > (/var/log/drill/node11/drillbits.out) doesn't contain anything about >> > > > the inclusion of Hive. >> > > > >> > > > (I'm assuming that SELECT * FROM INFORMATION_SCHEMA.TABLES will >> return >> > > more >> > > > than the usual 5 tables (VIEW/COLUMNS/TABLES/CATALOGS/SCHEMATA.) >> > > > >> > > > George >> > > > >> > > > >> > > > On Tue, Mar 25, 2014 at 6:12 PM, Timothy Chen >> > > > <[email protected]> >> > wrote: >> > > > >> > > > > Hi George, >> > > > > >> > > > > I believe we have a wiki section on setting up hive + drill : >> > > > > >> > > > > >> > https://cwiki.apache.org/confluence/display/DRILL/Hive+Storage+Engin >> > e >> > > > > >> > > > > I haven't personally tried it yet but see if this helps. >> > > > > >> > > > > Tim >> > > > > >> > > > > >> > > > > On Tue, Mar 25, 2014 at 6:10 PM, George Chow >> > > > > <[email protected]> >> > > > wrote: >> > > > > >> > > > > > Hi, >> > > > > > >> > > > > > So I'm trying to setup a drillbit to reach across to my Hive >> > > > metastore. I >> > > > > > dug into JIRA and see that DRILL-354/357 are both resolved. >> > > > > > I've >> > also >> > > > > > looked into my drillbit and found .../conf/storage-engines.json. >> > > > > > >> > > > > > Inside, I see an entry for Hive that's commented out. >> > > > > > However, it >> > is >> > > > for >> > > > > an >> > > > > > instance Hive with a Derby metastore. My instance is a >> HiveServer1 >> > > > with a >> > > > > > MySQL metastore. So both javax.jdo.option.ConnectionURL and >> > > > > > hive.metastore.sasl.enabled are clear to me. But what about >> > > > > > the >> > > > > remaining: >> > > > > > >> > > > > > hive.metastore.uris >> > > > > > hive.metastore.warehouse.dir >> > > > > > fs.default.name >> > > > > > >> > > > > > George >> > > > > > >> > > > > > -- >> > > > > > -- >> > > > > > "Not everything that can be counted counts, and not >> > > > > > everything >> that >> > > > > counts >> > > > > > can be counted." Albert Einstein >> > > > > > >> > > > > >> > > > >> > > > >> > > > >> > > > -- >> > > > -- >> > > > "Not everything that can be counted counts, and not everything >> > > > that >> > > counts >> > > > can be counted." Albert Einstein >> > > > >> > > >> > >> > >> > >> > -- >> > -- >> > "Not everything that can be counted counts, and not everything that >> counts >> > can be counted." Albert Einstein >> > >> > -- > -- > "Not everything that can be counted counts, and not everything that counts > can be counted." Albert Einstein
