This is an issue in 0.4 as well. The only difference between 0.4 and trunk is the error text itself is this:
2012-04-09 19:28:52,898 INFO hive.metastore: Trying to connect to metastore with URI thrift://hcatserver.example.com:9080 2012-04-09 19:28:52,937 INFO hive.metastore: Connected to metastore. 2012-04-09 19:28:52,963 ERROR hive.metastore: Unable to shutdown local metastore client 2012-04-09 19:28:52,963 ERROR hive.metastore: [Ljava.lang.StackTraceElement;@1127b52 Looks like https://issues.apache.org/jira/browse/HCATALOG-236 is the jira for this. On Fri, Apr 6, 2012 at 2:44 PM, Rohini Palaniswamy <[email protected]>wrote: > David, > I looked at the mapred log for your job. > > 2012-04-06 17:32:48,452 INFO hive.metastore: Trying to connect to metastore > with URI thrift://gsbl90831.blue.ygrid.yahoo.com:9080 > 2012-04-06 17:32:48,496 INFO hive.metastore: Connected to metastore. > 2012-04-06 17:32:48,522 ERROR hive.metastore: Unable to shutdown local > metastore client > org.apache.thrift.transport.TTransportException: SASL authentication not > complete > at > org.apache.thrift.transport.TSaslTransport.write(TSaslTransport.java:443) > at > org.apache.thrift.transport.TSaslClientTransport.write(TSaslClientTransport.java:37) > > ......... > > at > com.facebook.fb303.FacebookService$Client.shutdown(FacebookService.java:415) > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.close(HiveMetaStoreClient.java:310) > at > org.apache.hcatalog.mapreduce.FileOutputCommitterContainer.cleanupJob(FileOutputCommitterContainer.java:430) > at > org.apache.hcatalog.mapreduce.FileOutputCommitterContainer.commitJob(FileOutputCommitterContainer.java:255) > > > You are hitting HCATALOG-236. hcat branch-0.4 uses hive-0.8 and you should > not be hitting this with that. Trunk uses hive 0.9 snapshot and that throws > this error. As part of the patch to MultiOutputFormat (HCAT-36) I at least > made the code to ignore exceptions when connection to metastore fails and > just log a warning. So when that gets checked in you can probably use trunk > builds. Till then can use hcat branch-0.4 builds? > > Regards, > Rohini > > On Fri, Apr 6, 2012 at 1:21 PM, David Capwell <[email protected]> wrote: > >> I am using trunk and it seems that pig is not updating partitions for me. >> Is anyone else seeing pig scripts not updated the partitions? Below is >> the following commands run. >> >> CREATE TABLE tmp_table ( >> data string >> ) >> partitioned by ( >> datestamp string >> ,srcid string >> ,action string >> ,testid string >> ) >> stored as rcfile >> location '/tmp/hcat_tmp_tables/tmp_table'; >> >> The end of my pigscript looks like this: >> >> store b into 'default.tmp_table' using >> >> org.apache.hcatalog.pig.HCatStorer('datestamp=20091103,srcid=19174,action=click,testid=NOTESTID'); >> >> >> On HDFS I see the following: >> >> -rw-rw-r-- 3 dcapwell hdfs 13990632 2012-04-06 >> >> 17:32/tmp/hcat_tmp_tables/tmp_table/action=click/datestamp=20091103/srcid=19174/testid=NOTESTID/part-m-00000 >> >> >> When I check the partitions in Hive I see the following: >> ./bin/hive -e "show partitions >> tmp_table" >> OK >> Time taken: 1.461 seconds >> >> So it looks like the pig job finishes just fine, writes the files to HDFS >> but >> the partitions are not getting updated? >> >> >> thanks for your time reading this email. >> > >
