Could you please tell us which Hadoop and Hive version are you using? Looks like you might be using an older version of Hadoop (more specifically one which ships with old version of jets3t).
Thanks Vaibhav From: Wouter de Bie [mailto:wou...@spotify.com] Sent: Wednesday, July 06, 2011 9:07 AM To: user@hive.apache.org Subject: Hive session locking up after 4 queries using S3 Hi all, I'm using Hive with the s3native FS. Today, I noticed that hive locks up after 4 queries that directly access S3 (select * from mytable limit 10). With debug logging on, I get the following output: 2011-07-06 15:54:31,459 DEBUG s3native.NativeS3FileSystem (NativeS3FileSystem.java:getFileStatus(393)) - getFileStatus retrieving metadata for key 'tmp/hive-mapred/hive_2011-07-06_15-54-29_881_4253697128840334916/-mr-10000' 2011-07-06 15:54:31,459 DEBUG httpclient.RestS3Service (RestS3Service.java:getObjectImpl(1511)) - Retrieving Head information for bucket XXXXXXXX and object tmp/hive-mapred/hive_2011-07-06_15-54-29_881_4253697128840334916/-mr-10000 2011-07-06 15:54:31,460 DEBUG service.Jets3tProperties (Jets3tProperties.java:getBoolProperty(314)) - s3service.disable-dns-buckets=false 2011-07-06 15:54:31,460 DEBUG httpclient.RestS3Service (RestS3Service.java:setupConnection(811)) - S3 URL: https://XXXXXXXX:443/tmp%2Fhive-mapred%2Fhive_2011-07-06_15-54-29_881_4253697128840334916%2F-mr-10000 2011-07-06 15:54:31,460 DEBUG httpclient.RestS3Service (RestS3Service.java:performRequest(334)) - Performing HEAD request for 'https://XXXXXXXXXXXXXX/tmp%2Fhive-mapred%2Fhive_2011-07-06_15-54-29_881_4253697128840334916%2F-mr-10000', expecting response code 200 2011-07-06 15:54:31,461 DEBUG httpclient.RestS3Service (RestS3Service.java:buildAuthorizationString(872)) - Adding authorization for AWS Access Key 'XXXXXXXXXXXXX'. 2011-07-06 15:54:31,461 DEBUG httpclient.RestS3Service (RestS3Service.java:buildAuthorizationString(922)) - Canonical string ('|' is a newline): HEAD|||Wed, 06 Jul 2011 15:54:31 GMT|/XXXXXXXXX/tmp%2Fhive-mapred%2Fhive_2011-07-06_15-54-29_881_4253697128840334916%2F-mr-10000 2011-07-06 15:54:31,461 DEBUG httpclient.HttpClient (HttpClient.java:executeMethod(322)) - enter HttpClient.executeMethod(HttpMethod) 2011-07-06 15:54:31,462 DEBUG httpclient.HttpClient (HttpClient.java:executeMethod(373)) - enter HttpClient.executeMethod(HostConfiguration,HttpMethod,HttpState) 2011-07-06 15:54:31,462 DEBUG httpclient.MultiThreadedHttpConnectionManager (MultiThreadedHttpConnectionManager.java:getConnectionWithTimeout(383)) - enter HttpConnectionManager.getConnectionWithTimeout(HostConfiguration, long) 2011-07-06 15:54:31,462 DEBUG httpclient.MultiThreadedHttpConnectionManager (MultiThreadedHttpConnectionManager.java:getConnectionWithTimeout(390)) - HttpConnectionManager.getConnection: config = HostConfiguration[host=https://XXXXXXXXX.s3.amazonaws.com], timeout = 0 2011-07-06 15:54:31,462 DEBUG httpclient.MultiThreadedHttpConnectionManager (MultiThreadedHttpConnectionManager.java:getHostPool(775)) - enter HttpConnectionManager.ConnectionPool.getHostPool(HostConfiguration) 2011-07-06 15:54:31,463 DEBUG httpclient.MultiThreadedHttpConnectionManager (MultiThreadedHttpConnectionManager.java:doGetConnection(494)) - Unable to get a connection, waiting..., hostConfig=HostConfiguration[host=https://XXXXXXXXXX.s3.amazonaws.com] Does anyone know if I can do anything to prevent this? It looks like connections are not returned correctly to the pool.. // Wouter
