I am trying to get hbase to run directly over s3n (much preferable) or s3 (less
preferable). I ran into a few different problems that I solved (such as
getting jets3t installed), but now i am stuck!! Sorry for the long post, but I
wanted to include some error messages.
I am on 0.90.3 installed on a Mac using homebrew. I have hadoop, but that
doesn't come in to play as I am not using HDFS.
The configuration I am using is below (a few replacements of keys for
anonymity).
<configuration>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
<property>
<name>hbase.rootdir</name>
<value>s3n://mcorner-hbase/</value> (note below, I used different values
for this)
</property>
<property>
<name>fs.s3n.awsAccessKeyId</name>
<value>mykeyid</value>
</property>
<property>
<name>fs.s3n.awsSecretAccessKey</name>
<value>mysecret</value>
</property>
</configuration>
1) If I use this hbase.rootdir with s3n (this seems to be the preferred format…)
<value>s3n://mcorner-hbase.s3.aws.com:80/data
(http://mcorner-hbase.s3.aws.com:80/data)</value>
then I get this failure.
2011-11-27 10:03:23,572 WARN
org.jets3t.service.impl.rest.httpclient.RestS3Service: Error Response: HEAD
'/data' -- ResponseCode: 404, ResponseStatus: Not Found, Request Headers:
[Content-Type: , Date: Sun, 27 Nov 2011 15:03:22 GMT, Authorization: AWS
AKIAJSHPUUTMSSRCOTBA:R0zegLCQbQyGxLVjPbzUobQMqHE=, User-Agent: JetS3t/0.7.4-dev
(Mac OS X/10.7.2; x86_64; en; JVM 1.6.0_26), Host:
mcorner-hbase.s3.aws.com.s3.amazonaws.com
(http://mcorner-hbase.s3.aws.com.s3.amazonaws.com)], Response Headers:
[x-amz-request-id: 0BCBF072804E621A, x-amz-id-2:
KHIjDEAcpDjGCXuEQ0c/V7hGr/iH5vqo8utMokZp+YCIw7U6SbVNem3RvtmBESCm, Content-Type:
application/xml, Transfer-Encoding: chunked, Date: Sun, 27 Nov 2011 15:03:22
GMT, Server: AmazonS3]
2011-11-27 10:03:23,573 FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled
exception. Starting shutdown.
org.apache.hadoop.fs.s3.S3Exception: org.jets3t.service.S3ServiceException:
Request Error. HEAD '/data' on Host 'mcorner-hbase.s3.aws.com.s3.amazonaws.com
(http://mcorner-hbase.s3.aws.com.s3.amazonaws.com)' @ 'Sun, 27 Nov 2011
15:03:22 GMT' -- ResponseCode: 404, ResponseStatus: Not Found, RequestId:
0BCBF072804E621A, HostId:
KHIjDEAcpDjGCXuEQ0c/V7hGr/iH5vqo8utMokZp+YCIw7U6SbVNem3RvtmBESCm
at
org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.retrieveMetadata(Jets3tNativeFileSystemStore.java:122)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
So it looks like it is appending other things to the URL:
"mcorner-hbase.s3.aws.com.s3.amazonaws.com
(http://mcorner-hbase.s3.aws.com.s3.amazonaws.com)"
2) If I replace the rootdir with:
<value>s3n://mcorner-hbase:80/data</value>
same problem:
2011-11-27 10:18:21,848 WARN
org.jets3t.service.impl.rest.httpclient.RestS3Service: Error Response: HEAD
'/data' -- ResponseCode: 404, ResponseStatus: Not Found, Request Headers:
[Content-Type: , Date: Sun, 27 Nov 2011 15:18:20 GMT, Authorization: AWS
AKIAJSHPUUTMSSRCOTBA:3tlyWfqbSkZB9lKQUCbQ8DW2VKk=, User-Agent: JetS3t/0.7.4-dev
(Mac OS X/10.7.2; x86_64; en; JVM 1.6.0_26), Host:
mcorner-hbase.s3.amazonaws.com (http://mcorner-hbase.s3.amazonaws.com)],
Response Headers: [x-amz-request-id: 2AFB15695EF68E13, x-amz-id-2:
grz/Sfl7OWZEjSFd+MiiPe8he43g/LfZWrtt6zT3oOpg7DlWxqtyDq7vlCLUlcOy, Content-Type:
application/xml, Transfer-Encoding: chunked, Date: Sun, 27 Nov 2011 15:18:21
GMT, Server: AmazonS3]
2011-11-27 10:18:21,849 FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled
exception. Starting shutdown.
org.apache.hadoop.fs.s3.S3Exception: org.jets3t.service.S3ServiceException:
Request Error. HEAD '/data' on Host 'mcorner-hbase.s3.amazonaws.com
(http://mcorner-hbase.s3.amazonaws.com)' @ 'Sun, 27 Nov 2011 15:18:21 GMT' --
ResponseCode: 404, ResponseStatus: Not Found, RequestId: 2AFB15695EF68E13,
HostId: grz/Sfl7OWZEjSFd+MiiPe8he43g/LfZWrtt6zT3oOpg7DlWxqtyDq7vlCLUlcOy
3) If I switch to s3 (and change the key properties to match), with
s3n://mcorner-hbase:80/data, then I get:
2011-11-27 10:08:55,951 WARN
org.jets3t.service.impl.rest.httpclient.RestS3Service: Error Response: GET
'/%2Fdata' -- ResponseCode: 404, ResponseStatus: Not Found, Request Headers:
[Content-Type: , Date: Sun, 27 Nov 2011 15:08:54 GMT, Authorization: AWS
AKIAJSHPUUTMSSRCOTBA:nKXZEBy/WIinzNByPX7JKBasSVw=, User-Agent: JetS3t/0.7.4-dev
(Mac OS X/10.7.2; x86_64; en; JVM 1.6.0_26), Host:
mcorner-hbase.s3.aws.com.s3.amazonaws.com
(http://mcorner-hbase.s3.aws.com.s3.amazonaws.com)], Response Headers:
[x-amz-request-id: B8A5A618F8F86B87, x-amz-id-2:
7EbeGYqAaIC2XApd66aGsEHEq8cW4r2D1YycYSkLLk9WmnIX7HDPYhWpl+pm4u4B, Content-Type:
application/xml, Transfer-Encoding: chunked, Date: Sun, 27 Nov 2011 15:08:55
GMT, Server: AmazonS3]
2011-11-27 10:08:55,958 FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled
exception. Starting shutdown.
org.apache.hadoop.fs.s3.S3Exception: org.jets3t.service.S3ServiceException: S3
Error Message. GET '/%2Fdata' on Host
'mcorner-hbase.s3.aws.com.s3.amazonaws.com
(http://mcorner-hbase.s3.aws.com.s3.amazonaws.com)' @ 'Sun, 27 Nov 2011
15:08:55 GMT' -- ResponseCode: 404, ResponseStatus: Not Found, XML Error
Message: <?xml version="1.0"
encoding="UTF-8"?><Error><Code>NoSuchBucket</Code><Message>The specified bucket
does not exist</Message><BucketName>mcorner-hbase.s3.aws.com
(http://mcorner-hbase.s3.aws.com)</BucketName><RequestId>B8A5A618F8F86B87</RequestId><HostId>7EbeGYqAaIC2XApd66aGsEHEq8cW4r2D1YycYSkLLk9WmnIX7HDPYhWpl+pm4u4B</HostId></Error>
4) The closest I get is with <value>s3://mcorner-hbase/data</value>. Then I
get what appears to be more progress, but it appears to be creating files (I
can see a file called "block_-222…." in my bucket, and a directory with a blank
name, but the bootstrapping process eventually bombs out as you can se.
2011-11-27 10:11:12,793 WARN
org.jets3t.service.impl.rest.httpclient.RestS3Service: Error Response: GET
'/%2Fdata' -- ResponseCode: 404, ResponseStatus: Not Found, Request Headers:
[Content-Type: , Date: Sun, 27 Nov 2011 15:11:11 GMT, Authorization: AWS
AKIAJSHPUUTMSSRCOTBA:rRR32CQb1xmqLv9PgEwOIVaTLMQ=, User-Agent: JetS3t/0.7.4-dev
(Mac OS X/10.7.2; x86_64; en; JVM 1.6.0_26), Host:
mcorner-hbase.s3.amazonaws.com (http://mcorner-hbase.s3.amazonaws.com)],
Response Headers: [x-amz-request-id: 2296D45E1390890C, x-amz-id-2:
ZT9IpXdAIAqWu+owdtilWQWj+9nRQmH48IhOFoWajFhuRSEOcTplMjekzZa8wkXb, Content-Type:
application/xml, Transfer-Encoding: chunked, Date: Sun, 27 Nov 2011 15:11:11
GMT, Server: AmazonS3]
2011-11-27 10:11:12,905 WARN
org.jets3t.service.impl.rest.httpclient.RestS3Service: Error Response: GET
'/%2F' -- ResponseCode: 404, ResponseStatus: Not Found, Request Headers:
[Content-Type: , Date: Sun, 27 Nov 2011 15:11:12 GMT, Authorization: AWS
AKIAJSHPUUTMSSRCOTBA:Be5+Q5QRILCtbJzUA6HcHEP9YZ8=, User-Agent: JetS3t/0.7.4-dev
(Mac OS X/10.7.2; x86_64; en; JVM 1.6.0_26), Host:
mcorner-hbase.s3.amazonaws.com (http://mcorner-hbase.s3.amazonaws.com)],
Response Headers: [x-amz-request-id: 6B6159CBC16B4742, x-amz-id-2:
4mBHDq4YdtrIBQS63x8zKjjpt2md2raM+LcXpIaGwybGrRPUjxOM53ZuYBab37FO, Content-Type:
application/xml, Transfer-Encoding: chunked, Date: Sun, 27 Nov 2011 15:11:11
GMT, Server: AmazonS3]
2011-11-27 10:11:13,332 WARN
org.jets3t.service.impl.rest.httpclient.RestS3Service: Error Response: GET
'/%2Fdata' -- ResponseCode: 404, ResponseStatus: Not Found, Request Headers:
[Content-Type: , Date: Sun, 27 Nov 2011 15:11:13 GMT, Authorization: AWS
AKIAJSHPUUTMSSRCOTBA:xPkvn6VtlerDOhZL604gwM3848c=, User-Agent: JetS3t/0.7.4-dev
(Mac OS X/10.7.2; x86_64; en; JVM 1.6.0_26), Host:
mcorner-hbase.s3.amazonaws.com (http://mcorner-hbase.s3.amazonaws.com)],
Response Headers: [x-amz-request-id: 46EC664A7B09DC6B, x-amz-id-2:
9mb0kcVcdjm86x69r9QDbN6cATm3nUKSPIfcxs7rs79M5CZ527z25F2Y3c6CzRdW, Content-Type:
application/xml, Transfer-Encoding: chunked, Date: Sun, 27 Nov 2011 15:11:12
GMT, Server: AmazonS3]
2011-11-27 10:11:13,604 WARN
org.jets3t.service.impl.rest.httpclient.RestS3Service: Error Response: GET
'/%2Fdata%2Fhbase.version' -- ResponseCode: 404, ResponseStatus: Not Found,
Request Headers: [Content-Type: , Date: Sun, 27 Nov 2011 15:11:13 GMT,
Authorization: AWS AKIAJSHPUUTMSSRCOTBA:6iBF8bkt+roA4N5/1RvL7X4qP9M=,
User-Agent: JetS3t/0.7.4-dev (Mac OS X/10.7.2; x86_64; en; JVM 1.6.0_26), Host:
mcorner-hbase.s3.amazonaws.com (http://mcorner-hbase.s3.amazonaws.com)],
Response Headers: [x-amz-request-id: 640F58D7AAF536E3, x-amz-id-2:
QY3YqAO4+652mNCPaIIqMe3tKTigUs+YlOj/vzKuRPJXiCD7DXEmziqdX/145Xov, Content-Type:
application/xml, Transfer-Encoding: chunked, Date: Sun, 27 Nov 2011 15:11:12
GMT, Server: AmazonS3]
2011-11-27 10:11:14,023 DEBUG org.apache.hadoop.hbase.util.FSUtils: Created
version file at s3://mcorner-hbase/data set its version at:7
2011-11-27 10:11:14,281 WARN
org.jets3t.service.impl.rest.httpclient.RestS3Service: Error Response: GET
'/block_-2255577201702834920' -- ResponseCode: 404, ResponseStatus: Not Found,
Request Headers: [Content-Type: , Date: Sun, 27 Nov 2011 15:11:14 GMT,
Authorization: AWS AKIAJSHPUUTMSSRCOTBA:gICpmb7gELgfm2sW02O2OBQGSPw=,
User-Agent: JetS3t/0.7.4-dev (Mac OS X/10.7.2; x86_64; en; JVM 1.6.0_26), Host:
mcorner-hbase.s3.amazonaws.com (http://mcorner-hbase.s3.amazonaws.com)],
Response Headers: [x-amz-request-id: DE383F7CAFC6856E, x-amz-id-2:
y3ffN4iTYIkR+iOJI9uJkW8xntsdBZHq4miEsO1qz1YLdJlAlpBjNM30n2ASx9bs, Content-Type:
application/xml, Transfer-Encoding: chunked, Date: Sun, 27 Nov 2011 15:11:13
GMT, Server: AmazonS3]
2011-11-27 10:11:14,836 WARN
org.jets3t.service.impl.rest.httpclient.RestS3Service: Error Response: GET
'/%2Fdata%2F-ROOT-%2F70236052' -- ResponseCode: 404, ResponseStatus: Not Found,
Request Headers: [Content-Type: , Date: Sun, 27 Nov 2011 15:11:14 GMT,
Authorization: AWS AKIAJSHPUUTMSSRCOTBA:XbfWC0XVWS53IAEHGTNvkBZ9gQk=,
User-Agent: JetS3t/0.7.4-dev (Mac OS X/10.7.2; x86_64; en; JVM 1.6.0_26), Host:
mcorner-hbase.s3.amazonaws.com (http://mcorner-hbase.s3.amazonaws.com)],
Response Headers: [x-amz-request-id: 25AE3E24162F7850, x-amz-id-2:
Q2oOyCd2NDQPiY2FjJPuI8pvhTid/dLyhFGQymF3SDvHkT0krDh79JerjxBt4FGY, Content-Type:
application/xml, Transfer-Encoding: chunked, Date: Sun, 27 Nov 2011 15:11:14
GMT, Server: AmazonS3]
2011-11-27 10:11:14,838 INFO org.apache.hadoop.hbase.master.MasterFileSystem:
BOOTSTRAP: creating ROOT and first META regions
2011-11-27 10:11:15,462 WARN
org.jets3t.service.impl.rest.httpclient.RestS3Service: Error Response: GET
'/%2Fdata%2F-ROOT-' -- ResponseCode: 404, ResponseStatus: Not Found, Request
Headers: [Content-Type: , Date: Sun, 27 Nov 2011 15:11:15 GMT, Authorization:
AWS AKIAJSHPUUTMSSRCOTBA:GdG0M+6O3+Cbls3DzXSfEIrISDw=, User-Agent:
JetS3t/0.7.4-dev (Mac OS X/10.7.2; x86_64; en; JVM 1.6.0_26), Host:
mcorner-hbase.s3.amazonaws.com (http://mcorner-hbase.s3.amazonaws.com)],
Response Headers: [x-amz-request-id: AEEB8EFE863A9B5A, x-amz-id-2:
P+wk8B2p3sRb/MR3uwG1jjH8BFwT6xHM3GXmoXjRwt/sc01NVNrJVUpr48j3Df4a, Content-Type:
application/xml, Transfer-Encoding: chunked, Date: Sun, 27 Nov 2011 15:11:14
GMT, Server: AmazonS3]
2011-11-27 10:11:15,735 WARN
org.jets3t.service.impl.rest.httpclient.RestS3Service: Error Response: GET
'/%2Fdata%2F-ROOT-%2F70236052' -- ResponseCode: 404, ResponseStatus: Not Found,
Request Headers: [Content-Type: , Date: Sun, 27 Nov 2011 15:11:15 GMT,
Authorization: AWS AKIAJSHPUUTMSSRCOTBA:qgFz5nvPmwHYLIKudvclcGJK2gw=,
User-Agent: JetS3t/0.7.4-dev (Mac OS X/10.7.2; x86_64; en; JVM 1.6.0_26), Host:
mcorner-hbase.s3.amazonaws.com (http://mcorner-hbase.s3.amazonaws.com)],
Response Headers: [x-amz-request-id: 7856EA5209B7D4A8, x-amz-id-2:
lAQR13CX8ZfwSMKjl4SfErVtQPUwss5zCrcPg2tV5CpJ1bEmixNOpACIEVzwW/jf, Content-Type:
application/xml, Transfer-Encoding: chunked, Date: Sun, 27 Nov 2011 15:11:14
GMT, Server: AmazonS3]
2011-11-27 10:11:16,024 WARN
org.jets3t.service.impl.rest.httpclient.RestS3Service: Error Response: GET
'/%2Fdata%2F-ROOT-%2F70236052%2F.logs' -- ResponseCode: 404, ResponseStatus:
Not Found, Request Headers: [Content-Type: , Date: Sun, 27 Nov 2011 15:11:15
GMT, Authorization: AWS AKIAJSHPUUTMSSRCOTBA:go7R49fCUX01Gn5thKP4nl2emh0=,
User-Agent: JetS3t/0.7.4-dev (Mac OS X/10.7.2; x86_64; en; JVM 1.6.0_26), Host:
mcorner-hbase.s3.amazonaws.com (http://mcorner-hbase.s3.amazonaws.com)],
Response Headers: [x-amz-request-id: 9AB9F6F7CE9012CD, x-amz-id-2:
GifVpi0k+F/se1gjESuUyFzUtSY93GyvLggqpWn6ZgxTMhwEw0YMa5jG1FTHhA33, Content-Type:
application/xml, Transfer-Encoding: chunked, Date: Sun, 27 Nov 2011 15:11:14
GMT, Server: AmazonS3]
2011-11-27 10:11:17,636 WARN
org.jets3t.service.impl.rest.httpclient.RestS3Service: Error Response: GET
'/%2Fdata%2F-ROOT-%2F70236052%2F.logs' -- ResponseCode: 404, ResponseStatus:
Not Found, Request Headers: [Content-Type: , Date: Sun, 27 Nov 2011 15:11:16
GMT, Authorization: AWS AKIAJSHPUUTMSSRCOTBA:pUj2QwlQa7ypigvoUs4LHikhDd0=,
User-Agent: JetS3t/0.7.4-dev (Mac OS X/10.7.2; x86_64; en; JVM 1.6.0_26), Host:
mcorner-hbase.s3.amazonaws.com (http://mcorner-hbase.s3.amazonaws.com)],
Response Headers: [x-amz-request-id: 5120CAE72D290EB7, x-amz-id-2:
aCkMJu4F5FrvRAYRHac/CTxGsgSN3VgBfAEo/jVW0OeXaQNvZ09y6Qu8moV03fvp, Content-Type:
application/xml, Transfer-Encoding: chunked, Date: Sun, 27 Nov 2011 15:11:16
GMT, Server: AmazonS3]
2011-11-27 10:11:17,926 WARN
org.jets3t.service.impl.rest.httpclient.RestS3Service: Error Response: GET
'/%2Fdata%2F-ROOT-%2F70236052%2F.oldlogs' -- ResponseCode: 404, ResponseStatus:
Not Found, Request Headers: [Content-Type: , Date: Sun, 27 Nov 2011 15:11:17
GMT, Authorization: AWS AKIAJSHPUUTMSSRCOTBA:8fAMMGghzJs6ICOfqy5Euf0nuC8=,
User-Agent: JetS3t/0.7.4-dev (Mac OS X/10.7.2; x86_64; en; JVM 1.6.0_26), Host:
mcorner-hbase.s3.amazonaws.com (http://mcorner-hbase.s3.amazonaws.com)],
Response Headers: [x-amz-request-id: BEDBB9E931FEE069, x-amz-id-2:
R9sMP4IkMOXsUCCzZErJYt9qGHhadkKstadB7pcpo0XKX9l4L+Clna4SPfZhNzyp, Content-Type:
application/xml, Transfer-Encoding: chunked, Date: Sun, 27 Nov 2011 15:11:17
GMT, Server: AmazonS3]
2011-11-27 10:11:20,130 WARN
org.jets3t.service.impl.rest.httpclient.RestS3Service: Error Response: GET
'/%2Fdata%2F-ROOT-%2F70236052%2F.oldlogs' -- ResponseCode: 404, ResponseStatus:
Not Found, Request Headers: [Content-Type: , Date: Sun, 27 Nov 2011 15:11:19
GMT, Authorization: AWS AKIAJSHPUUTMSSRCOTBA:Hg1cqsBmprgya4GXSeAOVhVoju0=,
User-Agent: JetS3t/0.7.4-dev (Mac OS X/10.7.2; x86_64; en; JVM 1.6.0_26), Host:
mcorner-hbase.s3.amazonaws.com (http://mcorner-hbase.s3.amazonaws.com)],
Response Headers: [x-amz-request-id: 8752AD561C46847E, x-amz-id-2:
6bA9vjM4kLhQoL0uaY2hG30atmaYKYAXJaIb3utehSqXlIlH8UoN1KG0lnwe4UfR, Content-Type:
application/xml, Transfer-Encoding: chunked, Date: Sun, 27 Nov 2011 15:11:18
GMT, Server: AmazonS3]
2011-11-27 10:11:20,307 INFO org.apache.hadoop.hbase.regionserver.wal.HLog:
HLog configuration: blocksize=32 MB, rollsize=30.4 MB, enabled=true,
flushlogentries=1, optionallogflushinternal=1000ms
2011-11-27 10:11:20,403 WARN
org.jets3t.service.impl.rest.httpclient.RestS3Service: Error Response: GET
'/%2Fdata%2F-ROOT-%2F70236052%2F.logs%2Fhlog.1322406680307' -- ResponseCode:
404, ResponseStatus: Not Found, Request Headers: [Content-Type: , Date: Sun, 27
Nov 2011 15:11:20 GMT, Authorization: AWS
AKIAJSHPUUTMSSRCOTBA:IaE+/XqX0aUmfo0LJw9O7pCsV1A=, User-Agent: JetS3t/0.7.4-dev
(Mac OS X/10.7.2; x86_64; en; JVM 1.6.0_26), Host:
mcorner-hbase.s3.amazonaws.com (http://mcorner-hbase.s3.amazonaws.com)],
Response Headers: [x-amz-request-id: F84C1DC251EB6F09, x-amz-id-2:
viG+7saKo8ZiPro1oWyNNqh8kAnve/5nddl35mETpf6udDScbbEY3ZCqV0pB7uJR, Content-Type:
application/xml, Transfer-Encoding: chunked, Date: Sun, 27 Nov 2011 15:11:19
GMT, Server: AmazonS3]
2011-11-27 10:11:21,578 INFO
org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter: syncFs --
HDFS-200 -- not available, dfs.support.append=false
2011-11-27 10:11:21,837 WARN
org.jets3t.service.impl.rest.httpclient.RestS3Service: Error Response: GET
'/%2Fdata%2F-ROOT-%2F70236052%2F.logs%2Fhlog.1322406680307' -- ResponseCode:
404, ResponseStatus: Not Found, Request Headers: [Content-Type: , Date: Sun, 27
Nov 2011 15:11:21 GMT, Authorization: AWS
AKIAJSHPUUTMSSRCOTBA:5l0MMGpPTR3OCqmgji3W3bWoLgQ=, User-Agent: JetS3t/0.7.4-dev
(Mac OS X/10.7.2; x86_64; en; JVM 1.6.0_26), Host:
mcorner-hbase.s3.amazonaws.com (http://mcorner-hbase.s3.amazonaws.com)],
Response Headers: [x-amz-request-id: 9B1502504A01DCF4, x-amz-id-2:
Kdzu/QRSHRgrrURTYiwijDgEIHcdZZZzGhleYsgsuLYlSEQ5D5WKCrhYhcDuZwHw, Content-Type:
application/xml, Transfer-Encoding: chunked, Date: Sun, 27 Nov 2011 15:11:20
GMT, Server: AmazonS3]
2011-11-27 10:11:21,839 ERROR org.apache.hadoop.hbase.master.MasterFileSystem:
bootstrap
java.io.FileNotFoundException:
s3://mcorner-hbase/data/-ROOT-/70236052/.logs/hlog.1322406680307: No such file
or directory.
at org.apache.hadoop.fs.s3.S3FileSystem.getFileStatus(S3FileSystem.java:334)
at org.apache.hadoop.hbase.regionserver.wal.HLog.rollWriter(HLog.java:486)
at org.apache.hadoop.hbase.regionserver.wal.HLog.<init>(HLog.java:374)
5) I have also tried creating the data directory in my s3 bucket, and that
doesn't help.
2011-11-27 10:20:18,003 WARN
org.jets3t.service.impl.rest.httpclient.RestS3Service: Error Response: HEAD
'/data' -- ResponseCode: 404, ResponseStatus: Not Found, Request Headers:
[Content-Type: , Date: Sun, 27 Nov 2011 15:20:15 GMT, Authorization: AWS
AKIAJSHPUUTMSSRCOTBA:AqeMf6wmBZGJye7qwiYFrG6ozSA=, User-Agent: JetS3t/0.7.4-dev
(Mac OS X/10.7.2; x86_64; en; JVM 1.6.0_26), Host:
mcorner-hbase.s3.amazonaws.com (http://mcorner-hbase.s3.amazonaws.com)],
Response Headers: [x-amz-request-id: 168138BCA8FCBD1D, x-amz-id-2:
o4gIuMOBc1Jf+Gw2NI2rWYCNBf/5uDVk6bL9TURfhkrTYGJ5JaZQc4J0Lq9k0jko, Content-Type:
application/xml, Transfer-Encoding: chunked, Date: Sun, 27 Nov 2011 15:20:17
GMT, Server: AmazonS3]
2011-11-27 10:20:18,004 FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled
exception. Starting shutdown.
org.apache.hadoop.fs.s3.S3Exception: org.jets3t.service.S3ServiceException:
Request Error. HEAD '/data' on Host 'mcorner-hbase.s3.amazonaws.com
(http://mcorner-hbase.s3.amazonaws.com)' @ 'Sun, 27 Nov 2011 15:20:17 GMT' --
ResponseCode: 404, ResponseStatus: Not Found, RequestId: 168138BCA8FCBD1D,
HostId: o4gIuMOBc1Jf+Gw2NI2rWYCNBf/5uDVk6bL9TURfhkrTYGJ5JaZQc4J0Lq9k0jko
What is the magic combination here?? Do you need to create the directory?
What are you using for the rootdir url??
--
Mark Corner
