Christopher,
Thank for reporting this behaviour.
I committed the attached patch.
Jean-Louis
On 08/16/2011 06:50 PM, Christopher McCrory wrote:
Hello...
I ran into an error/bug using amanda with S3. It had been working great
for a while and then started erroring out last week.
Tue Aug 16 05:01:32 2011: taper: PUT https://s3.amazonaws.com/mybucket
failed with 400/TooManyBuckets
Tue Aug 16 05:01:32 2011: taper: Device s3:mybucket/slot-10/ error =
'While creating new S3 bucket: You have attempted to create more buckets
than allowed (TooManyBuckets) (HTTP 400)'
Tue Aug 16 05:01:32 2011: taper: Device s3:mybucket/slot-10/ setting
status flag(s): DEVICE_STATUS_DEVICE_ERROR
Tue Aug 16 05:01:32 2011: taper: error (fatal): file device.c: line 1097
(device_read_label): assertion failed: (self->access_mode ==
ACCESS_NULL)
I upgraded from 3.2.3 to 3.3.0 hoping for more insight and newer code.
The error messages are much better. But the error still happens, but
the reason shows up in the email!
in device-src/s3-device.c :
/* try creating the bucket, in case it doesn't exist */
if (mode != ACCESS_READ&& !s3_make_bucket(self->s3t[0].s3,
self->bucket)) {
guint response_code;
I think this is a logic error. In my case the bucket already exists and
we are at our bucket limit so the create bucket code returns an error
and nothing is ever written to S3. I think the more correct logic is to
test is the bucket exists and only try to create it if it does not
exist.
Unfortunately, my C codeing skills are not up to submitting a patch.
And a big thanks to the s3 code writer, this is an awesome feature.
diff --git a/device-src/s3-device.c b/device-src/s3-device.c
index b419ffa..1e0de8f 100644
--- a/device-src/s3-device.c
+++ b/device-src/s3-device.c
@@ -1652,7 +1652,9 @@ s3_device_start (Device * pself, DeviceAccessMode mode,
pself->in_file = FALSE;
/* try creating the bucket, in case it doesn't exist */
- if (mode != ACCESS_READ && !s3_make_bucket(self->s3t[0].s3, self->bucket))
{
+ if (mode != ACCESS_READ &&
+ !s3_is_bucket_exists(self->s3t[0].s3, self->bucket) &&
+ !s3_make_bucket(self->s3t[0].s3, self->bucket)) {
guint response_code;
s3_error_code_t s3_error_code;
s3_error(self->s3t[0].s3, NULL, &response_code, &s3_error_code, NULL,
NULL, NULL);
diff --git a/device-src/s3.c b/device-src/s3.c
index a8ff762..f0afa10 100644
--- a/device-src/s3.c
+++ b/device-src/s3.c
@@ -2130,6 +2130,25 @@ cleanup:
}
gboolean
+s3_is_bucket_exists(S3Handle *hdl,
+ const char *bucket)
+{
+ s3_result_t result = S3_RESULT_FAIL;
+ static result_handling_t result_handling[] = {
+ { 200, 0, 0, S3_RESULT_OK },
+ { 404, S3_ERROR_NoSuchBucket, 0, S3_RESULT_RETRY },
+ RESULT_HANDLING_ALWAYS_RETRY,
+ { 0, 0, 0, /* default: */ S3_RESULT_FAIL }
+ };
+
+ result = perform_request(hdl, "GET", bucket, NULL, NULL, NULL,
+ NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL,
+ NULL, NULL, result_handling);
+
+ return result == S3_RESULT_OK;
+}
+
+gboolean
s3_delete_bucket(S3Handle *hdl,
const char *bucket)
{
diff --git a/device-src/s3.h b/device-src/s3.h
index c5ec607..1b8cb2a 100644
--- a/device-src/s3.h
+++ b/device-src/s3.h
@@ -424,6 +424,16 @@ gboolean
s3_make_bucket(S3Handle *hdl,
const char *bucket);
+/* Check if a bucket exists.
+ *
+ * @param hdl: the S3Handle object
+ * @param bucket: the bucket to create
+ * @returns: FALSE if an error occur
+ */
+gboolean
+s3_is_bucket_exists(S3Handle *hdl,
+ const char *bucket);
+
/* Delete a bucket
*
* @note A bucket can not be deleted if it still contains keys
