It looks like there is an alternate explanation: attempting to sync the entire bucket fails, but syncing one folder in it works:
$ aws s3 --profile tempbucket sync --region us-east-1 's3://hcp-openaccess-temp' /mnt/myelin/tim/synctest A client error (AccessDenied) occurred when calling the ListObjects operation: Access Denied Completed 1 part(s) with ... file(s) remaining $ aws s3 --profile tempbucket sync --region us-east-1 's3://hcp-openaccess-temp/HCP_1200' /mnt/myelin/tim/synctest download: s3://hcp-openaccess-temp/HCP_1200/100206/.xdlm/100206_3T_Diffusion_unproc.json to ../../synctest/100206/.xdlm/100206_3T_Diffusion_unproc.json download: s3://hcp-openaccess-temp/HCP_1200/100206/.xdlm/100206_3T_rfMRI_REST_fix.json to ../../synctest/100206/.xdlm/100206_3T_rfMRI_REST_fix.json ... So, this may be similar to the fact that you can't mount the bucket's root with s3fs, but you can mount its folders. This may have been by design on our part, setting specific permissions on the bucket to give an error when someone tries something that they probably don't want, but it doesn't seem to be documented. Tim On Thu, Aug 9, 2018 at 2:22 PM, Hodge, Michael <[email protected]> wrote: > > > Hi Xavier, > > > > That message indicates that the access key in your credentials isn’t a > currently valid access key registered with AWS. I’m thinking that would > most likely either be a cut-paste issue or an issue or an issue where the > access and secret keys are switched or something similar. One thing that > does happen is that requesting a new access key invalidates or removes any > prior keys for the account. It seems unlikely, but it’s possible that a > double click on the key generation button could present you with an access > key that’s already been invalidated. > > > > I’d suggest trying again with a fresh set of credentials. If you’re still > seeing this issue, feel free to contact me directly. I’d be able to look > up your account in AWS and get information on what key’s active, the age of > the key, etc. I don’t have access to the secret key value, but the issue > here is that the key value stored in the credentials isn’t a currently > valid one. > > > > Regards, > > > > Mike > > > > *From:* [email protected] <hcp-users-bounces@ > humanconnectome.org> *On Behalf Of *Xavier Guell Paradis > *Sent:* Thursday, August 9, 2018 11:43 AM > *To:* [email protected] > *Subject:* [HCP-Users] AWS sync error > > > > Hello HCP, > > We are a team of researchers designing some deep learning tools we have > created for modality conversions, and wanting to test our library on the > HCP dataset (structural T1 and T2). We are having issues copying from the > s3 bucket. Currently after creating our credentials, running aws configure, > and attempting to sync via: > > > > $aws configure > > [entering access credentials] > > $aws s3 sync s3://hcp-openaccess-temp . > > Ïnvalid Access Key ID¨AWS key does not exist in the records. > > > > We appreciate any recommendations on how to proceed. > > > > Thank you, > > Patrick, Xavier, TJ, Anita, Shreyas, Saige. > > _______________________________________________ > HCP-Users mailing list > [email protected] > http://lists.humanconnectome.org/mailman/listinfo/hcp-users > > > ------------------------------ > > The materials in this message are private and may contain Protected > Healthcare Information or other information of a sensitive nature. If you > are not the intended recipient, be advised that any unauthorized use, > disclosure, copying or the taking of any action in reliance on the contents > of this information is strictly prohibited. If you have received this email > in error, please immediately notify the sender via telephone or return mail. > > _______________________________________________ > HCP-Users mailing list > [email protected] > http://lists.humanconnectome.org/mailman/listinfo/hcp-users > _______________________________________________ HCP-Users mailing list [email protected] http://lists.humanconnectome.org/mailman/listinfo/hcp-users
