Looks good to me but I'm not convinced that removing the file:// prefix is the right way of handling paths on Windows.
I was also suspecting an intermittent AWS failure. I'm glad they managed to fix it. Could you create a patch from that pull request and attach to WHIRR-349 so that we can also handle the multipart uploads before committing? Thanks, -- Andrei Savu / andreisavu.ro On Mon, Aug 1, 2011 at 11:37 PM, Karel Vervaeke <ka...@outerthought.org>wrote: > Hi there. > > Yesterday I have digging into this problem. Before the end of the day > my uploads to eu-west-1 started working intermittently, and by the end > of the day I couldn't reproduce the problem anymore (without any > changes to the code). > > I guess I don't have enough cloud-karma. > > For what it's worth I have sent a pull request with a partial request > for WHIRR-349 > What I've done is just remove the prefix 'file://' from the url. I > don't know if that's correct way to do it for Windows-style paths. > It also includes a testcase for uploading slightly larger blobs (a 1MB > random alphanumeric string). > I haven't gone into the multiparts uploads. > > Regards, > Karel > > On Fri, Jul 29, 2011 at 11:53 AM, Karel Vervaeke <ka...@outerthought.org> > wrote: > > I just got around to testing all 4 combinations: > > > > jclouds 1.0.0 + eu-west-1: doesn't work > > jclouds 1.0.0 + us-east-1: works > > jclouds 1.1.0-SNAPSHOT + eu-west-1: doesn't work > > jclouds 1.1.0-SNAPSHOT + us-east-1: works > > > > Would you be willing to try using the s3 blobstore on eu-west-1 so I > > know if it's just me or not? > > > > Regards, > > Karel > > > > On Fri, Jul 29, 2011 at 10:49 AM, Karel Vervaeke <ka...@outerthought.org> > wrote: > >> with jclouds 1.0.0 it failed with eu-west-1 (several attempts) > >> with jclouds 1.0.0 it worked with us-east-1 on the first try. > >> > >> I still have to try with jclouds 1.1.0-SNAPSHOT on us-east-1, but if > that works, > >> the problem is related to eu-west-1. > >> > >> Regards, > >> Karel > >> > >> On Fri, Jul 29, 2011 at 9:49 AM, Karel Vervaeke <ka...@outerthought.org> > wrote: > >>> Ah, Bruno just alerted me that I need to mention that I am using his > >>> (github) whirr fork, which is using a jclouds 1.1 snapshot. > >>> > >>> I can fall back to using jclouds 1.0 for a second (with the limitation > >>> that I won't be able to use eu-west-1. Just give me a second while I > >>> try that. (And in the mean time I'll open the upload-retry issue) > >>> > >>> Regards, > >>> Karel > >>> > >>> On Fri, Jul 29, 2011 at 9:28 AM, Andrei Savu <savu.and...@gmail.com> > wrote: > >>>> I've just took a quick look at the AWS service status dashboard and > >>>> everything seems to be working there. > >>>> > >>>> The error you are seeing when trying to upload a larger file is > >>>> something that I believe we should handle by retrying the upload from > >>>> scratch. It would be great if you could open an issue with this > >>>> suggestion. > >>>> > >>>> I have no idea why the small upload is failing because we are doing > >>>> the same thing in the integration test and it works. I will spend some > >>>> more time on this later tomorrow. Adrian any ideas? Have you seen > >>>> something similar? > >>>> > >>>> -- Andrei Savu / andreisavu.ro > >>>> > >>>> On Fri, Jul 29, 2011 at 12:06 AM, Karel Vervaeke < > ka...@outerthought.org> wrote: > >>>>> It is pretty consistent. > >>>>> > >>>>> Today there's something new on the menu... > >>>>> http://pastebin.com/MSKDn6rt > >>>>> (strange, because that's using the same file I was yesterday, so > >>>>> nothing really changed except today's date) > >>>>> > >>>>> The number of bytes already written (mentioned in the stack trace) is > >>>>> always different. > >>>>> > >>>>> When I replace the file with something very small (184 bytes, to be > exact), > >>>>> I get the original error again. > >>>>> > >>>>> Regards, > >>>>> Karel > >>>>> > >>>>> On Thu, Jul 28, 2011 at 7:17 PM, Andrei Savu <savu.and...@gmail.com> > wrote: > >>>>>> I've wrote that code but this is the first time I see this error. > >>>>>> > >>>>>> I tend to believe it's somehow related to jclouds. Does it happen > all > >>>>>> the time? How about with different files? > >>>>>> > >>>>>> -- Andrei Savu / andreisavu.ro > >>>>>> > >>>>>> On Thu, Jul 28, 2011 at 8:59 AM, Karel Vervaeke < > ka...@outerthought.org> wrote: > >>>>>>> I'm trying out the local file upload from WHIRR-220, but I'm > running > >>>>>>> in to a problem: > >>>>>>> > >>>>>>> While calling prepareRemoteFileUrl(...) (during the install phase), > I > >>>>>>> get this as the last bit of output from whirr on the console: > >>>>>>> http://pastebin.com/LZ92AUhQ > >>>>>>> > >>>>>>> After that there's no more output on the console for several > minutes > >>>>>>> (I just killed the process after that) > >>>>>>> > >>>>>>> I have tried the BlobCacheTest, which seems to work fine for my AWS > settings. > >>>>>>> Before I start digging into jcloud's code, does anyone have any > >>>>>>> thought as to what might cause this? > >>>>>>> > >>>>>>> -- > >>>>>>> Karel Vervaeke > >>>>>>> http://outerthought.org/ > >>>>>>> Open Source Content Applications > >>>>>>> Makers of Kauri, Daisy CMS and Lily > >>>>>>> > >>>>>> > >>>>> > >>>>> > >>>>> > >>>>> -- > >>>>> Karel Vervaeke > >>>>> http://outerthought.org/ > >>>>> Open Source Content Applications > >>>>> Makers of Kauri, Daisy CMS and Lily > >>>>> > >>>> > >>> > >>> > >>> > >>> -- > >>> Karel Vervaeke > >>> http://outerthought.org/ > >>> Open Source Content Applications > >>> Makers of Kauri, Daisy CMS and Lily > >>> > >> > >> > >> > >> -- > >> Karel Vervaeke > >> http://outerthought.org/ > >> Open Source Content Applications > >> Makers of Kauri, Daisy CMS and Lily > >> > > > > > > > > -- > > Karel Vervaeke > > http://outerthought.org/ > > Open Source Content Applications > > Makers of Kauri, Daisy CMS and Lily > > > > > > -- > Karel Vervaeke > http://outerthought.org/ > Open Source Content Applications > Makers of Kauri, Daisy CMS and Lily >
