I've been using S3QL for a few days now, and whenever I am copying over large amounts of data the mount point seems to 'lock up' for a period of time (as well as anything else trying to access it), and then I start getting 'transport endpoint not connected' errors. I "umount" the mount point, run fsck on it. and then continue archiving until the problem happens again. I have been using cp --update --archive but I'm not quite sure this is working to ensure that I end up with a complete replicated filesystem from my copy source to my copy destination - and MD5ing takes forever since it has to transfer all the data (not to mention, it's costly!). So, as you all can imagine, this transport endpoint disconnection thing is causing many headaches. Right now as I type this message, fsck is running and 'committing blocks' (correcting errors?) to the S3 backend.
Does anyone know what would cause these problems? The machine is connected via a dedicated 100mbit pipe and has low latency to the S3 servers. The problem only seems to occur when I'm copying large amounts of data. I have tried disabling threads, thinking perhaps the asynchronous writes were causing fault, but this did not help. The back-end is a regular old S3 bucket in the Oregon location. The errors from 'cp' looks like: ... successful copies ... cp: cannot create directory ‘/mnt/s3-df/Backup/Movies/Old’: Transport endpoint is not connected ... etc etc ... This issue is essentially making S3QL unusable as its so unstable that I cannot confidently mount a partition and let it be. Just now, prior to my currently running fsck, I had only mounted the S3QL filesystem just 45 seconds prior and ran a 'cp' of a 17GB directory to the S3QL fs when it then locked up and I started receiving the Transport Endpoint Not Connected errors. Any and all help and ideas are appreciated! Thanks -- You received this message because you are subscribed to the Google Groups "s3ql" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
