On Monday, October 20, 2014 9:03:27 PM UTC-4, Nikolaus Rath wrote:
>
> Rich B <[email protected] <javascript:>> writes:
> >> Could you give some more context? In particular, I'd like to see the
> logs
> >> from the 32st and 1st retry.
> >>
> >> Because of log rotations, I had to re-run the test to get the 1st and
> 32nd
> > retry. This time more of the blocks managed to get uploaded. Some of the
> > threads seemed to have better luck than others:
> [...]
>
> To me this looks just like a crappy network connection. You could try to
> increase the send/recv timeout (using the --backend-options argument,
> see manual).
>
> >> I'm stumped. Has anyone got an idea of what might be causing my
> problem?
> >>
> >>
> >> Do you have the same problem when you store data in Amazon S3 from the
> >> same system?
> >>
> > Yes, I just tried Amazon and I get similar results.
>
> This is consistent with a bad connection.
>
> >> If you have the necessary SSL-fu, you could also do a traffic capture
> using
> >> Wireshark. That's gonna be a bit tricky, but it should tell us a lot
> more.
> >
> > I've worked with tcpdump/wireshark before, but my SSL-fu is weak can
> > you tell me what I should look for?
>
> I'm afraid not, I never got it to work myself. But you're lucky, if you
> have the same problem with S3, just use --backend-options nossl, and
> capture that traffic.
>
> I've been playing around with the "no-ssl" option and it _SEEMS_ to work a
little better, but that could just be due to vagaries of an intermittently
poor connection. I'll do the packet capture when time permits.
In the mean time, I have some questions about the tcp-timeout backend
option. The docs don't say what the units are, what the default value is,
or how to use the option.
From the code, it looks like the units are seconds, and the syntax is
"--backend-options tcp-timeout=<integer>", however when I try this syntax,
python barfs:
$ /usr/bin/mount.s3ql --debug --cachedir /data/s3ql-cache --cachesize
1048567 --backend-options no-ssl,tcp-timeout=12 s3://myBucket/myPrefix
/mnt/myMount
Using 8 upload threads.
Autodetected 4040 file descriptors available for cache entries
Uncaught top-level exception:
Traceback (most recent call last):
File "/usr/bin/mount.s3ql", line 9, in <module>
load_entry_point('s3ql==2.11.1', 'console_scripts', 'mount.s3ql')()
File "/usr/lib/s3ql/s3ql/mount.py", line 121, in main
backend_factory = get_backend_factory(options)
File "/usr/lib/s3ql/s3ql/common.py", line 366, in get_backend_factory
backend.fetch('s3ql_passphrase')
File "/usr/lib/s3ql/s3ql/backends/common.py", line 272, in fetch
return self.perform_read(do_read, key)
File "/usr/lib/s3ql/s3ql/backends/common.py", line 46, in wrapped
return method(*a, **kw)
File "/usr/lib/s3ql/s3ql/backends/common.py", line 235, in perform_read
fh = self.open_read(key)
File "/usr/lib/s3ql/s3ql/backends/common.py", line 46, in wrapped
return method(*a, **kw)
File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 292, in open_read
resp = self._do_request('GET', '/%s%s' % (self.prefix, key))
File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 409, in _do_request
query_string=query_string, body=body)
File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 667, in _send_request
return read_response()
File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 630, in read_response
resp = self.conn.read_response()
File "/usr/lib/python3/dist-packages/dugong/__init__.py", line 673, in
read_response
return eval_coroutine(self.co_read_response(), self.timeout)
File "/usr/lib/python3/dist-packages/dugong/__init__.py", line 1361, in
eval_coroutine
if not next(crt).poll(timeout=timeout):
File "/usr/lib/python3/dist-packages/dugong/__init__.py", line 115, in
poll
return bool(poll.poll(timeout*1000)) # convert to ms
TypeError: timeout must be an integer or None
Am I using tcp-timeout incorrectly? What is the default timeout?
Thanks,
Rich B
--
You received this message because you are subscribed to the Google Groups
"s3ql" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
