On Mon, Oct 20, 2014 at 3:05 PM, Tim Dunphy bluethu...@gmail.com wrote:
Don't forget that the time taken to build the file list is a function
of
the number of files present, and not their size. If you have many
millions
of small files, it will indeed take a very long time. Over
On Mon, October 20, 2014 10:49, Denniston, Todd A CIV NAVSURFWARCENDIV Crane
wrote:
[OP: `they don't allow ssh between the datacenters` ...but... they nfs between
them...??? ME: much head scratching.]
We have a Value Added Network (VAN) provider who insists on a similar thing.
We were given
Are you allowed to temporarily run an ssh tunnel (or stunnel) on your jumpbox?
So connecting from host1 to jumpbox on port XXX would be tunneled to ssh port
on host2...
Or with netcat (if you can mkfifo)?
mkfifo backpipe
nc -l 12345 0backpipe | nc host2 22 1backpipeBut you will have to trick
-Original Message-
From: John Doe [mailto:jd...@yahoo.com]
Sent: Monday, October 20, 2014 5:30 AM
To: CentOS mailing list; Tim Dunphy
Subject: Re: [CentOS] rsync question: building list taking forever
Are you allowed to temporarily run an ssh tunnel (or stunnel) on your
jumpbox
On Sun, Oct 19, 2014 at 9:05 PM, Tim Dunphy bluethu...@gmail.com wrote:
Don't forget that the time taken to build the file list is a function of
the number of files present, and not their size. If you have many
millions
of small files, it will indeed take a very long time. Over sshfs with
On 2014-10-20, Les Mikesell lesmikes...@gmail.com wrote:
So, you probably want the 'source' side of the transfer to be local
for faster startup. But... in what universe is NFS mounting across
data centers considered more secure than ssh? Or even a reasonable
thing to do? How about a VPN
On Mon, Oct 20, 2014 at 1:12 PM, Keith Keller
kkel...@wombat.san-francisco.ca.us wrote:
On 2014-10-20, Les Mikesell lesmikes...@gmail.com wrote:
So, you probably want the 'source' side of the transfer to be local
for faster startup. But... in what universe is NFS mounting across
data centers
On 2014-10-20, Les Mikesell lesmikes...@gmail.com wrote:
On Mon, Oct 20, 2014 at 1:12 PM, Keith Keller
The OP said he was mounting the NFS over sshfs.
OK, I don't see how that is possible because something would either be
mounted as nfs or sshfs, not one over the other.
I'm just repeating
Guys,
I've setup an rsync between two directories that I've mounted locally on a
jump box. Long story short, the two directories are both NFS shares from
two different hosts. Our security dept won't allow us to SSH between the
two data centers, directly. But the jump host can contact both. So
2014-10-19 18:55 GMT+03:00 Tim Dunphy bluethu...@gmail.com:
Guys,
I've setup an rsync between two directories that I've mounted locally on a
jump box. Long story short, the two directories are both NFS shares from
two different hosts. Our security dept won't allow us to SSH between the
two
2014-10-19 20:03 GMT+03:00 Eero Volotinen eero.voloti...@iki.fi:
2014-10-19 18:55 GMT+03:00 Tim Dunphy bluethu...@gmail.com:
Guys,
I've setup an rsync between two directories that I've mounted locally on
a
jump box. Long story short, the two directories are both NFS shares from
two
... and remember to use tcp for nfs transfer ;)
Hmm you mean specify tcp for rsync? I thought that's default. But holy
crap, you were right about it taking a long time to build a file list! The
rsync just started a few minutes ago... !
dumps/dotmedia.031237.svndmp
dumps/dotmedia.031238.svndmp
2014-10-19 20:49 GMT+03:00 Tim Dunphy bluethu...@gmail.com:
... and remember to use tcp for nfs transfer ;)
Hmm you mean specify tcp for rsync? I thought that's default. But holy
crap, you were right about it taking a long time to build a file list! The
rsync just started a few minutes
On 10/19/2014 8:55 AM, Tim Dunphy wrote:
I've setup an rsync between two directories that I've mounted locally on a
jump box. Long story short, the two directories are both NFS shares from
two different hosts. Our security dept won't allow us to SSH between the
two data centers, directly. But
On 2014-10-19, Tim Dunphy bluethu...@gmail.com wrote:
... and remember to use tcp for nfs transfer ;)
Hmm you mean specify tcp for rsync? I thought that's default.
No, he means use TCP for NFS (which is also the default).
I suspect that sshfs's relatively poor performance is having an impact
On Sun, 19 Oct 2014, Keith Keller wrote:
I suspect that sshfs's relatively poor performance is having an impact
on your transfer. I have a 30TB filesystem which I rsync over an
OpenVPN link, and building the file list doesn't take that long (maybe
an hour?). (The links themselves are
On 2014/10/19 08:01, Keith Keller wrote:
On 2014-10-19, Tim Dunphy bluethu...@gmail.com wrote:
... and remember to use tcp for nfs transfer ;)
Hmm you mean specify tcp for rsync? I thought that's default.
No, he means use TCP for NFS (which is also the default).
I suspect that sshfs's
On 2014-10-19, Steve Thompson s...@vgersoft.com wrote:
Don't forget that the time taken to build the file list is a function of
the number of files present, and not their size. If you have many millions
of small files, it will indeed take a very long time. Over sshfs with
a slowish link, it
On Mon, Oct 20, 2014 at 7:57 AM, Steve Thompson s...@vgersoft.com wrote:
On Sun, 19 Oct 2014, Keith Keller wrote:
I suspect that sshfs's relatively poor performance is having an impact
on your transfer. I have a 30TB filesystem which I rsync over an
OpenVPN link, and building the file list
Don't forget that the time taken to build the file list is a function of
the number of files present, and not their size. If you have many
millions
of small files, it will indeed take a very long time. Over sshfs with
a slowish link, it could be days.
and it may end up failing
20 matches
Mail list logo
