Re: Problem reports for version control systems

[Johnny Billquist]

On 2021-05-02 18:21, Michael van Elst wrote:

b...@update.uu.se (Johnny Billquist) writes:


And as a "fun" fact. On my 4000/90, it takes about 3h after I start a
cvs update until I actually start having any network traffic...


A SCSI SSD could help. :)


Definitely. Because the disk is 100% busy all the time. Not a very 
modern disk either. RZ29 if I remember right.
SSD could make a big difference. And that kind of capacity isn't that 
much today. 4G or so...


  Johnny

--
Johnny Billquist  || "I'm on a bus
  ||  on a psychedelic trip
email: b...@softjar.se ||  Reading murder books
pdp is alive! ||  tryin' to stay hip" - B. Idol

Re: Problem reports for version control systems

[Havard Eidnes]

> I suspect what is commonly the problem here is related to the fact
> that cvs has such a phase at the beginning where it is scanning
> through the file system, which can take quite a while. Some NAT
> devices along the path sometimes have timeouts on existing connections
> that if no traffic is happening for a while, they are dropped, even
> though there hasn't been any FINs on the connection.
> So a connection that just don't have any traffic for a while are hit
> by this, which is exactly the pattern you have with cvs.

This is the reason some of my ~/.ssh/config files has

host *
  ServerAliveInterval 240

in them. :)

Regards,

- Håvard

Re: Problem reports for version control systems

[Michael van Elst]

b...@update.uu.se (Johnny Billquist) writes:

>And as a "fun" fact. On my 4000/90, it takes about 3h after I start a 
>cvs update until I actually start having any network traffic... 

A SCSI SSD could help. :)

Re: Problem reports for version control systems

[Johnny Billquist]

On 2021-05-02 16:32, Anders Magnusson wrote:

Den 2021-05-02 kl. 15:57, skrev Johnny Billquist:

On 2021-05-02 13:51, Anders Magnusson wrote:

Den 2021-05-02 kl. 13:44, skrev Johnny Billquist:


I suspect what is commonly the problem here is related to the fact 
that cvs has such a phase at the beginning where it is scanning 
through the file system, which can take quite a while. Some NAT 
devices along the path sometimes have timeouts on existing 
connections that if no traffic is happening for a while, they are 
dropped, even though there hasn't been any FINs on the connection.
So a connection that just don't have any traffic for a while are hit 
by this, which is exactly the pattern you have with cvs.


I've seen the same effect on a simple telnet session, where ssh 
survives fine. And there it's just that when the connection is idle, 
telnet is not creating any traffic at all, while ssh do generate a 
bit of traffic even if there is no activity.


So one obvious solution is to use something like ssh as a carries 
for the cvs traffic, if possible, or else see if some kind of 
keepalives can be enabled on a connection, to defeat NAT and similar 
devices which aggressively drop connections on which there is no 
traffic for a while.
(Or, of course, if there is a NAT you have control over, you might 
be able to change how it behaves...)

This is quite common, yes.
I ususlly add ssh keepalive to ssh_config for all hosts to avoid this 
problem (which may occur, as written, when doing cvs update).


And as a "fun" fact. On my 4000/90, it takes about 3h after I start a 
cvs update until I actually start having any network traffic... In 
total it takes something like 8h to do a cvs update on /usr/src.
(I guess I'm a bit masochistic is still insisting on trying to do 
things on my VAXen...)

Have you tried it on the 8650? :-)


Last time I did manage, I think it was close to a day. That with RA73 
drives. But you know there's been problems with having two unibuses 
lately, and so on. So it's been quite a while since I actually managed 
to much of anything natively on that machine. :-(


Another reason I'd like to get gcc working native again. Also trying to 
get a chance to fix things on the 8650. Not sure how much longer we will 
be able to keep that machine around ready to run. We might need to 
shrink our locales soon...


  Johnny

--
Johnny Billquist  || "I'm on a bus
  ||  on a psychedelic trip
email: b...@softjar.se ||  Reading murder books
pdp is alive! ||  tryin' to stay hip" - B. Idol

Re: Problem reports for version control systems

[Greg Troxel]

Robert Swindells  writes:

> Lloyd Parkes  wrote:
>>The network is a 1Gb/s LAN through to a smaller NetBSD router running 
>>NPF with MSS clamping enabled so that I can get Netflix. My ISP does not 
>>use CGN for my IPv4 connection. My IPv6 connection is tunnelled through 
>>to Hurricane Electric in Sydney, Australia.
>
> Have you tried disabling IPv6 or explicitly connecting using IPv4 ?
>
> I don't see any problems using IPv6 through NPF to update cvs but I have
> native IPv6 and can use a 1500 byte MTU. I'm also using cvs.n.o instead
> of anoncvs.n.o but they have adjacent IPv4 addresses.
>
> I'm guessing that your IPv6 tunnel has a lower MTU than your IPv4
> connection to your ISP.

I update over HE tunnels all the time with no issues (cvs, not anoncvs).
IPv6 tends to 1280 MTU; my gif for the tunnel is that, and wthat has all
apparently worked fine.   So always good to look with tcpdump, but I
suspect tunenl MTU is not really an issue.

There are sometimes problems with reachability via v6, and sometimes the
speeds are lower.  More recently, I had one case of v4 reachability
failure while v6 worked, and have sometimes seen lower ping times on v6.


signature.asc
Description: PGP signature

Re: Problem reports for version control systems

[Anders Magnusson]

Den 2021-05-02 kl. 15:57, skrev Johnny Billquist:

On 2021-05-02 13:51, Anders Magnusson wrote:

Den 2021-05-02 kl. 13:44, skrev Johnny Billquist:


I suspect what is commonly the problem here is related to the fact 
that cvs has such a phase at the beginning where it is scanning 
through the file system, which can take quite a while. Some NAT 
devices along the path sometimes have timeouts on existing 
connections that if no traffic is happening for a while, they are 
dropped, even though there hasn't been any FINs on the connection.
So a connection that just don't have any traffic for a while are hit 
by this, which is exactly the pattern you have with cvs.


I've seen the same effect on a simple telnet session, where ssh 
survives fine. And there it's just that when the connection is idle, 
telnet is not creating any traffic at all, while ssh do generate a 
bit of traffic even if there is no activity.


So one obvious solution is to use something like ssh as a carries 
for the cvs traffic, if possible, or else see if some kind of 
keepalives can be enabled on a connection, to defeat NAT and similar 
devices which aggressively drop connections on which there is no 
traffic for a while.
(Or, of course, if there is a NAT you have control over, you might 
be able to change how it behaves...)

This is quite common, yes.
I ususlly add ssh keepalive to ssh_config for all hosts to avoid this 
problem (which may occur, as written, when doing cvs update).


And as a "fun" fact. On my 4000/90, it takes about 3h after I start a 
cvs update until I actually start having any network traffic... In 
total it takes something like 8h to do a cvs update on /usr/src.
(I guess I'm a bit masochistic is still insisting on trying to do 
things on my VAXen...)

Have you tried it on the 8650? :-)

-- R

Re: Problem reports for version control systems

[Johnny Billquist]

On 2021-05-02 13:51, Anders Magnusson wrote:

Den 2021-05-02 kl. 13:44, skrev Johnny Billquist:


I suspect what is commonly the problem here is related to the fact 
that cvs has such a phase at the beginning where it is scanning 
through the file system, which can take quite a while. Some NAT 
devices along the path sometimes have timeouts on existing connections 
that if no traffic is happening for a while, they are dropped, even 
though there hasn't been any FINs on the connection.
So a connection that just don't have any traffic for a while are hit 
by this, which is exactly the pattern you have with cvs.


I've seen the same effect on a simple telnet session, where ssh 
survives fine. And there it's just that when the connection is idle, 
telnet is not creating any traffic at all, while ssh do generate a bit 
of traffic even if there is no activity.


So one obvious solution is to use something like ssh as a carries for 
the cvs traffic, if possible, or else see if some kind of keepalives 
can be enabled on a connection, to defeat NAT and similar devices 
which aggressively drop connections on which there is no traffic for a 
while.
(Or, of course, if there is a NAT you have control over, you might be 
able to change how it behaves...)

This is quite common, yes.
I ususlly add ssh keepalive to ssh_config for all hosts to avoid this 
problem (which may occur, as written, when doing cvs update).


And as a "fun" fact. On my 4000/90, it takes about 3h after I start a 
cvs update until I actually start having any network traffic... In total 
it takes something like 8h to do a cvs update on /usr/src.
(I guess I'm a bit masochistic is still insisting on trying to do things 
on my VAXen...)


  Johnny

--
Johnny Billquist  || "I'm on a bus
  ||  on a psychedelic trip
email: b...@softjar.se ||  Reading murder books
pdp is alive! ||  tryin' to stay hip" - B. Idol

Re: Problem reports for version control systems

[Robert Swindells]

Lloyd Parkes  wrote:
>The network is a 1Gb/s LAN through to a smaller NetBSD router running 
>NPF with MSS clamping enabled so that I can get Netflix. My ISP does not 
>use CGN for my IPv4 connection. My IPv6 connection is tunnelled through 
>to Hurricane Electric in Sydney, Australia.

Have you tried disabling IPv6 or explicitly connecting using IPv4 ?

I don't see any problems using IPv6 through NPF to update cvs but I have
native IPv6 and can use a 1500 byte MTU. I'm also using cvs.n.o instead
of anoncvs.n.o but they have adjacent IPv4 addresses.

I'm guessing that your IPv6 tunnel has a lower MTU than your IPv4
connection to your ISP.

Re: Problem reports for version control systems

[Anders Magnusson]

Den 2021-05-02 kl. 13:44, skrev Johnny Billquist:


I suspect what is commonly the problem here is related to the fact 
that cvs has such a phase at the beginning where it is scanning 
through the file system, which can take quite a while. Some NAT 
devices along the path sometimes have timeouts on existing connections 
that if no traffic is happening for a while, they are dropped, even 
though there hasn't been any FINs on the connection.
So a connection that just don't have any traffic for a while are hit 
by this, which is exactly the pattern you have with cvs.


I've seen the same effect on a simple telnet session, where ssh 
survives fine. And there it's just that when the connection is idle, 
telnet is not creating any traffic at all, while ssh do generate a bit 
of traffic even if there is no activity.


So one obvious solution is to use something like ssh as a carries for 
the cvs traffic, if possible, or else see if some kind of keepalives 
can be enabled on a connection, to defeat NAT and similar devices 
which aggressively drop connections on which there is no traffic for a 
while.
(Or, of course, if there is a NAT you have control over, you might be 
able to change how it behaves...)

This is quite common, yes.
I ususlly add ssh keepalive to ssh_config for all hosts to avoid this 
problem (which may occur, as written, when doing cvs update).


-- R

Re: Problem reports for version control systems

[Johnny Billquist]

On 2021-05-01 23:54, Brett Lymn wrote:

On Sat, May 01, 2021 at 12:58:50PM +1200, Lloyd Parkes wrote:


Germany is pretty much the opposite of New Zealand. It's close to
everywhere, but its last mile access speeds are a bit infamous.



Just for you info... there are a few NetBSD developers in .au, my self 
included.  I haven't
had any issues with cvs disconnects.  Not to deny you have an issue, just 
letting you know
it works ok for people near you.


Not anywhere near such a location, but just adding that cvs works fine 
for me too, but yes, there is a lot of disk activity on the local 
machine before anything even starts downloading, and a lot of activity 
at the end where it updates file metadata as well as clean out empty 
directories (if you added pruning).



I'm running some tests on other local clients and against other CVS mirrors
in the hope that come up with a better characterisation of the problem than
"it doesn't work".



If you have the space, a tcpdump from both sides of your firewall may provide a 
clue.


I suspect what is commonly the problem here is related to the fact that 
cvs has such a phase at the beginning where it is scanning through the 
file system, which can take quite a while. Some NAT devices along the 
path sometimes have timeouts on existing connections that if no traffic 
is happening for a while, they are dropped, even though there hasn't 
been any FINs on the connection.
So a connection that just don't have any traffic for a while are hit by 
this, which is exactly the pattern you have with cvs.


I've seen the same effect on a simple telnet session, where ssh survives 
fine. And there it's just that when the connection is idle, telnet is 
not creating any traffic at all, while ssh do generate a bit of traffic 
even if there is no activity.


So one obvious solution is to use something like ssh as a carries for 
the cvs traffic, if possible, or else see if some kind of keepalives can 
be enabled on a connection, to defeat NAT and similar devices which 
aggressively drop connections on which there is no traffic for a while.
(Or, of course, if there is a NAT you have control over, you might be 
able to change how it behaves...)


  Johnny

--
Johnny Billquist  || "I'm on a bus
  ||  on a psychedelic trip
email: b...@softjar.se ||  Reading murder books
pdp is alive! ||  tryin' to stay hip" - B. Idol

Re: Problem reports for version control systems

[Andreas Gustafsson]

Brett Lymn wrote:
> Just for you info... there are a few NetBSD developers in .au, my self 
> included.  I haven't
> had any issues with cvs disconnects.  Not to deny you have an issue, just 
> letting you know
> it works ok for people near you.

For what it's worth, my connections to anoncvs from Finland frequently
break in the middle of a transfer, though I'm using rsync rather than
cvs.  I have an admins@ ticket open about this since a couple of years
ago (#160795).
-- 
Andreas Gustafsson, g...@gson.org

Re: Problem reports for version control systems

[Brett Lymn]

On Sat, May 01, 2021 at 12:58:50PM +1200, Lloyd Parkes wrote:
> 
> Germany is pretty much the opposite of New Zealand. It's close to
> everywhere, but its last mile access speeds are a bit infamous.
> 

Just for you info... there are a few NetBSD developers in .au, my self 
included.  I haven't
had any issues with cvs disconnects.  Not to deny you have an issue, just 
letting you know
it works ok for people near you.

> I'm running some tests on other local clients and against other CVS mirrors
> in the hope that come up with a better characterisation of the problem than
> "it doesn't work".
> 

If you have the space, a tcpdump from both sides of your firewall may provide a 
clue.

-- 
Brett Lymn
--
Sent from my NetBSD device.

"We are were wolves",
"You mean werewolves?",
"No we were wolves, now we are something else entirely",
"Oh"

Re: Problem reports for version control systems

[Greg A. Woods]

At Fri, 30 Apr 2021 15:23:03 +0200, Christian Groessler  
wrote:
Subject: Re: Problem reports for version control systems
>
> On 4/30/21 7:31 AM, Lloyd Parkes wrote:
>
> > Hi all,
> > The problem reports people have in their
> > emails are completely inadequate for
> > trying to determine what is going wrong
> > for people trying to access the NetBSD
> > source.
> >
> > 
>
>
> I'm rsync'ing the CVS tree to my local
> server and then run CVS against my server
> on the LAN. No problems...

Same here, since 2001 or so:

RCS file: RCS/rsync-netbsd-cvs,v
revision 1.1
date: 2001/06/06 17:52:06;  author: woods;  state: Exp;
Initial revision

This script can now be found here:

 https://github.com/robohack/rsync-netbsd-cvs

--
Greg A. Woods 

Kelowna, BC +1 250 762-7675   RoboHack 
Planix, Inc.  Avoncote Farms 


pgpzTB5zfwE1y.pgp
Description: OpenPGP Digital Signature

Re: Problem reports for version control systems

[Rhialto]

On Fri 30 Apr 2021 at 19:51:27 +1000, matthew green wrote:
> > I too get long pauses with cvs, both at the beginning,
> > and even longer at the end after update is complete.
> 
> the end part is most likely cvs cleaning up after itslf by
> removing all the subdirs it created but doesn't need.

And the pause at the beginning is likely to be a full scan of the tree
for "extra" files. Note that those get reported before you see any
progress on the update itself.

> .mrg.
-Olaf.
-- 
___ Q: "What's an anagram of Banach-Tarski?"  -- Olaf "Rhialto" Seibert
\X/ A: "Banach-Tarski Banach-Tarski." -- rhialto at falu dot nl


signature.asc
Description: PGP signature

Re: Problem reports for version control systems

[matthew sporleder]

On Fri, Apr 30, 2021 at 5:50 AM Joerg Sonnenberger  wrote:
>
> On Fri, Apr 30, 2021 at 05:31:53PM +1200, Lloyd Parkes wrote:
> > ceph4% hg --version
> > Mercurial Distributed SCM (version 5.3.2)
>
> Please note that this is quite an old version and a lot of work on
> improving both CPU time and memory use has been spend since then.
>
> Joerg

I confirmed this was the case - btw.  mercurial 5.7 actually works for
a clone and should stay under 1G of memory but will eat about 8GB of
space (git is 2GB)

These older versions all failed for me 9/10 times.

Re: Problem reports for version control systems

[Hauke Fath]

On Sat, 1 May 2021 12:58:50 +1200, Lloyd Parkes wrote:
> I'm not using a SLOG. I couldn't be bothered setting one up on my 
> crash and burn systems. It doesn't seem to be too bad, except for 
> when I try and run "rm -rf src".

Looking at 'systat -w2 vmstat' during writes to zfs, the activity on my 
machine's ZLOG is amazing. And for the end phase of a 'cvs update', it 
would make a huge difference.
 
>> Both from home (16 MBit DSL) and $WORKPLACE I am frequently running cvs
>> updates through filtering routers (pf(4) here), and basically never see
>> connection issues.
> 
> Germany is pretty much the opposite of New Zealand. It's close to 
> everywhere, but its last mile access speeds are a bit infamous.

More speed is available where I live, I just don't have an imminent 
need. 'cvs update' works anyway.  ;)

Cheerio,
Hauke

-- 
Hauke Fath
Linnéweg 7
64342 Seeheim-Jugenheim
Germany

Re: Problem reports for version control systems

[Lloyd Parkes]

On 30/04/21 9:50 pm, Joerg Sonnenberger wrote:

On Fri, Apr 30, 2021 at 05:31:53PM +1200, Lloyd Parkes wrote:

     ceph4% hg --version
     Mercurial Distributed SCM (version 5.3.2)


Please note that this is quite an old version and a lot of work on
improving both CPU time and memory use has been spend since then.


I have a pkgsrc from about the 9.0 days that I keep copying around for 
"reasons". Maybe it's time to upgrade.


Cheers,
Lloyd

Re: Problem reports for version control systems

[Lloyd Parkes]

On 30/04/21 8:36 pm, Hauke Fath wrote:

Out of curiosity: Do you use a ZIL SLOG* volume with that setup? I
remember cvs operations used to be a lot slower on spinning rust than
on SSD.


I'm not using a SLOG. I couldn't be bothered setting one up on my crash 
and burn systems. It doesn't seem to be too bad, except for when I try 
and run "rm -rf src".



Both from home (16 MBit DSL) and $WORKPLACE I am frequently running cvs
updates through filtering routers (pf(4) here), and basically never see
connection issues.


Germany is pretty much the opposite of New Zealand. It's close to 
everywhere, but its last mile access speeds are a bit infamous.


I'm running some tests on other local clients and against other CVS 
mirrors in the hope that come up with a better characterisation of the 
problem than "it doesn't work".


Cheers,
Lloyd

Re: Problem reports for version control systems

[Christian Groessler]

On 4/30/21 7:31 AM, Lloyd Parkes wrote:


Hi all,
The problem reports people have in their emails are completely 
inadequate for trying to determine what is going wrong for people 
trying to access the NetBSD source.






I'm rsync'ing the CVS tree to my local server and then run CVS against 
my server on the LAN. No problems...


regards,
chris

Re: Problem reports for version control systems

[Hauke Fath]

On Fri, 30 Apr 2021 19:51:27 +1000, matthew green wrote:
>> I too get long pauses with cvs, both at the beginning,
>> and even longer at the end after update is complete.
> 
> the end part is most likely cvs cleaning up after itslf by
> removing all the subdirs it created but doesn't need.

... which is mostly metadata operations that take a _lot_ longer on 
spinning rust. Which is why I asked about SLOG.

Cheerio,
Hauke

-- 
Hauke Fath
Linnéweg 7
64342 Seeheim-Jugenheim
Germany

re: Problem reports for version control systems

[matthew green]

> I too get long pauses with cvs, both at the beginning,
> and even longer at the end after update is complete.

the end part is most likely cvs cleaning up after itslf by
removing all the subdirs it created but doesn't need.

check disk io or ktrace for this part -- it's usually a
local iops issue, than a network issue.


.mrg.

Re: Problem reports for version control systems

[Joerg Sonnenberger]

On Fri, Apr 30, 2021 at 05:31:53PM +1200, Lloyd Parkes wrote:
>     ceph4% hg --version
>     Mercurial Distributed SCM (version 5.3.2)

Please note that this is quite an old version and a lot of work on
improving both CPU time and memory use has been spend since then.

Joerg

Re: Problem reports for version control systems

[Thomas Mueller]

> Hi all,
> The problem reports people have in their emails are completely inadequate for
> trying to determine what is going wrong for people trying to access the 
> NetBSD 
> source. 
 
> Since I was the first person to post an inadequate report in this first batch,
> I'll go first at trying to do better. There are three sections to this email.
> First, a description of the host and network I am running this on. Second, a
> description of what I did, what I got, and what I expected to get. Third, I'll
> speculate on possible causes.
 
> Host and Network
> 
 
> I am in New Zealand, which tends to have good internet access to the home and
> Long Fat Networks (LFNs) to the rest of the world.
 
> The host is a Xeon E3-1241 v3 @ 3.50GHz with eight hyperthreads and 32GB RAM. 
> It has a 128GB SSD for / and 4x4TB disks in a raidz zpool on /vol. All work is
> being done on /vol. The OS is NetBSD 9.99.79.

> The network is a 1Gb/s LAN through to a smaller NetBSD router running NPF with
> MSS clamping enabled so that I can get Netflix. My ISP does not use CGN for my
> IPv4 connection. My IPv6 connection is tunnelled through to Hurricane Electric
> in Sydney, Australia.

>     ceph4% cvs -v

>     Concurrent Versions System (CVS) 1.12.13 (client/server)
>     with CVSACL Patch 1.2.5 (cvsacl.sourceforge.net)
 
>     ceph4% hg --version
>     Mercurial Distributed SCM (version 5.3.2)
 
> What I Did, Got, and Expected
> =

> In all cases, I pretty much copied instructions from the NetBSD web site.
 
>     ceph4% cd /vol/src/cylc/src-CVS
>     ceph4% export CVSROOT="anon...@anoncvs.netbsd.org:/cvsroot"
>     ceph4% export CVS_RSH="ssh"
>     ceph4% cvs checkout -A -P src
>     cvs checkout: Updating src
>     U src/BUILDING
>     U src/Makefile
>     U src/Makefile.inc
>     U src/README.md
>     ...
>     U src/crypto/external/bsd/heimdal/dist/lib/wind/test-utf8.c
>     [very long pause here]
>     client_loop: send disconnect: Broken pipe
>     cvs [checkout aborted]: end of file from server (consult above 
> messages if any)

> I expected the output to not pause too much and for the listing to run through
> to the end of src/usr.sbin without disconnecting and aborting.

>     ceph4% hg clone -U https://anonhg.NetBSD.org/src src-hg
>     [lots of progress stuff gets displayed here]
>     applying clone bundle from 
> https://cdn.NetBSD.org/_bundles/src/77d2a2ece3a06d837da45acd0fda80086ab4113c.zstd.hg
 
>     adding changesets 
>     adding manifests
>     adding file changes 
>     added 931876 changesets with 2425841 changes to 439702 files (+417 
> heads) 
>     finished applying clone bundle
>     searching for changes
>     adding changesets
>     adding manifests
>     adding file changes
>     adding changesets
>     adding manifests
>     adding file changes
>     added 22612 changesets with 161374 changes to 110024 files (+6 heads)
>     new changesets 26c8f37631b6:6f32acf2c5e1 (230 drafts) 
>     280602 local changesets published

> This is what I expected.

> Speculation
> ===
 
> I'm suspicious of the long pause in the CVS example. Maybe one end or the
> other of the CVS connection spent too long doing disk I/O or calculating stuff
> and then NPF timed-out the connection?

> Cheers, 
> Lloyd

I too get long pauses with cvs, both at the beginning, and even longer at the 
end after update is complete.

This is equally true whether I update using cvs in NetBSD or cvs (built from 
FreeBSD ports, not in base system) in FreeBSD.

There was a similar but much shorter delay in FreeBSD when updating by svn, 
before they switched to git.

So I look forward to a switch from cvs to hg when everything is in good working 
order.

Switching FreeBSD src, doc and ports trees from svn to git was decidedly 
nontrivial.

Tom

Re: Problem reports for version control systems

[Hauke Fath]

On Fri, 30 Apr 2021 17:31:53 +1200, Lloyd Parkes wrote:
> The host is a Xeon E3-1241 v3 @ 3.50GHz with eight hyperthreads and 
> 32GB RAM. It has a 128GB SSD for / and 4x4TB disks in a raidz zpool 
> on /vol. All work is being done on /vol.

Out of curiosity: Do you use a ZIL SLOG* volume with that setup? I 
remember cvs operations used to be a lot slower on spinning rust than 
on SSD.

Both from home (16 MBit DSL) and $WORKPLACE I am frequently running cvs 
updates through filtering routers (pf(4) here), and basically never see 
connection issues.

Cheerio,
Hauke

* 


-- 
Hauke Fath
Linnéweg 7
64342 Seeheim-Jugenheim
Germany

Problem reports for version control systems

[Lloyd Parkes]

Hi all,
The problem reports people have in their emails are completely 
inadequate for trying to determine what is going wrong for people trying 
to access the NetBSD source.


Since I was the first person to post an inadequate report in this first 
batch, I'll go first at trying to do better. There are three sections to 
this email. First, a description of the host and network I am running 
this on. Second, a description of what I did, what I got, and what I 
expected to get. Third, I'll speculate on possible causes.


Host and Network


I am in New Zealand, which tends to have good internet access to the 
home and Long Fat Networks (LFNs) to the rest of the world.


The host is a Xeon E3-1241 v3 @ 3.50GHz with eight hyperthreads and 32GB 
RAM. It has a 128GB SSD for / and 4x4TB disks in a raidz zpool on /vol. 
All work is being done on /vol. The OS is NetBSD 9.99.79.


The network is a 1Gb/s LAN through to a smaller NetBSD router running 
NPF with MSS clamping enabled so that I can get Netflix. My ISP does not 
use CGN for my IPv4 connection. My IPv6 connection is tunnelled through 
to Hurricane Electric in Sydney, Australia.


    ceph4% cvs -v

    Concurrent Versions System (CVS) 1.12.13 (client/server)
    with CVSACL Patch 1.2.5 (cvsacl.sourceforge.net)

    ceph4% hg --version
    Mercurial Distributed SCM (version 5.3.2)

What I Did, Got, and Expected
=

In all cases, I pretty much copied instructions from the NetBSD web site.

    ceph4% cd /vol/src/cylc/src-CVS
    ceph4% export CVSROOT="anon...@anoncvs.netbsd.org:/cvsroot"
    ceph4% export CVS_RSH="ssh"
    ceph4% cvs checkout -A -P src
    cvs checkout: Updating src
    U src/BUILDING
    U src/Makefile
    U src/Makefile.inc
    U src/README.md
    ...
    U src/crypto/external/bsd/heimdal/dist/lib/wind/test-utf8.c
    [very long pause here]
    client_loop: send disconnect: Broken pipe
    cvs [checkout aborted]: end of file from server (consult above 
messages if any)


I expected the output to not pause too much and for the listing to run 
through to the end of src/usr.sbin without disconnecting and aborting.


    ceph4% hg clone -U https://anonhg.NetBSD.org/src src-hg
    [lots of progress stuff gets displayed here]
    applying clone bundle from 
https://cdn.NetBSD.org/_bundles/src/77d2a2ece3a06d837da45acd0fda80086ab4113c.zstd.hg 


    adding changesets
    adding manifests
    adding file changes
    added 931876 changesets with 2425841 changes to 439702 files (+417 
heads)

    finished applying clone bundle
    searching for changes
    adding changesets
    adding manifests
    adding file changes
    adding changesets
    adding manifests
    adding file changes
    added 22612 changesets with 161374 changes to 110024 files (+6 heads)
    new changesets 26c8f37631b6:6f32acf2c5e1 (230 drafts)
    280602 local changesets published

This is what I expected.

Speculation
===

I'm suspicious of the long pause in the CVS example. Maybe one end or 
the other of the CVS connection spent too long doing disk I/O or 
calculating stuff and then NPF timed-out the connection?


Cheers,
Lloyd