Re: freebsd-update and speed

2021-04-19 Thread Rainer Duffner 


> Am 17.04.2021 um 23:09 schrieb Cejka Rudolf :
> 
> Did you try aws.update.freebsd.org  also?


The problem with that is that it probably doesn’t have a fixed IP and not all 
our firewalls can whitelist a domain (and freebsd-update doesn’t work through 
our sophos-proxie)….


But thanks for taking update4 out of the rotation, Philip.




Best Regards
Rainer
___
[email protected] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "[email protected]"

Re: freebsd-update and speed

2021-04-19 Thread Ferdinand Goldmann 

On Sat, 17 Apr 2021, Cejka Rudolf wrote:


The other day, freebsd-update even suffered a timeout.

What are other European users experiences and is there anything to do about it?


Did you try aws.update.freebsd.org also?


I just did now with an update of 11.4 to 12.2. It was really fast, so this
definitely is an option. Thanks!

Would be nice if the FreeBSD project could include this in their config files
for documentation or create a mechanism for freebsd-update to choose a fast
update server.

Regards
Ferdinand

smime.p7s
Description: S/MIME Cryptographic Signature

Re: freebsd-update and speed

2021-04-17 Thread Philip Paeps 

On 2021-04-18 13:57:45 (+0800), Jason Tubnor wrote:


On Sun, 18 Apr 2021 at 10:51, Philip Paeps  wrote:




It looks like there were at least experiments with pointing
freebsd-update at AWS, similar to how portsnap currently works.  I 
will

check if these experiments went anywhere and possibly point
freebsd-update there instead.



The AWS freebsd-update has been working fine for quite a while.  All
project mirrors are slow in Australia so we have been using the AWS 
one
since Colin brought it online to make it a better experience for our 
team

to update the fleet.

FWIW adjust update.FreeBSD.org to aws.update.FreeBSD.org in
/etc/freebsd-update.conf and you are good to go.

Can the project look at offering traditional mirrors for base and pkgs
rather than the current offering?  Those that want to stand up 
un-official

mirrors can do so by pointing rsync to a Tier 2 mirror for updating
purposes so they can provide faster, localised access.  From Melbourne 
to
our closest geo mirror is 240ms, this latency really drags out 
updates, so

having a mirror out of a Melbourne DC would be beneficial.


We've got an ongoing action item to set up a traditional pkg and 
download mirror at IX Australia.  This has kept stalling out over the 
past year-and-a-bit because the world is on fire.  I'll try to pick this 
up again Soon.


Currently, from Australia you're either sent to a pkg or download mirror 
in Malaysia or on the west coast of America.  Neither of those are 
great.  Australia is a big island, far away from everywhere. :)


Glad to hear the AWS stuff is working for you.  I'll see what needs to 
happen to put that in the SRV record for everyone.


Philip

--
Philip Paeps
Senior Reality Engineer
Alternative Enterprises
___
[email protected] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "[email protected]"

Re: freebsd-update and speed

2021-04-17 Thread Jason Tubnor 
On Sun, 18 Apr 2021 at 10:51, Philip Paeps  wrote:

>
>
> It looks like there were at least experiments with pointing
> freebsd-update at AWS, similar to how portsnap currently works.  I will
> check if these experiments went anywhere and possibly point
> freebsd-update there instead.
>

The AWS freebsd-update has been working fine for quite a while.  All
project mirrors are slow in Australia so we have been using the AWS one
since Colin brought it online to make it a better experience for our team
to update the fleet.

FWIW adjust update.FreeBSD.org to aws.update.FreeBSD.org in
/etc/freebsd-update.conf and you are good to go.

Can the project look at offering traditional mirrors for base and pkgs
rather than the current offering?  Those that want to stand up un-official
mirrors can do so by pointing rsync to a Tier 2 mirror for updating
purposes so they can provide faster, localised access.  From Melbourne to
our closest geo mirror is 240ms, this latency really drags out updates, so
having a mirror out of a Melbourne DC would be beneficial.

Cheers,

Jason.
___
[email protected] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "[email protected]"

Re: freebsd-update and speed

2021-04-17 Thread Philip Paeps 

On 2021-04-18 08:51:05 (+0800), Philip Paeps wrote:

On 2021-04-18 03:12:35 (+0800), Rainer Duffner wrote:
I’m cc-ing clusteradm and dnsadmin, in hope that there’s somebody 
there who can either fix it or take update4 out of the srv record…


I can take update4 out of the DNS if it's misbehaving consistently.  
If at all possible though, I'd prefer to fix the actual problem rather 
than simply make it disappear from the DNS.


I've taken update4 out of the SRV record for the time being since it was 
doing more harm than good.


Philip

--
Philip Paeps
Senior Reality Engineer
Alternative Enterprises
___
[email protected] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "[email protected]"

Re: freebsd-update and speed

2021-04-17 Thread Philip Paeps 

On 2021-04-18 03:12:35 (+0800), Rainer Duffner wrote:

Am 16.04.2021 um 10:17 schrieb Ferdinand Goldmann 
:


On Thu, 15 Apr 2021, Rainer Duffner wrote:




It’s OK-ish most of the time here (CH).

It does *NOT* work through a proxy, due to the use of pipelined 
http-requests.


What’s your internet-connection?


The 10Gbit uplink of my university, directly connected to the 
internet, not
behind a proxy. I don't think that's the problem. When update3 was 
still online

I'd always use that and updates were really fast back then.

Now that update3 is gone all update servers seem to be in the US or 
Australia.


After waiting for nearly one hour:

..853085408550856085708580859086008610862086308640865086608670868086908700 
 done.

Applying patches... done.
Fetching 9628 files... gunzip: (stdin): unexpected end of file
0a4626107f3700cf5f87bd9c123bf427bd5a8561aadc2eca1d1605465c090935 has 
incorrect hash.


This is getting kind of tiresome. :(

Regards
Ferdinand





There seems to be a problem with update4.

I now have this, too.


I’m cc-ing clusteradm and dnsadmin, in hope that there’s somebody 
there who can either fix it or take update4 out of the srv record…


I can take update4 out of the DNS if it's misbehaving consistently.  If 
at all possible though, I'd prefer to fix the actual problem rather than 
simply make it disappear from the DNS.


It looks like there were at least experiments with pointing 
freebsd-update at AWS, similar to how portsnap currently works.  I will 
check if these experiments went anywhere and possibly point 
freebsd-update there instead.


I believe the problem with update4 is load-related.


:-(

I would rather just mirror the update server but I think this is not 
supposed to be done?


I think you can set up your own freebsd-update servers, but that won't 
fix this problem.


I'll see what can be done to fix this.

Watch this space.

Philip [hat: clusteradm firefighter]


--
Philip Paeps
Senior Reality Engineer
Alternative Enterprises
___
[email protected] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "[email protected]"

Re: freebsd-update and speed

2021-04-17 Thread Cejka Rudolf 
Ferdinand Goldmann wrote (2021/04/15):
> Hello,
> 
> I've noticed that ever since update3.freebsd.org is gone (which was in Czech
> republic I think), FreeBSD updates are often quite slow for me (= 
> Austria/Europe)
> Especially so for major release upgrades. In fact so slow that I have time
> to type this mail while waiting for '8778 patches'.

Hello, you are right, it was ;o)

> The other day, freebsd-update even suffered a timeout.
> 
> What are other European users experiences and is there anything to do about 
> it?

Did you try aws.update.freebsd.org also?

-- 
Rudolf Cejka  http://www.fit.vut.cz/~cejkar
Brno University of Technology, Faculty of Information Technology
Bozetechova 2, 612 66  Brno, Czech Republic
___
[email protected] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "[email protected]"

Re: freebsd-update and speed

2021-04-17 Thread Rainer Duffner 


> Am 16.04.2021 um 10:17 schrieb Ferdinand Goldmann :
> 
> On Thu, 15 Apr 2021, Rainer Duffner wrote:
> 
>> 
>> 
>> It’s OK-ish most of the time here (CH).
>> 
>> It does *NOT* work through a proxy, due to the use of pipelined 
>> http-requests.
>> 
>> What’s your internet-connection?
> 
> The 10Gbit uplink of my university, directly connected to the internet, not
> behind a proxy. I don't think that's the problem. When update3 was still 
> online
> I'd always use that and updates were really fast back then.
> 
> Now that update3 is gone all update servers seem to be in the US or Australia.
> 
> After waiting for nearly one hour:
> 
> ..853085408550856085708580859086008610862086308640865086608670868086908700
>   done.
> Applying patches... done.
> Fetching 9628 files... gunzip: (stdin): unexpected end of file
> 0a4626107f3700cf5f87bd9c123bf427bd5a8561aadc2eca1d1605465c090935 has 
> incorrect hash.
> 
> This is getting kind of tiresome. :(
> 
> Regards
> Ferdinand




There seems to be a problem with update4.

I now have this, too.


I’m cc-ing clusteradm and dnsadmin, in hope that there’s somebody there who can 
either fix it or take update4 out of the srv record…


:-(

I would rather just mirror the update server but I think this is not supposed 
to be done?





___
[email protected] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "[email protected]"

Re: freebsd-update and speed

2021-04-16 Thread Christos Chatzaras 


> On 16 Apr 2021, at 13:46, Stefan Esser  wrote:
> 
> There was a discussion about adding another mirror in Europe, but
> it was decided that a suitable system already existed.
> 
> Not sure whether this mirror actually has been provided, but I do
> remember that it should have been a well connected system (maybe in
> NL?) that has been selected to perform all FreeBSD mirror services
> for users in Europe with little latency and high throughput.
> 
> Regards, STefan
> 

I use gitup with git.freebsd.org and it connects to gitmir.pkt.freebsd.org 
which is in Netherlands. My servers are in Germany and I get good speed.

portsnap uses "AWS Global Accelerator" and while I use it I was getting good 
speeds too.

I am not familiar with freebsd-update as I build from source, but 
update.freebsd.org points to update1.freebsd.org , update2.freebsd.org and 
update4.freebsd.org which are not in Europe. Does any freebsd-update mirror 
exist in Europe?
___
[email protected] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "[email protected]"

Re: freebsd-update and speed

2021-04-16 Thread Ferdinand Goldmann 

On Fri, 16 Apr 2021, Andrea Brancatelli wrote:


I was experiencing the same problem and modified freebsd-update's config file 
to point directly to one of the other server, can't remember
if update1 or update2 and it was fast.


I just tried update1 and it really was considerably faster. Might have been a
coincidence... I'll see when I do the next updates in a few days.

Anyway this should be something I'd like the freebsd-update utility to by
itsself: Choose a fast mirror. Or at least have a documented way on how to
adjust this instead of trial and error.

Regards
Ferdinand

smime.p7s
Description: S/MIME Cryptographic Signature

Re: freebsd-update and speed

2021-04-16 Thread Stefan Esser 

Am 16.04.21 um 10:17 schrieb Ferdinand Goldmann:

On Thu, 15 Apr 2021, Rainer Duffner wrote:




It’s OK-ish most of the time here (CH).

It does *NOT* work through a proxy, due to the use of pipelined http-requests.

What’s your internet-connection?


The 10Gbit uplink of my university, directly connected to the internet, 

not

behind a proxy. I don't think that's the problem. When update3 was still online
I'd always use that and updates were really fast back then.

Now that update3 is gone all update servers seem to be in the US or Australia.

After waiting for nearly one hour:

..853085408550856085708580859086008610862086308640865086608670868086908700  
done.

Applying patches... done.
Fetching 9628 files... gunzip: (stdin): unexpected end of file
0a4626107f3700cf5f87bd9c123bf427bd5a8561aadc2eca1d1605465c090935 has incorrect 
hash.


This is getting kind of tiresome. :(


There was a discussion about adding another mirror in Europe, but
it was decided that a suitable system already existed.

Not sure whether this mirror actually has been provided, but I do
remember that it should have been a well connected system (maybe in
NL?) that has been selected to perform all FreeBSD mirror services
for users in Europe with little latency and high throughput.

Regards, STefan



OpenPGP_signature
Description: OpenPGP digital signature

Re: freebsd-update and speed

2021-04-16 Thread Ferdinand Goldmann 

On Thu, 15 Apr 2021, Rainer Duffner wrote:




It’s OK-ish most of the time here (CH).

It does *NOT* work through a proxy, due to the use of pipelined http-requests.

What’s your internet-connection?


The 10Gbit uplink of my university, directly connected to the internet, not
behind a proxy. I don't think that's the problem. When update3 was still online
I'd always use that and updates were really fast back then.

Now that update3 is gone all update servers seem to be in the US or Australia.

After waiting for nearly one hour:

..853085408550856085708580859086008610862086308640865086608670868086908700
  done.
Applying patches... done.
Fetching 9628 files... gunzip: (stdin): unexpected end of file
0a4626107f3700cf5f87bd9c123bf427bd5a8561aadc2eca1d1605465c090935 has incorrect 
hash.

This is getting kind of tiresome. :(

Regards
Ferdinand

smime.p7s
Description: S/MIME Cryptographic Signature

Re: freebsd-update and speed

2021-04-16 Thread Andrea Brancatelli via freebsd-stable 
On 2021-04-15 14:20, Ferdinand Goldmann wrote:

> I've noticed that ever since update3.freebsd.org is gone (which was in Czech
> republic I think), FreeBSD updates are often quite slow for me (= 
> Austria/Europe)
> Especially so for major release upgrades. In fact so slow that I have time
> to type this mail while waiting for '8778 patches'.

I was experiencing the same problem and modified freebsd-update's config
file to point directly to one of the other server, can't remember if
update1 or update2 and it was fast.

---

Andrea Brancatelli
___
[email protected] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "[email protected]"

Re: freebsd-update and speed

2021-04-15 Thread Karl Denninger 


On 4/15/2021 08:28, Ferdinand Goldmann wrote:

Following up on my own mail:


to type this mail while waiting for '8778 patches'.


Which has ended in:

71107120 done.
Applying patches... done.
Fetching 1965 files... failed.

and after restarting it:

Fetching 1750 patches
[...]
Applying patches... done.
Fetching 326 files...

This does not seem very reassuring to me. :(


It already got the others, so it now only has to fetch 326 more.

--
Karl Denninger
[email protected] 
/The Market Ticker/
/[S/MIME encrypted email preferred]/


smime.p7s
Description: S/MIME Cryptographic Signature

Re: freebsd-update and speed

2021-04-15 Thread Rainer Duffner 


> Am 15.04.2021 um 14:20 schrieb Ferdinand Goldmann :
> 
> Hello,
> 
> I've noticed that ever since update3.freebsd.org is gone (which was in Czech
> republic I think), FreeBSD updates are often quite slow for me (= 
> Austria/Europe)
> Especially so for major release upgrades. In fact so slow that I have time
> to type this mail while waiting for '8778 patches'.
> 
> The other day, freebsd-update even suffered a timeout.
> 
> What are other European users experiences and is there anything to do about 
> it?
> 
> Regards
> Ferdinand


It’s OK-ish most of the time here (CH).

It does *NOT* work through a proxy, due to the use of pipelined http-requests.

What’s your internet-connection?


___
[email protected] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "[email protected]"

Re: freebsd-update and speed

2021-04-15 Thread Matthias Gamsjager 
Yeah it took some time. One machine (Netherlands) took 4 hours to download
the patches, failed multiple times but it finished after all.
today another box was faster but failed one time.

Far from perfect
___
[email protected] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "[email protected]"

Re: freebsd-update and speed

2021-04-15 Thread Ferdinand Goldmann 

Following up on my own mail:


to type this mail while waiting for '8778 patches'.


Which has ended in:

71107120 done.
Applying patches... done.
Fetching 1965 files... failed.

and after restarting it:

Fetching 1750 patches
[...]
Applying patches... done.
Fetching 326 files...

This does not seem very reassuring to me. :(



The other day, freebsd-update even suffered a timeout.

What are other European users experiences and is there anything to do about 
it?


Regards
Ferdinand


--
Ferdinand Goldmann
System Administrator
Information Management

JOHANNES KEPLER
UNIVERSITY LINZ
Altenberger Straße 69
Hochschulfond Building, HF9902
4040 Linz, Austria
P +43 732 2468 3925
[email protected]
www.jku.at/im

smime.p7s
Description: S/MIME Cryptographic Signature