Re: [Bf-committers] move prebuilt libs from svn to git?

[Ryan Inch via Bf-committers]

Congratulations Dan!  I know this has been something that's been 
problematic for a long time, so it's great you were able to improve 
things (and by the sound of it, things have improved by a lot)


Ryan Inch (Imaginer)

On 2022-06-17 17:45, Dan McGrath via Bf-committers wrote:

Just a heads-up that Ray and I took a packet capture on both ends for me.

Thanks to his capture, I noticed that the SVN server host was not sending
TCP window scale or SACK options. Ultimately this turned out to be due to
that host using the PF firewall "synproxy" state option, which apparently
was causing the TCP header options to be removed, which limited the window
size to 64k.

I have corrected this change to match the www host, and we did observe some
improvements that were hidden from me before due to my and the studio IPs
special treatment that bypassed this feature.

While it should help, I also tuned a few sysctl settings for send and
receive buffers. In addition, I noticed that the SVN server is also not
sending gzip/deflate accept headers for the files. No doubt this is going
to be a factor. However, there was a note from 2016 from myself explaining
that Brecht asked for the SVN compression to be disabled due to some client
problems. So, this will require some research, perhaps next week.

Anyway, hopefully it helps a bit until I can review some more settings.

Enjoy your weekend everyone!


Dan

On Fri, Jun 17, 2022, 2:22 PM Ray Molenkamp via Bf-committers <
bf-committers@blender.org> wrote:


=8<==[Editors note]=8<==
I'll be honest I'm 100% convinced mailinator will ruin
the little formatting I have done, I put a copy of this
email on https://developer.blender.org/P3014.
=8<8<8<8<==

How much as I like the sentiment of "lets move to git will it
solve all these problems" lets be honest here, git.blender.org's
speed is nothing to write home about either it may or may not be
as glitchy as svn, but it still wouldn't be fast.

from my 300mbit connection in western Canada

Receiving objects:   6% (135931/2078159), 33.34 MiB | 458.00 KiB/s

total time for a clone off git.blender.org 27m39s

Cloning off https://github.com/blender/blender.git however

total time for a clone off github.com 1m 52s

Now amount of finger pointing to client settings will convince me
it's a client setting at this point, but I'll be honest, I am all
the way in western Canada and that could definitely be a factor.

so, lets investigate!

Let’s spin up an AWS EC2 instance in London, I'd say that be close
enough to Amsterdam, and let’s rule out any other CPU or Bandwidth
related factors as well, I threw a few dollars at it and got a
cn5.18xlarge instance (72 CPU’s 192gb memory, 100gbit ethernet)
overkill to do a git/svn test? yes.. yes indeed

first to rule out the blender.org servers lets grab a ubuntu iso
off an .nl mirror (
https://mirror.nl.datapacket.com/ubuntu-releases/22.04/ )

2022-06-17 17:13:21 (232 MB/s) - ‘ubuntu-22.04-desktop-amd64.iso’ saved
[3654957056/3654957056]

all right bandwidth is NOT going to be an issue on this box!

On to blender stuff:

cloning from git.blender.org  2m32.895s
svn checkout of lib/win64_vc15 2m56.388s (iftop said 65Mbit peak rate)

yowza!

All right, so in London, safe to say all is well

lets move closer to my location, us-east-2 Ohio (same instance type and os)

grabbing ubuntu from that same mirror

2022-06-17 17:31:00 (29.5 MB/s) - ‘ubuntu-22.04-desktop-amd64.iso’ saved
[3654957056/3654957056]

all right, that has lost quite a bit of steam, but it's
still nothing to sneeze at, just to be sure lets grab
an iso off the Princeton university mirror 120MB/s ok..
good enough...

onto cloning blender

Receiving objects:   5% (104235/2078173), 25.44 MiB | 692.00 KiB/s

well... that's not looking promising... lets wait it out

cloning from git.blender.org  17m23.449s

sadface... lets try svn next, i lost patience and did not let
it finish.. iftop reported a top RX of 5.59Mbit though..

To summarize:

Ubuntu iso from mirror.nl.datapacket.com (Speed taken from wget)
My home - western Canada - 1.3  MB/s
AWS - London - 232  MB/S
AWS - Ohio   - 29.5 MB/s

Clone of git.blender.org (time taken from linux time command)
My home - western Canada - 27m39
AWS - London -  2m32
AWS - Ohio   - 17m23

SVN Clone: (peak RX bitrate taken from iftop)
My home - western Canada -  1.5 Mbit/s
AWS - London -   64 Mbit/s
AWS - Ohio   - 5.59 Mbit/s

Think the only thing we really can conclude is
that being further from the server is leading
to an "unhappy" time for the developer. Given
the fact the EC2 instances were 100% identical
between London and Ohio, it's unlikely to be a
client configuration issue.

--Ray

On 2022-06-17 10:10 a.m., Dan McGrath via Bf-committers wrote:

Hi,

Really, I would need to see a capture from both sides to dig into it. I
need to see what is being sent, and what is received on each side. There
could be some weird firewall 

Re: [Bf-committers] move prebuilt libs from svn to git?

[Ray Molenkamp via Bf-committers]

Except for you just now i don't think anyone is
questioning the need for version control on the
3rd party dependencies. This tree was there long
before i joined the blender project so can't give
any insights on any controversy in creating it.

However whenever this subject (why svn) comes up,
it tends to be one or both of these scenario's

1) svn is giving people a hard time, calling it
"quirky" would be generous,  it doesn't recover
nicely whenever it runs into trouble, and with
the large size of the libraries and the slow
download speeds for some people it's just bound
to run into trouble. And these people rightfully
question "Why is it like this?"

2) SVN is old, everything uses git these days
it's the hip and trendy new kid on the block.
they never have these kinds of issues with git!
why is blender being so silly, just use git!

on the other hand, no-one is using pure git to manage
a large collection of binaries since it is just not
designed for this job, in much the same way the
hand basket at your local hardware store is not
designed to pick up that 25kg bag of cement.

git-lfs is one of the proposed solutions, but
also not free of problems [1] (written by the
maintainer of an different version control
system, be aware of "some" bias there)

Yet another alternative is to go, 3rd party
dependencies are no concern of us, developers
can go obtain/build these on their own. This is
the stance taken by the majority of opensource
projects. And I'll admit, as one of the people
responsible for the contents of this repository
I can't help feeling some envy there, maintaining
the libs and guaranteeing a smooth experience for
developers is *VERY* time consuming. However seeing
new people join the blender project going "building
blender was much easier than I expected, I finished
my small first patch fixing something that had been
bothering me in under a day" negates those feelings
and makes me realize as a project the blender project
is better for everyone by having these libs available.

SVN is what we have and it's mostly working for us
there's alternatives but there's a lot more nuance
to it than "just use X I never have problems with X"

[1] https://gregoryszorc.com/blog/2021/05/12/why-you-shouldn%27t-use-git-lfs/
 

On 2022-06-18 11:24 a.m., Zack Brown wrote:
> Thanks, I appreciate the explanation. It still seems weird. I'm sure the 
> original conversation that led to this particular svn tree must have involved 
> some amount of controversy.
>
> But at this point in the project's history there's definitely something to be 
> said for "if it ain't broke, don't fix it." Especially with so many amazing 
> blender features in rapid development.
>
> Be well,
> Zack
>
>
> On Sat, Jun 18, 2022 at 6:45 PM Ray Molenkamp  wrote:
>
> libraries change over time, sometimes they have large api changes
> where we have to do reintegration work into blender.
>
> Which then leads to, if we have a single folder with libs, what
> do the 2 concurrent LTS releases use? that same lib folder? they
> need major rework to use them, we can't land rework in an lts release
> brand new bugs will likely come with it.  option b: well we'll give
> every release their own folder on that shared drive so they won't have
> issues when the libs change.
>
> wait a minute... we've just reinvented version control :)
>
> --Ray
>
> On 2022-06-18 1:04 a.m., Zack Brown wrote:
>> Hi everyone,
>>
>> It's fascinating to see this discussion. It never occurred to me that 
>> anyone might continue to prefer svn once git came into existence, and yet 
>> here is a sample case, right under my nose! I guess svn is still relevant 
>> even after all these years!
>>
>> But actually I'd be curious to know why revision control is used at all 
>> in this particular case. We're talking about precompiled binaries, right? 
>> That's sort of the poster child for situations where revision control is 
>> *not* what one wants, exactly for the reason why git would be a bad choice.
>>
>> Please, could someone explain why the blender project doesn't simply use 
>> a dev-accessible shared folder somewhere? I love revision control, but it's 
>> only useful if it's useful.
>>
>> Be well,
>> Zack
>
>
___
Bf-committers mailing list
Bf-committers@blender.org
List details, subscription details or unsubscribe:
https://lists.blender.org/mailman/listinfo/bf-committers

Re: [Bf-committers] move prebuilt libs from svn to git?

[Ankit via Bf-committers]

Zack
- To track library versions https://developer.blender.org/T88438
- To tag corresponding versions for each release 
https://svn.blender.org/svnroot/bf-blender/tags/
- Some bugs show up due to deps also, 
https://developer.blender.org/rBL62677 . VCS can point to the commit 
that caused it.


Ankit

On 18/06/22 12:34, Zack Brown via Bf-committers wrote:

Hi everyone,

It's fascinating to see this discussion. It never occurred to me that
anyone might continue to prefer svn once git came into existence, and yet
here is a sample case, right under my nose! I guess svn is still relevant
even after all these years!

But actually I'd be curious to know why revision control is used at all in
this particular case. We're talking about precompiled binaries, right?
That's sort of the poster child for situations where revision control is
*not* what one wants, exactly for the reason why git would be a bad choice.

Please, could someone explain why the blender project doesn't simply use a
dev-accessible shared folder somewhere? I love revision control, but it's
only useful if it's useful.

Be well,
Zack


On Sat, Jun 18, 2022 at 1:10 AM Ray Molenkamp via Bf-committers <
bf-committers@blender.org> wrote:


Just an update with some new numbers, dan's changes
brought down the time for a full clone from my home
connection down to 5m48s (down from 27m39s previously)

well done dan!

--Ray

On 2022-06-17 3:45 p.m., Dan McGrath wrote:

Just a heads-up that Ray and I took a packet capture on both ends for me.

Thanks to his capture, I noticed that the SVN server host was not

sending TCP window scale or SACK options. Ultimately this turned out to be
due to that host using the PF firewall "synproxy" state option, which
apparently was causing the TCP header options to be removed, which limited
the window size to 64k.


I have corrected this change to match the www host, and we did observe

some improvements that were hidden from me before due to my and the studio
IPs special treatment that bypassed this feature.


While it should help, I also tuned a few sysctl settings for send and

receive buffers. In addition, I noticed that the SVN server is also not
sending gzip/deflate accept headers for the files. No doubt this is going
to be a factor. However, there was a note from 2016 from myself explaining
that Brecht asked for the SVN compression to be disabled due to some client
problems. So, this will require some research, perhaps next week.


Anyway, hopefully it helps a bit until I can review some more settings.

Enjoy your weekend everyone!


Dan

On Fri, Jun 17, 2022, 2:22 PM Ray Molenkamp via Bf-committers <

bf-committers@blender.org> wrote:


 =8<==[Editors note]=8<==
 I'll be honest I'm 100% convinced mailinator will ruin
 the little formatting I have done, I put a copy of this
 email on https://developer.blender.org/P3014.
 =8<8<8<8<==

 How much as I like the sentiment of "lets move to git will it
 solve all these problems" lets be honest here, git.blender.org <

http://git.blender.org>'s

 speed is nothing to write home about either it may or may not be
 as glitchy as svn, but it still wouldn't be fast.

 from my 300mbit connection in western Canada

 Receiving objects:   6% (135931/2078159), 33.34 MiB | 458.00 KiB/s

 total time for a clone off git.blender.org 

27m39s


 Cloning off https://github.com/blender/blender.git however

 total time for a clone off github.com  1m 52s

 Now amount of finger pointing to client settings will convince me
 it's a client setting at this point, but I'll be honest, I am all
 the way in western Canada and that could definitely be a factor.

 so, lets investigate!

 Let’s spin up an AWS EC2 instance in London, I'd say that be close
 enough to Amsterdam, and let’s rule out any other CPU or Bandwidth
 related factors as well, I threw a few dollars at it and got a
 cn5.18xlarge instance (72 CPU’s 192gb memory, 100gbit ethernet)
 overkill to do a git/svn test? yes.. yes indeed

 first to rule out the blender.org  servers lets

grab a ubuntu iso

 off an .nl mirror (

https://mirror.nl.datapacket.com/ubuntu-releases/22.04/ )


 2022-06-17 17:13:21 (232 MB/s) - ‘ubuntu-22.04-desktop-amd64.iso’

saved [3654957056/3654957056]


 all right bandwidth is NOT going to be an issue on this box!

 On to blender stuff:

 cloning from git.blender.org   2m32.895s
 svn checkout of lib/win64_vc15 2m56.388s (iftop said 65Mbit peak

rate)


 yowza!

 All right, so in London, safe to say all is well

 lets move closer to my location, us-east-2 Ohio (same instance type

and os)


 grabbing ubuntu from that same mirror

 2022-06-17 17:31:00 (29.5 MB/s) - ‘ubuntu-22.04-desktop-amd64.iso’

saved [3654957056/3654957056]


 all right, that has lost quite a bit of steam, 

Re: [Bf-committers] move prebuilt libs from svn to git?

[Zack Brown via Bf-committers]

Hi everyone,

It's fascinating to see this discussion. It never occurred to me that
anyone might continue to prefer svn once git came into existence, and yet
here is a sample case, right under my nose! I guess svn is still relevant
even after all these years!

But actually I'd be curious to know why revision control is used at all in
this particular case. We're talking about precompiled binaries, right?
That's sort of the poster child for situations where revision control is
*not* what one wants, exactly for the reason why git would be a bad choice.

Please, could someone explain why the blender project doesn't simply use a
dev-accessible shared folder somewhere? I love revision control, but it's
only useful if it's useful.

Be well,
Zack


On Sat, Jun 18, 2022 at 1:10 AM Ray Molenkamp via Bf-committers <
bf-committers@blender.org> wrote:

> Just an update with some new numbers, dan's changes
> brought down the time for a full clone from my home
> connection down to 5m48s (down from 27m39s previously)
>
> well done dan!
>
> --Ray
>
> On 2022-06-17 3:45 p.m., Dan McGrath wrote:
> > Just a heads-up that Ray and I took a packet capture on both ends for me.
> >
> > Thanks to his capture, I noticed that the SVN server host was not
> sending TCP window scale or SACK options. Ultimately this turned out to be
> due to that host using the PF firewall "synproxy" state option, which
> apparently was causing the TCP header options to be removed, which limited
> the window size to 64k.
> >
> > I have corrected this change to match the www host, and we did observe
> some improvements that were hidden from me before due to my and the studio
> IPs special treatment that bypassed this feature.
> >
> > While it should help, I also tuned a few sysctl settings for send and
> receive buffers. In addition, I noticed that the SVN server is also not
> sending gzip/deflate accept headers for the files. No doubt this is going
> to be a factor. However, there was a note from 2016 from myself explaining
> that Brecht asked for the SVN compression to be disabled due to some client
> problems. So, this will require some research, perhaps next week.
> >
> > Anyway, hopefully it helps a bit until I can review some more settings.
> >
> > Enjoy your weekend everyone!
> >
> >
> > Dan
> >
> > On Fri, Jun 17, 2022, 2:22 PM Ray Molenkamp via Bf-committers <
> bf-committers@blender.org> wrote:
> >
> > =8<==[Editors note]=8<==
> > I'll be honest I'm 100% convinced mailinator will ruin
> > the little formatting I have done, I put a copy of this
> > email on https://developer.blender.org/P3014.
> > =8<8<8<8<==
> >
> > How much as I like the sentiment of "lets move to git will it
> > solve all these problems" lets be honest here, git.blender.org <
> http://git.blender.org>'s
> > speed is nothing to write home about either it may or may not be
> > as glitchy as svn, but it still wouldn't be fast.
> >
> > from my 300mbit connection in western Canada
> >
> > Receiving objects:   6% (135931/2078159), 33.34 MiB | 458.00 KiB/s
> >
> > total time for a clone off git.blender.org 
> 27m39s
> >
> > Cloning off https://github.com/blender/blender.git however
> >
> > total time for a clone off github.com  1m 52s
> >
> > Now amount of finger pointing to client settings will convince me
> > it's a client setting at this point, but I'll be honest, I am all
> > the way in western Canada and that could definitely be a factor.
> >
> > so, lets investigate!
> >
> > Let’s spin up an AWS EC2 instance in London, I'd say that be close
> > enough to Amsterdam, and let’s rule out any other CPU or Bandwidth
> > related factors as well, I threw a few dollars at it and got a
> > cn5.18xlarge instance (72 CPU’s 192gb memory, 100gbit ethernet)
> > overkill to do a git/svn test? yes.. yes indeed
> >
> > first to rule out the blender.org  servers lets
> grab a ubuntu iso
> > off an .nl mirror (
> https://mirror.nl.datapacket.com/ubuntu-releases/22.04/ )
> >
> > 2022-06-17 17:13:21 (232 MB/s) - ‘ubuntu-22.04-desktop-amd64.iso’
> saved [3654957056/3654957056]
> >
> > all right bandwidth is NOT going to be an issue on this box!
> >
> > On to blender stuff:
> >
> > cloning from git.blender.org   2m32.895s
> > svn checkout of lib/win64_vc15 2m56.388s (iftop said 65Mbit peak
> rate)
> >
> > yowza!
> >
> > All right, so in London, safe to say all is well
> >
> > lets move closer to my location, us-east-2 Ohio (same instance type
> and os)
> >
> > grabbing ubuntu from that same mirror
> >
> > 2022-06-17 17:31:00 (29.5 MB/s) - ‘ubuntu-22.04-desktop-amd64.iso’
> saved [3654957056/3654957056]
> >
> > all right, that has lost quite a bit of steam, but it's
> > still nothing to sneeze at, just to be sure lets grab
> > an iso off the Princeton 

Re: [Bf-committers] move prebuilt libs from svn to git?

[Ray Molenkamp via Bf-committers]

Just an update with some new numbers, dan's changes
brought down the time for a full clone from my home
connection down to 5m48s (down from 27m39s previously)

well done dan!

--Ray

On 2022-06-17 3:45 p.m., Dan McGrath wrote:
> Just a heads-up that Ray and I took a packet capture on both ends for me.
>
> Thanks to his capture, I noticed that the SVN server host was not sending TCP 
> window scale or SACK options. Ultimately this turned out to be due to that 
> host using the PF firewall "synproxy" state option, which apparently was 
> causing the TCP header options to be removed, which limited the window size 
> to 64k.
>
> I have corrected this change to match the www host, and we did observe some 
> improvements that were hidden from me before due to my and the studio IPs 
> special treatment that bypassed this feature.
>
> While it should help, I also tuned a few sysctl settings for send and receive 
> buffers. In addition, I noticed that the SVN server is also not sending 
> gzip/deflate accept headers for the files. No doubt this is going to be a 
> factor. However, there was a note from 2016 from myself explaining that 
> Brecht asked for the SVN compression to be disabled due to some client 
> problems. So, this will require some research, perhaps next week.
>
> Anyway, hopefully it helps a bit until I can review some more settings.
>
> Enjoy your weekend everyone!
>
>
> Dan
>
> On Fri, Jun 17, 2022, 2:22 PM Ray Molenkamp via Bf-committers 
>  wrote:
>
> =8<==[Editors note]=8<==
> I'll be honest I'm 100% convinced mailinator will ruin
> the little formatting I have done, I put a copy of this
> email on https://developer.blender.org/P3014.
> =8<8<8<8<==
>
> How much as I like the sentiment of "lets move to git will it
> solve all these problems" lets be honest here, git.blender.org 
> 's
> speed is nothing to write home about either it may or may not be
> as glitchy as svn, but it still wouldn't be fast.
>
> from my 300mbit connection in western Canada
>
> Receiving objects:   6% (135931/2078159), 33.34 MiB | 458.00 KiB/s
>
> total time for a clone off git.blender.org  27m39s
>
> Cloning off https://github.com/blender/blender.git however
>
> total time for a clone off github.com  1m 52s
>
> Now amount of finger pointing to client settings will convince me
> it's a client setting at this point, but I'll be honest, I am all
> the way in western Canada and that could definitely be a factor.
>
> so, lets investigate!
>
> Let’s spin up an AWS EC2 instance in London, I'd say that be close
> enough to Amsterdam, and let’s rule out any other CPU or Bandwidth
> related factors as well, I threw a few dollars at it and got a
> cn5.18xlarge instance (72 CPU’s 192gb memory, 100gbit ethernet)
> overkill to do a git/svn test? yes.. yes indeed
>
> first to rule out the blender.org  servers lets grab 
> a ubuntu iso
> off an .nl mirror ( 
> https://mirror.nl.datapacket.com/ubuntu-releases/22.04/ )
>
> 2022-06-17 17:13:21 (232 MB/s) - ‘ubuntu-22.04-desktop-amd64.iso’ saved 
> [3654957056/3654957056]
>
> all right bandwidth is NOT going to be an issue on this box!
>
> On to blender stuff:
>
> cloning from git.blender.org   2m32.895s
> svn checkout of lib/win64_vc15 2m56.388s (iftop said 65Mbit peak rate)
>
> yowza!
>
> All right, so in London, safe to say all is well
>
> lets move closer to my location, us-east-2 Ohio (same instance type and 
> os)
>
> grabbing ubuntu from that same mirror
>
> 2022-06-17 17:31:00 (29.5 MB/s) - ‘ubuntu-22.04-desktop-amd64.iso’ saved 
> [3654957056/3654957056]
>
> all right, that has lost quite a bit of steam, but it's
> still nothing to sneeze at, just to be sure lets grab
> an iso off the Princeton university mirror 120MB/s ok..
> good enough...
>
> onto cloning blender
>
> Receiving objects:   5% (104235/2078173), 25.44 MiB | 692.00 KiB/s
>
> well... that's not looking promising... lets wait it out
>
> cloning from git.blender.org   17m23.449s
>
> sadface... lets try svn next, i lost patience and did not let
> it finish.. iftop reported a top RX of 5.59Mbit though..
>
> To summarize:
>
> Ubuntu iso from mirror.nl.datapacket.com 
>  (Speed taken from wget)
> My home - western Canada - 1.3  MB/s
> AWS - London - 232  MB/S
> AWS - Ohio   - 29.5 MB/s
>
> Clone of git.blender.org  (time taken from linux 
> time command)
> My home - western Canada - 27m39
> AWS - London -  2m32
> AWS - Ohio   - 17m23
>
> SVN Clone: (peak RX bitrate taken from iftop)
> My home - western Canada -  1.5 Mbit/s
> AWS - London   

Re: [Bf-committers] move prebuilt libs from svn to git?

[Dan McGrath via Bf-committers]

Just a heads-up that Ray and I took a packet capture on both ends for me.

Thanks to his capture, I noticed that the SVN server host was not sending
TCP window scale or SACK options. Ultimately this turned out to be due to
that host using the PF firewall "synproxy" state option, which apparently
was causing the TCP header options to be removed, which limited the window
size to 64k.

I have corrected this change to match the www host, and we did observe some
improvements that were hidden from me before due to my and the studio IPs
special treatment that bypassed this feature.

While it should help, I also tuned a few sysctl settings for send and
receive buffers. In addition, I noticed that the SVN server is also not
sending gzip/deflate accept headers for the files. No doubt this is going
to be a factor. However, there was a note from 2016 from myself explaining
that Brecht asked for the SVN compression to be disabled due to some client
problems. So, this will require some research, perhaps next week.

Anyway, hopefully it helps a bit until I can review some more settings.

Enjoy your weekend everyone!


Dan

On Fri, Jun 17, 2022, 2:22 PM Ray Molenkamp via Bf-committers <
bf-committers@blender.org> wrote:

> =8<==[Editors note]=8<==
> I'll be honest I'm 100% convinced mailinator will ruin
> the little formatting I have done, I put a copy of this
> email on https://developer.blender.org/P3014.
> =8<8<8<8<==
>
> How much as I like the sentiment of "lets move to git will it
> solve all these problems" lets be honest here, git.blender.org's
> speed is nothing to write home about either it may or may not be
> as glitchy as svn, but it still wouldn't be fast.
>
> from my 300mbit connection in western Canada
>
> Receiving objects:   6% (135931/2078159), 33.34 MiB | 458.00 KiB/s
>
> total time for a clone off git.blender.org 27m39s
>
> Cloning off https://github.com/blender/blender.git however
>
> total time for a clone off github.com 1m 52s
>
> Now amount of finger pointing to client settings will convince me
> it's a client setting at this point, but I'll be honest, I am all
> the way in western Canada and that could definitely be a factor.
>
> so, lets investigate!
>
> Let’s spin up an AWS EC2 instance in London, I'd say that be close
> enough to Amsterdam, and let’s rule out any other CPU or Bandwidth
> related factors as well, I threw a few dollars at it and got a
> cn5.18xlarge instance (72 CPU’s 192gb memory, 100gbit ethernet)
> overkill to do a git/svn test? yes.. yes indeed
>
> first to rule out the blender.org servers lets grab a ubuntu iso
> off an .nl mirror (
> https://mirror.nl.datapacket.com/ubuntu-releases/22.04/ )
>
> 2022-06-17 17:13:21 (232 MB/s) - ‘ubuntu-22.04-desktop-amd64.iso’ saved
> [3654957056/3654957056]
>
> all right bandwidth is NOT going to be an issue on this box!
>
> On to blender stuff:
>
> cloning from git.blender.org  2m32.895s
> svn checkout of lib/win64_vc15 2m56.388s (iftop said 65Mbit peak rate)
>
> yowza!
>
> All right, so in London, safe to say all is well
>
> lets move closer to my location, us-east-2 Ohio (same instance type and os)
>
> grabbing ubuntu from that same mirror
>
> 2022-06-17 17:31:00 (29.5 MB/s) - ‘ubuntu-22.04-desktop-amd64.iso’ saved
> [3654957056/3654957056]
>
> all right, that has lost quite a bit of steam, but it's
> still nothing to sneeze at, just to be sure lets grab
> an iso off the Princeton university mirror 120MB/s ok..
> good enough...
>
> onto cloning blender
>
> Receiving objects:   5% (104235/2078173), 25.44 MiB | 692.00 KiB/s
>
> well... that's not looking promising... lets wait it out
>
> cloning from git.blender.org  17m23.449s
>
> sadface... lets try svn next, i lost patience and did not let
> it finish.. iftop reported a top RX of 5.59Mbit though..
>
> To summarize:
>
> Ubuntu iso from mirror.nl.datapacket.com (Speed taken from wget)
> My home - western Canada - 1.3  MB/s
> AWS - London - 232  MB/S
> AWS - Ohio   - 29.5 MB/s
>
> Clone of git.blender.org (time taken from linux time command)
> My home - western Canada - 27m39
> AWS - London -  2m32
> AWS - Ohio   - 17m23
>
> SVN Clone: (peak RX bitrate taken from iftop)
> My home - western Canada -  1.5 Mbit/s
> AWS - London -   64 Mbit/s
> AWS - Ohio   - 5.59 Mbit/s
>
> Think the only thing we really can conclude is
> that being further from the server is leading
> to an "unhappy" time for the developer. Given
> the fact the EC2 instances were 100% identical
> between London and Ohio, it's unlikely to be a
> client configuration issue.
>
> --Ray
>
> On 2022-06-17 10:10 a.m., Dan McGrath via Bf-committers wrote:
> > Hi,
> >
> > Really, I would need to see a capture from both sides to dig into it. I
> > need to see what is being sent, and what is received on each side. There
> > could be some weird firewall stuff happening that is causing
> fragmentation
> > or blocking TCP options, or window scaling 

Re: [Bf-committers] move prebuilt libs from svn to git?

[Chad Fraleigh via Bf-committers]

On 6/16/2022 11:51 PM, Ankit via Bf-committers wrote:
> Tom:
> 1. Unlike SVN checkout, git checkout cannot be resumed. It starts
>   over if interrupted.
> 2. .git folder's size will blow up with every release. Git is not
>   so good for binary files.

Yes, plain vanilla git would be bad with them.

If I understand LFS correctly, it only holds object references. So even if the 
pull can't be restarted, it would only have the [small] references to download. 
The actual downloading of the [large] binary files is moved to the checkout 
phase (which doesn't appear to be an all-or-nothing step). But I could be wrong.

> 
> Chad:
> 
> https://developer.blender.org/T73511
> https://developer.blender.org/T73394

These are from about 2 years ago. So if there were still any blocking issues 
back then, support for LFS should have grown since (at least enough to 
re-evaluate the feasibility of migration).

> 
> Cheers
> Ankit
> 
> 
> On 17/06/22 00:45, Chad Fraleigh via Bf-committers wrote:
>> I've been kind of wondering if they would eventually be migrated too. I only 
>> update my local libs occasionally, so it has a tenancy to fail if doing a 
>> large update spanning many revisions. Plus the benefit of not having to use 
>> two different tools.
>>
>> Now with Git LFS being more widespread, it should mitigate the issue with 
>> all the large binary libraries.
>>
>> On 6/16/2022 11:43 AM, Tom M via Bf-committers wrote:
>>> When checking out the libraries from SVN following the directions on
>>>
>>> https://wiki.blender.org/wiki/Building_Blender/Windows
>>>
>>> The svn checkout of the libraries fails with significant frequency,
>>>
>>> if you google you might stumble across using make svnfix
>>>
>>> https://www.mail-archive.com/bf-blender-cvs@blender.org/msg151996.html
>>>
>>> Even after using it, I'm still having to do Ctrl C, and then make
>>> svnfix repeatedly (it downloads some files, than randomly freezes,
>>> sometimes after 20-50 files, sometimes after 100's of files).
>>>
>>> Git doesn't seem to have any problems, though.
>>>
>>> So might we consider migrating the libs to git, rather than using SVN?
>>>
>>> LetterRip
>>> ___
>>> Bf-committers mailing list
>>> Bf-committers@blender.org
>>> List details, subscription details or unsubscribe:
>>> https://lists.blender.org/mailman/listinfo/bf-committers
>> ___
>> Bf-committers mailing list
>> Bf-committers@blender.org
>> List details, subscription details or unsubscribe:
>> https://lists.blender.org/mailman/listinfo/bf-committers
> ___
> Bf-committers mailing list
> Bf-committers@blender.org
> List details, subscription details or unsubscribe:
> https://lists.blender.org/mailman/listinfo/bf-committers
___
Bf-committers mailing list
Bf-committers@blender.org
List details, subscription details or unsubscribe:
https://lists.blender.org/mailman/listinfo/bf-committers

Re: [Bf-committers] move prebuilt libs from svn to git?

[Ray Molenkamp via Bf-committers]

=8<==[Editors note]=8<==
I'll be honest I'm 100% convinced mailinator will ruin
the little formatting I have done, I put a copy of this
email on https://developer.blender.org/P3014.
=8<8<8<8<==

How much as I like the sentiment of "lets move to git will it
solve all these problems" lets be honest here, git.blender.org's
speed is nothing to write home about either it may or may not be
as glitchy as svn, but it still wouldn't be fast.

from my 300mbit connection in western Canada

Receiving objects:   6% (135931/2078159), 33.34 MiB | 458.00 KiB/s

total time for a clone off git.blender.org 27m39s

Cloning off https://github.com/blender/blender.git however

total time for a clone off github.com 1m 52s

Now amount of finger pointing to client settings will convince me
it's a client setting at this point, but I'll be honest, I am all
the way in western Canada and that could definitely be a factor.

so, lets investigate!

Let’s spin up an AWS EC2 instance in London, I'd say that be close
enough to Amsterdam, and let’s rule out any other CPU or Bandwidth
related factors as well, I threw a few dollars at it and got a
cn5.18xlarge instance (72 CPU’s 192gb memory, 100gbit ethernet)
overkill to do a git/svn test? yes.. yes indeed

first to rule out the blender.org servers lets grab a ubuntu iso
off an .nl mirror ( https://mirror.nl.datapacket.com/ubuntu-releases/22.04/ )

2022-06-17 17:13:21 (232 MB/s) - ‘ubuntu-22.04-desktop-amd64.iso’ saved 
[3654957056/3654957056]

all right bandwidth is NOT going to be an issue on this box!

On to blender stuff:

cloning from git.blender.org  2m32.895s
svn checkout of lib/win64_vc15 2m56.388s (iftop said 65Mbit peak rate)

yowza!

All right, so in London, safe to say all is well

lets move closer to my location, us-east-2 Ohio (same instance type and os)

grabbing ubuntu from that same mirror

2022-06-17 17:31:00 (29.5 MB/s) - ‘ubuntu-22.04-desktop-amd64.iso’ saved 
[3654957056/3654957056]

all right, that has lost quite a bit of steam, but it's
still nothing to sneeze at, just to be sure lets grab
an iso off the Princeton university mirror 120MB/s ok..
good enough...

onto cloning blender

Receiving objects:   5% (104235/2078173), 25.44 MiB | 692.00 KiB/s

well... that's not looking promising... lets wait it out

cloning from git.blender.org  17m23.449s

sadface... lets try svn next, i lost patience and did not let
it finish.. iftop reported a top RX of 5.59Mbit though..

To summarize:

Ubuntu iso from mirror.nl.datapacket.com (Speed taken from wget)
My home - western Canada - 1.3  MB/s
AWS - London - 232  MB/S
AWS - Ohio   - 29.5 MB/s

Clone of git.blender.org (time taken from linux time command)
My home - western Canada - 27m39
AWS - London -  2m32
AWS - Ohio   - 17m23

SVN Clone: (peak RX bitrate taken from iftop)
My home - western Canada -  1.5 Mbit/s
AWS - London -   64 Mbit/s
AWS - Ohio   - 5.59 Mbit/s

Think the only thing we really can conclude is
that being further from the server is leading
to an "unhappy" time for the developer. Given
the fact the EC2 instances were 100% identical
between London and Ohio, it's unlikely to be a
client configuration issue.

--Ray

On 2022-06-17 10:10 a.m., Dan McGrath via Bf-committers wrote:
> Hi,
>
> Really, I would need to see a capture from both sides to dig into it. I
> need to see what is being sent, and what is received on each side. There
> could be some weird firewall stuff happening that is causing fragmentation
> or blocking TCP options, or window scaling issues, etc.
>
> As for the default settings for tortoisesvn, the connection stuff I was
> referring to was about TCP/IP behaviour, rather than an application
> setting. Either way, I need to see what is actually happening. The fact
> that I can pull 5-8MB/sec (~40mbit) from Canada and not have a single
> interruption is telling. Granted, the host only has around 100 or 200mbit
> allocated, iirc. Multiply that by a few dozen people and you have a problem.
>
> For a simpler test, maybe try a large svn check, perhaps some old FreeBSD
> ports svn repo if you can find one, something also in .NL, and see how it
> goes, just to help eliminate your PC/firewall. Worst case scenario, you can
> also poke me in chat and we can try some server side and client side
> captures.
>
> TBH though, 1gig uplink just isn't a whole lot to serve our user base ;)
>
> On Fri, Jun 17, 2022 at 11:45 AM Tom M  wrote:
>
>> On Fri, Jun 17, 2022 at 5:20 AM Dan McGrath 
>> wrote:
>>> Hi Tom,
>>>
>>> Long time no see :)
>>>
>>> I did some tests from home and found that, aside from slow speeds, I was
>> able to checkout at least 5 or 6 gig of the bf-blender repo without error
>> or having to retransmit.
>>
>> Yeah, looks like I was mistaking really slow download of the larger
>> (.7 GB) libraries as freezing.  I was averaging about .3 MB/s.  I used
>> a different SVN client and saw that it was indeed downloading (the
>> 

Re: [Bf-committers] move prebuilt libs from svn to git?

[Jacob Merrill via Bf-committers]

What about p2p?

Can we have users seed lib downloads?

On Fri, Jun 17, 2022, 9:11 AM Dan McGrath via Bf-committers <
bf-committers@blender.org> wrote:

> Hi,
>
> Really, I would need to see a capture from both sides to dig into it. I
> need to see what is being sent, and what is received on each side. There
> could be some weird firewall stuff happening that is causing fragmentation
> or blocking TCP options, or window scaling issues, etc.
>
> As for the default settings for tortoisesvn, the connection stuff I was
> referring to was about TCP/IP behaviour, rather than an application
> setting. Either way, I need to see what is actually happening. The fact
> that I can pull 5-8MB/sec (~40mbit) from Canada and not have a single
> interruption is telling. Granted, the host only has around 100 or 200mbit
> allocated, iirc. Multiply that by a few dozen people and you have a
> problem.
>
> For a simpler test, maybe try a large svn check, perhaps some old FreeBSD
> ports svn repo if you can find one, something also in .NL, and see how it
> goes, just to help eliminate your PC/firewall. Worst case scenario, you can
> also poke me in chat and we can try some server side and client side
> captures.
>
> TBH though, 1gig uplink just isn't a whole lot to serve our user base ;)
>
> On Fri, Jun 17, 2022 at 11:45 AM Tom M  wrote:
>
> > On Fri, Jun 17, 2022 at 5:20 AM Dan McGrath 
> > wrote:
> > >
> > > Hi Tom,
> > >
> > > Long time no see :)
> > >
> > > I did some tests from home and found that, aside from slow speeds, I
> was
> > able to checkout at least 5 or 6 gig of the bf-blender repo without error
> > or having to retransmit.
> >
> > Yeah, looks like I was mistaking really slow download of the larger
> > (.7 GB) libraries as freezing.  I was averaging about .3 MB/s.  I used
> > a different SVN client and saw that it was indeed downloading (the
> > sliksvn doesn't provide any feedback on files in progress, only once a
> > file completes) - It was taking about 40 minutes for each of the
> > larger files.  It did complete eventually after about 5 hours.
> >
> > > I took the liberty of packet capturing the stream from my PC's POV, and
> > placed them in the ftp download area[1], if you care to take a look.
> > >
> > > The gist is that I was able to get around 5MB/sec for at least 5G of
> the
> > checkout, before I cancelled it.
> > >
> > Interesting, I usually get up to 3 MB/s download here normally from
> > other sources, but was typically getting .3 MB/s, with occasional
> > bursts of higher speed to a max of .8 MB/s when it wasn't downloading
> > the large files.
> >
> > > Now, my IP is a bit of a special case, so I am wondering if you are
> just
> > hitting the server so hard with http requests in separate sockets, or
> > something extreme, that is causing the system to block you for a period
> of
> > time. For comparison, my PC only had about 4 sockets open to the server
> for
> > the transfer. I would be curious to know if you are doing pipelining, or
> > sending each request in a separate connection.
> > >
> > Whatever the sliksvn and tortoisesvn defaults are, I'd assume they are
> > setup for typical usage.
> >
> > > If you can, the next time it happens, see if you are unable to visit
> > Phabricator or resume the svn checkout process. It may not be a firewall
> > issue, but without seeing some packet captures, I wouldn't be able to
> > really guess. Ideally, try to capture a few packets, until failure. At
> the
> > very least, you will need to snap 20b+20b=40b for IP+TCP headers, but I
> > would suggest a tad higher, maybe 64b.
> > >
> > > Cheers,
> > >
> > > Dan
> > >
> > > [1] - https://download.blender.org/ftp/dan/svn-checkout/
> > >
> > >
> > >
> > > On Thu, Jun 16, 2022 at 2:43 PM Tom M via Bf-committers <
> > bf-committers@blender.org> wrote:
> > >>
> > >> When checking out the libraries from SVN following the directions on
> > >>
> > >> https://wiki.blender.org/wiki/Building_Blender/Windows
> > >>
> > >> The svn checkout of the libraries fails with significant frequency,
> > >>
> > >> if you google you might stumble across using make svnfix
> > >>
> > >>
> https://www.mail-archive.com/bf-blender-cvs@blender.org/msg151996.html
> > >>
> > >> Even after using it, I'm still having to do Ctrl C, and then make
> > >> svnfix repeatedly (it downloads some files, than randomly freezes,
> > >> sometimes after 20-50 files, sometimes after 100's of files).
> > >>
> > >> Git doesn't seem to have any problems, though.
> > >>
> > >> So might we consider migrating the libs to git, rather than using SVN?
> > >>
> > >> LetterRip
> > >> ___
> > >> Bf-committers mailing list
> > >> Bf-committers@blender.org
> > >> List details, subscription details or unsubscribe:
> > >> https://lists.blender.org/mailman/listinfo/bf-committers
> > >
> > >
> > >
> > > --
> > > Cheers,
> > > Danny
> > >
> > > --
> > > Danny McGrath - 

Re: [Bf-committers] move prebuilt libs from svn to git?

[Dan McGrath via Bf-committers]

Hi,

Really, I would need to see a capture from both sides to dig into it. I
need to see what is being sent, and what is received on each side. There
could be some weird firewall stuff happening that is causing fragmentation
or blocking TCP options, or window scaling issues, etc.

As for the default settings for tortoisesvn, the connection stuff I was
referring to was about TCP/IP behaviour, rather than an application
setting. Either way, I need to see what is actually happening. The fact
that I can pull 5-8MB/sec (~40mbit) from Canada and not have a single
interruption is telling. Granted, the host only has around 100 or 200mbit
allocated, iirc. Multiply that by a few dozen people and you have a problem.

For a simpler test, maybe try a large svn check, perhaps some old FreeBSD
ports svn repo if you can find one, something also in .NL, and see how it
goes, just to help eliminate your PC/firewall. Worst case scenario, you can
also poke me in chat and we can try some server side and client side
captures.

TBH though, 1gig uplink just isn't a whole lot to serve our user base ;)

On Fri, Jun 17, 2022 at 11:45 AM Tom M  wrote:

> On Fri, Jun 17, 2022 at 5:20 AM Dan McGrath 
> wrote:
> >
> > Hi Tom,
> >
> > Long time no see :)
> >
> > I did some tests from home and found that, aside from slow speeds, I was
> able to checkout at least 5 or 6 gig of the bf-blender repo without error
> or having to retransmit.
>
> Yeah, looks like I was mistaking really slow download of the larger
> (.7 GB) libraries as freezing.  I was averaging about .3 MB/s.  I used
> a different SVN client and saw that it was indeed downloading (the
> sliksvn doesn't provide any feedback on files in progress, only once a
> file completes) - It was taking about 40 minutes for each of the
> larger files.  It did complete eventually after about 5 hours.
>
> > I took the liberty of packet capturing the stream from my PC's POV, and
> placed them in the ftp download area[1], if you care to take a look.
> >
> > The gist is that I was able to get around 5MB/sec for at least 5G of the
> checkout, before I cancelled it.
> >
> Interesting, I usually get up to 3 MB/s download here normally from
> other sources, but was typically getting .3 MB/s, with occasional
> bursts of higher speed to a max of .8 MB/s when it wasn't downloading
> the large files.
>
> > Now, my IP is a bit of a special case, so I am wondering if you are just
> hitting the server so hard with http requests in separate sockets, or
> something extreme, that is causing the system to block you for a period of
> time. For comparison, my PC only had about 4 sockets open to the server for
> the transfer. I would be curious to know if you are doing pipelining, or
> sending each request in a separate connection.
> >
> Whatever the sliksvn and tortoisesvn defaults are, I'd assume they are
> setup for typical usage.
>
> > If you can, the next time it happens, see if you are unable to visit
> Phabricator or resume the svn checkout process. It may not be a firewall
> issue, but without seeing some packet captures, I wouldn't be able to
> really guess. Ideally, try to capture a few packets, until failure. At the
> very least, you will need to snap 20b+20b=40b for IP+TCP headers, but I
> would suggest a tad higher, maybe 64b.
> >
> > Cheers,
> >
> > Dan
> >
> > [1] - https://download.blender.org/ftp/dan/svn-checkout/
> >
> >
> >
> > On Thu, Jun 16, 2022 at 2:43 PM Tom M via Bf-committers <
> bf-committers@blender.org> wrote:
> >>
> >> When checking out the libraries from SVN following the directions on
> >>
> >> https://wiki.blender.org/wiki/Building_Blender/Windows
> >>
> >> The svn checkout of the libraries fails with significant frequency,
> >>
> >> if you google you might stumble across using make svnfix
> >>
> >> https://www.mail-archive.com/bf-blender-cvs@blender.org/msg151996.html
> >>
> >> Even after using it, I'm still having to do Ctrl C, and then make
> >> svnfix repeatedly (it downloads some files, than randomly freezes,
> >> sometimes after 20-50 files, sometimes after 100's of files).
> >>
> >> Git doesn't seem to have any problems, though.
> >>
> >> So might we consider migrating the libs to git, rather than using SVN?
> >>
> >> LetterRip
> >> ___
> >> Bf-committers mailing list
> >> Bf-committers@blender.org
> >> List details, subscription details or unsubscribe:
> >> https://lists.blender.org/mailman/listinfo/bf-committers
> >
> >
> >
> > --
> > Cheers,
> > Danny
> >
> > --
> > Danny McGrath - danmcgrath...@gmail.com
> > GPG key: EDF6 AFF5 2086 F93A 1F59 36A5 44B6 26F3 6968 71CA
>


-- 
Cheers,
Danny

--
Danny McGrath - danmcgrath...@gmail.com
GPG key: EDF6 AFF5 2086 F93A 1F59 36A5 44B6 26F3 6968 71CA
___
Bf-committers mailing list
Bf-committers@blender.org
List details, subscription 

Re: [Bf-committers] move prebuilt libs from svn to git?

[Tom M via Bf-committers]

On Fri, Jun 17, 2022 at 5:20 AM Dan McGrath  wrote:
>
> Hi Tom,
>
> Long time no see :)
>
> I did some tests from home and found that, aside from slow speeds, I was able 
> to checkout at least 5 or 6 gig of the bf-blender repo without error or 
> having to retransmit.

Yeah, looks like I was mistaking really slow download of the larger
(.7 GB) libraries as freezing.  I was averaging about .3 MB/s.  I used
a different SVN client and saw that it was indeed downloading (the
sliksvn doesn't provide any feedback on files in progress, only once a
file completes) - It was taking about 40 minutes for each of the
larger files.  It did complete eventually after about 5 hours.

> I took the liberty of packet capturing the stream from my PC's POV, and 
> placed them in the ftp download area[1], if you care to take a look.
>
> The gist is that I was able to get around 5MB/sec for at least 5G of the 
> checkout, before I cancelled it.
>
Interesting, I usually get up to 3 MB/s download here normally from
other sources, but was typically getting .3 MB/s, with occasional
bursts of higher speed to a max of .8 MB/s when it wasn't downloading
the large files.

> Now, my IP is a bit of a special case, so I am wondering if you are just 
> hitting the server so hard with http requests in separate sockets, or 
> something extreme, that is causing the system to block you for a period of 
> time. For comparison, my PC only had about 4 sockets open to the server for 
> the transfer. I would be curious to know if you are doing pipelining, or 
> sending each request in a separate connection.
>
Whatever the sliksvn and tortoisesvn defaults are, I'd assume they are
setup for typical usage.

> If you can, the next time it happens, see if you are unable to visit 
> Phabricator or resume the svn checkout process. It may not be a firewall 
> issue, but without seeing some packet captures, I wouldn't be able to really 
> guess. Ideally, try to capture a few packets, until failure. At the very 
> least, you will need to snap 20b+20b=40b for IP+TCP headers, but I would 
> suggest a tad higher, maybe 64b.
>
> Cheers,
>
> Dan
>
> [1] - https://download.blender.org/ftp/dan/svn-checkout/
>
>
>
> On Thu, Jun 16, 2022 at 2:43 PM Tom M via Bf-committers 
>  wrote:
>>
>> When checking out the libraries from SVN following the directions on
>>
>> https://wiki.blender.org/wiki/Building_Blender/Windows
>>
>> The svn checkout of the libraries fails with significant frequency,
>>
>> if you google you might stumble across using make svnfix
>>
>> https://www.mail-archive.com/bf-blender-cvs@blender.org/msg151996.html
>>
>> Even after using it, I'm still having to do Ctrl C, and then make
>> svnfix repeatedly (it downloads some files, than randomly freezes,
>> sometimes after 20-50 files, sometimes after 100's of files).
>>
>> Git doesn't seem to have any problems, though.
>>
>> So might we consider migrating the libs to git, rather than using SVN?
>>
>> LetterRip
>> ___
>> Bf-committers mailing list
>> Bf-committers@blender.org
>> List details, subscription details or unsubscribe:
>> https://lists.blender.org/mailman/listinfo/bf-committers
>
>
>
> --
> Cheers,
> Danny
>
> --
> Danny McGrath - danmcgrath...@gmail.com
> GPG key: EDF6 AFF5 2086 F93A 1F59 36A5 44B6 26F3 6968 71CA
___
Bf-committers mailing list
Bf-committers@blender.org
List details, subscription details or unsubscribe:
https://lists.blender.org/mailman/listinfo/bf-committers

Re: [Bf-committers] move prebuilt libs from svn to git?

[Dan McGrath via Bf-committers]

Hi Tom,

Long time no see :)

I did some tests from home and found that, aside from slow speeds, I was
able to checkout at least 5 or 6 gig of the bf-blender repo without error
or having to retransmit.

I took the liberty of packet capturing the stream from my PC's POV, and
placed them in the ftp download area[1], if you care to take a look.

The gist is that I was able to get around 5MB/sec for at least 5G of the
checkout, before I cancelled it.

Now, my IP is a bit of a special case, so I am wondering if you are just
hitting the server so hard with http requests in separate sockets, or
something extreme, that is causing the system to block you for a period of
time. For comparison, my PC only had about 4 sockets open to the server for
the transfer. I would be curious to know if you are doing pipelining, or
sending each request in a separate connection.

If you can, the next time it happens, see if you are unable to visit
Phabricator or resume the svn checkout process. It may not be a firewall
issue, but without seeing some packet captures, I wouldn't be able to
really guess. Ideally, try to capture a few packets, until failure. At the
very least, you will need to snap 20b+20b=40b for IP+TCP headers, but I
would suggest a tad higher, maybe 64b.

Cheers,

Dan

[1] - https://download.blender.org/ftp/dan/svn-checkout/



On Thu, Jun 16, 2022 at 2:43 PM Tom M via Bf-committers <
bf-committers@blender.org> wrote:

> When checking out the libraries from SVN following the directions on
>
> https://wiki.blender.org/wiki/Building_Blender/Windows
>
> The svn checkout of the libraries fails with significant frequency,
>
> if you google you might stumble across using make svnfix
>
> https://www.mail-archive.com/bf-blender-cvs@blender.org/msg151996.html
>
> Even after using it, I'm still having to do Ctrl C, and then make
> svnfix repeatedly (it downloads some files, than randomly freezes,
> sometimes after 20-50 files, sometimes after 100's of files).
>
> Git doesn't seem to have any problems, though.
>
> So might we consider migrating the libs to git, rather than using SVN?
>
> LetterRip
> ___
> Bf-committers mailing list
> Bf-committers@blender.org
> List details, subscription details or unsubscribe:
> https://lists.blender.org/mailman/listinfo/bf-committers
>


-- 
Cheers,
Danny

--
Danny McGrath - danmcgrath...@gmail.com
GPG key: EDF6 AFF5 2086 F93A 1F59 36A5 44B6 26F3 6968 71CA
___
Bf-committers mailing list
Bf-committers@blender.org
List details, subscription details or unsubscribe:
https://lists.blender.org/mailman/listinfo/bf-committers

Re: [Bf-committers] move prebuilt libs from svn to git?

[Ankit via Bf-committers]

Tom:
1. Unlike SVN checkout, git checkout cannot be resumed. It starts
  over if interrupted.
2. .git folder's size will blow up with every release. Git is not
  so good for binary files.

Chad:

https://developer.blender.org/T73511
https://developer.blender.org/T73394

Cheers
Ankit


On 17/06/22 00:45, Chad Fraleigh via Bf-committers wrote:

I've been kind of wondering if they would eventually be migrated too. I only 
update my local libs occasionally, so it has a tenancy to fail if doing a large 
update spanning many revisions. Plus the benefit of not having to use two 
different tools.

Now with Git LFS being more widespread, it should mitigate the issue with all 
the large binary libraries.

On 6/16/2022 11:43 AM, Tom M via Bf-committers wrote:

When checking out the libraries from SVN following the directions on

https://wiki.blender.org/wiki/Building_Blender/Windows

The svn checkout of the libraries fails with significant frequency,

if you google you might stumble across using make svnfix

https://www.mail-archive.com/bf-blender-cvs@blender.org/msg151996.html

Even after using it, I'm still having to do Ctrl C, and then make
svnfix repeatedly (it downloads some files, than randomly freezes,
sometimes after 20-50 files, sometimes after 100's of files).

Git doesn't seem to have any problems, though.

So might we consider migrating the libs to git, rather than using SVN?

LetterRip
___
Bf-committers mailing list
Bf-committers@blender.org
List details, subscription details or unsubscribe:
https://lists.blender.org/mailman/listinfo/bf-committers

___
Bf-committers mailing list
Bf-committers@blender.org
List details, subscription details or unsubscribe:
https://lists.blender.org/mailman/listinfo/bf-committers

___
Bf-committers mailing list
Bf-committers@blender.org
List details, subscription details or unsubscribe:
https://lists.blender.org/mailman/listinfo/bf-committers

Re: [Bf-committers] move prebuilt libs from svn to git?

[Chad Fraleigh via Bf-committers]

I've been kind of wondering if they would eventually be migrated too. I only 
update my local libs occasionally, so it has a tenancy to fail if doing a large 
update spanning many revisions. Plus the benefit of not having to use two 
different tools.

Now with Git LFS being more widespread, it should mitigate the issue with all 
the large binary libraries.

On 6/16/2022 11:43 AM, Tom M via Bf-committers wrote:
> When checking out the libraries from SVN following the directions on
> 
> https://wiki.blender.org/wiki/Building_Blender/Windows
> 
> The svn checkout of the libraries fails with significant frequency,
> 
> if you google you might stumble across using make svnfix
> 
> https://www.mail-archive.com/bf-blender-cvs@blender.org/msg151996.html
> 
> Even after using it, I'm still having to do Ctrl C, and then make
> svnfix repeatedly (it downloads some files, than randomly freezes,
> sometimes after 20-50 files, sometimes after 100's of files).
> 
> Git doesn't seem to have any problems, though.
> 
> So might we consider migrating the libs to git, rather than using SVN?
> 
> LetterRip
> ___
> Bf-committers mailing list
> Bf-committers@blender.org
> List details, subscription details or unsubscribe:
> https://lists.blender.org/mailman/listinfo/bf-committers
___
Bf-committers mailing list
Bf-committers@blender.org
List details, subscription details or unsubscribe:
https://lists.blender.org/mailman/listinfo/bf-committers

[Bf-committers] move prebuilt libs from svn to git?

[Tom M via Bf-committers]

When checking out the libraries from SVN following the directions on

https://wiki.blender.org/wiki/Building_Blender/Windows

The svn checkout of the libraries fails with significant frequency,

if you google you might stumble across using make svnfix

https://www.mail-archive.com/bf-blender-cvs@blender.org/msg151996.html

Even after using it, I'm still having to do Ctrl C, and then make
svnfix repeatedly (it downloads some files, than randomly freezes,
sometimes after 20-50 files, sometimes after 100's of files).

Git doesn't seem to have any problems, though.

So might we consider migrating the libs to git, rather than using SVN?

LetterRip
___
Bf-committers mailing list
Bf-committers@blender.org
List details, subscription details or unsubscribe:
https://lists.blender.org/mailman/listinfo/bf-committers