Re: The vim port needs a refresh
On 25 May 2013 15:24, Chris Rees cr...@freebsd.org wrote: On 25 May 2013 11:54, Niclas Zeising zeising+free...@daemonic.se wrote: On 05/25/13 10:50, Chris Rees wrote: Alternatively, perhaps we need an editors/vim-options port Just for the record, editors/vim was (and shells/bash) was converted to optionsNG not too long ago. Ah, that's at least some good news. I notice that it was on yet another maintainer timeout, so that criticism stands. It appears that David is no longer interested. As I see it is not regular maintainer timeout as we hoped. May be really David should be removed from maintainer list. -- Eir Nym ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: The vim port needs a refresh
On 06/07/2013 12:56 PM, David O'Brien wrote: On Sat, May 25, 2013 at 09:50:50AM +0100, Chris Rees wrote: For years, people have been begging him to get over his fear of OPTIONS, and he sits in the way of progress against almost everyone's wishes. It's funny -- it's not just my fear of options -- every FreeBSD using co-worker I talk to frequently about OPTIONS do not like them either. Um... What?! If you don't like options, use packages. The ports system is specifically designed to allow the user to *CHOOSE THEIR BUILD OPTIONS*. How is this so difficult to understand? -Janketh Jay ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: The vim port needs a refresh
On 7 June 2013 19:56, David O'Brien obr...@freebsd.org wrote: On Sat, May 25, 2013 at 09:50:50AM +0100, Chris Rees wrote: For years, people have been begging him to get over his fear of OPTIONS, and he sits in the way of progress against almost everyone's wishes. It's funny -- it's not just my fear of options -- every FreeBSD using co-worker I talk to frequently about OPTIONS do not like them either. What's not funny is your dreadful attitude at replying to email. The number of timeouts on the vim and bash ports are shameful, and we're stuck with two bash ports because you're too lazy to keep the main one up to date. I really have no idea why you're still allowed to plant your flag in the ports tree and stand in the way of progress. Please consider dropping the ports, so people who are happy to keep them up to date and don't ignore bug reports can work on them. Chris ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: The vim port needs a refresh
On Sat, May 25, 2013 at 09:50:50AM +0100, Chris Rees wrote: For years, people have been begging him to get over his fear of OPTIONS, and he sits in the way of progress against almost everyone's wishes. It's funny -- it's not just my fear of options -- every FreeBSD using co-worker I talk to frequently about OPTIONS do not like them either. -- -- David (obr...@freebsd.org) ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: The vim port needs a refresh
On Fri, May 24, 2013 at 05:23:18PM -0400, Kenta Suzumoto wrote: - It fetches almost 700 patches from what seems like a dial-up connection in AUSTRALIA. You might as well be downloading a 1080p movie from a rock in the north pole, because that's about how fast it is. This can be very easily avoided by putting all the patches into a single tarball Please take this up with the Vim folks. I agree the time it takes to fetch the number of patches are very anonying. I've perodically asked for an update of the distfiles to include the patches to date. Please join in asking for this of the Vim developers. But I believe vim.org is a very reliable offical distribution source and I do not want to get in the middle of their distribution methods. P.S. we're now at 7.3.1011 - the port could use a normal update as well. /minor complaint I agree. Now that we have have all the releases we've had in flight out the door, new packages sets built; and finally I can use pkgNG I am doing an update. -- -- David (obr...@freebsd.org) ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: The vim port needs a refresh
On Tue, May 28, 2013 at 1:05 AM, RW rwmailli...@googlemail.com wrote: On Mon, 27 May 2013 22:33:53 +0200 John Marino wrote: On 5/27/2013 22:09, RW wrote: On Mon, 27 May 2013 20:38:11 +0200 John Marino wrote: No, that's something you just made up. It is however vague and anecdotal. We have only one data point that we know is from this year and not self-inflicted, even if the others are, for all we know it could still be fast most of the time. Some monitoring would be useful. However you slice it, a distinfo file with 1000+ entries is completely absurd. 95% of the blame goes to Vim developers. However, it is within the realm of feasibility to pre-package patches in batches of 100 (or conversely 1 tarball of patches rolled for every time patch count hits multiple of 100). In other words downloading every patch twice. As a side note: Patches 100-199 (as an example) are 600KB, while a tar.xz of them is 115KB. Data-wise, it's closer to downloading each patch 1.2 times. -- Daniel let's poke the fresh paint on this bikeshed Nebdal ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: The vim port needs a refresh
On Tue, May 28, 2013 at 4:14 PM, Chris Rees utis...@gmail.com wrote:
On 28 May 2013 06:08, Jeremy Messenger mezz.free...@gmail.com wrote:
On Sat, May 25, 2013 at 3:50 AM, Chris Rees cr...@freebsd.org wrote:
On 24 May 2013 22:23, Kenta Suzumoto ken...@hush.com wrote:
Hello all. The editors/vim port is currently a mess and needs some
changes.
- It fetches almost 700 patches from what seems like a dial-up
connection in AUSTRALIA.
You might as well be downloading a 1080p movie from a rock in the north
pole, because that's about how fast it is.
This can be very easily avoided by putting all the patches into a
single tarball and hosting it anywhere decent. I've
seen someone in ##freebsd on freenode handing out a tarball with all
the patches many times, and everyone asks
why isn't this the default? why is some random guy giving me
distfiles? etc. Seems like a no-brainer.
- By default, it builds lots of gui stuff that certainly almost no one
wants
It almost seems like the vim-lite port should be renamed vim and the
vim port should be renamed gvim. I had to
google to come up with this solution, because I can't even disable that
stuff in make config (another problem!)
.if ${.CURDIR}==/usr/ports/editors/vim
WITH_VIM_OPTIONS=yes
WITHOUT_X11=yes
.endif
People shouldn't have to find this hack to be able to install vim
normally (and no, telling them to use vim-lite isn't normal).
I'm surprised that none of these changes have been made yet. I've heard
it's because the maintainer won't listen to reason
but I have no way to know if that's the case or not. I also heard bapt@
had an optionsNG patch that he wouldn't
integrate into the port for some reason. Please, let's get this stuff
fixed once and for all. None of it requires a large amount
of work on anyone's part.
I'm very sad to talk of a fellow developer like this, but I'm afraid
the maintainer of vim is a contrarian who thinks he knows better than
everyone else on the matter.
For years, people have been begging him to get over his fear of
OPTIONS, and he sits in the way of progress against almost everyone's
wishes.
FYI, the OPTIONS is not required to have. I agree with him pretty much
everything about the OPTIONS. I have refused to add OPTIONS in any of
my ports before I gave up a lot of them long time ago. All of his
thought of OPTIONS are very valid. The OPTIONS still has bugs.
BTW: I always have BATCH=yes in my make.conf, because I hate OPTIONS a
lot.
Putting BATCH=yes in your environment is entirely up to you, but forcing
every user of the ports tree to learn a new way of dealing with certain
ports because They're mine and they're special is absolutely wrong.
Actually, it's not wrong when OPTIONS has bugs.
If you don't like OPTIONS, fix them,
I did by BATCH=yes. ;-)
but please don't labour under the
misapprehension that users are happy to have an inconsistent ports tree and
unpredictable ports tree on the whim of a few maverick developers.
Fix the OPTIONS first and I will accept it in my ports. Pretty simple.
Since I don't like OPTIONS, so I am not required to fix it. If you do
really want OPTIONS to be added in my port then please fix it.
Although, I have lost in track of which bugs have been fixed in
OPTIONS. I know one important bug is:
http://lists.freebsd.org/pipermail/freebsd-ports/2013-April/083035.html
As same as with the LICENSE. I will not add in my ports until he
writes document of it. But I will accept the patch of it though. Well,
again, I might be out of date but I seem still can't find it in the
porter handbook as today.
Chris
--
mezz.free...@gmail.com - m...@freebsd.org
FreeBSD GNOME Team
http://www.FreeBSD.org/gnome/ - gn...@freebsd.org
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: The vim port needs a refresh
On 5/29/2013 21:28, Jeremy Messenger wrote: Fix the OPTIONS first and I will accept it in my ports. Pretty simple. Since I don't like OPTIONS, so I am not required to fix it. If you do really want OPTIONS to be added in my port then please fix it. Although, I have lost in track of which bugs have been fixed in OPTIONS. I know one important bug is: http://lists.freebsd.org/pipermail/freebsd-ports/2013-April/083035.html Do any of your ports defer setting of PKGNAMEPREFIX? In other words, is your stance on principle or does this bug actually affect you? As same as with the LICENSE. I will not add in my ports until he writes document of it. But I will accept the patch of it though. Well, again, I might be out of date but I seem still can't find it in the porter handbook as today. I thought LICENSE was totally optional anyway John ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: The vim port needs a refresh
On 5/28/2013 02:44, Martin Wilke wrote: On the first note, complain about the patches to the upstream, not to us. This patches problem has been around since forever and so long the upstream is not changing anything about it, nor do we. Hi Martin, This statement is hand-waives the entire discussion, which took as a *given* that upstream is the problem who crazy policies will not change. I already said that 95% of the blame (probably more) should get allocated there. The whole discussion started because upstream will clearly not change. About rolling your own distfile, I completely disagree because we do not know what the maintaner has changed, and seeing it from the security view, I prefer to get all my patches from the original mirrors. 1. Yes, some amount of trust is necessary but hopefully we don't have maintainers that aren't trusted. 2. A trivial script can verify the roll-up contains only official patches, which could be executed by the committer prior to committing changes to distfile. That's a pretty easy security issue to address. I get your point but that issue could be solved. John ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: The vim port needs a refresh
On Fri, 24 May 2013 16:23:18 -0500, Kenta Suzumoto ken...@hush.com wrote: - It fetches almost 700 patches from what seems like a dial-up connection in AUSTRALIA. Australia's deploying fiber, so joke's on you! But honestly this is horrible. I'm sitting at my desk at a well-peered ISP with plenty of bandwidth and low, low latency and these patches are taking forever. Someone should just add a pre-fetch routine that downloads the first 1000 patches in a tarball, puts them in distfiles, and they well all be verified and the remaining few be fetched normally. Someone should teach upstream a serious lesson about versioning. Maybe someone just needs to fork vim and tag releases on github so we can actually have a sane upstream. Good grief! ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: The vim port needs a refresh
On 5/28/2013 14:09, Mark Felder wrote: On Fri, 24 May 2013 16:23:18 -0500, Kenta Suzumoto ken...@hush.com wrote: - It fetches almost 700 patches from what seems like a dial-up connection in AUSTRALIA. Australia's deploying fiber, so joke's on you! But honestly this is horrible. I'm sitting at my desk at a well-peered ISP with plenty of bandwidth and low, low latency and these patches are taking forever. Someone should just add a pre-fetch routine that downloads the first 1000 patches in a tarball, puts them in distfiles, and they well all be verified and the remaining few be fetched normally. Someone should teach upstream a serious lesson about versioning. Maybe someone just needs to fork vim and tag releases on github so we can actually have a sane upstream. Good grief! Well Mark, haven't you realized yet that there's actually no problem and this is almost completely about (your) laziness[1]? All patches only take 74 seconds to download[2] so there is no sympathy for your obviously single data point anecdote, you're clearly doing something wrong. You need to stop complaining and start think about folks with slow connections[3] who also rebuild Vim frequently. Your prefetch/fork idea is obviously unworkable because the security issues can't be solved.[4] [1]http://lists.freebsd.org/pipermail/freebsd-ports/2013-May/083880.html [2]http://lists.freebsd.org/pipermail/freebsd-ports/2013-May/083849.html [3]http://lists.freebsd.org/pipermail/freebsd-ports/2013-May/083844.html [4]http://lists.freebsd.org/pipermail/freebsd-ports/2013-May/083882.html Regards, John P.S. Hopefully it's obvious this is tongue in cheek. ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: The vim port needs a refresh
On Tue, 28 May 2013 15:14:52 +0200 John Marino wrote: All patches only take 74 seconds to download[2] so there is no sympathy for your obviously single data point anecdote, Well at the point you provided one data-point there was only one data point. And it was like pulling teeth to get you to eliminate the alternative explanations. Was it really too much to ask that you provided some actual evidence. you're clearly doing something wrong. You need to stop complaining and start think about folks with slow connections[3] who also rebuild Vim frequently. Don't make things up. I never said anything about frequent rebuilds, the patches all get redownloaded on the next rebuild. ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: The vim port needs a refresh
On Tue, 28 May 2013 15:16:00 +0100 RW rwmailli...@googlemail.com wrote: On Tue, 28 May 2013 15:14:52 +0200 John Marino wrote: All patches only take 74 seconds to download[2] so there is no sympathy for your obviously single data point anecdote, Well at the point you provided one data-point there was only one data point. And it was like pulling teeth to get you to eliminate the alternative explanations. Was it really too much to ask that you provided some actual evidence. you're clearly doing something wrong. You need to stop complaining and start think about folks with slow connections[3] who also rebuild Vim frequently. Don't make things up. I never said anything about frequent rebuilds, the patches all get redownloaded on the next rebuild. The real issue is not the number of patches, but the fact that every single patch is downloaded by invoking the fetch(1) command, creating lots of overhead not limited to the fetch command itself. The ports system wasn't designed for such an amount of distfiles in a single port I guess. I just timed fetching the patches through ports vs. fetching over HTTP/1.1 using ftp/curl vs calling fetch directly. The VIM tarball was already downloaded, so this is really just the patches (plus downloading 6mb is barely noticeable on a fast line). It's a slow machine on a fast line. Fetch: [user@server /usr/ports/editors/vim]$ time sudo make fetch real4m57.327s user0m17.010s sys 0m39.588s Curl: [user@server /tmp]$ longcurlcommandline real0m15.291s user0m0.026s sys 0m0.272s Fetch on the command line (after initial make fetch, so this is only measuring transmission of the files): cd /usr/ports/editors/distfiles time for name in 7.3.*; do fetch http://artfiles.org/vim.org/patches/7.3/$name done real1m25.329s user0m0.660s sys 0m3.174s So just the fact we're invoking fetch for every file costs us about one minute - I assume the time lost is much bigger on a slow line with long latency. The remaining 3.5 minutes are spent somewhere in the ports infrastructure and clearly depend on the performance of the machine used. For comparison I timed make fetch on a reasonably fast server (good IO, fast datacenter connection), make fetch still took about 120 seconds(!). So the bottomline is: - Using HTTP/1.1 and keepalive could safe a lot of time - The ports infrastructure creates a lot of overhead per patch file Maybe there's something we can do to improve the situation. Cheers, Michael PS: I don't use vim myself and have no stake in this discussion whatsoever. -- Michael Gmelin ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: The vim port needs a refresh
On Tue, May 28, 2013 at 07:51:37PM +0200, Michael Gmelin wrote:
On Tue, 28 May 2013 15:16:00 +0100
RW rwmailli...@googlemail.com wrote:
On Tue, 28 May 2013 15:14:52 +0200
John Marino wrote:
All
patches only take 74 seconds to download[2] so there is no sympathy
for your obviously single data point anecdote,
Well at the point you provided one data-point there was only one data
point. And it was like pulling teeth to get you to eliminate the
alternative explanations. Was it really too much to ask that you
provided some actual evidence.
you're clearly doing
something wrong. You need to stop complaining and start think about
folks with slow connections[3] who also rebuild Vim frequently.
Don't make things up. I never said anything about frequent rebuilds,
the patches all get redownloaded on the next rebuild.
The real issue is not the number of patches, but the fact that every
single patch is downloaded by invoking the fetch(1) command, creating
lots of overhead not limited to the fetch command itself. The ports
system wasn't designed for such an amount of distfiles in a single
port I guess.
I just timed fetching the patches through ports vs. fetching over
HTTP/1.1 using ftp/curl vs calling fetch directly. The VIM tarball was
already downloaded, so this is really just the patches (plus
downloading 6mb is barely noticeable on a fast line). It's a slow
machine on a fast line.
Fetch:
[user@server /usr/ports/editors/vim]$ time sudo make fetch
real4m57.327s
user0m17.010s
sys 0m39.588s
Curl:
[user@server /tmp]$ longcurlcommandline
real0m15.291s
user0m0.026s
sys 0m0.272s
Fetch on the command line (after initial make fetch, so this is only
measuring transmission of the files):
cd /usr/ports/editors/distfiles
time for name in 7.3.*; do
fetch http://artfiles.org/vim.org/patches/7.3/$name
done
real1m25.329s
user0m0.660s
sys 0m3.174s
So just the fact we're invoking fetch for every file costs us about one
minute - I assume the time lost is much bigger on a slow line with
long latency. The remaining 3.5 minutes are spent somewhere in the
ports infrastructure and clearly depend on the performance of the
machine used. For comparison I timed make fetch on a reasonably fast
server (good IO, fast datacenter connection), make fetch still took
about 120 seconds(!).
So the bottomline is:
- Using HTTP/1.1 and keepalive could safe a lot of time
- The ports infrastructure creates a lot of overhead per patch file
Maybe there's something we can do to improve the situation.
Cheers,
Michael
PS: I don't use vim myself and have no stake in this discussion
whatsoever.
Someone in this thread proposed to change the port to use phttpget, so I
gave it a try using a German mirror nearby with 6 Mbit/s downlink:
$ time /usr/libexec/phttpget ftp.vim.ossmirror.de $(eval echo
/pub/vim/patches/7.3/{$(make -C /usr/ports/editors/vim -VPATCHFILES | sed 's/\
/,/g')})
http://ftp.vim.ossmirror.de//pub/vim/patches/7.3/7.3.001: 200 OK
http://ftp.vim.ossmirror.de//pub/vim/patches/7.3/7.3.002: 200 OK
http://ftp.vim.ossmirror.de//pub/vim/patches/7.3/7.3.003: 200 OK
http://ftp.vim.ossmirror.de//pub/vim/patches/7.3/7.3.004: 200 OK
http://ftp.vim.ossmirror.de//pub/vim/patches/7.3/7.3.005: 200 OK
[...]
http://ftp.vim.ossmirror.de//pub/vim/patches/7.3/7.3.974: 200 OK
http://ftp.vim.ossmirror.de//pub/vim/patches/7.3/7.3.984: 200 OK
http://ftp.vim.ossmirror.de//pub/vim/patches/7.3/7.3.985: 200 OK
http://ftp.vim.ossmirror.de//pub/vim/patches/7.3/7.3.1000: 200 OK
real0m12.509s
user0m0.154s
sys 0m0.089s
That's really nice!
Compare this to the current version using fetch(1):
time make PATCH_SITES=http://ftp.vim.ossmirror.de/pub/vim/patches/7.3/ fetch
=== Found saved configuration for vim-7.3.669_1
=== vim-7.3.1014 depends on file: /usr/local/sbin/pkg - found
= 7.3.002 doesn't seem to exist in /usr/ports/distfiles/vim.
= Attempting to fetch http://ftp.vim.ossmirror.de/pub/vim/patches/7.3/7.3.002
7.3.002 100% of 1610 B 16 MBps 00m00s
= 7.3.003 doesn't seem to exist in /usr/ports/distfiles/vim.
= Attempting to fetch http://ftp.vim.ossmirror.de/pub/vim/patches/7.3/7.3.003
7.3.003 100% of 1299 B 1281 kBps 00m00s
= 7.3.004 doesn't seem to exist in /usr/ports/distfiles/vim.
[...]
= 7.3.984 doesn't seem to exist in /usr/ports/distfiles/vim.
= Attempting to fetch http://ftp.vim.ossmirror.de/pub/vim/patches/7.3/7.3.984
7.3.984 100% of 1706 B 2852 kBps 00m00s
= 7.3.985 doesn't seem to exist in /usr/ports/distfiles/vim.
= Attempting to fetch http://ftp.vim.ossmirror.de/pub/vim/patches/7.3/7.3.985
7.3.985 100% of 1691 B 14 MBps 00m00s
= 7.3.1000 doesn't seem to exist in /usr/ports/distfiles/vim.
= Attempting to fetch
Re: The vim port needs a refresh
Am 28.05.2013 19:51, schrieb Michael Gmelin: On Tue, 28 May 2013 15:16:00 +0100 RW rwmailli...@googlemail.com wrote: On Tue, 28 May 2013 15:14:52 +0200 John Marino wrote: All patches only take 74 seconds to download[2] so there is no sympathy for your obviously single data point anecdote, Well at the point you provided one data-point there was only one data point. And it was like pulling teeth to get you to eliminate the alternative explanations. Was it really too much to ask that you provided some actual evidence. you're clearly doing something wrong. You need to stop complaining and start think about folks with slow connections[3] who also rebuild Vim frequently. Don't make things up. I never said anything about frequent rebuilds, the patches all get redownloaded on the next rebuild. The real issue is not the number of patches, but the fact that every single patch is downloaded by invoking the fetch(1) command, creating lots of overhead not limited to the fetch command itself. The ports system wasn't designed for such an amount of distfiles in a single port I guess. I just timed fetching the patches through ports vs. fetching over HTTP/1.1 using ftp/curl vs calling fetch directly. The VIM tarball was already downloaded, so this is really just the patches (plus downloading 6mb is barely noticeable on a fast line). It's a slow machine on a fast line. Fetch: [user@server /usr/ports/editors/vim]$ time sudo make fetch real4m57.327s user0m17.010s sys 0m39.588s Curl: [user@server /tmp]$ longcurlcommandline real0m15.291s user0m0.026s sys 0m0.272s Fetch on the command line (after initial make fetch, so this is only measuring transmission of the files): cd /usr/ports/editors/distfiles time for name in 7.3.*; do fetch http://artfiles.org/vim.org/patches/7.3/$name done real1m25.329s user0m0.660s sys 0m3.174s So just the fact we're invoking fetch for every file costs us about one minute - I assume the time lost is much bigger on a slow line with long latency. The remaining 3.5 minutes are spent somewhere in the ports infrastructure and clearly depend on the performance of the machine used. For comparison I timed make fetch on a reasonably fast server (good IO, fast datacenter connection), make fetch still took about 120 seconds(!). So the bottomline is: - Using HTTP/1.1 and keepalive could safe a lot of time - The ports infrastructure creates a lot of overhead per patch file Maybe there's something we can do to improve the situation. Probably. On the fetching side, we have: - /usr/src/usr.sbin/portsnap/phttpget/phttpget.c /usr/libexec/phttpget - the possibility to specify multiple URLs on fetch(1)'s command line - the xargs command to assemble command lines with a decent amount of URL arguments Given that connection setup for FTP costs considerable amounts of time especially with FTP. On the URL list generation, we have excessive external command in shell scripts; try make -nC /usr/ports/editors/vim fetch-url-list-int to see the commands. I suppose fewer external commands and more make-internal processing could help quite a bit. ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: The vim port needs a refresh
On 28 May 2013 06:08, Jeremy Messenger mezz.free...@gmail.com wrote:
On Sat, May 25, 2013 at 3:50 AM, Chris Rees cr...@freebsd.org wrote:
On 24 May 2013 22:23, Kenta Suzumoto ken...@hush.com wrote:
Hello all. The editors/vim port is currently a mess and needs some
changes.
- It fetches almost 700 patches from what seems like a dial-up
connection in AUSTRALIA.
You might as well be downloading a 1080p movie from a rock in the
north pole, because that's about how fast it is.
This can be very easily avoided by putting all the patches into a
single tarball and hosting it anywhere decent. I've
seen someone in ##freebsd on freenode handing out a tarball with all
the patches many times, and everyone asks
why isn't this the default? why is some random guy giving me
distfiles? etc. Seems like a no-brainer.
- By default, it builds lots of gui stuff that certainly almost no one
wants
It almost seems like the vim-lite port should be renamed vim and the
vim port should be renamed gvim. I had to
google to come up with this solution, because I can't even disable
that stuff in make config (another problem!)
.if ${.CURDIR}==/usr/ports/editors/vim
WITH_VIM_OPTIONS=yes
WITHOUT_X11=yes
.endif
People shouldn't have to find this hack to be able to install vim
normally (and no, telling them to use vim-lite isn't normal).
I'm surprised that none of these changes have been made yet. I've
heard it's because the maintainer won't listen to reason
but I have no way to know if that's the case or not. I also heard bapt@had
an optionsNG patch that he wouldn't
integrate into the port for some reason. Please, let's get this stuff
fixed once and for all. None of it requires a large amount
of work on anyone's part.
I'm very sad to talk of a fellow developer like this, but I'm afraid
the maintainer of vim is a contrarian who thinks he knows better than
everyone else on the matter.
For years, people have been begging him to get over his fear of
OPTIONS, and he sits in the way of progress against almost everyone's
wishes.
FYI, the OPTIONS is not required to have. I agree with him pretty much
everything about the OPTIONS. I have refused to add OPTIONS in any of
my ports before I gave up a lot of them long time ago. All of his
thought of OPTIONS are very valid. The OPTIONS still has bugs.
BTW: I always have BATCH=yes in my make.conf, because I hate OPTIONS a
lot.
Putting BATCH=yes in your environment is entirely up to you, but forcing
every user of the ports tree to learn a new way of dealing with certain
ports because They're mine and they're special is absolutely wrong.
If you don't like OPTIONS, fix them, but please don't labour under the
misapprehension that users are happy to have an inconsistent ports tree and
unpredictable ports tree on the whim of a few maverick developers.
Chris
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: The vim port needs a refresh
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Am 28.05.2013 22:24, schrieb Lars Engels:
Someone in this thread proposed to change the port to use phttpget, so I
gave it a try using a German mirror nearby with 6 Mbit/s downlink:
$ time /usr/libexec/phttpget ftp.vim.ossmirror.de $(eval echo
/pub/vim/patches/7.3/{$(make -C /usr/ports/editors/vim -VPATCHFILES | sed
's/\ /,/g')})
http://ftp.vim.ossmirror.de//pub/vim/patches/7.3/7.3.001: 200 OK
http://ftp.vim.ossmirror.de//pub/vim/patches/7.3/7.3.002: 200 OK
http://ftp.vim.ossmirror.de//pub/vim/patches/7.3/7.3.003: 200 OK
http://ftp.vim.ossmirror.de//pub/vim/patches/7.3/7.3.004: 200 OK
http://ftp.vim.ossmirror.de//pub/vim/patches/7.3/7.3.005: 200 OK
[...]
http://ftp.vim.ossmirror.de//pub/vim/patches/7.3/7.3.974: 200 OK
http://ftp.vim.ossmirror.de//pub/vim/patches/7.3/7.3.984: 200 OK
http://ftp.vim.ossmirror.de//pub/vim/patches/7.3/7.3.985: 200 OK
http://ftp.vim.ossmirror.de//pub/vim/patches/7.3/7.3.1000: 200 OK
real 0m12.509s
user 0m0.154s
sys 0m0.089s
That's really nice!
Compare this to the current version using fetch(1):
This incurs massive overhead from the generation of the fetch URLs, on
top of fetch being slower.
You would have to come up with something similar to your phttpget line,
only that it runs fetch(1) instead. Try:
$ make -C /usr/ports/editors/vim fetch FETCH_CMD=true
to see how slow URL generation is.
On one of my reasonably fast amd64 machines (dual-core 2.1 GHz AMD
BE-2350), residing on the T-Home domestic backbone near the Ruhr area in
Germany with a 3000/384 kBit/s ADSL that has roughly 50 ms ping round
trips, it generates approximately 1.7 fetch commands in 1 s, and see
below for fetch speed:
time make PATCH_SITES=http://ftp.vim.ossmirror.de/pub/vim/patches/7.3/ fetch
=== Found saved configuration for vim-7.3.669_1
=== vim-7.3.1014 depends on file: /usr/local/sbin/pkg - found
= 7.3.002 doesn't seem to exist in /usr/ports/distfiles/vim.
= Attempting to fetch http://ftp.vim.ossmirror.de/pub/vim/patches/7.3/7.3.002
7.3.002 100% of 1610 B 16 MBps 00m00s
= 7.3.003 doesn't seem to exist in /usr/ports/distfiles/vim.
= Attempting to fetch http://ftp.vim.ossmirror.de/pub/vim/patches/7.3/7.3.003
7.3.003 100% of 1299 B 1281 kBps 00m00s
= 7.3.004 doesn't seem to exist in /usr/ports/distfiles/vim.
[...]
= 7.3.984 doesn't seem to exist in /usr/ports/distfiles/vim.
= Attempting to fetch http://ftp.vim.ossmirror.de/pub/vim/patches/7.3/7.3.984
7.3.984 100% of 1706 B 2852 kBps 00m00s
= 7.3.985 doesn't seem to exist in /usr/ports/distfiles/vim.
= Attempting to fetch http://ftp.vim.ossmirror.de/pub/vim/patches/7.3/7.3.985
7.3.985 100% of 1691 B 14 MBps 00m00s
= 7.3.1000 doesn't seem to exist in /usr/ports/distfiles/vim.
= Attempting to fetch
http://ftp.vim.ossmirror.de/pub/vim/patches/7.3/7.3.1000
7.3.1000 100% of 1637 B 1715 kBps 00m00s
=== Fetching all distfiles required by vim-7.3.1014 for building
Total time : 3:48.55s
CPU utilisation (percentage): 54.5%
Now, using a similar approach as yours, I get, with fetch:
$ time fetch $(eval echo -O
http://ftp.vim.ossmirror.de/pub/vim/patches/7.3/{$(make -C
/usr/ports/editors/vim -VPATCHFILES | tr ' ' ',')})
...
real2m2.841s
user0m1.083s
sys 0m5.238s
Which takes some 2 min of wasted CPU time out of your calculation.
Not that it helps overly in terms of wallclock time.
Using curl with -O cuts this down to:
real1m5.195s
user0m0.668s
sys 0m1.479s
And wget -nv to:
real1m15.426s
user0m0.845s
sys 0m1.536s
Finally, so you can compare gains whilst taking line speed out of the
picture, phttpget:
real0m22.673s
user0m0.530s
sys 0m0.659s
The computer on its link should be able to fetch a tarball in c. 11 s
wallclock time, we have 3.7 MB of payload in patches and the link
manages roughly 355 kB/s.
Now, any takers for make make fetch more efficient jobs?
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with undefined - http://www.enigmail.net/
iEYEARECAAYFAlGlHjAACgkQvmGDOQUufZWaNgCgjERowRAqRo2Rgv8rQOZQiA7f
ahcAoMkjJxeFHGBe7hcZIjZb8rrLh+0/
=XoQd
-END PGP SIGNATURE-
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: The vim port needs a refresh
On Fri, 24 May 2013 17:23:18 -0400 Kenta Suzumoto wrote: - It fetches almost 700 patches from what seems like a dial-up connection in AUSTRALIA. You might as well be downloading a 1080p movie from a rock in the north pole, because that's about how fast it is. This can be very easily avoided by putting all the patches into a single tarball and hosting it anywhere decent. I've seen someone in ##freebsd on freenode handing out a tarball with all the patches many times, and everyone asks why isn't this the default? why is some random guy giving me distfiles? etc. Seems like a no-brainer. I prefer it the way it is; those patch files are cached in the distfiles directory, so only new patches need be downloaded. I can't say I've ever noticed it being slow. If you roll them up into one file the whole thing needs to be download every time a patch is added. If you combine a tarball with individual newer patch, it's no better than the current situation with caching. ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: The vim port needs a refresh
RW rwmailli...@googlemail.com writes: On Fri, 24 May 2013 17:23:18 -0400 Kenta Suzumoto wrote: - It fetches almost 700 patches from what seems like a dial-up connection in AUSTRALIA. You might as well be downloading a 1080p movie from a rock in the north pole, because that's about how fast it is. This can be very easily avoided by putting all the patches into a single tarball and hosting it anywhere decent. I've seen someone in ##freebsd on freenode handing out a tarball with all the patches many times, and everyone asks why isn't this the default? why is some random guy giving me distfiles? etc. Seems like a no-brainer. I prefer it the way it is; those patch files are cached in the distfiles directory, so only new patches need be downloaded. I can't say I've ever noticed it being slow. If you roll them up into one file the whole thing needs to be download every time a patch is added. If you combine a tarball with individual newer patch, it's no better than the current situation with caching. There's plenty of middle ground. Re-rolling the tarball every time a new patch is added would definitely be worse than the current situation, but rolling lots of long-standing patches into a much-smaller number of collective downloads would be an improvement for some people without hurting anyone else. -- Rick Astley was not harmed in the making of this communication. ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: The vim port needs a refresh
On Mon, 27 May 2013 09:36:20 -0400 Lowell Gilbert wrote: RW rwmailli...@googlemail.com writes: I prefer it the way it is; those patch files are cached in the distfiles directory, so only new patches need be downloaded. I can't say I've ever noticed it being slow. If you roll them up into one file the whole thing needs to be download every time a patch is added. If you combine a tarball with individual newer patch, it's no better than the current situation with caching. There's plenty of middle ground. Re-rolling the tarball every time a new patch is added would definitely be worse than the current situation, but rolling lots of long-standing patches into a much-smaller number of collective downloads would be an improvement for some people without hurting anyone else. It would hurt people with a slow connections who would end-up having to download most of the patches twice. I've a lot more sympathy with people in that situation than with someone who doesn't cache and then complains it's slow. It wouldn't matter if this were KDE4, but VIM is the kind of port that's likely to be present on a minimalist install. ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: The vim port needs a refresh
On 5/27/2013 16:34, RW wrote: It would hurt people with a slow connections who would end-up having to download most of the patches twice. I've a lot more sympathy with people in that situation than with someone who doesn't cache and then complains it's slow. Trust me. If you get the wrong mirror, it is horrifically slow. Like 4 patches per minute slow. You have no sympathy for somebody that has to download all 900+ patches from the beginning? The trick, of course, is to override MASTER_SITES in the environment, e.g. make MASTER_SITES= fetch and then force fetching from FreeBSD. Then it gets all the patches pretty quickly. But most users wouldn't figure that out (and I'm sure you don't want it broadcast either). The slow complaint is not trivial, it's very, very real. Saying you haven't seen it doesn't make it less real, you probably just didn't sit there and watch it from patch#1. John ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: The vim port needs a refresh
On Mon, 27 May 2013 17:19:40 +0200 John Marino wrote: On 5/27/2013 16:34, RW wrote: It would hurt people with a slow connections who would end-up having to download most of the patches twice. I've a lot more sympathy with people in that situation than with someone who doesn't cache and then complains it's slow. Trust me. If you get the wrong mirror, ... I'm not sure what you mean by that. Unless something has changed everyone tries the mirrors list in the same order. Have you set something in make.conf to alter that? In particular RANDOMIZE_MASTER_SITES is well known for poor speeds. Like 4 patches per minute slow. You have no sympathy for somebody that has to download all 900+ patches from the beginning? A little if it's the first time they've ever built vim on FreeBSD, and they have have genuine good reason for not being able to wait an extra minute. The trick, of course, is to override MASTER_SITES in the environment, e.g. make MASTER_SITES= fetch and then force fetching from FreeBSD. Then it gets all the patches pretty quickly. It looks to me like you've just proposed a more sensible solution: change PATCH_SITES or MASTER_SITE_VIM to exclude slow servers and/or bring the fast ones to the front of the list. The slow complaint is not trivial, it's very, very real. Saying you haven't seen it doesn't make it less real, you probably just didn't sit there and watch it from patch#1. No, it's because I've been using FreeBSD since before August 2010 when that patch was created. I just tried deleting all the patches and refetching and it took 74 seconds, it's scarcely a major problem. ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: The vim port needs a refresh
On 5/27/2013 18:36, RW wrote: Like 4 patches per minute slow. You have no sympathy for somebody that has to download all 900+ patches from the beginning? A little if it's the first time they've ever built vim on FreeBSD, and they have have genuine good reason for not being able to wait an extra minute. 900 patches /4 patches/min = 225 minutes = 3.75 hours hardly an extra minute The slow complaint is not trivial, it's very, very real. Saying you haven't seen it doesn't make it less real, you probably just didn't sit there and watch it from patch#1. No, it's because I've been using FreeBSD since before August 2010 when that patch was created. I've been using FreeBSD since version 4.10, but that doesn't imply that I've every downloaded vim patches before. I just tried deleting all the patches and refetching and it took 74 seconds, it's scarcely a major problem. Great. With the previous mirror I had it would have taken well over an hour back when the patch count was 700. That is a major problem for others, despite the fact that it's not a problem for you. It's obviously true since multiple users are seeing it. (the whole 1080 movie analogy, remember?) John ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: The vim port needs a refresh
On Mon, 27 May 2013 18:44:55 +0200 John Marino wrote: Great. With the previous mirror I had it would have taken well over an hour back when the patch count was 700. By default it should be the same mirror if you tested it this year. Slow and dead mirrors were removed at the beginning of January. And you still haven't said whether you have any make.conf settings that affect the order. It's obviously true since multiple users are seeing it. (the whole 1080 movie analogy, remember?) It not obviously true that you aren't all either making this problem with your make.conf settings or referring to a problem that existed last year. ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: The vim port needs a refresh
On 5/27/2013 19:36, RW wrote: On Mon, 27 May 2013 18:44:55 +0200 John Marino wrote: Great. With the previous mirror I had it would have taken well over an hour back when the patch count was 700. By default it should be the same mirror if you tested it this year. Slow and dead mirrors were removed at the beginning of January. And you still haven't said whether you have any make.conf settings that affect the order. I didn't change any settings. It may have been an FTP site rather than an HTTP site. I seem to recall some work to move HTTP over FTP fairly recently. It's obviously true since multiple users are seeing it. (the whole 1080 movie analogy, remember?) It not obviously true that you aren't all either making this problem with your make.conf settings or referring to a problem that existed last year. It was THIS year and it wasn't that long ago. January in the worst case. But now you're explicitly telling multiple users that they didn't in fact experience this? ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: The vim port needs a refresh
On Mon, 27 May 2013 20:38:11 +0200 John Marino wrote: It's obviously true since multiple users are seeing it. (the whole 1080 movie analogy, remember?) It not obviously true that you aren't all either making this problem with your make.conf settings or referring to a problem that existed last year. It was THIS year and it wasn't that long ago. January in the worst case. But now you're explicitly telling multiple users that they didn't in fact experience this? No, that's something you just made up. It is however vague and anecdotal. We have only one data point that we know is from this year and not self-inflicted, even if the others are, for all we know it could still be fast most of the time. Some monitoring would be useful. ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: The vim port needs a refresh
On 5/27/2013 22:09, RW wrote: On Mon, 27 May 2013 20:38:11 +0200 John Marino wrote: No, that's something you just made up. It is however vague and anecdotal. We have only one data point that we know is from this year and not self-inflicted, even if the others are, for all we know it could still be fast most of the time. Some monitoring would be useful. However you slice it, a distinfo file with 1000+ entries is completely absurd. 95% of the blame goes to Vim developers. However, it is within the realm of feasibility to pre-package patches in batches of 100 (or conversely 1 tarball of patches rolled for every time patch count hits multiple of 100). At the very, very least maybe only HTTP hosts are listed for VIM (I just checked bsd.sites.mk, the ftp sites are all at the end of the list now) It looks like some of this was addressed in January though: Revision 309899 - (view) (download) (annotate) - [select for diffs] Modified Thu Jan 3 17:38:30 2013 UTC (4 months, 3 weeks ago) by rm File length: 63730 byte(s) Diff to previous 309846 - sync list of vim mirrors with official list on http://www.vim.org/mirrors.php - remove dead vim sites - remove vim sites with missing files - remove slow vim sites ( 10 seconds to serve a file) after repeated complaints by blakkheim (I'm not sure what's that) PR: 174875 Submitted by: 4...@hushmail.com I may have still been on the old bsd.sites.mk with a site 10 seconds per file. (this is yet another data point) so maybe the situation is much better now. ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: The vim port needs a refresh
On Mon, 27 May 2013 22:33:53 +0200 John Marino wrote: On 5/27/2013 22:09, RW wrote: On Mon, 27 May 2013 20:38:11 +0200 John Marino wrote: No, that's something you just made up. It is however vague and anecdotal. We have only one data point that we know is from this year and not self-inflicted, even if the others are, for all we know it could still be fast most of the time. Some monitoring would be useful. However you slice it, a distinfo file with 1000+ entries is completely absurd. 95% of the blame goes to Vim developers. However, it is within the realm of feasibility to pre-package patches in batches of 100 (or conversely 1 tarball of patches rolled for every time patch count hits multiple of 100). In other words downloading every patch twice. At the very, very least maybe only HTTP hosts are listed for VIM (I just checked bsd.sites.mk, the ftp sites are all at the end of the list now) All 13 http links would have to fail before the ftp links are tried. It looks like some of this was addressed in January though: I already told you that. I may have still been on the old bsd.sites.mk with a site 10 seconds per file. (this is yet another data point) We already knew that it was slow before January, so that's irrelevant. ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: The vim port needs a refresh
On 5/28/2013 01:05, RW wrote: On Mon, 27 May 2013 22:33:53 +0200 John Marino wrote: On 5/27/2013 22:09, RW wrote: On Mon, 27 May 2013 20:38:11 +0200 John Marino wrote: No, that's something you just made up. It is however vague and anecdotal. We have only one data point that we know is from this year and not self-inflicted, even if the others are, for all we know it could still be fast most of the time. Some monitoring would be useful. However you slice it, a distinfo file with 1000+ entries is completely absurd. 95% of the blame goes to Vim developers. However, it is within the realm of feasibility to pre-package patches in batches of 100 (or conversely 1 tarball of patches rolled for every time patch count hits multiple of 100). In other words downloading every patch twice. No. That's not what those words mean. Please stop assuming that somebody builds Vim repeatedly and start assuming it's built for the very first time. Also, given these patches are a couple of kilobytes at most, a compressed tarball of 100 patches (or even 700 patches) is negligible. Even if somebody with a cache downloaded it twice, so what? It's not even noticeable. At the very, very least maybe only HTTP hosts are listed for VIM (I just checked bsd.sites.mk, the ftp sites are all at the end of the list now) All 13 http links would have to fail before the ftp links are tried. So what's the point of having them on the list? Isn't 13 mirrors enough? I may have still been on the old bsd.sites.mk with a site 10 seconds per file. (this is yet another data point) We already knew that it was slow before January, so that's irrelevant. It validated my story as more than anecdotal. ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: The vim port needs a refresh
On Tue, 28 May 2013 01:13:43 +0200 John Marino wrote: On 5/28/2013 01:05, RW wrote: On Mon, 27 May 2013 22:33:53 +0200 John Marino wrote: In other words downloading every patch twice. No. That's not what those words mean. Please stop assuming that somebody builds Vim repeatedly and start assuming it's built for the very first time. Why wouldn't I? Are you seriously suggesting that it's the norm to build a port once and then never build it again? Also, given these patches are a couple of kilobytes at most, a compressed tarball of 100 patches (or even 700 patches) is negligible. Even if somebody with a cache downloaded it twice, so what? It's not even noticeable. They add up to 3 MB which is noticeable to someone on dialup even when compressed. Ordinarily, it wouldn't matter, but as I said before VIM is something that could be part of a very minimal build - something that might be maintained even over very slow dial-up. At the very, very least maybe only HTTP hosts are listed for VIM (I just checked bsd.sites.mk, the ftp sites are all at the end of the list now) All 13 http links would have to fail before the ftp links are tried. So what's the point of having them on the list? Isn't 13 mirrors enough? Some people may find ftp faster or more reliable - it depends on your circumstances. I may have still been on the old bsd.sites.mk with a site 10 seconds per file. (this is yet another data point) We already knew that it was slow before January, so that's irrelevant. It validated my story as more than anecdotal. No it didn't because I already told you that there unreliable servers then. ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: The vim port needs a refresh
On 5/28/2013 01:48, RW wrote: On Tue, 28 May 2013 01:13:43 +0200 No. That's not what those words mean. Please stop assuming that somebody builds Vim repeatedly and start assuming it's built for the very first time. Why wouldn't I? Are you seriously suggesting that it's the norm to build a port once and then never build it again? 1. Yes, that can happen. I'm working on some servers with 1600 days uptime (should be 2300 days but the data center relocated them a few years ago) and most of the software on them is from 2007. 2. Every software built from source is built the first time on each server. 3. It is nice to cater to new users. 4. It's good practice to target the lowest common denominator They add up to 3 MB which is noticeable to someone on dialup even when compressed. Ordinarily, it wouldn't matter, but as I said before VIM is something that could be part of a very minimal build - something that might be maintained even over very slow dial-up. If you are going to use dialup as an example, then it's much, much worse to download them all individually. Unless you're building vim repeatedly and often, the opportunity for double-downloads isn't that high. If it's a real worry then the 100-patch rollups would be better than the full aggregates. Some people may find ftp faster or more reliable - it depends on your circumstances. That's not my experience but for the sake of argument I'll accept the point. It still seems like overkill though. It validated my story as more than anecdotal. No it didn't because I already told you that there unreliable servers then. That doesn't invalidate what I said. You can't assume everyone portsnaps daily. A commit in January might not trickle down for months. All you can say is, yes, that was the case but a PR was written against it and since closed, please try again with a current port tree. Plus I think you said it after I told the story. ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: The vim port needs a refresh
On the first note, complain about the patches to the upstream, not to us. This patches problem has been around since forever and so long the upstream is not changing anything about it, nor do we. About rolling your own distfile, I completely disagree because we do not know what the maintaner has changed, and seeing it from the security view, I prefer to get all my patches from the original mirrors. - Martin P.S. This is solely my personal view and does not reflect the official portmgr's stand. On May 28, 2013, at 8:02 AM, John Marino freebs...@marino.st wrote: On 5/28/2013 01:48, RW wrote: On Tue, 28 May 2013 01:13:43 +0200 No. That's not what those words mean. Please stop assuming that somebody builds Vim repeatedly and start assuming it's built for the very first time. Why wouldn't I? Are you seriously suggesting that it's the norm to build a port once and then never build it again? 1. Yes, that can happen. I'm working on some servers with 1600 days uptime (should be 2300 days but the data center relocated them a few years ago) and most of the software on them is from 2007. 2. Every software built from source is built the first time on each server. 3. It is nice to cater to new users. 4. It's good practice to target the lowest common denominator They add up to 3 MB which is noticeable to someone on dialup even when compressed. Ordinarily, it wouldn't matter, but as I said before VIM is something that could be part of a very minimal build - something that might be maintained even over very slow dial-up. If you are going to use dialup as an example, then it's much, much worse to download them all individually. Unless you're building vim repeatedly and often, the opportunity for double-downloads isn't that high. If it's a real worry then the 100-patch rollups would be better than the full aggregates. Some people may find ftp faster or more reliable - it depends on your circumstances. That's not my experience but for the sake of argument I'll accept the point. It still seems like overkill though. It validated my story as more than anecdotal. No it didn't because I already told you that there unreliable servers then. That doesn't invalidate what I said. You can't assume everyone portsnaps daily. A commit in January might not trickle down for months. All you can say is, yes, that was the case but a PR was written against it and since closed, please try again with a current port tree. Plus I think you said it after I told the story. ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org +-oOO--(_)--OOo-+ With best Regards, Martin Wilke (miwi_(at)_FreeBSD.org) Mess with the Best, Die like the Rest ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: The vim port needs a refresh
On Tue, 28 May 2013 02:02:00 +0200 John Marino wrote: On 5/28/2013 01:48, RW wrote: On Tue, 28 May 2013 01:13:43 +0200 No. That's not what those words mean. Please stop assuming that somebody builds Vim repeatedly and start assuming it's built for the very first time. Why wouldn't I? Are you seriously suggesting that it's the norm to 2. Every software built from source is built the first time on each server. That's not the same as first time, you do have access to other distfile directories. This is what I suspected from the start, that this is almost completely about laziness. ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: The vim port needs a refresh
On Sat, May 25, 2013 at 3:50 AM, Chris Rees cr...@freebsd.org wrote:
On 24 May 2013 22:23, Kenta Suzumoto ken...@hush.com wrote:
Hello all. The editors/vim port is currently a mess and needs some changes.
- It fetches almost 700 patches from what seems like a dial-up connection in
AUSTRALIA.
You might as well be downloading a 1080p movie from a rock in the north
pole, because that's about how fast it is.
This can be very easily avoided by putting all the patches into a single
tarball and hosting it anywhere decent. I've
seen someone in ##freebsd on freenode handing out a tarball with all the
patches many times, and everyone asks
why isn't this the default? why is some random guy giving me distfiles?
etc. Seems like a no-brainer.
- By default, it builds lots of gui stuff that certainly almost no one wants
It almost seems like the vim-lite port should be renamed vim and the vim
port should be renamed gvim. I had to
google to come up with this solution, because I can't even disable that
stuff in make config (another problem!)
.if ${.CURDIR}==/usr/ports/editors/vim
WITH_VIM_OPTIONS=yes
WITHOUT_X11=yes
.endif
People shouldn't have to find this hack to be able to install vim normally
(and no, telling them to use vim-lite isn't normal).
I'm surprised that none of these changes have been made yet. I've heard it's
because the maintainer won't listen to reason
but I have no way to know if that's the case or not. I also heard bapt@ had
an optionsNG patch that he wouldn't
integrate into the port for some reason. Please, let's get this stuff fixed
once and for all. None of it requires a large amount
of work on anyone's part.
I'm very sad to talk of a fellow developer like this, but I'm afraid
the maintainer of vim is a contrarian who thinks he knows better than
everyone else on the matter.
For years, people have been begging him to get over his fear of
OPTIONS, and he sits in the way of progress against almost everyone's
wishes.
FYI, the OPTIONS is not required to have. I agree with him pretty much
everything about the OPTIONS. I have refused to add OPTIONS in any of
my ports before I gave up a lot of them long time ago. All of his
thought of OPTIONS are very valid. The OPTIONS still has bugs.
BTW: I always have BATCH=yes in my make.conf, because I hate OPTIONS a lot.
He has also impeded progress on the bash port, resulting in the
ridiculous situation where we now have two bash ports, where one will
do. For historical reasons, people seem reluctant to confront him
about this, and he ignores all attempts to reason about it.
It's far beyond time to remove David O'Brien from MAINTAINER lines--
he doesn't do the job properly anyway; several PRs he's timed out on
for his ports:
http://www.freebsd.org/cgi/query-pr.cgi?pr=ports/177597
http://www.freebsd.org/cgi/query-pr.cgi?pr=ports/174965
http://www.freebsd.org/cgi/query-pr.cgi?pr=ports/175447
http://www.freebsd.org/cgi/query-pr.cgi?pr=ports/178462
Last time I timed him out on a PR I was subjected to a tirade from
him, with questionable justification, but I may process these too when
I have time.
Alternatively, perhaps we need an editors/vim-options port
Chris
--
mezz.free...@gmail.com - m...@freebsd.org
FreeBSD GNOME Team
http://www.FreeBSD.org/gnome/ - gn...@freebsd.org
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: The vim port needs a refresh
On 5/25/2013 03:32, Jimmy wrote: Perhaps one might ask the VIM developers themselves to provide a tarball/zip of the patches along with the individual patches in their distributions, pointing out the huge number of patches that piled up for this particular VIM release and the problems it caused trying to download them individually... (if they're going to wait for another 2 1/2 years / 1000+ patches for their next release, they might consider it) As I previously mentioned, Bram Moolenaar said on 9 May that the patch level for 7.3 will not exceed 1000 patches. ( https://groups.google.com/forum/#!topic/vim_announce/ZWWgK9aXQ2Y ) Seeing how they apparently refuse to have normal releases and think 900 patches is normally, I don't see how Vim developers would ever create a special tarball. If they were willing to do that, they'd just do normal releases, right? This suggestion to talk to Vim is destined to go down in flames, especially as version 7.4 is imminent (beta scheduled for end of May). ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: The vim port needs a refresh
On 24 May 2013 22:23, Kenta Suzumoto ken...@hush.com wrote:
Hello all. The editors/vim port is currently a mess and needs some changes.
- It fetches almost 700 patches from what seems like a dial-up connection in
AUSTRALIA.
You might as well be downloading a 1080p movie from a rock in the north pole,
because that's about how fast it is.
This can be very easily avoided by putting all the patches into a single
tarball and hosting it anywhere decent. I've
seen someone in ##freebsd on freenode handing out a tarball with all the
patches many times, and everyone asks
why isn't this the default? why is some random guy giving me distfiles?
etc. Seems like a no-brainer.
- By default, it builds lots of gui stuff that certainly almost no one wants
It almost seems like the vim-lite port should be renamed vim and the vim port
should be renamed gvim. I had to
google to come up with this solution, because I can't even disable that stuff
in make config (another problem!)
.if ${.CURDIR}==/usr/ports/editors/vim
WITH_VIM_OPTIONS=yes
WITHOUT_X11=yes
.endif
People shouldn't have to find this hack to be able to install vim normally
(and no, telling them to use vim-lite isn't normal).
I'm surprised that none of these changes have been made yet. I've heard it's
because the maintainer won't listen to reason
but I have no way to know if that's the case or not. I also heard bapt@ had
an optionsNG patch that he wouldn't
integrate into the port for some reason. Please, let's get this stuff fixed
once and for all. None of it requires a large amount
of work on anyone's part.
I'm very sad to talk of a fellow developer like this, but I'm afraid
the maintainer of vim is a contrarian who thinks he knows better than
everyone else on the matter.
For years, people have been begging him to get over his fear of
OPTIONS, and he sits in the way of progress against almost everyone's
wishes.
He has also impeded progress on the bash port, resulting in the
ridiculous situation where we now have two bash ports, where one will
do. For historical reasons, people seem reluctant to confront him
about this, and he ignores all attempts to reason about it.
It's far beyond time to remove David O'Brien from MAINTAINER lines--
he doesn't do the job properly anyway; several PRs he's timed out on
for his ports:
http://www.freebsd.org/cgi/query-pr.cgi?pr=ports/177597
http://www.freebsd.org/cgi/query-pr.cgi?pr=ports/174965
http://www.freebsd.org/cgi/query-pr.cgi?pr=ports/175447
http://www.freebsd.org/cgi/query-pr.cgi?pr=ports/178462
Last time I timed him out on a PR I was subjected to a tirade from
him, with questionable justification, but I may process these too when
I have time.
Alternatively, perhaps we need an editors/vim-options port
Chris
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: The vim port needs a refresh
On 05/25/13 10:50, Chris Rees wrote: Alternatively, perhaps we need an editors/vim-options port Just for the record, editors/vim was (and shells/bash) was converted to optionsNG not too long ago. Regards! -- Niclas ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: The vim port needs a refresh
On 25 May 2013 11:54, Niclas Zeising zeising+free...@daemonic.se wrote: On 05/25/13 10:50, Chris Rees wrote: Alternatively, perhaps we need an editors/vim-options port Just for the record, editors/vim was (and shells/bash) was converted to optionsNG not too long ago. Ah, that's at least some good news. I notice that it was on yet another maintainer timeout, so that criticism stands. It appears that David is no longer interested. Chris ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: The vim port needs a refresh
On 5/25/2013 13:24, Chris Rees wrote: On 25 May 2013 11:54, Niclas Zeisingzeising+free...@daemonic.se wrote: On 05/25/13 10:50, Chris Rees wrote: Alternatively, perhaps we need an editors/vim-options port Just for the record, editors/vim was (and shells/bash) was converted to optionsNG not too long ago. Ah, that's at least some good news. I notice that it was on yet another maintainer timeout, so that criticism stands. It appears that David is no longer interested. FWIW, the default on the vim port have taken the dports users by surprise. I've gotten several complaints about the boatload of ports that get sucked in (and the amount of bandwidth it requires) by vim. They didn't know vim-lite existed. I agree the default should be light and the kitchen sink version should be explicitly requested (if two ports are indeed needed for pre-built binary reasons). John ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: The vim port needs a refresh
Am 25.05.2013 13:28, schrieb John Marino: On 5/25/2013 13:24, Chris Rees wrote: On 25 May 2013 11:54, Niclas Zeisingzeising+free...@daemonic.se wrote: On 05/25/13 10:50, Chris Rees wrote: Alternatively, perhaps we need an editors/vim-options port Just for the record, editors/vim was (and shells/bash) was converted to optionsNG not too long ago. Ah, that's at least some good news. I notice that it was on yet another maintainer timeout, so that criticism stands. It appears that David is no longer interested. FWIW, the default on the vim port have taken the dports users by surprise. I've gotten several complaints about the boatload of ports that get sucked in (and the amount of bandwidth it requires) by vim. They didn't know vim-lite existed. I agree the default should be light and the kitchen sink version should be explicitly requested (if two ports are indeed needed for pre-built binary reasons). I routinely install vim-lite on FreeBSD, and move on. Yes, it may surprise first-time users, but either you have the GUI stuff installed anyways and don't care, or you get into the habit of installing vim-lite on headless and otherwise low-end machines. Frequent timeouts are a reason to reset the maintainer, though. We have policies for that, and we have been re-rolling distributions all the way, from SVN repos, or Github, for example, so why bother. As to the patch hosting, if some port fails and we modified it in any way, we're the first contact of the end user anyhow to pre-filter between what we've botched, and what needs forwarding upstream -- and there should be plenty of vim mirrors around -- perhaps a case to deploy phttpget for bulk fetching of a thousand patches from HTTP-based mirrors. Oh, and look at how snappy portsnap has become on -HEAD. Not sure what it does differently than 9.1-REL, but it sure feels a lot snappier. If that's not phttpget we should consider if we can leverage that. ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: The vim port needs a refresh
On Sat, May 25, 2013 at 12:23 AM, Kenta Suzumoto ken...@hush.com wrote:
Hello all. The editors/vim port is currently a mess and needs some changes.
- It fetches almost 700 patches from what seems like a dial-up connection in
AUSTRALIA.
You might as well be downloading a 1080p movie from a rock in the north pole,
because that's about how fast it is.
This can be very easily avoided by putting all the patches into a single
tarball and hosting it anywhere decent. I've
seen someone in ##freebsd on freenode handing out a tarball with all the
patches many times, and everyone asks
why isn't this the default? why is some random guy giving me distfiles?
etc. Seems like a no-brainer.
- By default, it builds lots of gui stuff that certainly almost no one wants
It almost seems like the vim-lite port should be renamed vim and the vim port
should be renamed gvim. I had to
google to come up with this solution, because I can't even disable that stuff
in make config (another problem!)
.if ${.CURDIR}==/usr/ports/editors/vim
WITH_VIM_OPTIONS=yes
WITHOUT_X11=yes
.endif
People shouldn't have to find this hack to be able to install vim normally
(and no, telling them to use vim-lite isn't normal).
I'm surprised that none of these changes have been made yet. I've heard it's
because the maintainer won't listen to reason
but I have no way to know if that's the case or not. I also heard bapt@ had
an optionsNG patch that he wouldn't
integrate into the port for some reason. Please, let's get this stuff fixed
once and for all. None of it requires a large amount
of work on anyone's part.
P.S. we're now at 7.3.1011 - the port could use a normal update as well.
/minor complaint
- kenta
As far as I know FreeBSD does not roll custom distfiles because of
very obvious issues with authenticity of the files. If you create a
custom distfile from let's say editors/vim as you suggest then who is
going to trust you to provide authentic sources of someone else's
work? Now when everything is separate and downloadable and verifiable
individually from the upstream vendor there's no problem with
authenticity.
-Kimmo
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: The vim port needs a refresh
On 5/25/2013 00:29, Kimmo Paasiala wrote: P.S. we're now at 7.3.1011 - the port could use a normal update as well./minor complaint - kenta As far as I know FreeBSD does not roll custom distfiles because of very obvious issues with authenticity of the files. If you create a custom distfile from let's say editors/vim as you suggest then who is going to trust you to provide authentic sources of someone else's work? Now when everything is separate and downloadable and verifiable individually from the upstream vendor there's no problem with authenticity. Supposedly the current patch level is approaching 1000 patches and vim 7.4 is supposed to come out before that happens. That will reset to number of patches to zero, so at least it won't continually get worse. ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: The vim port needs a refresh
On Sat, May 25, 2013 at 1:29 AM, Kimmo Paasiala kpaas...@gmail.com wrote:
On Sat, May 25, 2013 at 12:23 AM, Kenta Suzumoto ken...@hush.com wrote:
Hello all. The editors/vim port is currently a mess and needs some changes.
- It fetches almost 700 patches from what seems like a dial-up connection in
AUSTRALIA.
You might as well be downloading a 1080p movie from a rock in the north
pole, because that's about how fast it is.
This can be very easily avoided by putting all the patches into a single
tarball and hosting it anywhere decent. I've
seen someone in ##freebsd on freenode handing out a tarball with all the
patches many times, and everyone asks
why isn't this the default? why is some random guy giving me distfiles?
etc. Seems like a no-brainer.
- By default, it builds lots of gui stuff that certainly almost no one wants
It almost seems like the vim-lite port should be renamed vim and the vim
port should be renamed gvim. I had to
google to come up with this solution, because I can't even disable that
stuff in make config (another problem!)
.if ${.CURDIR}==/usr/ports/editors/vim
WITH_VIM_OPTIONS=yes
WITHOUT_X11=yes
.endif
People shouldn't have to find this hack to be able to install vim normally
(and no, telling them to use vim-lite isn't normal).
I'm surprised that none of these changes have been made yet. I've heard it's
because the maintainer won't listen to reason
but I have no way to know if that's the case or not. I also heard bapt@ had
an optionsNG patch that he wouldn't
integrate into the port for some reason. Please, let's get this stuff fixed
once and for all. None of it requires a large amount
of work on anyone's part.
P.S. we're now at 7.3.1011 - the port could use a normal update as well.
/minor complaint
- kenta
As far as I know FreeBSD does not roll custom distfiles because of
very obvious issues with authenticity of the files. If you create a
custom distfile from let's say editors/vim as you suggest then who is
going to trust you to provide authentic sources of someone else's
work? Now when everything is separate and downloadable and verifiable
individually from the upstream vendor there's no problem with
authenticity.
-Kimmo
One way around the problem that would not compromise authenticity
would be adding support for wrapper distfiles (if there's not already
something like that) that combine all distfiles of one port into one
tarball that gets extracted to $DISTDIR before the distfile
verification takes place.
-Kimmo
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: The vim port needs a refresh
On Sat, May 25, 2013 at 01:29:09AM +0300, Kimmo Paasiala wrote: As far as I know FreeBSD does not roll custom distfiles because of very obvious issues with authenticity of the files. If you create a custom distfile from let's say editors/vim as you suggest then who is going to trust you to provide authentic sources of someone else's work? Now when everything is separate and downloadable and verifiable individually from the upstream vendor there's no problem with authenticity. There are some exceptions. For instance, www/chromium has a custom distfile. I think it would be reasonable to create one for editors/vim as well. -- Denny Lin ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: The vim port needs a refresh
Perhaps one might ask the VIM developers themselves to provide a tarball/zip of the patches along with the individual patches in their distributions, pointing out the huge number of patches that piled up for this particular VIM release and the problems it caused trying to download them individually... (if they're going to wait for another 2 1/2 years / 1000+ patches for their next release, they might consider it) Jimmy On Sat, May 25, 2013 at 08:20:19AM +0800, Denny Lin wrote: On Sat, May 25, 2013 at 01:29:09AM +0300, Kimmo Paasiala wrote: As far as I know FreeBSD does not roll custom distfiles because of very obvious issues with authenticity of the files. If you create a custom distfile from let's say editors/vim as you suggest then who is going to trust you to provide authentic sources of someone else's work? Now when everything is separate and downloadable and verifiable individually from the upstream vendor there's no problem with authenticity. There are some exceptions. For instance, www/chromium has a custom distfile. I think it would be reasonable to create one for editors/vim as well. -- Denny Lin ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
