Re: INDEX build optimizations - please review

2008-06-23 Thread Doug Barton
Kris Kennaway wrote: Unfortunately it doesn't DTRT with files terminated with DOS-style CRLF (e.g. devel/p5-Tie-Restore, others). First, I certainly would not have any problem with a policy that says files in ports shouldn't have CRLF line endings. Second, I am sure we could probably do

INDEX build optimizations

2008-06-22 Thread Vladimir . Chukharev
Since this subject has been discussed recently, I want to tell my observations. I played a bit with INDEX_JOBS environment variable, and I found that for my notebook the best value is 8. The limiting factor seems to be grep'ing for libssl through packages. I believe this is easy to reproduce,

Re: INDEX build optimizations - please review

2008-06-22 Thread Kris Kennaway
Doug Barton wrote: Kris Kennaway wrote: The new 'make describe' target runs entirely using shell builtins apart from the need to sed pkg-descr to extract the WWW [2] [2] Actually I am not happy with this but couldn't think of a way to do it better. Having to fork the subshell costs about

Re: INDEX build optimizations - please review

2008-06-19 Thread Kris Kennaway
Pietro Cerutti wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Kris Kennaway wrote: | ** | * NOTE TO PORT DEVELOPERS |

Re: INDEX build optimizations - please review

2008-06-19 Thread Pietro Cerutti
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Kris Kennaway wrote: | Pietro Cerutti wrote: | -BEGIN PGP SIGNED MESSAGE- | Hash: SHA512 | | Kris Kennaway wrote: | | | ** | | * NOTE TO PORT DEVELOPERS

Re: INDEX build optimizations - please review

2008-06-19 Thread Pietro Cerutti
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Kris Kennaway wrote: | ** | * NOTE TO PORT DEVELOPERS | ** | |

Re: INDEX build optimizations - please review

2008-06-11 Thread Alex Kozlov
On Tue, Jun 10, 2008 at 04:49:18PM -0700, Doug Barton wrote: Kris Kennaway wrote: The new 'make describe' target runs entirely using shell builtins apart from the need to sed pkg-descr to extract the WWW [2] [2] Actually I am not happy with this but couldn't think of a way to do it

Re: INDEX build optimizations - please review

2008-06-11 Thread Doug Barton
Alex Kozlov wrote: Good idea. I also do something like that [1]. But before we can use this method, we have to fix ports with bad pkg-descr: Yeah, in a brief glance at Kris' sed routines I would think at least some of this work would have to be done no matter what (and it should be done

Re: INDEX build optimizations - please review

2008-06-11 Thread Alex Kozlov
On Wed, Jun 11, 2008 at 10:22:18AM -0700, Doug Barton wrote: Alex Kozlov wrote: Good idea. I also do something like that [1]. But before we can use this method, we have to fix ports with bad pkg-descr: Yeah, in a brief glance at Kris' sed routines I would think at least some of this

INDEX build optimizations - please review

2008-06-10 Thread Kris Kennaway
Please review and test the following patches that optimize port INDEX builds (and as a side-effect, other recursive tree traversals). I am particularly interested in a comparison between old and new indexes built locally: the only diff should be in audio/festvox-hvs [1]. The patches remove

Re: INDEX build optimizations - please review

2008-06-10 Thread Pietro Cerutti
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Kris Kennaway wrote: | Another important optimization is to use /rescue/sh instead of /bin/sh | for index builds. The former is statically linked and this is much | faster to execute. True, but the for is not even guaranteed to exist

Re: INDEX build optimizations - please review

2008-06-10 Thread Pietro Cerutti
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Pietro Cerutti wrote: | Kris Kennaway wrote: | | | Another important optimization is to use /rescue/sh instead of /bin/sh | | for index builds. The former is statically linked and this is much | | faster to execute. | | True, but the for is not

Re: INDEX build optimizations - please review

2008-06-10 Thread Jeremy Messenger
On Tue, 10 Jun 2008 11:45:32 -0500, Kris Kennaway [EMAIL PROTECTED] wrote: Another important optimization is to use /rescue/sh instead of /bin/sh for index builds. The former is statically linked and this is much faster to execute. I don't have INDEX in all of my systems, so I am only

Re: INDEX build optimizations - please review

2008-06-10 Thread Kris Kennaway
Pietro Cerutti wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Pietro Cerutti wrote: | Kris Kennaway wrote: | | | Another important optimization is to use /rescue/sh instead of /bin/sh | | for index builds. The former is statically linked and this is much | | faster to execute. | |

Re: INDEX build optimizations - please review

2008-06-10 Thread Kris Kennaway
Jeremy Messenger wrote: On Tue, 10 Jun 2008 11:45:32 -0500, Kris Kennaway [EMAIL PROTECTED] wrote: Another important optimization is to use /rescue/sh instead of /bin/sh for index builds. The former is statically linked and this is much faster to execute. I don't have INDEX in all of my

Re: INDEX build optimizations - please review

2008-06-10 Thread Kris Kennaway
Alex Kozlov wrote: On Tue, Jun 10, 2008 at 06:45:32PM +0200, Kris Kennaway wrote: Please review and test the following patches that optimize port INDEX builds (and as a side-effect, other recursive tree traversals). I am particularly interested in a comparison between old and new indexes

Re: INDEX build optimizations - please review

2008-06-10 Thread Alex Kozlov
On Tue, Jun 10, 2008 at 11:52:01PM +0200, Kris Kennaway wrote: Alex Kozlov wrote: On Tue, Jun 10, 2008 at 06:45:32PM +0200, Kris Kennaway wrote: Please review and test the following patches that optimize port INDEX builds (and as a side-effect, other recursive tree traversals). I am

Re: INDEX build optimizations - please review

2008-06-10 Thread Kris Kennaway
Alex Kozlov wrote: On Tue, Jun 10, 2008 at 11:52:01PM +0200, Kris Kennaway wrote: Alex Kozlov wrote: On Tue, Jun 10, 2008 at 06:45:32PM +0200, Kris Kennaway wrote: Please review and test the following patches that optimize port INDEX builds (and as a side-effect, other recursive tree

Re: INDEX build optimizations - please review

2008-06-10 Thread Doug Barton
Kris Kennaway wrote: The new 'make describe' target runs entirely using shell builtins apart from the need to sed pkg-descr to extract the WWW [2] [2] Actually I am not happy with this but couldn't think of a way to do it better. Having to fork the subshell costs about 60 seconds of system