Re: /usr/ports/ too big?
On Wed, 9 Feb 2000, Kai Voigt wrote: Hello, I'm just doing a cvsup update of my system and -as many times before- I realize that /usr/ports/ takes a lot of time and also disk space to sync. # du -sk /usr/ports 71118 /usr/ports Am I the only one being little annoyed by this fact? Would it make any sense to offer some "castrated" ports repository. Like putting a target "overview" into each /usr/ports/*/Makefile to list all available subdiretories. Then, with some other command, one could fetch the current port's directory from the cvs server to install the port. Do these thoughts make any sense? I know this is a few days old, but I want to suggest this anyway. Restructure the ports tree so that at the top level are languages and then there are categories beneath that. Like this: en/ archivers/ astro/ ... jp/ archivers/ astro/ ... And so forth. I do know that foreign (I speak English natively, therefore everything else is foreign to me) have been seperated off into their own directories, but still a straight CVSup on ports will fetch them. Configuring CVSup to only get those non-language directories is both time consuming and problematic if a new directory is added and nobody tells you. This also takes a step towards removing the English bias that clearly exists within the ports tree (and elseware). I know this will only reduce the problem of an overly large ports tree for some time, but this is a good idea anyway, especially from an organizational standpoint. Has this been discussed before? Will I be laughed to scorn? Jamie To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: /usr/ports/ too big?
An even more radical approach, and more controversial, would be to remove /usr/ports entirely and use the concept of source packages. pkg_add -r aumix would install the binary, and something along the lines of: pkg-source_add -r aumix would download the source, patches, and whatever else needed. Considering most of us out there, myself included, have fallen madly in love with ports the way it is, I doubt that the current system won't go away for a long, long time. -Dan Papasian [EMAIL PROTECTED] On Wed, Feb 09, 2000 at 09:58:06PM +0100, Kai Voigt wrote: Hello, I'm just doing a cvsup update of my system and -as many times before- I realize that /usr/ports/ takes a lot of time and also disk space to sync. # du -sk /usr/ports 71118 /usr/ports Am I the only one being little annoyed by this fact? Would it make any sense to offer some "castrated" ports repository. Like putting a target "overview" into each /usr/ports/*/Makefile to list all available subdiretories. Then, with some other command, one could fetch the current port's directory from the cvs server to install the port. Do these thoughts make any sense? Kai To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: /usr/ports/ too big?
On Wed, 09 Feb 2000, Dan Papasian wrote:
An even more radical approach, and more controversial, would
be to remove /usr/ports entirely and use the concept of source packages.
pkg_add -r aumix would install the binary, and something along the lines of:
pkg-source_add -r aumix would download the source, patches, and whatever else
needed.
This is the direction that my thinking is headed. Let the actual developers
keep things (pretty much) as is. Repackage the distribution into a multi-level
hierarchy.
The top level would be a description of what's available.
{basically the DESCR files}
The second level would be the details.
{the rest of the stuff in /usr/ports/xxx/yyy/}
The third level would be the distribution tarballs.
{files presently fetched to /usr/ports/distfiles}
The ports maintainers would commit to the expanded tree just as they do now.
However, instead of distributing that tree, we would derive (automatically) the
level 2 tarballs and distribute them. The top level Makefile in /usr/ports/
would expand the level 2 build tree and continue down into it just as it does
now.
--
Richard Wackerbarth
[EMAIL PROTECTED]
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message
Re: /usr/ports/ too big?
TAKE THIS TO [EMAIL PROTECTED] * This is NOT a -current issue!! And the people that can actually effect change hang out on [EMAIL PROTECTED], not necessarly on [EMAIL PROTECTED] On Sat, Feb 12, 2000 at 12:19:01PM -0600, Richard Wackerbarth wrote: This is the direction that my thinking is headed. Let the actual developers keep things (pretty much) as is. Repackage the distribution into a multi-level hierarchy. *IF* people were to take this to [EMAIL PROTECTED], they would have learned there was a discussion about a month ago in which the conclusion was to use one dir per port, but remove the subdirs w/in each ports subdir. Thus rather than foo/Makefile foo/files/md5 foo/pkg/PLIST foo/pkg/DESCR foo/pkg/COMMENT foo/patch/patch-aa foo/patch/patch-ab we would have foo/Makefile foo/md5 foo/pkg_PLIST foo/pkg_DESCR foo/pkg_COMMENT foo/patch-aa foo/patch-ab To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: /usr/ports/ too big?
On Sat, 12 Feb 2000, David O'Brien wrote:
TAKE THIS TO [EMAIL PROTECTED] *
Agreed. This is where the depth of the discussion should take place.
This is NOT a -current issue!!
I beg to differ. Any significant change to the status-quo is a -current issue.
To adopt ANY SIGNIFICANT CHANGE without widespread public notice is just
inviting grumblings of "backroom politics". Just see what happens if the City
Council votes to close Main Street and explains " this was discussed at a Public
Hearing before the Public Works Commission"
And some of us, myself included, are advocating making FreeBSD into a small set
of ports!
I guess that doesn't affect very many people :-)
--
Richard Wackerbarth
[EMAIL PROTECTED]
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message
Re: /usr/ports/ too big?
On Sat, Feb 12, 2000 at 01:58:14PM -0600, Richard Wackerbarth wrote:
On Sat, 12 Feb 2000, David O'Brien wrote:
TAKE THIS TO [EMAIL PROTECTED] *
Agreed. This is where the depth of the discussion should take place.
This is NOT a -current issue!!
I beg to differ. Any significant change to the status-quo is a -current issue.
To adopt ANY SIGNIFICANT CHANGE without widespread public notice is just
inviting grumblings of "backroom politics". Just see what happens if the City
Council votes to close Main Street and explains " this was discussed at a Public
Hearing before the Public Works Commission"
And some of us, myself included, are advocating making FreeBSD into a small set
of ports!
I guess that doesn't affect very many people :-)
You are very wrong. We are not "adopting significant change", we are discussing
possibilities. Discussions related to ports/packages belong on freebsd-ports.
Please read the mail charters.
--
Bill Fumerola - Network Architect
Computer Horizons Corp - CVM
e-mail: [EMAIL PROTECTED] / [EMAIL PROTECTED]
Office: 800-252-2421 x128 / Cell: 248-761-7272
PS. The reason that "Public Works Commissions" of the world exist is because
City Councils trust their departments to make informed reccomendations.
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message
Re: /usr/ports/ too big?
On Thu, 10 Feb 2000, John Polstra wrote: In article [EMAIL PROTECTED], Leif Neland [EMAIL PROTECTED] wrote: Just FYI, a cvsup of ports over a single ISDN-line took 22 min on a soft-update'd disk. Something is seriously wrong over there then, because I can update my entire CVS repository in 1.5-2 minutes. And my Internet link is a wimpy 56 Kbit frame relay connection Normally a cvsup at 3am to cvsup.dk.freebsd.org takes around that too. But I wiped the entire ports-tree and re-cvsup'ped it; this took around 22minutes at 23-o'clock. However, a normal cvsup here at 13:40 took 9m58s on a only half-loaded 1Mb line, and a loadlevel of 0.01. Perhaps Jesper was using it too :-) Leif To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: /usr/ports/ too big?
At 05:55 PM 2/10/00 -0800, Matthew Dillon wrote: :Sounds good, but again how will the CVSup file for ports and CVSup itself :deal with this. Either a "refuse" file would need to be created and then :populated or there would need to be other changes. Not sure Mr Wraith or :the CVS maintainers would like to break down all the ports and have a :*huge* CVSup file for ports. Or some other method would be needed that :would increase the complexity of how the ports source is handled. You don't. The CVS tree for ports stays the way it is, and you wouldn't use cvsup to download a broken out version. Ahhh... had me wondering there for bit. Here's what I would do: * create /usr/src/ports * create /usr/src/ports/Makefile * make targets would be: make install Install a new /usr/ports. Deletes anything previously in /usr/ports (?) and constructs a new first-level directory hierarchy, first-level Makefiles, /usr/ports/Mk, and aggregate DESCR file. make update Updates /usr/ports. Locates any broken-out subdirectories in /usr/ports and updates them (equivalent to cvs update in those subdirectories), updates the Makefile's in the first-level directories, updates the aggregate DESCR file, and updates /usr/ports/Mk. That's it. Most normal users can install/update their ports collection the same way they install/update a kernel or bin or sbin, by CD'ing into a (minimally populated) /usr/src/ports directory and typing 'make install'. /usr/src/ports would contain nothing more then a Makefile which cvs checkout's or cvsup's just the top level directory structure. That handles everything except the aggregate DESCR file. I can think of a number of trivial ways to handle the aggregate DESCR file. Those people who are actively working with the ports hierarchy can cvsup the whole blessed thing as they currently do. The ports maintainers would not have to lift a finger. The ports structure is not changing at all except for adding the ability to create and populate a subdirectory on the fly, something that ought to be easy to incorporate into /usr/ports/Mk, and adding /usr/src/ports as a launching pad for standard users to install /usr/ports. Sounds better now. So, when does it debut? ;) Jeff Mountin - [EMAIL PROTECTED] Systems/Network Administrator FreeBSD - the power to serve '86 Yamaha MaxiumX (not FBSD powered) To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: /usr/ports/ too big?
At 10:15 PM 2/10/00 -0800, John Polstra wrote: In article [EMAIL PROTECTED], Jeffrey J. Mountin [EMAIL PROTECTED] wrote: In the context of CVSup server connections it would not be. Have to chuckle when I hear someone doing CVSup for ports-all. Unless they have a reason, but as we know many will do man things blindly. In my experience, CVSup is not slow for the ports tree. CVS is slow, but not CVSup. I can typically update my entire CVS repository (CVSROOT + distrib + doc + ports + src + www) in 1.5-2 minutes on a 56 Kbit link. Of course the "cvs upd" afterwards does take a long time. Yes, the CVSup "client" it is quick, even with a 33.6K modem to my surprise (not sure if I'll go back to ISDN or get ADSL). Don't use CVS too often. Was thinking more about all the extra work that the servers are doing in light of your request to spread the load around. Reducing the size of the initial distrubution and explaining the wonders of refuse files to trim down the port tree are my main gist. Less to look at, clean out, refuse, and serve. Or just sit there collecting dust. I'm more than happy with how the ports and CVSup work for me, but think it could be made better to help new users. Matt's idea or something similar that is self-contained sounds like the way to go for the future. And at a guess would not take too much hackery for the install. Something BTW, other than a few odd quirks a few months back have not seen any connections hanging. Transient poltergeist perhaps. shrug Jeff Mountin - [EMAIL PROTECTED] Systems/Network Administrator FreeBSD - the power to serve '86 Yamaha MaxiumX (not FBSD powered) To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: /usr/ports/ too big?
On Thu, Feb 10, 2000 at 11:45:44AM -0800, Matthew Dillon wrote: All I would propose is that those subdirectories do not need to be part of the base distribution -- that typing 'make modulename' in the parent directory (e.g. typing 'make ssh' in ports/security) would first download the subdirectory and then do a normal make within that subdirectory. Something of this general idea exists in the portcheckout port. I haven't looked at the newest version of the port and I imagine there are improvements to be made, still, but it does implement the general idea of demand based downloading of port skeletons. Expanding it to have a full interface that works from the INDEX file and produces a set of skeletons guaranteed to compile should not be too hard. -- Signature withheld by request of author. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: /usr/ports/ too big?
On Fri, 11 Feb 2000, Tim Vanderhoek wrote: Something of this general idea exists in the portcheckout port. If we were to have a stripped down skeleton of the ports, is it generally felt that the INDEX contains enough information? Or do we really need to have the DESCRiptions available for browsing before we go online to actually fetch the required files? -- Richard Wackerbarth [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: /usr/ports/ too big?
:think of a number of trivial ways to handle the aggregate DESCR file. : :Those people who are actively working with the ports hierarchy can :cvsup the whole blessed thing as they currently do. : :The ports maintainers would not have to lift a finger. The ports :structure is not changing at all except for adding the ability to :create and populate a subdirectory on the fly, something that ought :to be easy to incorporate into /usr/ports/Mk, and adding /usr/src/ports :as a launching pad for standard users to install /usr/ports. : :Sounds better now. So, when does it debut? ;) : : :Jeff Mountin - [EMAIL PROTECTED] :Systems/Network Administrator :FreeBSD - the power to serve :'86 Yamaha MaxiumX (not FBSD powered) I wish I had time to do it, but I don't. But if pepole are serious about reducing the ports footprint this is the only thing that I believe will fly. -Matt Matthew Dillon [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: /usr/ports/ too big?
At 09:58 PM 2/9/00 +0100, Kai Voigt wrote: Hello, I'm just doing a cvsup update of my system and -as many times before- I realize that /usr/ports/ takes a lot of time and also disk space to sync. # du -sk /usr/ports 71118 /usr/ports Is that just source or with some distfiles and port/work dirs? Seems a bit high. By far most of the space is taken up with distfiles once you populate a system. The number of directories is the killer, which slows down installing the collection. Am I the only one being little annoyed by this fact? Would it make any sense to offer some "castrated" ports repository. Like putting a target "overview" into each /usr/ports/*/Makefile to list all available subdiretories. Then, with some other command, one could fetch the current port's directory from the cvs server to install the port. Do these thoughts make any sense? A bit. Not certain what you are suggesting here, but something having only Makefile, DESCR and possibly README.html files for the port. Upon the first 'make' the rest of the port's directory would be first fetched with business as usual after that. My opinion is that from the start when someone installs a system they should not have to install the entire directory structure. Compared to a full source install (no X) it takes much too long. And will only increase. Breaking up ports.tgz has been suggested before. Speeding up the install should keep the impatient newcomer around and will most likely be installing the ports collection. Then there is problem of CVSup. If someone were to install the entire tree as above and use "ports-all" in their supfile... More practical to break ports.tar into their respective categories. One the "base" and other choosen categories are installed. Having the choice from the start, IMO, would mean less users with the entire, mostly unused (and certainly not needed!), tree who would later CVSup everything when updating. Then add a clear mention of of updating the ports that explains using "refuse" files along with a handy script that would use sup/whatever/refuse file for further pruning. Then if you keep the refuse files they could be used to clean fresh installs either by installing and then deleting or by removing them from ports.tar from the start, which is what I've been doing for a while. sigh There is the ever present problem of getting users to actually read documention, but at least an effort would be made. Only just recently trimmed down what I pull from ports. Did so a while back with doc and www, but didn't take the time for anything further until recently. Guess the size of the tree and CVSup logs didn't really register until a few days back (rather ironic) when I pulled for the first time in a while. Figured then it was time to make things a bit more neat and clean. Now only need to watch for checkouts and possibly add to the refuse files, which still need cleaning. The best example being ports/games, which I only need those that are called for when KDE is installed. Might want others at some time, but can check out either DESCR or README.html for candidates. Hmmm... there could be a port of these files that would compliment the INDEX. As for the initial install, taking time to trim down the original tarball saves time. (At least the CVS subdirs are gone.) Yeah, one can also use NFS and copy the tree, but I prefer to start fresh from a release and re-CVSup to -stable (or -current), so it's something I every few months. 56279040 ports-3_4.tar 48312320 ports.tar-nolang (only English spoken here :) 44206080 ports.tar-no_lang-no_README.html 39229440 ports.tar (dropped more categories) 34048000 ports.tar (nukes most games) Getting there. Glad to take some time and work out some more trimming, since 4.0-RC (2/8 snap) is on it's way over and it will be installed fresh makeing 'tar --delete -f ports.tar long_list' my friend. Trying to figure out where the time went. Last ran -current prior to 3.0. 8-) 'Nuff ramble. Jeff Mountin - [EMAIL PROTECTED] Systems/Network Administrator FreeBSD - the power to serve '86 Yamaha MaxiumX (not FBSD powered) To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: /usr/ports/ too big?
On 2000-Feb-10 07:59:48 +1100, Kai Voigt [EMAIL PROTECTED] wrote: # du -sk /usr/ports 71118 /usr/ports Seems reasonable. Last time I checked (1st June 1999), I found 79967 - of which 35215 was CVS-related files/directories. There were also nearly 62,000 inodes. It'll get worse - PHK has changed the FS defaults from 8K/1K to 16K/4K, which will roughly triple the space. Am I the only one being little annoyed by this fact? This comes up regularly. The last I recall was a thread "a two-level port system?" in -hackers last May/June. My favourite solution (because it's mine) would be to replace the existing each port skeleton directory with a single ar(5) file, which is unpacked into the directory structure when you make the port. (I think ar(5) would be a good choice because (a) it is text, and so can be easily managed by CVS; (b) it includes a tool - ar(1) - for easily managing the files). As an example, /usr/ports/cad currently looks like (without CVS): /usr/ports/cad: Makefilefelt/ magic/ qcad/ xcircuit/ acs/geda/ mars/ sis/ chipmunk/ irsim/ pcb/spice/ cider/ kaskade/pkg/tkgate/ It would turn into: /usr/ports/cad: ARCHIVE/Makefile ./ARCHIVE: acs.ar geda.ar mars.ar sis.ar chipmunk.ar irsim.arpcb.ar spice.ar cider.arkaskade.ar pkg.ar tkgate.ar felt.ar magic.arqcad.ar xcircuit.ar Where the Makefile knew how to unpack/pack each ar into the existing structure as necessary. What's need to change the existing structure is: 1) A completely implemented replacement, including the tools necessary to manage the new structure. 2) Agreement from Asami-san (and maybe others) to implement the changed structure. Peter To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: /usr/ports/ too big?
On Thu, 10 Feb 2000, Peter Jeremy wrote: Seems reasonable. Last time I checked (1st June 1999), I found 79967 - of which 35215 was CVS-related files/directories. There were also nearly 62,000 inodes. It'll get worse - PHK has changed the FS defaults from 8K/1K to 16K/4K, which will roughly triple the space. My favourite solution (because it's mine) would be to replace the existing each port skeleton directory with a single ar(5) file There are two problems in the size of the ports system. 1) The large number of inodes. Your proposal certainly addresses this. 2) The huge size of our ports collection. Unfortunately, this will only get worse^H^H^H^H^Hbetter. In addition to the files needed to build a port, there is a description which is quite useful. My variation on your theme is to have two files for each of the present ports. The first would be a Makefile that has the description imbedded in it. This would provide the hook to build the port and a browsable "library of available programs" The second would be your `ar` file of the patches, build instructions, etc. These would get unpacked only during the actual building of a particular port. For distribution, I would have the top level descriptions as one set. The archives could be obtained either individually or as sets corresponding to each of the present top level directories. Now, here is a really "silly" idea. Why don't we make a `port` collection of the FreeBSD kernel and standard userland utilities? That would lead to the next step of having the "standard distribution" become just a meta package much like 'kde' pulls in 'kdebase', 'kdeutils', 'kdegraphics', etc. -- Richard Wackerbarth [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: /usr/ports/ too big?
On Thu, Feb 10, 2000 at 10:19:18PM +1100, Peter Jeremy wrote: Am I the only one being little annoyed by this fact? This comes up regularly. The last I recall was a thread "a two-level port system?" in -hackers last May/June. Actually, -ports discussed this quite recently, and it was suggested that we combine some of the directories to reduce the number of inodes in half. This discussion belongs on -ports anyway.. so I'm bcc'ing -current. My favourite solution (because it's mine) would be to replace the existing each port skeleton directory with a single ar(5) file, which is unpacked into the directory structure when you make the port. (I think ar(5) would be a good choice because (a) it is text, and so can be easily managed by CVS; (b) it includes a tool - ar(1) - for easily managing the files). So, what you'd do is archive all of these directories into ar files, and have the Makefile unpack the archive whenever a port is needed? It would preserve the current Makefile, pkg/, scripts/, files/, etc. hierarchy? (How the hell would you pull that off? I've only known ar(1) to be used for creating library archives later ranlib'd..) Seems like this idea would make an initial install much faster and the inode/directory creation would be spread over time. Am I right? How would this affect the CVS repository? Would we still have to deal with the current hierarchy in the ports tree as it is? Or would we deal with it in ar(5) form? Which format would CVSUP update - ar(5) or current hierarchy? If it updates ar(5) form, how will bsd.port.mk know to update the directory tree for a particular port if the particular port is already unarchived? What's need to change the existing structure is: 1) A completely implemented replacement, including the tools necessary to manage the new structure. 2) Agreement from Asami-san (and maybe others) to implement the changed structure. I'm sure if Satoshi heard the answers to the above questions (among others asked), we'd be well on our way to having a new ports hierarchy for 5.0-CURRENT. :-) But it probably won't happen before 4.0-RELEASE since that's just too close to implement something big like this.. -- Will Andrews [EMAIL PROTECTED] GCS/E/S @d- s+:++:- a---+++ C++ UB P+ L- E--- W+++ !N !o ?K w--- ?O M+ V-- PS+ PE++ Y+ PGP t++ 5 X++ R+ tv+ b++ DI+++ D+ G+ e- h! r--+++ y? To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: /usr/ports/ too big?
On Thu, Feb 10, 2000 at 08:44:09AM -0500, Will Andrews wrote: On Thu, Feb 10, 2000 at 10:19:18PM +1100, Peter Jeremy wrote: Am I the only one being little annoyed by this fact? This comes up regularly. The last I recall was a thread "a two-level port system?" in -hackers last May/June. Actually, -ports discussed this quite recently, and it was suggested that we combine some of the directories to reduce the number of inodes in half. This discussion belongs on -ports anyway.. so I'm bcc'ing -current. Oops. I'll leave all the context in so the -ports people can see it. :-) My favourite solution (because it's mine) would be to replace the existing each port skeleton directory with a single ar(5) file, which is unpacked into the directory structure when you make the port. (I think ar(5) would be a good choice because (a) it is text, and so can be easily managed by CVS; (b) it includes a tool - ar(1) - for easily managing the files). So, what you'd do is archive all of these directories into ar files, and have the Makefile unpack the archive whenever a port is needed? It would preserve the current Makefile, pkg/, scripts/, files/, etc. hierarchy? (How the hell would you pull that off? I've only known ar(1) to be used for creating library archives later ranlib'd..) Seems like this idea would make an initial install much faster and the inode/directory creation would be spread over time. Am I right? How would this affect the CVS repository? Would we still have to deal with the current hierarchy in the ports tree as it is? Or would we deal with it in ar(5) form? Which format would CVSUP update - ar(5) or current hierarchy? If it updates ar(5) form, how will bsd.port.mk know to update the directory tree for a particular port if the particular port is already unarchived? What's need to change the existing structure is: 1) A completely implemented replacement, including the tools necessary to manage the new structure. 2) Agreement from Asami-san (and maybe others) to implement the changed structure. I'm sure if Satoshi heard the answers to the above questions (among others asked), we'd be well on our way to having a new ports hierarchy for 5.0-CURRENT. :-) But it probably won't happen before 4.0-RELEASE since that's just too close to implement something big like this.. -- Will Andrews [EMAIL PROTECTED] GCS/E/S @d- s+:++:- a---+++ C++ UB P+ L- E--- W+++ !N !o ?K w--- ?O M+ V-- PS+ PE++ Y+ PGP t++ 5 X++ R+ tv+ b++ DI+++ D+ G+ e- h! r--+++ y? To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message -- Will Andrews [EMAIL PROTECTED] GCS/E/S @d- s+:++:- a---+++ C++ UB P+ L- E--- W+++ !N !o ?K w--- ?O M+ V-- PS+ PE++ Y+ PGP t++ 5 X++ R+ tv+ b++ DI+++ D+ G+ e- h! r--+++ y? To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: /usr/ports/ too big?
On Wed, Feb 09, 2000 at 09:45:42PM -0500, Chuck Robey wrote: Flattening out the unecessarily deep ports directory structure would help, too. Probably, 98 percent of it could be done with a script, and it would greatly decrease cvsup time and space. I've often thought that it might be better if each port were a single tar file or something instead of the 30+ files that many of them now contain. From there, it seems like a straightforward step to not keep the tar files on your machine, much like you don't keep the distfiles. "make-port xmms" or whatever could go out and grab the xmms port tar file from ftpX.freebsd.org, extract it to the appropriate place, then do a make as usual. I haven't had time to try to implement it, though. -- Christopher Masto Senior Network Monkey NetMonger Communications [EMAIL PROTECTED][EMAIL PROTECTED]http://www.netmonger.net Free yourself, free your machine, free the daemon -- http://www.freebsd.org/ To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: /usr/ports/ too big?
Christopher Masto wrote: On Wed, Feb 09, 2000 at 09:45:42PM -0500, Chuck Robey wrote: Flattening out the unecessarily deep ports directory structure would help, too. Probably, 98 percent of it could be done with a script, and it would greatly decrease cvsup time and space. I've often thought that it might be better if each port were a single tar file or something instead of the 30+ files that many of them now contain. Here's what we can do. We keep all the "major" subdirectories in place, such as audio, devel, etc. BUT, instead of branching out into separate subdirectories, we can just put everything into the Makefile. For example, here are some subdirectories in /usr/ports/audio: amp/ ascd/ aumix/ bladeenc/ cam/ cd-console/ cdd/ We just do away with the subdirectories, flatten all the directories' contents out into one makefile, including DESCR, etc. (I really don't see why we need to have a separate DESC, md5 files, etc.) We can have md5 = xx and DESC=x inside the Makefiles, for example. Here's how we can flatten this: Makefile.amp Makefile.ascd Makefile.aumix Makefile.bladeenc Makefile.cam Makefile.cd-console Makefile.cdd Then, to build the port, one would do make -f Makefile.aumix to build the aumix port. All these makefiles would go inside of audio. To do the building each port, we can have the "work" be done inside the user's home directory. This would eliminate the need to log in as root in order to do the actual building. The benefits of having the port build inside the user's home directory are: * the user could have the option of installing the port in his own personal directory, in case the sysadmin doesn't see fit to put the port on the system * eliminates the security risk of having to do long compiles as root Then, to install the port, the port mechanism could check the uid of the user. If it is 0, the port gets installed in the system, under /usr/local, /usr/X11R6, or whatever. If it is nonzero, the port gets installed in the user's home directory. - Donn To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: /usr/ports/ too big?
: contain. : :Here's what we can do. We keep all the "major" subdirectories in :place, such as audio, devel, etc. BUT, instead of branching out into :separate subdirectories, we can just put everything into the :Makefile. For example, here are some subdirectories in :/usr/ports/audio: Ouch. I think this is a big mistake. The one-directory-per-port scheme works extremely well, it doesn't make much sense to get rid of it, and it doesn't make much sense ripping the *working* ports scheme to shreds when a simpler solution is available. All I would propose is that those subdirectories do not need to be part of the base distribution -- that typing 'make modulename' in the parent directory (e.g. typing 'make ssh' in ports/security) would first download the subdirectory and then do a normal make within that subdirectory. The initial ports distribution would thus only be a few dozen directories and a single Makefile in each one. -Matt To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: /usr/ports/ too big?
On Thu, 10 Feb 2000, I wrote: PHK has changed the FS defaults from 8K/1K to 16K/4K, Ooops. I mis-remembered a commit pkh made last August (src/release/sysinstall/install.c 1.91), I thought he had changed the defaults, but he just commented that 16K/4K was more sensible... My apologies to Poul-Henning and I'll try to remember not to trust my memory without double-checking :-(. Peter To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: /usr/ports/ too big?
Richard Wackerbarth writes: There are two problems in the size of the ports system. 1) The large number of inodes. I don't see the ports tree as the problem. The problem is that FreeBSD does not handle a very large directory hierarchy like that presented by the ports tree very well. The right angle of attack IMHO is to better optimize the kernel and maybe the filesystem. I don't know enough to know how you would do this though. For example, there's the thing about how we don't cache filesystem data and filesystem meta-data (directory blocks) the same way (this is the best I can describe it). Now, here is a really "silly" idea. Why don't we make a `port` collection of the FreeBSD kernel and standard userland utilities? That would lead to the next step of having the "standard distribution" become just a meta package much like 'kde' pulls in 'kdebase', 'kdeutils', 'kdegraphics', etc. This idea makes a lot of sense. All of FreeBSD could be packagable as ports/packages. It might even simplify the installer. -Archie ___ Archie Cobbs * Whistle Communications, Inc. * http://www.whistle.com To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: /usr/ports/ too big?
On Thu, 10 Feb 2000, Donn Miller wrote: All these makefiles would go inside of audio. To do the building each port, we can have the "work" be done inside the user's home directory. This would eliminate the need to log in as root in order to do the actual building. The benefits of having the port build It is already possible to do this. You simply need to set $WRKDIRPREFIX and $WRKDIR. inside the user's home directory are: * the user could have the option of installing the port in his own personal directory, in case the sysadmin doesn't see fit to put the port on the system Already possible. Set $DESTDIR The stuff in /usr/ports/Mk is pretty impressive. David To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: /usr/ports/ too big?
At 11:45 AM 2/10/00 -0800, Matthew Dillon wrote: : contain. : :Here's what we can do. We keep all the "major" subdirectories in :place, such as audio, devel, etc. BUT, instead of branching out into :separate subdirectories, we can just put everything into the :Makefile. For example, here are some subdirectories in :/usr/ports/audio: Ouch. I think this is a big mistake. The one-directory-per-port scheme works extremely well, it doesn't make much sense to get rid of it, and it doesn't make much sense ripping the *working* ports scheme to shreds when a simpler solution is available. In the context of CVSup server connections it would not be. Have to chuckle when I hear someone doing CVSup for ports-all. Unless they have a reason, but as we know many will do man things blindly. All I would propose is that those subdirectories do not need to be part of the base distribution -- that typing 'make modulename' in the parent directory (e.g. typing 'make ssh' in ports/security) would first download the subdirectory and then do a normal make within that subdirectory. Sounds good, but again how will the CVSup file for ports and CVSup itself deal with this. Either a "refuse" file would need to be created and then populated or there would need to be other changes. Not sure Mr Wraith or the CVS maintainers would like to break down all the ports and have a *huge* CVSup file for ports. Or some other method would be needed that would increase the complexity of how the ports source is handled. The initial ports distribution would thus only be a few dozen directories and a single Makefile in each one. But there are several goals: A faster, more simple install would be good. Reduced inode usage, less dirs to walk, etc.. Done in a manner to keep updating simple. What you (and others) have suggested covers the first 2, but then what happens when someone decides to update their source. Your suggestion would mean that to first get the source for the port, but then how does one keep it up to date. Not to mention the ports maintainer(s) might end up jumping through more hoops. Guess that should be added to the list, as well as not relying more the source repositories and mirrors. All irrelevant if hitting a mirror every time 'make port' to grab the source (initial or update) will not be an ongoing load issue for the servers. Still, what if I just want the source? Something like 'make source' will be needed or 'make fetch' will need to cover that. Just trying to cover all bases here and should sub to -install and see what's going on. Jeff Mountin - [EMAIL PROTECTED] Systems/Network Administrator FreeBSD - the power to serve '86 Yamaha MaxiumX (not FBSD powered) To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: /usr/ports/ too big?
On Thu, 10 Feb 2000, Archie Cobbs wrote: Richard Wackerbarth writes: There are two problems in the size of the ports system. 1) The large number of inodes. I don't see the ports tree as the problem. The problem is that FreeBSD does not handle a very large directory hierarchy like that presented by the ports tree very well. We HAVE to live in the house. The question is "how do we make the best use of the hand that was dealt us?" Fundamentally, I object to being required/expected to maintain a copy of a large amount of information that does not impact my system. I don't care about the patches to X unless I decide to install it. Similarly, I think that it is a stupid design to require everyone to keep the ENTIRE history of a file (per cvs). I have CD roms which have the old versions in case I need to reference them. Why cannot the 4.0 branch simply "end" with a reference to the 3.x CD for those who want to dig deeper. Now, here is a really "silly" idea. Why don't we make a `port` collection of the FreeBSD kernel and standard userland utilities? This idea makes a lot of sense. All of FreeBSD could be packagable as ports/packages. It might even simplify the installer. And `make` can pull in the dependencies (-: Sorry, you cannot reuse the existing tools. You must write a new one :-) -- Richard Wackerbarth [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: /usr/ports/ too big?
Just FYI, a cvsup of ports over a single ISDN-line took 22 min on a soft-update'd disk. .cvsignore INDEX LEGAL Makefile Mk README Templates Tools YEAR2000 archivers astro audio benchmarks cad comms converters databases deskutils devel distfiles editors emulators ftp games graphics lang mail math misc net news print security shells sysutils textproc www x11 x11-clocks x11-fm x11-fonts x11-toolkits x11-wm Leif To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: /usr/ports/ too big?
In article [EMAIL PROTECTED], Leif Neland [EMAIL PROTECTED] wrote: Just FYI, a cvsup of ports over a single ISDN-line took 22 min on a soft-update'd disk. Something is seriously wrong over there then, because I can update my entire CVS repository in 1.5-2 minutes. And my Internet link is a wimpy 56 Kbit frame relay connection John -- John Polstra [EMAIL PROTECTED] John D. Polstra Co., Inc.Seattle, Washington USA "Disappointment is a good sign of basic intelligence." -- Chögyam Trungpa To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: /usr/ports/ too big?
On Thu, Feb 10, 2000 at 22:18:12 -0800, John Polstra wrote: In article [EMAIL PROTECTED], Leif Neland [EMAIL PROTECTED] wrote: Just FYI, a cvsup of ports over a single ISDN-line took 22 min on a soft-update'd disk. Something is seriously wrong over there then, because I can update my entire CVS repository in 1.5-2 minutes. And my Internet link is a wimpy 56 Kbit frame relay connection For me, located behind a 64Kbit ISDN connection (sppp), it takes 3min for the ports + 1min for src/crypto. Regards -- Udo Schweigert, Siemens AG | Voice : +49 89 636 42170 ZT IK 3, Siemens CERT| Fax: +49 89 636 41166 D-81730 Muenchen / Germany | email : [EMAIL PROTECTED] PGP-2/5 fingerprint | 2A 53 F6 A6 30 59 64 02 6B C4 E0 73 B2 C9 6C E7 To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: /usr/ports/ too big?
On Thu, Feb 10, 2000 at 10:18:12PM -0800, John Polstra wrote: In article [EMAIL PROTECTED], Leif Neland [EMAIL PROTECTED] wrote: Just FYI, a cvsup of ports over a single ISDN-line took 22 min on a soft-update'd disk. Something is seriously wrong over there then, because I can update my entire CVS repository in 1.5-2 minutes. And my Internet link is a wimpy 56 Kbit frame relay connection Pretty much same results here... 3 mins approx on a 10Mb link this morning:-) /doc and www not counted/ The amount of data is largely dependent on how often you do the cvsup, of course. I do it every day... also it dependes on how loaded the cvsup server is. In Europe we have quite a lot of servers to choose from.:-) Regards: Szilveszter ADAM -- --- * Szilveszter ADAM * JATE Szeged * email: [EMAIL PROTECTED] * * Homepage : none * alternate email: [EMAIL PROTECTED] * * Finger [EMAIL PROTECTED] for PGP key. * * I prefer using the door instead of Windows(tm)... * To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
/usr/ports/ too big?
Hello, I'm just doing a cvsup update of my system and -as many times before- I realize that /usr/ports/ takes a lot of time and also disk space to sync. # du -sk /usr/ports 71118 /usr/ports Am I the only one being little annoyed by this fact? Would it make any sense to offer some "castrated" ports repository. Like putting a target "overview" into each /usr/ports/*/Makefile to list all available subdiretories. Then, with some other command, one could fetch the current port's directory from the cvs server to install the port. Do these thoughts make any sense? Kai -- kai voigt hamburger chaussee 36 24113 kiel 04 31 - 22 19 98 69 http://k.123.org/ To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: /usr/ports/ too big?
* Kai Voigt [EMAIL PROTECTED] [000209 13:26] wrote: Hello, I'm just doing a cvsup update of my system and -as many times before- I realize that /usr/ports/ takes a lot of time and also disk space to sync. # du -sk /usr/ports 71118 /usr/ports Am I the only one being little annoyed by this fact? Would it make any sense to offer some "castrated" ports repository. Like putting a target "overview" into each /usr/ports/*/Makefile to list all available subdiretories. Then, with some other command, one could fetch the current port's directory from the cvs server to install the port. Do these thoughts make any sense? Yes, this has been desired for some time, but without an actual implementation we're kinda stuck. :) -Alfred To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: /usr/ports/ too big?
: a target "overview" into each /usr/ports/*/Makefile to list all available : subdiretories. Then, with some other command, one could fetch the : current port's directory from the cvs server to install the port. : : Do these thoughts make any sense? : :Yes, this has been desired for some time, but without an actual :implementation we're kinda stuck. :) : :-Alfred It's a nice problem to have, I guess :-) I really like the idea of having a target overview. It would be utterly trivial to have a module list in the Makefile and to change the dependancies to run 'make modulename' in the parent directory rather then in the subdirectory (which might not exist). There's only one person who can make something like this happen. -Matt Matthew Dillon [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: /usr/ports/ too big?
On Wed, 9 Feb 2000, Matthew Dillon wrote: : a target "overview" into each /usr/ports/*/Makefile to list all available : subdiretories. Then, with some other command, one could fetch the : current port's directory from the cvs server to install the port. : : Do these thoughts make any sense? : :Yes, this has been desired for some time, but without an actual :implementation we're kinda stuck. :) : :-Alfred It's a nice problem to have, I guess :-) I really like the idea of having a target overview. It would be utterly trivial to have a module list in the Makefile and to change the dependancies to run 'make modulename' in the parent directory rather then in the subdirectory (which might not exist). There's only one person who can make something like this happen. Flattening out the unecessarily deep ports directory structure would help, too. Probably, 98 percent of it could be done with a script, and it would greatly decrease cvsup time and space. Chuck Robey| Interests include C Java programming, FreeBSD, [EMAIL PROTECTED] | electronics, communications, and signal processing. New Year's Resolution: I will not sphroxify gullible people into looking up fictitious words in the dictionary. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
