Re: OptionalObsoleteFiles.inc completeness improvement, try 2
* Dmitry Marakasov (amd...@amdmi3.ru) wrote:
The patch is complete, see https://reviews.freebsd.org/D1600.
- With it, for all knobs from src.conf(5) there are no difference in
file tree contents between `KNOB make installworld` and `make
installworld KNOB make delete-old delete-old-libs`
- Note that some OLD_DIRS were removed - they were incorrect as most
directories are installed unconditionally by mtree. Since empty
directores are not really needed, the install process should be
probably improved
- Some knobs which delete massive number of files (WITHOUT_MAN,
WITHOUT_INCLUDES) were implemented with
FILES!=find ...
OLD_FILES+=${FILES}
(similar construct was already used in WITHOUT_TESTS)
I though of more effective implementation for this, such as
introducing OLD_DIRS_RECURSIVE to fill from OptionalObsoleteFiles,
in which delete-old will just call `find ... -delete`
- Unneeded conditions such as
.if ${TARGET_ARCH} == amd64 || ${TARGET_ARCH} == powerpc64
OLD_FILES+=usr/lib32/...
.else
were not removed yet. Maybe with the next patch.
--
Dmitry Marakasov . 55B5 0596 FF1E 8D84 5F56 9510 D35A 80DD F9D2 F77D
amd...@amdmi3.ru ..: jabber: amd...@jabber.ruhttp://www.amdmi3.ru
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: OptionalObsoleteFiles.inc completeness improvement, try 2
In other words, if a user has some WITHOUT_* set in src.conf, specific files are not installed by installworld, but not removed by remove-old, which I try to fix. Please carry on, I saw quite a few times that this behaviour was assumed by new users, only it is not (yet), to their surprise :) -- View this message in context: http://freebsd.1045724.n5.nabble.com/OptionalObsoleteFiles-inc-completeness-improvement-try-2-tp5983192p5983627.html Sent from the freebsd-current mailing list archive at Nabble.com. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: OptionalObsoleteFiles.inc completeness improvement, try 2
* Garrett Cooper (yaneurab...@gmail.com) wrote: Hi Dmitry, Seems like we’ve duplicated work a bit. Have you looked at ^/projects/building-blocks yet ? Hm, seems so, partly. How do you gather missing entries? My way is pretty dumb, I just do bunch of installworlds + delete-old's and add diff to the file, you probably do it more cleverly. Will committing my changes interfere with your work? If so, it may be better to direct them to your branch instead. Especially if you are more aware of knob combinations and their effects. I wrote this script to track what files get installed by directories: https://svnweb.freebsd.org/base/projects/building-blocks/tools/add-optional-obsolete-files-entries.sh?revision=275238view=markup . It’s not perfect (doesn’t work for “kitchen sink” directories like etc/ and share/…), but it’s a reasonable starting point for grabbing all of the files. I’m going to hold off on the rc.d deletions/rewrites for dependent services after I do more targeted review/testing, as it might screw up boot order in unexpected ways with services and programs that get run out of order, but I’m reasonably confident that the contents of the branch work. I was going to start cherry-picking changes from the branch this weekend. Forgot to respond on one important point: reviewing and committing the higher-level changes (make remove-old) shouldn’t be a problem, and I’ll be sure to commit the overlapping changes with OptionalObsoleteFiles.inc first to reduce the diff between our branches. Thanks! Okay, the I'll just go on. Nice to know it's useful. Note that the branch may have bugs as on the first run I just extend it, and it's tested on another run, while run takes couple of days on the box I use. -- Dmitry Marakasov . 55B5 0596 FF1E 8D84 5F56 9510 D35A 80DD F9D2 F77D amd...@amdmi3.ru ..: jabber: amd...@jabber.ruhttp://www.amdmi3.ru ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: OptionalObsoleteFiles.inc completeness improvement, try 2
* Dmitry Marakasov (amdmi3@hades.panopticon) wrote:
Also there is a question of delete-old-dirs removing directories
which are created by mtree run by installworld unconditionally.
This seems to be incorrect - either directories should be installed
conditionally or not removed by delete-old-dirs. My patch will
address this issue as well, by not remiving unconditionally installed
dirs.
I'd really like to hear opinions on this.
Another issue is conditional lib32 files:
.if ${MK_ATM} == no
...
.if ${TARGET_ARCH} == amd64 || ${TARGET_ARCH} == powerpc64
...
.endif
...
.endif
I think that this may be simplified by dropping internal condition - it
won't change behavior for end-user.
--
Dmitry Marakasov . 55B5 0596 FF1E 8D84 5F56 9510 D35A 80DD F9D2 F77D
amd...@amdmi3.ru ..: jabber: amd...@jabber.ruhttp://www.amdmi3.ru
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: OptionalObsoleteFiles.inc completeness improvement, try 2
On Jan 23, 2015, at 16:29, Dmitry Marakasov amd...@amdmi3.ru wrote: Hi! Some years ago I've started a project of improving OptionalObsoleteFiles.inc completeness, which allows make delete-old / delete-old-libs / delete-old-dirs targets completelty remove files which are normally installed when specific src.conf WITHOUT_* knobs are set. In other words, if a user has some WITHOUT_* set in src.conf, specific files are not installed by installworld, but not removed by remove-old, which I try to fix. In yet other words, I want to make it so `make installworld -DWITHOUT_foo=yes` and `make installworld make delete-old -DWITHOUT_foo=yes` result in the very same file sets. Though the project seems to be useful and have real demand (added to IdeasPage by netchild@, though removed later by brooks@ [1]) and interest ([2]), the change was ignored back then and now the patch is completely rotten. I can redo it, but I need a reviewer. Here's a first small part of the patch: https://reviews.freebsd.org/D1600 The WIP branch with other changes is [3] Also there is a question of delete-old-dirs removing directories which are created by mtree run by installworld unconditionally. This seems to be incorrect - either directories should be installed conditionally or not removed by delete-old-dirs. My patch will address this issue as well, by not remiving unconditionally installed dirs. [1] https://wiki.freebsd.org/action/diff/IdeasPage?action=diffrev1=260rev2=261 [2] https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=168341#c6 [3] https://github.com/AMDmi3/freebsd/compare/obsolete-files Hi Dmitry, Seems like we’ve duplicated work a bit. Have you looked at ^/projects/building-blocks yet ? Thanks! signature.asc Description: Message signed with OpenPGP using GPGMail
Re: OptionalObsoleteFiles.inc completeness improvement, try 2
On Jan 23, 2015, at 17:20, Garrett Cooper yaneurab...@gmail.com wrote: On Jan 23, 2015, at 17:16, Dmitry Marakasov amd...@amdmi3.ru wrote: * Garrett Cooper (yaneurab...@gmail.com) wrote: Some years ago I've started a project of improving OptionalObsoleteFiles.inc completeness, which allows make delete-old / delete-old-libs / delete-old-dirs targets completelty remove files which are normally installed when specific src.conf WITHOUT_* knobs are set. In other words, if a user has some WITHOUT_* set in src.conf, specific files are not installed by installworld, but not removed by remove-old, which I try to fix. In yet other words, I want to make it so `make installworld -DWITHOUT_foo=yes` and `make installworld make delete-old -DWITHOUT_foo=yes` result in the very same file sets. Though the project seems to be useful and have real demand (added to IdeasPage by netchild@, though removed later by brooks@ [1]) and interest ([2]), the change was ignored back then and now the patch is completely rotten. I can redo it, but I need a reviewer. Here's a first small part of the patch: https://reviews.freebsd.org/D1600 The WIP branch with other changes is [3] Also there is a question of delete-old-dirs removing directories which are created by mtree run by installworld unconditionally. This seems to be incorrect - either directories should be installed conditionally or not removed by delete-old-dirs. My patch will address this issue as well, by not remiving unconditionally installed dirs. [1] https://wiki.freebsd.org/action/diff/IdeasPage?action=diffrev1=260rev2=261 [2] https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=168341#c6 [3] https://github.com/AMDmi3/freebsd/compare/obsolete-files Hi Dmitry, Seems like we’ve duplicated work a bit. Have you looked at ^/projects/building-blocks yet ? Hm, seems so, partly. How do you gather missing entries? My way is pretty dumb, I just do bunch of installworlds + delete-old's and add diff to the file, you probably do it more cleverly. Will committing my changes interfere with your work? If so, it may be better to direct them to your branch instead. Especially if you are more aware of knob combinations and their effects. I wrote this script to track what files get installed by directories: https://svnweb.freebsd.org/base/projects/building-blocks/tools/add-optional-obsolete-files-entries.sh?revision=275238view=markup . It’s not perfect (doesn’t work for “kitchen sink” directories like etc/ and share/…), but it’s a reasonable starting point for grabbing all of the files. I’m going to hold off on the rc.d deletions/rewrites for dependent services after I do more targeted review/testing, as it might screw up boot order in unexpected ways with services and programs that get run out of order, but I’m reasonably confident that the contents of the branch work. I was going to start cherry-picking changes from the branch this weekend. Forgot to respond on one important point: reviewing and committing the higher-level changes (make remove-old) shouldn’t be a problem, and I’ll be sure to commit the overlapping changes with OptionalObsoleteFiles.inc first to reduce the diff between our branches. Thanks! signature.asc Description: Message signed with OpenPGP using GPGMail
Re: OptionalObsoleteFiles.inc completeness improvement, try 2
* Garrett Cooper (yaneurab...@gmail.com) wrote: Some years ago I've started a project of improving OptionalObsoleteFiles.inc completeness, which allows make delete-old / delete-old-libs / delete-old-dirs targets completelty remove files which are normally installed when specific src.conf WITHOUT_* knobs are set. In other words, if a user has some WITHOUT_* set in src.conf, specific files are not installed by installworld, but not removed by remove-old, which I try to fix. In yet other words, I want to make it so `make installworld -DWITHOUT_foo=yes` and `make installworld make delete-old -DWITHOUT_foo=yes` result in the very same file sets. Though the project seems to be useful and have real demand (added to IdeasPage by netchild@, though removed later by brooks@ [1]) and interest ([2]), the change was ignored back then and now the patch is completely rotten. I can redo it, but I need a reviewer. Here's a first small part of the patch: https://reviews.freebsd.org/D1600 The WIP branch with other changes is [3] Also there is a question of delete-old-dirs removing directories which are created by mtree run by installworld unconditionally. This seems to be incorrect - either directories should be installed conditionally or not removed by delete-old-dirs. My patch will address this issue as well, by not remiving unconditionally installed dirs. [1] https://wiki.freebsd.org/action/diff/IdeasPage?action=diffrev1=260rev2=261 [2] https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=168341#c6 [3] https://github.com/AMDmi3/freebsd/compare/obsolete-files Hi Dmitry, Seems like we’ve duplicated work a bit. Have you looked at ^/projects/building-blocks yet ? Hm, seems so, partly. How do you gather missing entries? My way is pretty dumb, I just do bunch of installworlds + delete-old's and add diff to the file, you probably do it more cleverly. Will committing my changes interfere with your work? If so, it may be better to direct them to your branch instead. Especially if you are more aware of knob combinations and their effects. -- Dmitry Marakasov . 55B5 0596 FF1E 8D84 5F56 9510 D35A 80DD F9D2 F77D amd...@amdmi3.ru ..: jabber: amd...@jabber.ruhttp://www.amdmi3.ru ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: OptionalObsoleteFiles.inc completeness improvement, try 2
On Jan 23, 2015, at 17:16, Dmitry Marakasov amd...@amdmi3.ru wrote: * Garrett Cooper (yaneurab...@gmail.com) wrote: Some years ago I've started a project of improving OptionalObsoleteFiles.inc completeness, which allows make delete-old / delete-old-libs / delete-old-dirs targets completelty remove files which are normally installed when specific src.conf WITHOUT_* knobs are set. In other words, if a user has some WITHOUT_* set in src.conf, specific files are not installed by installworld, but not removed by remove-old, which I try to fix. In yet other words, I want to make it so `make installworld -DWITHOUT_foo=yes` and `make installworld make delete-old -DWITHOUT_foo=yes` result in the very same file sets. Though the project seems to be useful and have real demand (added to IdeasPage by netchild@, though removed later by brooks@ [1]) and interest ([2]), the change was ignored back then and now the patch is completely rotten. I can redo it, but I need a reviewer. Here's a first small part of the patch: https://reviews.freebsd.org/D1600 The WIP branch with other changes is [3] Also there is a question of delete-old-dirs removing directories which are created by mtree run by installworld unconditionally. This seems to be incorrect - either directories should be installed conditionally or not removed by delete-old-dirs. My patch will address this issue as well, by not remiving unconditionally installed dirs. [1] https://wiki.freebsd.org/action/diff/IdeasPage?action=diffrev1=260rev2=261 [2] https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=168341#c6 [3] https://github.com/AMDmi3/freebsd/compare/obsolete-files Hi Dmitry, Seems like we’ve duplicated work a bit. Have you looked at ^/projects/building-blocks yet ? Hm, seems so, partly. How do you gather missing entries? My way is pretty dumb, I just do bunch of installworlds + delete-old's and add diff to the file, you probably do it more cleverly. Will committing my changes interfere with your work? If so, it may be better to direct them to your branch instead. Especially if you are more aware of knob combinations and their effects. I wrote this script to track what files get installed by directories: https://svnweb.freebsd.org/base/projects/building-blocks/tools/add-optional-obsolete-files-entries.sh?revision=275238view=markup . It’s not perfect (doesn’t work for “kitchen sink” directories like etc/ and share/…), but it’s a reasonable starting point for grabbing all of the files. I’m going to hold off on the rc.d deletions/rewrites for dependent services after I do more targeted review/testing, as it might screw up boot order in unexpected ways with services and programs that get run out of order, but I’m reasonably confident that the contents of the branch work. I was going to start cherry-picking changes from the branch this weekend. Thanks! signature.asc Description: Message signed with OpenPGP using GPGMail
Re: OptionalObsoleteFiles.inc completeness
* Dmitry Marakasov (amd...@amdmi3.ru) wrote: So would someone please commit it? -- Dmitry Marakasov . 55B5 0596 FF1E 8D84 5F56 9510 D35A 80DD F9D2 F77D amd...@amdmi3.ru ..: jabber: amd...@jabber.ruhttp://www.amdmi3.ru ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: OptionalObsoleteFiles.inc completeness
On Fri, 2012-06-01 at 07:20:42 +1000, Peter Jeremy wrote: On 2012-May-30 13:27:03 +1000, Peter Jeremy pe...@rulingia.com wrote: On 2012-May-29 02:18:25 +0400, Dmitry Marakasov amd...@amdmi3.ru wrote: Then you should try to profile it - my script basically runs delete-old delete-old-libs for every knob (131 of them), and it hadn't taken more than 4 seconds even once. I've done some investigating and the problem is that xargs -n1 fork()/exec()s /bin/echo on each file (and there are 5538 files for me). Changing this to tr ' ' '\n' reduces make delete-old runtime to 1.75s - which is much nicer. I've checked a variety of other systems running 8.x 9.x and the 97s seems to be anomalously long so I'll do some more investigating. I've tracked the problem down to excessive VM faults caused by jemalloc. Whilst executing /bin/echo, jemalloc mmap()s two 4MiB chunks of memory. Unless you build with MALLOC_PRODUCTION (which I hadn't), it then proceeds to verify that both blocks are zero-filled. This causes 2048 (unnecessary) page faults (out of a total of 2133). When I rebuilt jemalloc with MALLOC_PRODUCTION, this dropped to 87 page faults (cf 76 an 8.x and 62 on 9.x) and the elapsed time for make delete-old dropped to slightly more than 8.x 9.x. xargs -n1 is probably a worst case scenario for jemalloc but this probably similarly affects other short-lived processes (and the shell scripts that invoke them). It's a pity that this particular test is a compile-time option. I still think that saving 5500 fork()/exec() pairs is a good reason to switch from xargs -n1 to tr ' ' '\n'. Why is xargs even calling /bin/echo when utility is not specified. Shouldn't it just print a certain number of arguments (one in this case)? Uli pgpLas7LXhp2V.pgp Description: PGP signature
Re: OptionalObsoleteFiles.inc completeness
On 2012-Jun-01 20:50:24 +0200, Ulrich Spörlein u...@freebsd.org wrote: Why is xargs even calling /bin/echo when utility is not specified. Because that's what it's documented as doing. Shouldn't it just print a certain number of arguments (one in this case)? The current approach is simpler - there's always utility and it defaults to /bin/echo. Therefore xargs can just always fork/exec. I agree that special-casing the default to have xargs print the relevant number of arguments would be more efficient. -- Peter Jeremy pgpjWzNyZgd8T.pgp Description: PGP signature
Re: OptionalObsoleteFiles.inc completeness
On 2012-May-30 13:27:03 +1000, Peter Jeremy pe...@rulingia.com wrote: On 2012-May-29 02:18:25 +0400, Dmitry Marakasov amd...@amdmi3.ru wrote: Then you should try to profile it - my script basically runs delete-old delete-old-libs for every knob (131 of them), and it hadn't taken more than 4 seconds even once. I've done some investigating and the problem is that xargs -n1 fork()/exec()s /bin/echo on each file (and there are 5538 files for me). Changing this to tr ' ' '\n' reduces make delete-old runtime to 1.75s - which is much nicer. I've checked a variety of other systems running 8.x 9.x and the 97s seems to be anomalously long so I'll do some more investigating. I've tracked the problem down to excessive VM faults caused by jemalloc. Whilst executing /bin/echo, jemalloc mmap()s two 4MiB chunks of memory. Unless you build with MALLOC_PRODUCTION (which I hadn't), it then proceeds to verify that both blocks are zero-filled. This causes 2048 (unnecessary) page faults (out of a total of 2133). When I rebuilt jemalloc with MALLOC_PRODUCTION, this dropped to 87 page faults (cf 76 an 8.x and 62 on 9.x) and the elapsed time for make delete-old dropped to slightly more than 8.x 9.x. xargs -n1 is probably a worst case scenario for jemalloc but this probably similarly affects other short-lived processes (and the shell scripts that invoke them). It's a pity that this particular test is a compile-time option. I still think that saving 5500 fork()/exec() pairs is a good reason to switch from xargs -n1 to tr ' ' '\n'. -- Peter Jeremy pgp66hvYrS7pF.pgp Description: PGP signature
Re: OptionalObsoleteFiles.inc completeness
On 2012-May-29 02:18:25 +0400, Dmitry Marakasov amd...@amdmi3.ru wrote: * Peter Jeremy (pe...@rulingia.com) wrote: My experience is that it now takes about 2½ minutes on 10.x with warm caches, compared to less than 1 second on 8.x. Now = after applying my patch or after changing system? Which knobs were enabled? Now as in -current as against 8.x. But, that 2½ mins was wrong, sorry. I recalled 150s but actually checking, it's really 1:50 (100s). It occurred to me that was an oldish -current (r235127) so I updated to r236183 and the time dropped to 107s. Since this is an oldish P4, I tried a UP kernel and that reduced it to 96s. Your patch made no noticable change (ministat reported no difference with 95% confidence). The system is amd64 with no MK_* knobs defined. Then you should try to profile it - my script basically runs delete-old delete-old-libs for every knob (131 of them), and it hadn't taken more than 4 seconds even once. I've done some investigating and the problem is that xargs -n1 fork()/exec()s /bin/echo on each file (and there are 5538 files for me). Changing this to tr ' ' '\n' reduces make delete-old runtime to 1.75s - which is much nicer. I've checked a variety of other systems running 8.x 9.x and the 97s seems to be anomalously long so I'll do some more investigating. -- Peter Jeremy pgp23vtZvpadf.pgp Description: PGP signature
Re: OptionalObsoleteFiles.inc completeness
On 05/27/2012 07:05, Dmitry Marakasov wrote: Hi! I'm running a little pet project of improving completeness of tools/build/mk/OptionalObsoleteFiles.inc file and thus delete-old* targets with regard to all possible WITHOUT_* knobs. E.g. when WITHOUT_foo is defined in src.conf, make delete-old should remove related files completely, to make a system look exactly like it's world was installed with that knob set. First of all, an automatic script to check for leftovers after delete-old for all possible knobs is available from [2]. Feel free to run in on different architectures and FreeBSD branches; I'm currently running it on amd64. I also think that it should be run during preparation of each FreeBSD release. There are some questions I'd like to discuss. 1) named config file var/named/etc/namedb/named.conf was intentionally left out from OptionalObsoleteFiles.inc, so I did the same for other configs which may be changed by users. That's one reason to omit it. Another is that the bind ports use the same set of config files. -- This .signature sanitized for your protection ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: OptionalObsoleteFiles.inc completeness
On 2012-May-27 18:05:41 +0400, Dmitry Marakasov amd...@amdmi3.ru wrote: 2) Is this ok to backport the list from current to stable branches? Pro - it's really simple, con - it will contain files never installed with this (old) branch. Another con: make delete-old on -current takes about 2 orders of magnitude longer to run than on 8.x. I would prefer to see some effort put into speeding it up before it was backported. -- Peter Jeremy pgptJtyQZ4Lv8.pgp Description: PGP signature
Re: OptionalObsoleteFiles.inc completeness
I'm pro improving completeness, as most people are really surprised when after setting WITHOUT* they are left with old cruft from first install, what's more important- it's getting left as is with all possible (security) bugs. Just be careful, as after recent expansion it looked here like setting WITHOUT_GCC= was breaking gcc47 from ports also, but couldn't really reproduce it reliably. -- View this message in context: http://freebsd.1045724.n5.nabble.com/OptionalObsoleteFiles-inc-completeness-tp5712560p5712976.html Sent from the freebsd-current mailing list archive at Nabble.com. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: OptionalObsoleteFiles.inc completeness
* Doug Barton (do...@freebsd.org) wrote: I'm running a little pet project of improving completeness of tools/build/mk/OptionalObsoleteFiles.inc file and thus delete-old* targets with regard to all possible WITHOUT_* knobs. E.g. when WITHOUT_foo is defined in src.conf, make delete-old should remove related files completely, to make a system look exactly like it's world was installed with that knob set. First of all, an automatic script to check for leftovers after delete-old for all possible knobs is available from [2]. Feel free to run in on different architectures and FreeBSD branches; I'm currently running it on amd64. I also think that it should be run during preparation of each FreeBSD release. There are some questions I'd like to discuss. 1) named config file var/named/etc/namedb/named.conf was intentionally left out from OptionalObsoleteFiles.inc, so I did the same for other configs which may be changed by users. That's one reason to omit it. Another is that the bind ports use the same set of config files. But putting in under OLD_CONFIGS+= would still be ok? -- Dmitry Marakasov . 55B5 0596 FF1E 8D84 5F56 9510 D35A 80DD F9D2 F77D amd...@amdmi3.ru ..: jabber: amd...@jabber.ruhttp://www.amdmi3.ru ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: OptionalObsoleteFiles.inc completeness
* Peter Jeremy (pe...@rulingia.com) wrote: 2) Is this ok to backport the list from current to stable branches? Pro - it's really simple, con - it will contain files never installed with this (old) branch. Another con: make delete-old on -current takes about 2 orders of magnitude longer to run than on 8.x. I would prefer to see some effort put into speeding it up before it was backported. Is that really a reason while it is still under 4 seconds and is not usually run more often than updates (which take minutes if not hours)? -- Dmitry Marakasov . 55B5 0596 FF1E 8D84 5F56 9510 D35A 80DD F9D2 F77D amd...@amdmi3.ru ..: jabber: amd...@jabber.ruhttp://www.amdmi3.ru ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: OptionalObsoleteFiles.inc completeness
On 05/28/2012 12:52, Dmitry Marakasov wrote: * Doug Barton (do...@freebsd.org) wrote: I'm running a little pet project of improving completeness of tools/build/mk/OptionalObsoleteFiles.inc file and thus delete-old* targets with regard to all possible WITHOUT_* knobs. E.g. when WITHOUT_foo is defined in src.conf, make delete-old should remove related files completely, to make a system look exactly like it's world was installed with that knob set. First of all, an automatic script to check for leftovers after delete-old for all possible knobs is available from [2]. Feel free to run in on different architectures and FreeBSD branches; I'm currently running it on amd64. I also think that it should be run during preparation of each FreeBSD release. There are some questions I'd like to discuss. 1) named config file var/named/etc/namedb/named.conf was intentionally left out from OptionalObsoleteFiles.inc, so I did the same for other configs which may be changed by users. That's one reason to omit it. Another is that the bind ports use the same set of config files. But putting in under OLD_CONFIGS+= would still be ok? I have no idea, I don't use the Obsolete stuff, don't like it, and have said since the beginning that it's the totally wrong approach to this issue. The numerous problems we've had with it ever since it was introduced seem to bear me out. :) My point is, until such time as we remove BIND from the base (or I add the config files as a port) I do not want the named config files removed from a user's system. Thanks, Doug -- This .signature sanitized for your protection ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: OptionalObsoleteFiles.inc completeness
On 2012-May-28 23:55:42 +0400, Dmitry Marakasov amd...@amdmi3.ru wrote: * Peter Jeremy (pe...@rulingia.com) wrote: 2) Is this ok to backport the list from current to stable branches? Pro - it's really simple, con - it will contain files never installed with this (old) branch. Another con: make delete-old on -current takes about 2 orders of magnitude longer to run than on 8.x. I would prefer to see some effort put into speeding it up before it was backported. Is that really a reason while it is still under 4 seconds and is not usually run more often than updates (which take minutes if not hours)? My experience is that it now takes about 2½ minutes on 10.x with warm caches, compared to less than 1 second on 8.x. For most of that time, there's no output and there's no warning of the increased time. I actually wrote about the poor performance here a couple of weeks ago. -- Peter Jeremy pgpj1hAqZ4ktC.pgp Description: PGP signature
Re: OptionalObsoleteFiles.inc completeness
On Mon, 28 May 2012 12:59:17 -0700 Doug Barton do...@freebsd.org wrote: this issue. The numerous problems we've had with it ever since it was introduced seem to bear me out. :) Can you list them? A missing obsolete file doesn't count. Bye, Alexander. -- http://www.Leidinger.netAlexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: OptionalObsoleteFiles.inc completeness
On Tue, 29 May 2012 06:04:03 +1000 Peter Jeremy pe...@rulingia.com wrote: My experience is that it now takes about 2½ minutes on 10.x with warm caches, compared to less than 1 second on 8.x. For most of that time, there's no output and there's no warning of the increased time. I actually wrote about the poor performance here a couple of weeks ago. One solution may be to remove some really old files. If we assume that we only support updates to major version X from the last release of major version X-1 we could maybe remove everything which was already obsolete in major version X-1 (or X-2 if you want to play safe). Bye, Alexander. -- http://www.Leidinger.netAlexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: OptionalObsoleteFiles.inc completeness
On 05/28/2012 13:23, Alexander Leidinger wrote: On Mon, 28 May 2012 12:59:17 -0700 Doug Barton do...@freebsd.org wrote: this issue. The numerous problems we've had with it ever since it was introduced seem to bear me out. :) Can you list them? A missing obsolete file doesn't count. It doesn't catch things it needs to It catches things it shouldn't The current incarnation is painfully slow (so I've heard) ... and the biggest problem ... It needs to be updated manually Doug -- This .signature sanitized for your protection ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: OptionalObsoleteFiles.inc completeness
* Doug Barton (do...@freebsd.org) wrote: this issue. The numerous problems we've had with it ever since it was introduced seem to bear me out. :) Can you list them? A missing obsolete file doesn't count. It doesn't catch things it needs to It catches things it shouldn't The current incarnation is painfully slow (so I've heard) ... and the biggest problem ... It needs to be updated manually Pretty true. Still I'd like to fix what we have now, than not to have a useful feature. -- Dmitry Marakasov . 55B5 0596 FF1E 8D84 5F56 9510 D35A 80DD F9D2 F77D amd...@amdmi3.ru ..: jabber: amd...@jabber.ruhttp://www.amdmi3.ru ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: OptionalObsoleteFiles.inc completeness
* Peter Jeremy (pe...@rulingia.com) wrote: 2) Is this ok to backport the list from current to stable branches? Pro - it's really simple, con - it will contain files never installed with this (old) branch. Another con: make delete-old on -current takes about 2 orders of magnitude longer to run than on 8.x. I would prefer to see some effort put into speeding it up before it was backported. Is that really a reason while it is still under 4 seconds and is not usually run more often than updates (which take minutes if not hours)? My experience is that it now takes about 2½ minutes on 10.x with warm caches, compared to less than 1 second on 8.x. Now = after applying my patch or after changing system? Which knobs were enabled? OptionalObsoleteFiles.inc is 3x larger after the patch - it's expected for things to become 3x slower, but definitely not 250x. For most of that time, there's no output and there's no warning of the increased time. I actually wrote about the poor performance here a couple of weeks ago. Then you should try to profile it - my script basically runs delete-old delete-old-libs for every knob (131 of them), and it hadn't taken more than 4 seconds even once. -- Dmitry Marakasov . 55B5 0596 FF1E 8D84 5F56 9510 D35A 80DD F9D2 F77D amd...@amdmi3.ru ..: jabber: amd...@jabber.ruhttp://www.amdmi3.ru ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: OptionalObsoleteFiles.inc completeness
On 5/28/2012 3:05 PM, Dmitry Marakasov wrote: * Doug Barton (do...@freebsd.org) wrote: this issue. The numerous problems we've had with it ever since it was introduced seem to bear me out. :) Can you list them? A missing obsolete file doesn't count. It doesn't catch things it needs to It catches things it shouldn't The current incarnation is painfully slow (so I've heard) ... and the biggest problem ... It needs to be updated manually Pretty true. Still I'd like to fix what we have now, than not to have a useful feature. A question was raised about named.conf, so I answered it. A question was raised about why I don't like/use Obsolete, so I answered it. At no point did I say don't work on Obsolete. That said, my concern about this is the same as my concern about effort being placed into other less-than-desirable solutions. 1. The effort could be better placed elsewhere 2. The fact that $SOMEONE is working on $SOMETHING gives people a warm fuzzy feeling that has a tendency to diminish the urgency towards putting real fixes to real problems. So once again, I'm not saying don't do it. But since someone actually asked for my opinion ... :) Doug -- This .signature sanitized for your protection ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
