Re: Upgrading libs with many dependent ports
Kevin Oberman wrote: I have just been cleaning up the mess caused by the upgrade of xcb-utils. On my systmes I have hundreds of ports that will be re-built by the methods listed in UPDATING, even though the vast majority of them are only dependent on other ports that are dependent on xcb-utils, but don't actually load any of the libraries in xcb-utils. It is a huge waste of time and CPU cycles. I think that I can see two ways of eliminating the rebuilding of ports that don't need it. One is rather manual but can be done now while the other wou;d be automatic, but would need to be written by someone who is far better at writing shell scripts than I. The manual method would be to install sysutils/bsdadminscripts and use a command like `pkg_libchk | grep -E xcb-.+.so | sort tmpfile` to provide a list of ports that actually are linked to the libraries in question. This would be fed into portmaster to rebuild just these ports. (I guess I could use awk and uniq to remove repeats.) Should this become a preferred method of handling this problem? You can use pkg_libchk -oc to have it report rather port name and skip status junk. You don't need uniq, use sort -u. Anyway portmaster will uniq them anyway. -- Sphinx of black quartz judge my vow. ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: Upgrading libs with many dependent ports
On Tue, Jan 17, 2012 at 06:43:27PM -0800 I heard the voice of Kevin Oberman, and lo! it spake thus: Take a look at pkg_chklib. It is quite optimized and runs multiple checks in parallel so that you can run it on 1100 ports in about 1.5 minutes. I've looked at it. From the samples and docs I've seen, it seems to do both more and less than I want. That's not a knock on it, it just goes in a slightly different direction than I want. -- Matthew Fuller (MF4839) | fulle...@over-yonder.net Systems/Network Administrator | http://www.over-yonder.net/~fullermd/ On the Internet, nobody can hear you scream. ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Upgrading libs with many dependent ports
I have just been cleaning up the mess caused by the upgrade of xcb-utils. On my systmes I have hundreds of ports that will be re-built by the methods listed in UPDATING, even though the vast majority of them are only dependent on other ports that are dependent on xcb-utils, but don't actually load any of the libraries in xcb-utils. It is a huge waste of time and CPU cycles. I think that I can see two ways of eliminating the rebuilding of ports that don't need it. One is rather manual but can be done now while the other wou;d be automatic, but would need to be written by someone who is far better at writing shell scripts than I. The manual method would be to install sysutils/bsdadminscripts and use a command like `pkg_libchk | grep -E xcb-.+.so | sort tmpfile` to provide a list of ports that actually are linked to the libraries in question. This would be fed into portmaster to rebuild just these ports. (I guess I could use awk and uniq to remove repeats.) Should this become a preferred method of handling this problem? -- R. Kevin Oberman, Network Engineer E-mail: kob6...@gmail.com ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: Upgrading libs with many dependent ports
On Tue, Jan 17, 2012 at 05:51:11PM -0800 I heard the voice of
Kevin Oberman, and lo! it spake thus:
The manual method would be to install sysutils/bsdadminscripts and
use a command like `pkg_libchk | grep -E xcb-.+.so | sort
tmpfile` to provide a list of ports that actually are linked to the
libraries in question.
FWIW, I some years ago wrote up a quickdirty perl script to find
missing or out of date libs. It pulls out and warns about missing
libs, stuff in compat/pkg (held over after upgrade by
portupgrade/portmaster), and stuff in the base /usr/lib/compat (handy
when crossing major versions, and potentially other big upheavals).
It's only about a k; I'll attach it.
I pretty much wind up ldd'ing /usr/local/{bin/*,sbin/*,lib/*.so*} and
running the results through the script. Usually something like `cd
/usr/local/bin ; ldd * /tmp/ldd.bin ; lddchk.pl /tmp/ldd.bin`.
That tells me the files; then I can use my brain or pkg_which to tell
me which packages are involved. I'm happy with that level of
automation, because I like keeping my brain firmly in the loop on such
things, but it wouldn't be too hard to extend it to do its own walks
over the filesystem, etc.
--
Matthew Fuller (MF4839) | fulle...@over-yonder.net
Systems/Network Administrator | http://www.over-yonder.net/~fullermd/
On the Internet, nobody can hear you scream.
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: Upgrading libs with many dependent ports
On Tue, Jan 17, 2012 at 6:09 PM, Matthew D. Fuller fulle...@over-yonder.net
wrote:
On Tue, Jan 17, 2012 at 05:51:11PM -0800 I heard the voice of
Kevin Oberman, and lo! it spake thus:
The manual method would be to install sysutils/bsdadminscripts and
use a command like `pkg_libchk | grep -E xcb-.+.so | sort
tmpfile` to provide a list of ports that actually are linked to the
libraries in question.
FWIW, I some years ago wrote up a quickdirty perl script to find
missing or out of date libs. It pulls out and warns about missing
libs, stuff in compat/pkg (held over after upgrade by
portupgrade/portmaster), and stuff in the base /usr/lib/compat (handy
when crossing major versions, and potentially other big upheavals).
It's only about a k; I'll attach it.
I pretty much wind up ldd'ing /usr/local/{bin/*,sbin/*,lib/*.so*} and
running the results through the script. Usually something like `cd
/usr/local/bin ; ldd * /tmp/ldd.bin ; lddchk.pl /tmp/ldd.bin`.
That tells me the files; then I can use my brain or pkg_which to tell
me which packages are involved. I'm happy with that level of
automation, because I like keeping my brain firmly in the loop on such
things, but it wouldn't be too hard to extend it to do its own walks
over the filesystem, etc.
Take a look at pkg_chklib. It is quite optimized and runs multiple checks
in parallel so that
you can run it on 1100 ports in about 1.5 minutes. Here is a sample o this
output:
%pkg_libchk | grep -E xcb-.+.so | sort
gok-2.30.1,1: /usr/local/bin/create-branching-keyboard misses
libxcb-atom.so.1
gok-2.30.1,1: /usr/local/bin/create-branching-keyboard misses
libxcb-aux.so.0
gok-2.30.1,1: /usr/local/bin/create-branching-keyboard misses
libxcb-event.so.1
gok-2.30.1,1: /usr/local/bin/gok misses libxcb-atom.so.1
gok-2.30.1,1: /usr/local/bin/gok misses libxcb-aux.so.0
gok-2.30.1,1: /usr/local/bin/gok misses libxcb-event.so.1
nautilus-open-terminal-0.18_4:
/usr/local/lib/nautilus/extensions-2.0/libnautilus-open-terminal.so misses
libxcb-atom.so.1
nautilus-open-terminal-0.18_4:
/usr/local/lib/nautilus/extensions-2.0/libnautilus-open-terminal.so misses
libxcb-aux.so.0
nautilus-open-terminal-0.18_4:
/usr/local/lib/nautilus/extensions-2.0/libnautilus-open-terminal.so misses
libxcb-event.so.1
vlc-1.1.13,3: /usr/local/lib/vlc/plugins/control/libglobalhotkeys_plugin.so
misses libxcb-keysyms.so.1
vlc-1.1.13,3:
/usr/local/lib/vlc/plugins/video_output/libxcb_window_plugin.so misses
libxcb-keysyms.so.1
yelp-2.30.2_1: /usr/local/bin/yelp misses libxcb-atom.so.1
yelp-2.30.2_1: /usr/local/bin/yelp misses libxcb-aux.so.0
yelp-2.30.2_1: /usr/local/bin/yelp misses libxcb-event.so.1
%
And it is already in ports.
--
R. Kevin Oberman, Network Engineer
E-mail: kob6...@gmail.com
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: Upgrading libs with many dependent ports
On 01/17/2012 07:51 PM, Kevin Oberman wrote:
I have just been cleaning up the mess caused by the upgrade of xcb-utils.
On my systmes I have hundreds of ports that will be re-built by the methods
listed in UPDATING, even though the vast majority of them are only
dependent on other ports that are dependent on xcb-utils, but don't
actually load any of the libraries in xcb-utils. It is a huge waste of time
and CPU cycles.
I think that I can see two ways of eliminating the rebuilding of ports that
don't need it. One is rather manual but can be done now while the other
wou;d be automatic, but would need to be written by someone who is far
better at writing shell scripts than I.
The manual method would be to install sysutils/bsdadminscripts and use a
command like `pkg_libchk | grep -E xcb-.+.so | sort tmpfile` to provide
a list of ports that actually are linked to the libraries in question. This
would be fed into portmaster to rebuild just these ports. (I guess I could
use awk and uniq to remove repeats.)
Should this become a preferred method of handling this problem?
You mean something like the attached script?
Type
perl pkg_libchk xcb-util-0.3.6,1
and it tells you all the ports that have xcb-util-0.3.6,1 as a lib-depends.
On my system I get 38 ports that require upgrading. All the other 248
dependencies installed on my system are only run-depends, and so don't
need to be upgraded.
I wrote it using perl, because I think a shell script would be quite a
bit slower. And it is already quite slow as it is.
It looks worth implementing, because it looks like it could save
considerable compilation time. If other people think this tool is
useful, someone could clean up the script (e.g. it isn't PREFIX
friendly, and assumes the port database is /var/db/pkg, etc). It could
also be written as a C program without too much trouble.
#!/usr/bin/perl -w
use strict;
my %lib_found;
my %ldd;
sub pkg_libchk {
my $req=/var/db/pkg/$_[0]/+REQUIRED_BY;
return if ! -e $req;
my @libs;
open(my $C,/var/db/pkg/$_[0]/+CONTENTS) || die;
while ($C) {
if (!/^\@/ /\/(lib[^\/]+\.so)$/) {
push @libs,$1;
}
}
close $C;
return if $#libs==-1;
open(my $R,$req) || die;
while (my $r=$R) {
next if defined($lib_found{$r});
chomp $r;
open(my $C,/var/db/pkg/$r/+CONTENTS) || die;
while (my $f=$C) {
chomp $f;
if ($f !~ /^@/) {
if (!defined($ldd{$f})) {
$ldd{$f} = `ldd /usr/local/$f 2/dev/null`;
}
foreach my $l (@libs) {
if ($ldd{$f} =~/$l/) {
if (!defined($lib_found{$r})) {
print $r\n;
$lib_found{$r}=1;
pkg_libchk($r);
goto DONE;
}
}
}
}
}
DONE:
close($C);
}
close($R);
}
die if ! -e /var/db/pkg/$ARGV[0];
pkg_libchk($ARGV[0]);
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: Upgrading libs with many dependent ports
On 01/17/2012 08:43 PM, Kevin Oberman wrote:
On Tue, Jan 17, 2012 at 6:09 PM, Matthew D. Fullerfulle...@over-yonder.net
wrote:
On Tue, Jan 17, 2012 at 05:51:11PM -0800 I heard the voice of
Kevin Oberman, and lo! it spake thus:
The manual method would be to install sysutils/bsdadminscripts and
use a command like `pkg_libchk | grep -E xcb-.+.so | sort
tmpfile` to provide a list of ports that actually are linked to the
libraries in question.
FWIW, I some years ago wrote up a quickdirty perl script to find
missing or out of date libs. It pulls out and warns about missing
libs, stuff in compat/pkg (held over after upgrade by
portupgrade/portmaster), and stuff in the base /usr/lib/compat (handy
when crossing major versions, and potentially other big upheavals).
It's only about a k; I'll attach it.
I pretty much wind up ldd'ing /usr/local/{bin/*,sbin/*,lib/*.so*} and
running the results through the script. Usually something like `cd
/usr/local/bin ; ldd * /tmp/ldd.bin ; lddchk.pl /tmp/ldd.bin`.
That tells me the files; then I can use my brain or pkg_which to tell
me which packages are involved. I'm happy with that level of
automation, because I like keeping my brain firmly in the loop on such
things, but it wouldn't be too hard to extend it to do its own walks
over the filesystem, etc.
Take a look at pkg_chklib. It is quite optimized and runs multiple checks
in parallel so that
you can run it on 1100 ports in about 1.5 minutes. Here is a sample o this
output:
%pkg_libchk | grep -E xcb-.+.so | sort
gok-2.30.1,1: /usr/local/bin/create-branching-keyboard misses
libxcb-atom.so.1
gok-2.30.1,1: /usr/local/bin/create-branching-keyboard misses
libxcb-aux.so.0
gok-2.30.1,1: /usr/local/bin/create-branching-keyboard misses
libxcb-event.so.1
gok-2.30.1,1: /usr/local/bin/gok misses libxcb-atom.so.1
gok-2.30.1,1: /usr/local/bin/gok misses libxcb-aux.so.0
gok-2.30.1,1: /usr/local/bin/gok misses libxcb-event.so.1
nautilus-open-terminal-0.18_4:
/usr/local/lib/nautilus/extensions-2.0/libnautilus-open-terminal.so misses
libxcb-atom.so.1
nautilus-open-terminal-0.18_4:
/usr/local/lib/nautilus/extensions-2.0/libnautilus-open-terminal.so misses
libxcb-aux.so.0
nautilus-open-terminal-0.18_4:
/usr/local/lib/nautilus/extensions-2.0/libnautilus-open-terminal.so misses
libxcb-event.so.1
vlc-1.1.13,3: /usr/local/lib/vlc/plugins/control/libglobalhotkeys_plugin.so
misses libxcb-keysyms.so.1
vlc-1.1.13,3:
/usr/local/lib/vlc/plugins/video_output/libxcb_window_plugin.so misses
libxcb-keysyms.so.1
yelp-2.30.2_1: /usr/local/bin/yelp misses libxcb-atom.so.1
yelp-2.30.2_1: /usr/local/bin/yelp misses libxcb-aux.so.0
yelp-2.30.2_1: /usr/local/bin/yelp misses libxcb-event.so.1
%
And it is already in ports.
And in my other email, I might have reinvented the wheel!
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
