On Thu, 26 Jul 2007 02:17:50 -0700
Garrett Cooper <[EMAIL PROTECTED]> wrote:
> Yoshihiro Ota wrote:
> > To Whom Slowness of Portupgrade Concerns a Lot:
> >
> > As I got tired of long waiting of portupgrade trying to resolve
> > dependencies, I came up with
> > yet another tool for upgrading FreeBSD ports system. Unlink other tools,
> > it tries to maximize
> > existing resource to maximize its performance. This program attempts to
> > wrap around with
> > another 'make' and expand use of FreeBSD ports system. The heart of ports+
> > is parsing INDEX
> > and +CONTENTS files. The rest is handed to GNU make. I think it comes to
> > a point where I seek
> > wider audiences to test with it.
> >
>
> Hiro,
> You're correct when you say that reading INDEX* does take a long time
> for portupgrade, but the problem is partly with how portupgrade formats
> its version of the INDEX database in BDB format, as well as how it
> doesn't buffer up proper information (pkg database information for
> instance), and guesses at port dependencies (origins, deporigins, etc).
Portupgrade is not only slow reading INDEX file but also on dependency
resolving and updating +CONTENTS files, too. I think portmaster is also one
tries to read and do the same things but with shell script. I personally
didn't have good luck with portmaster and haven't really used to evaluate.
However, "portmaster -a -n" wasn't not fast, neither. By the way, it builds
ports in background, doesn't it?
> 1. Can your solution be made bsdmake and bsd awk compatible?
I am not as familiar in newer BSD awk and make extensions. I can point out
what kinds of GNU extensions are being used so that someone can tell it is also
available in BSD make/awk.
For awk, this feature is required, quoted from "man gawk". I am not sure if
BSD awk implements this feature.
The book indicates that command line variable assignment happens when
awk would otherwise open the argument as a file, which is after the
BEGIN block is executed. However, in earlier implementations, when
such an assignment appeared before any file names, the assignment would
happen before the BEGIN block was run. Applications came to depend on
this "feature." When awk was changed to match its documentation, the
-v option for assigning variables before program execution was added to
accommodate applications that depended upon the old behavior. (This
feature was agreed upon by both the Bell Laboratories and the GNU
developers.)
For make, 1.1 and 1.2 below are in the same topic. I use GNU extension which
keeps generating makefiles until dependency rule saticifies and use it part of
the make rule. GNU extension reloads newer makefiles and reconstruct
dependencies. I am not sure if it is possible in BSD make; however, this is
not a heart of this program. We can work around with bootstrap script or
something.
1.3 This is recursive expansion of variables. If you take a look, every port
needs upgrade uses $(UPGRADE). $(UPGRADE) is expanded multiple times using the
target variable, $@, such that it backups up correct old package names and so
on. This is very important and unless this works in BSD make, my tool really
needed full redesign from scratch.
1.4 is just how GNU make does shell invocations. It is available in most of
Make tools.
1.5 is something I am not sure which makes support these. I don't think GUN
make recommends using special characters as variable names. However, when I
tried, it was too difficult to normalize to alphabets and underscore only. If
we cannot use variable like PORTS+IGNORE as a variable name, it will be very
difficult and personally I don't even think about trying that myself. If you
take a look at compat.gmk which ports+ generates as a part of its build rule,
you can find something like "COMPAT_$(PKG_DIR)Hermes-1.3.3_2 = cp -f
/usr/local/lib/libHermes.so.1 /usr/local/lib/compat/pkg&&" being variable.
1.1 How makefiles get remade:
http://uw713doc.sco.com/cgi-bin/info2html?(make.info)Remaking%2520Makefiles&lang=en
1.2 Including Other Makefiles:
http://uw713doc.sco.com/cgi-bin/info2html?(make.info)Include&lang=en
1.3 Computed Variable Names or recursive expandsion of variables:
http://uw713doc.sco.com/cgi-bin/info2html?(make.info)Computed%2520Names&lang=en
1.4 The `shell' Function
http://uw713doc.sco.com/cgi-bin/info2html?(make.info)Shell%2520Function&lang=en
1.5 Special characters in variable names such as dot, ., comma, ,, slash, /,
and so on.
> 2. Does your solution account for cases when you're trying to install
> package a, which depends on package b, but because you built package b
> while trying to build a, and are at an intermediate package c (between a
> and b), the dependencies for a are only partially complete. Thus when
> you try to install direct or indirect dependences, the install fails?
Some of answers are explain in the GNUmakefile. Anyway, ports+ categorizes all
ports into three, NEW as ones not installed, UPGRADE as installed and newer
version is available, NOOP as installed an no newer version is available. I
only started with upgrading only and haven't really though thought installing
new ones. I am not ready to talk about installing new ports via ports+.
Short answer for upgrade is, "Yes, it does." Here are some examples. Let's
say 'a' depends on 'b'. Then, it builds 'b' first and than 'a'. If new
version of 'b' has new requirement of 'c'. Making 'a' will result installing
'c' and than upgrading 'b.' If 'e' requires 'f', 'g', and 'h' and these are
independent from each other, upgrading 'e' with -j 3 will start upgrading 'f',
'g', and 'h' at the same time and once all three are upgraded, it will start on
'e'.
> 3. How is this different from pkg_version combined with similar scripts?
I wasn't aware of pkg_version, but it looks like it only tells if port is up to
date or not. It doesn't tell anything about dependencies among ports, does it?
If so, it doesn't provide enough information to upgrade.
The difference from other two is the following. Portupgrade and portmaster
"reads" what's in ports, INDEX, and what's installed, +CONTENTS, "solves"
dependencies itself, and "manages" upgrading port itself. Instead, ports+
"converts" INDEX and +CONTENTS to (GNU) Makefile and let the dependency expert
solve the rest.
Hiro
> -Garrett
_______________________________________________
[email protected] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
