Anyone building Seamonkey locally?
Hi. It has been a while since seamonkey was removed from ports. I attempted to build latest seamonkey but it fails with compiler error. I'm wondering if anyone keeping up to date local seamonkey port and also wishing to share. Hiro ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: lang/gawk
Hi Walery, As a maintainer of Japanese/gawk, I tested both Japanese/gawk and lang/gawk. Neither of ports had problems you described. Thanks, Hiro On Wed, 26 Aug 2009 17:53:18 +0400 Walery Kokarev w65...@gmail.com wrote: Hi everybody! Today I tried to 'make install' in /usr/ports/lang/gawk but failed. At the same time 'make build' works properly. Seems gawk needs gmake to install itself. Here is the patch for ports/lang/Makefile. -- Index: Makefile 6a7 22a24,25 USE_GMAKE=yes -- I tried to send this patch to port maintainer Andreas Klemm andr...@klemm.gtn.com, but failed since domain klemm.gtn.com does not exsist. ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: HEADS UP multi processor compilations for everyone
On Wed, 25 Mar 2009 12:57:51 -0400 Coleman Kane cok...@freebsd.org wrote: On Wed, 2009-03-25 at 19:30 +0300, Dmitry Marakasov wrote: * Pav Lucistnik (p...@freebsd.org) wrote: This would break very fast -- it's passing -j3 to port Makefile instead of vendor Makefile. This has worked fine for me for countless years, except where the vendor's Makefiles were not parallel-safe. This has been my trick to get larger things (like mysql or xorg-server) to make in parallel. It *did* work. If this has changed, then it definitely warrants mention in UPDATING. Then it must have worked all these years by pure chance :) Be the way, could anyone clarify how this works? My idea was that passing -j to port Makefile does nothing, as make/gmake on vendor's Makefile is ran without any -j flags - you get usual singlethreaded build. However, I have a broken port, which uses gmake and something like that: sometarget: (cd xxx; make) and that fails with -j (make: illegal option -- -). So is there some magic with recursive make calls and -j? When processing a Makefile, make's that support concurrent operation look for targets that will execute the $(MAKE) program. Whenever a compliant make is run (make or gmake), if it detects $(MAKE) in a rule then it will automatically expand that rule into a child process that has some sort of magical connection to the parent process. The connection allows the different make processes to share the same pool of process count resources amongst each other. I am not sure what the implementation is, but this communication mechanism allows child make processes called with $(MAKE) ... from inside a Makefile to globally only use N children (from -j N), and otherwise block until more free jobs are available amongst their shared job pool. I hope that's clear... Probably more so than the explanation given on GNU make's manual. -- Coleman Kane Indeed, that is why (cd xxx; make) fails where gmake is running this command. The way GNU make and BSD make passes -j related options are different. They are not compatible. If I remember correctly, if GNU make calls BSD make (which is an error of writing Makefiles in most of vendor codes), the differences are significant and fails. Usually, changing make to $(MAKE) fixes the problems. Why did it build when -j was not supplied. That is because makefile rule was not GNU makefile specific such that BSD make could also executed it without problems. With -j switch presented, these two make commands became incompatible each other. I hope this helps if you guys haven't found this facts; I hope I had replied earlier. Regards, Hiro ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: Call for potential ports maintainers
On Wed, 18 Feb 2009 00:56:01 -0500 Thomas Abthorpe tabtho...@freebsd.org wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On February 17, 2009 10:19:34 pm Yoshihiro Ota wrote: Let me try these, first. japanese/gawk lang/mawk lang/nawk Thanks, Hiro They are yours! Thomas Let me take these, too. japanese/kakashi sysutils/phantom Thanks, Hiro ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: Call for potential ports maintainers
On Thu, 12 Feb 2009 12:32:13 -0500 Thomas Abthorpe tabtho...@freebsd.org wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 How do I know this? I was one, a silent observer on the mailing lists, and in on IRC. Then one day, I answered a similar plea, http://lists.freebsd.org/pipermail/freebsd-announce/2006-May/001065.html. The gauntlet has been thrown down, who among you is prepared to pick it up? Thomas Thank you for your hard work, Thomas. Can I make 2 suggestions? 1. Can you ask PR senders if they are intersted in becoming maintainer if port doesn't have a maintaner? Sender was moticvated enough to make a patch. 2. Can you assign two or more maintaners to a port? This may improve response time of PRs. Regards, Hiro ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: Call for potential ports maintainers
On Thu, 12 Feb 2009 12:32:13 -0500 Thomas Abthorpe tabtho...@freebsd.org wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 The gauntlet has been thrown down, who among you is prepared to pick it up? Let me try these, first. japanese/gawk lang/mawk lang/nawk Thanks, Hiro ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: jakarta-tomcat v.s. tomcat
On Tue, 11 Nov 2008 07:44:17 +0100 Gábor Kövesdán [EMAIL PROTECTED] wrote: Yoshihiro Ota escribió: Hello, forks. There are 2 directories for apache-tomcat like the following. % ls -d */*tomcat* java/eclipse-sysdeo-tomcat www/tomcat41 www/jakarta-tomcat4 www/tomcat55 www/jakarta-tomcat5 www/tomcat6 www/tomcat-native When I diffed ver. 5 dirs, they looked similar except versions. Does anyone know the differences between jakarta-tomcat and tomcat under the ports? Afaik, it used to be known as Jakarta-Tomcat, but its new name is Tomcat. If you observe it well www/tomcat41 and www/tomcat55 are much newer version than www/jakarta-tomcat4 and www/jakarta-tomcat5 and there isn't even www/jakarta-tomcat6. I suggest you should just use www/tomcat6. I'm also planning to use it and I succeeded to install it without problems and I can reach it through the port 8180. Indeed, that is one of the answer I had expected. So, if www/tomcat* are the new ones, why don't we remove www/jakarta-tomcat*? Hiro ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to [EMAIL PROTECTED]
jakarta-tomcat v.s. tomcat
Hello, forks. There are 2 directories for apache-tomcat like the following. % ls -d */*tomcat* java/eclipse-sysdeo-tomcat www/tomcat41 www/jakarta-tomcat4 www/tomcat55 www/jakarta-tomcat5 www/tomcat6 www/tomcat-native When I diffed ver. 5 dirs, they looked similar except versions. Does anyone know the differences between jakarta-tomcat and tomcat under the ports? Thanks, Hiro ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: ports/113132 (make -j patch)
On Wed, 12 Mar 2008 15:45:46 +0100 Benjamin Lutz [EMAIL PROTECTED] wrote: On Wednesday 12 March 2008 14:01:57 Florent Thoumie wrote: On Wed, Mar 12, 2008 at 12:11 PM, Benjamin Lutz [EMAIL PROTECTED] wrote: This patch has been sitting in GNATS for a couple of months now: http://www.freebsd.org/cgi/query-pr.cgi?pr=ports/113132 I've received a few mails from people reporting success, and none reporting that bad things have happened. Is it possible to get this committed? It needs to go through an experimental build first. IMHO, this is an ugly hack. Ultimately, we're talking about marking almost 20k ports as parallel-safe. Because in requires modifications of individual port Makefiles? Or is there something else in it that you don't like? snip... Also, the amount of work required to gete an advantage from this change is actually not as high as you think. Of the 20k ports we have, only the big ones (e.g., kde*) gain a substantial benefit from -j building. Small ports that only compile for a few seconds anyway can be left as they are now. You may do better by or together with ports+, http://uyota.asablo.jp/blog/cat/portsplus/ . It allows to fetch and build independent ports at parallel. So, not only big ones but also small ones make benefit. It does not build ports in parallel. So, it won't break anything as they are now. Cheers, Hiro ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: please test qemu-devel port update (2008-03-02 snapshot)
On Tue, 4 Mar 2008 23:38:22 +0100 Juergen Lock [EMAIL PROTECTED] wrote: On Sun, Mar 02, 2008 at 09:21:20PM +0100, Juergen Lock wrote: Hi! Anyway the update is here (also queued on tb3): http://people.freebsd.org/~nox/qemu/qemu-devel-20080302.patch Updated patch available, fixed the 7.0 amd64 iso faults (indeed it was a tcg bug, I can boot livefs amd64 also on i386 now and enter fixit), and added the gnutls dependency knob that I just commited to the 0.9.1 qemu port too. enjoy, Juergen I also started using with your patch. I haven't had any problems so far; although I haven't tried anything new, yet. Thanks, Hiro ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Problems with icu - 3.8
On Sun, 17 Feb 2008 14:23:37 -0700 James [EMAIL PROTECTED] wrote: On Sun, 2008-02-17 at 21:14 +, Matthew Seaman wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Atanas Gendov wrote: Greetings to all from FreeBSD Ports!!! :) I have some problems with icu - 3.8. After portupgrade of icu and some others packages I can't start Gnome. Because of Gnome needs module from icu - 3.6. Unfortunately there is no new port for gdm which doesn't have depends on icu-3.6. Lots of people are getting bitten by the effects of the bump in the libicu* shlib ABI version numbers -- there really should be a note in UPDATING about it. It affects all sorts of different packages -- I've even got one system where it took out OpenLDAP. Do you know who to ask for the note? I was bitten by it, but fortunately there had just been a large thread in questions@ about it. It does seem to be a common issue. James Why are so many people are bitten by this? Is that the jobs of port-upgrading tool to safe copy these libraries to compat so that all programs using the old libraries works? Hiro ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to [EMAIL PROTECTED]
Parallelized Port Upgrade Tool Announcement
Dear Multi-core Computer Users: Ports+ is the first and the only port upgrading tool to fetch, configure, compile, and install packages via ports in parallel. There are some tools that allows downloading archives in parallel. However, none of others builds, nor installs packages in concurrent fashion, despite its high demand. This is an announcement of the first stable ports+ tool for these who wants to utilize all CPUs, disk actives, and network bandwidth at maximum. Indeed, I asked some inputs on this list about a half year ago. Thank you for the feedbacks. Now, ports+ has its front-end, fixed number of bugs, and more polished. The purpose of Ports+ is not a complete solution but rather co-existence with others tools such as portupgrade. Ports+ provides speed at cost of user interaction. That is like a car is a better vehicle than an airplane for daily life. However, you would like to take an airplane for a long trip. You won't drive from New York to California for a Thanksgiving weekend. You have big steps with ports+ and adjust small pieces with like portupgrade. More reading materials are available via following links. The Ports+ Project Design and Goal http://uyota.asablo.jp/blog/2008/01/02/2541411 The Ports+ Architecture/Implementation http://uyota.asablo.jp/blog/2008/01/03/2543196 Daily Ports+ http://uyota.asablo.jp/blog/2008/01/16/2563161 http://uyota.asablo.jp/blog/2008/01/17/2564930 http://uyota.asablo.jp/blog/2008/01/18/2566268 http://uyota.asablo.jp/blog/2008/01/20/2569358 http://uyota.asablo.jp/blog/2008/01/21/2571368 http://uyota.asablo.jp/blog/2008/01/22/2572683 http://uyota.asablo.jp/blog/2008/02/01/2594680 Feedbacks are welcome. Thanks, Hiro ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Compiling utilizing multiple CPUs
Yes, there is. Take a look at http://uyota.asablo.jp/blog/cat/portsplus/. I have written the program and finally came to a satisfying stage. I was thinking to seek for testers after writing some more tips and examples. You can give a try. I've been using/developing it for a half year. Hiro On Tue, 15 Jan 2008 00:55:29 +0100 Christoffer Strömblad [EMAIL PROTECTED] wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, I've recently begun to explore the world of FreeBSD and all its wonderful things. (Who thought it would even be possible to actually get excited about a port upgrade?) Having looked through much of the available documentation one thing continues to elude me... Is it possible to specify globally how many CPUs are available when compiling a port? When I compile a port now it seems as if only one CPU is used, even though more are available. My apologies if I am missing something obvious, but I've tried to the best of my abilities to find the answer but without success. Many thanks, Chris ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Limitations of Ports System
On Thu, 13 Dec 2007 21:58:57 +0100 Erik Trulsson [EMAIL PROTECTED] wrote: One shortcoming is the lack of locking making parallell builds a bit unsafe. If you try to build both port A and port B at the same time, and both A and B depends (directly or indirectly) on port C which is not installed, then you can esily end up having two processes both trying to build C at the same time. This usually confuses both builds very badly making them fail. I also don't think there is any locking on /var/db/pkg making possibly somewhat unsafe trying to register the installation of two ports/packages at the same time. I have never noticed any actual problems with this though. Some sort of locking, making parallel builds safe, should be possible to add to the ports system without doing any sweeping changes. (I did look briefly at the makefiles, but did not find any obvious place to put the locking. I probably just did not look hard enough.) The ports system is to install a new port. It won't be easy to accomplish what you suggest. For example, dependencies are checked one at a time. So, even if you want to run multiple processes on LIB_DEPENDS, there is no easy way to control CPU load. It is a better idea for other ports UPGRADE utilities to take care of your suggestions. Indeed, I have been developing such utility myself. If you want to try, I can give out for testing. There are 2 known issues with my tool, yet: 1. no good way to run 'make config', yet, and 2. even if less LIB_DEPENDS are required due to less selected OPTIONS, my tool does not fully eliminate these dependencies. Hiro ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: problem with mount_ntfs-3g
This a bit off the topic; however, I pointed out the problem of this mount_XXX
behaviour quite long time ago. It currently maintains a list of NON-supported
FS by nmount(2). Instead, it's better to maintain a list of SUPPORTED FS by
nmount.
In the current scheme, mount has to be aware of all the FSs that nmount(2)
cannot
support. In other words, if one attempts to add a new FS, he/she has to modify
mount(8) program. That doesn't sound right, does it?
I attached the patch, again, if anyone is interested in commiting this fix.
Thanks,
Hiro
On Fri, 30 Nov 2007 11:18:08 +0800
Yi Wang [EMAIL PROTECTED] wrote:
Yes, I think it does the trick.
some words from mount(8):
The default behavior of mount is to pass the -t option directly to the
nmount(2) system call in the fstype option.
However, for the following file system types: cd9660, mfs, msdosfs,
nfs, nfs4, ntfs, nwfs, nullfs, portalfs, smbfs, udf, and unionfs,
mount will not call nmount(2) directly and will instead attempt to
execute a program in /sbin/mount_XXX where XXX is replaced by the file
system type name.
according to this, your patch should works fine.
I didn't test it yet because I delete /usr/obj before. So I'm build world now.
I'll post the result when it finish.
On 11/30/07, Suhaib . [EMAIL PROTECTED] wrote:
- Original Message -
From: Alejandro Pulver [EMAIL PROTECTED]
To: Aryeh M. Friedman [EMAIL PROTECTED]
Subject: Re: problem with mount_ntfs-3g
Date: Thu, 29 Nov 2007 20:16:14 -0300
On Thu, 29 Nov 2007 19:42:00 -0300
Alejandro Pulver [EMAIL PROTECTED] wrote:
On Tue, 27 Nov 2007 09:35:48 -0500
Aryeh M. Friedman [EMAIL PROTECTED] wrote:
Then, I execute this command:
mount -t ntfs-3g -o ro /dev/ad6s10 /media/g
but is also says:
mount: /dev/ad6s10 : Operation not supported by device
Is this a bug of mount?
I get exactly the same thing on -current AMD64.
uname -a:
FreeBSD wangyi.com 7.0-BETA3 FreeBSD 7.0-BETA3 #0: Fri Nov 23
12:18:30
CST 2007 [EMAIL PROTECTED]:/usr/obj/usr/src/sys/MYKERNEL i386
Can't reproduce in my machine, please try with the latest version
(attached as shell archive).
Sorry, it was stripped. It's available here:
ftp://ftp.alepulver.com.ar/fusefs-ntfs.shar
Best Regards,
Ale
signature.asc
I had the same problem on RELENG_7 and made a minor edit to mount
for ntfs-3g to work. However,I have not had the time to investigate
whether or not it is the correct solution, see the comment about
nmount() in mount.c
To try out the change apply the attached patch file from /usr/src,
then cd to /usr/src/sbin/mount and run make. A quick test...
# mount -t ntfs-3g -o ro /dev/da8s1 /mnt
mount: /dev/da8s1 : Operation not supported by device
# /usr/obj/usr/src/sbin/mount/mount -t ntfs-3g -o ro /dev/da8s1 /mnt
--
Suhaib
--
___
Surf the Web in a faster, safer and easier way:
Download Opera 9 at http://www.opera.com
Powered by Outblaze
--
Regards,
Wang Yi
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to [EMAIL PROTECTED]
Index: mount.c
===
RCS file: /home/ncvs/src/sbin/mount/mount.c,v
retrieving revision 1.92
diff -u -r1.92 mount.c
--- mount.c 14 Nov 2006 01:07:42 - 1.92
+++ mount.c 1 Feb 2007 05:25:55 -
@@ -133,18 +133,19 @@
*/
unsigned int i;
const char *fs[] = {
- cd9660, mfs, msdosfs, nfs, nfs4, ntfs,
- nwfs, nullfs, portalfs, smbfs, udf, umapfs,
- unionfs,
+ ufs,
+ ext2fs,
+ devfs, fdescfs, procfs, linprocfs, linsysfs,
+ std,
NULL
};
for (i = 0; fs[i] != NULL; ++i) {
if (strcmp(vfstype, fs[i]) == 0)
- return (1);
+ return (0);
}
- return (0);
+ return (1);
}
static int
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Call for testers for yet another ports upgrade program, ports+
On Sun, 29 Jul 2007 02:35:30 -0500
Matthew D. Fuller [EMAIL PROTECTED] wrote:
On Sat, Jul 28, 2007 at 10:14:28PM -0400 I heard the voice of
Yoshihiro Ota, and lo! it spake thus:
Matthew D. Fuller [EMAIL PROTECTED] wrote:
No, it's not. Lots of ports change dependancies based on what's
installed, so if you install one thing you have to assume that any
other thing you intend to build in the future may change its list
because of that. For a trivial demonstration, look at how many
use the .if ${HAVE_GNOME:Msomething} pattern.
It doesn't matter how many you have. There are only finite number
of ports and configurations. You will finish configuring at one
point.
No, you mistake me. I'm not talking about ports _configuring_. I'm
talking about ports _installing_.
You can recursively configure all the ports, but the dependancies can
still change AFTER that, when other ports get INSTALLED. As soon as
you install one port in your process, every other port you're dealing
with can potentially change its dependancies, with NO actual user
interaction or configuration.
I don't think I understand very well on this issue. I have a couple
of questions.
1. Is this about RUN_DEPENDS in makefiles?
2. Is this about DEPENDS in makefiles?
3. Or, is this about the configure scripts detect extra
libraries at run time?
4. If so, does the ports system keep track of such information?
At least, I couldn't find anything alarming of this in the makefiles.
Is that something the pkg_* tools detect?
5. Do you know how portupgrade or portmaster extract such dependencies?
6. Can you give me an easy example of such port or scenario?
Thanks,
Hiro
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Call for testers for yet another ports upgrade program, ports+
On Fri, 27 Jul 2007 21:49:39 -0700 Doug Barton [EMAIL PROTECTED] wrote: Hiro, I'm happy to respond to you, but first I'd like to make clear that I'm not trying to talk you out of anything. If there is a better way to manage ports, or even just a different approach, I'm all for it. I don't think portmaster is a one size fits all tool, and I'm not trying to make it one. Doug, The same goes here. With my approach, if I can handle let's say 90% of cases, that's fair enough. But, let it do as FAST as possible. If one claims he has 32 CPU or 32 cores, let him use all. And let other tools like portupgrade and portmaster handle difficult cases. They are better for intarctive use. For instance, I can never do --recursive option in portupgrade in ports+. It is something impossible for a 'make' utiltiy to do. Yoshihiro Ota wrote: On Fri, 27 Jul 2007 00:57:34 -0700 Doug Barton [EMAIL PROTECTED] wrote: Yoshihiro Ota wrote: Could you tell me a bit more or point to a source if already written on how portmaster works. The man page has a good overview, I have documented it relatively thoroughly, and I keep it up to date. You can either install the port, or do: I will do so. However, portmaster -a -n wasn't not fast, neither. I should probably point out that this is the worst case scenario, since by definition portmaster -a has to evaluate each installed port. The vast majority of the time spent doing that though is in 'make -V PORTVERSION'. I see. When I just started ports+, I also attempted doing like that. However, when I looked into bsd.port.mk, I found that the ports system uses the INDEX file for pretty-print-run-depends-list. Then, I decided to read the INDEX file myself for 1. it's faster to parse it myself, 2. more information is available, and 3. prevent generating processes. The benefit comes when you actually start building stuff and because all the information about the up to date ports is cached, it won't have to be reevaluated. What do you mean by cached? Is it file-cache by VM? Is it something the ports system do? Or is it something else? I think it's useful and healthy to discuss where both the various tools, and the infrastructure can be improved. Sure. I took what portupgrade is doing to upgrade ports to make ports+. I.e, saving dynamic libraries, and so on. When you mention something like 'make -V PORTVERSION', it gives a good idea how to get something out of the port system. It just occurred that as the ports system generates INDEX and ports+ uses it to write out Makefiles, the ports system might be able to write Makefiles for upgrading, too. Hiro ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Call for testers for yet another ports upgrade program, ports+
On Sat, 28 Jul 2007 10:59:36 -0500
Matthew D. Fuller [EMAIL PROTECTED] wrote:
On Fri, Jul 27, 2007 at 09:48:47PM -0400 I heard the voice of
Yoshihiro Ota, and lo! it spake thus:
Q. Is it safe to assume all dependencies are STATIC?
A. Yes, it is.
No, it's not. Lots of ports change dependancies based on what's
installed, so if you install one thing you have to assume that any
other thing you intend to build in the future may change its list
because of that. For a trivial demonstration, look at how many use
the .if ${HAVE_GNOME:Msomething} pattern.
It doesn't matter how many you have. There are only finite number of
ports and configurations. You will finish configuring at one point.
With GNU extension, one can write makefile dependencies such
that it keeps tracking and generating makefiles until all ports
to be installed are configured.
The key is how one will process /var/db/ports/* files.
I'd say it wouldn't be an easy one, though.
Ports+ doesn't do it, yet.
Check the link below if you'd like to know more. The 2nd sentence
explains the concept of doing this.
Remaking Makefiles:
http://uw713doc.sco.com/cgi-bin/info2html?(make.info)Remaking%2520Makefileslang=en
Hiro
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Call for testers for yet another ports upgrade program, ports+
On Fri, 27 Jul 2007 01:47:14 -0700 Garrett Cooper [EMAIL PROTECTED] wrote: Yoshihiro Ota wrote: On Thu, 26 Jul 2007 02:17:50 -0700 Garrett Cooper [EMAIL PROTECTED] wrote: Yoshihiro Ota wrote: Portupgrade is not only slow reading INDEX file but also on dependency resolving and updating +CONTENTS files, too. I think portmaster is also one tries to read and do the same things but with shell script. I personally didn't have good luck with portmaster and haven't really used to evaluate. However, portmaster -a -n wasn't not fast, neither. By the way, it builds ports in background, doesn't it? Yes, but a lot of time is spent reading the INDEX file, especially since portupgrade / portinstall don't keep that information in memory (probably for the better though). I, so far, only observe the behavior of portupgrade. I know it took so long doing something and guessed it was trying to resolve dependencies. It seems my guess wasn't quite right. Like Doug said, portmaster doesn't built ports in the background, and the way that the package system is currently setup I don't advise doing that unless you plan everything out properly (I see potential for duplicated installs, race conditions, and other unwanted possibilities). I'm working on getting away from that at the moment. Even portupgrade doesn't work on some of these cases anyway. I had problems with: The default ghostscript port is changed from ghostscript-gnu to ghostscript-gpl. To upgrade portupgrade -o print/ghostscript-gpl ghostscript-gnu I had two ports required ghostscript-gnu and portupgrade couldn't handle this situation with this command. I guess I also need -r or something; now I come to think about it. I will try -r later. I do see your variable naming scheme as a cause for concern though, unfortunately. I steer clear of shell-like configurations like this when you can potentially get malicious text inside a variable name / declaration. I didn't like it, either. ;) Normalizing got so ugly, I gave up and decided to use them as it is. 2. Does your solution account for cases when you're trying to install package a, which depends on package b, but because you built package b while trying to build a, and are at an intermediate package c (between a and b), the dependencies for a are only partially complete. Thus when you try to install direct or indirect dependences, the install fails? Some of answers are explain in the GNUmakefile. Anyway, ports+ categorizes all ports into three, NEW as ones not installed, UPGRADE as installed and newer version is available, NOOP as installed an no newer version is available. I only started with upgrading only and haven't really though thought installing new ones. I am not ready to talk about installing new ports via ports+. Short answer for upgrade is, Yes, it does. Here are some examples. Let's say 'a' depends on 'b'. Then, it builds 'b' first and than 'a'. If new version of 'b' has new requirement of 'c'. Making 'a' will result installing 'c' and than upgrading 'b.' If 'e' requires 'f', 'g', and 'h' and these are independent from each other, upgrading 'e' with -j 3 will start upgrading 'f', 'g', and 'h' at the same time and once all three are upgraded, it will start on 'e'. That's really good thinking sir for single installs. Props to you for that :). 3. How is this different from pkg_version combined with similar scripts? I wasn't aware of pkg_version, but it looks like it only tells if port is up to date or not. It doesn't tell anything about dependencies among ports, does it? If so, it doesn't provide enough information to upgrade. Every time you run make install, portmaster, or portupgrade, there's a lot of stuff going on behind the scenes. One of the things involved is installing package information in /var/db/pkg. That's where pkg_version gets its information from, along with portmaster (AFAIK) and the makefiles for ports (for sure). I could tell that 'make install' does a lots of things looking at top(1) how quickly 'last pid' increases during that time. I sometime picked up that pkg_create and pkg_info were running during that time in top(1). I was also surprised that as I upgraded KDE or GNOME, pkg_delete started taking longer and longer as I reached to the end of packages. It does provide enough info to tell whether or not something needs to be upgraded, based on the output (, =, or ). It doesn't tell you the official target you're upgrading to, but it doesn't really need to do that though. It tells 'which one' to upgrade but not 'in which order' to upgrade, doesn't it? The difference from other two is the following. Portupgrade and portmaster reads what's in ports, INDEX, and what's installed, +CONTENTS, solves dependencies itself, and manages upgrading port itself. Instead, ports
Re: Call for testers for yet another ports upgrade program, ports+
On Fri, 27 Jul 2007 00:57:34 -0700 Doug Barton [EMAIL PROTECTED] wrote: Yoshihiro Ota wrote: I think portmaster is also one tries to read and do the same things but with shell script. Not exactly the same things. Portmaster doesn't keep any external database, it only updates what is in /var/db/pkg. Could you tell me a bit more or point to a source if already written on how portmaster works. I personally didn't have good luck with portmaster and haven't really used to evaluate. I'm sorry to hear that. If you're interested, please feel free to start another thread that describes your issues. My problem was obsolete ports. I think I need to put +IGNORE_ME file for such ports, but I haven't spent much time on portmaster so far yet. However, portmaster -a -n wasn't not fast, neither. Well, I'm not sure when you last tried it, but I've implemented a lot of caching features in the past year, so nowadays almost all of the time spent running portmaster is actually spent in the ports tree, most of that in building the port. I did about a half year ago and a couple days ago. I don't think I am familar enough to evaluate portsmaster. It is true that for big ports such as KDE, openoffice, java, mozilla-drivens like seamonkey and firefox take long time to compile from source. That is where most time is spent for such ports for ports+ as well. However, when I upgraded X11 to 7.2 and perl script ports, portupgrade spent most of time, like 5 to 10 minutes for each port, and each port build is done in a coupe seconds to a couple minutes. That's when I started looking into speeding up port-upgrading-management processes. My goal is to minimize the resource by which ever port-upgrading tool is and maximize the resource for the 'FreeBSD ports' system doing its job. For example, for instance, use all CPUs and cores if possible. By the way, it builds ports in background, doesn't it? No, it only builds in the foreground. I've thought about various ways to try and implement parallel builds, but it's a pretty hard problem to solve in a reliable way. The reasons why have been covered in other threads, so you might want to check the archives. Portmaster _does_ download new distfiles in the background though, as well as a few other functions that aren't typically user visible. I see. That's where my confusion was. Hiro ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Call for testers for yet another ports upgrade program, ports+
On Thu, 26 Jul 2007 02:17:50 -0700 Garrett Cooper [EMAIL PROTECTED] wrote: Yoshihiro Ota wrote: To Whom Slowness of Portupgrade Concerns a Lot: As I got tired of long waiting of portupgrade trying to resolve dependencies, I came up with yet another tool for upgrading FreeBSD ports system. Unlink other tools, it tries to maximize existing resource to maximize its performance. This program attempts to wrap around with another 'make' and expand use of FreeBSD ports system. The heart of ports+ is parsing INDEX and +CONTENTS files. The rest is handed to GNU make. I think it comes to a point where I seek wider audiences to test with it. Hiro, You're correct when you say that reading INDEX* does take a long time for portupgrade, but the problem is partly with how portupgrade formats its version of the INDEX database in BDB format, as well as how it doesn't buffer up proper information (pkg database information for instance), and guesses at port dependencies (origins, deporigins, etc). Portupgrade is not only slow reading INDEX file but also on dependency resolving and updating +CONTENTS files, too. I think portmaster is also one tries to read and do the same things but with shell script. I personally didn't have good luck with portmaster and haven't really used to evaluate. However, portmaster -a -n wasn't not fast, neither. By the way, it builds ports in background, doesn't it? 1. Can your solution be made bsdmake and bsd awk compatible? I am not as familiar in newer BSD awk and make extensions. I can point out what kinds of GNU extensions are being used so that someone can tell it is also available in BSD make/awk. For awk, this feature is required, quoted from man gawk. I am not sure if BSD awk implements this feature. The book indicates that command line variable assignment happens when awk would otherwise open the argument as a file, which is after the BEGIN block is executed. However, in earlier implementations, when such an assignment appeared before any file names, the assignment would happen before the BEGIN block was run. Applications came to depend on this feature. When awk was changed to match its documentation, the -v option for assigning variables before program execution was added to accommodate applications that depended upon the old behavior. (This feature was agreed upon by both the Bell Laboratories and the GNU developers.) For make, 1.1 and 1.2 below are in the same topic. I use GNU extension which keeps generating makefiles until dependency rule saticifies and use it part of the make rule. GNU extension reloads newer makefiles and reconstruct dependencies. I am not sure if it is possible in BSD make; however, this is not a heart of this program. We can work around with bootstrap script or something. 1.3 This is recursive expansion of variables. If you take a look, every port needs upgrade uses $(UPGRADE). $(UPGRADE) is expanded multiple times using the target variable, $@, such that it backups up correct old package names and so on. This is very important and unless this works in BSD make, my tool really needed full redesign from scratch. 1.4 is just how GNU make does shell invocations. It is available in most of Make tools. 1.5 is something I am not sure which makes support these. I don't think GUN make recommends using special characters as variable names. However, when I tried, it was too difficult to normalize to alphabets and underscore only. If we cannot use variable like PORTS+IGNORE as a variable name, it will be very difficult and personally I don't even think about trying that myself. If you take a look at compat.gmk which ports+ generates as a part of its build rule, you can find something like COMPAT_$(PKG_DIR)Hermes-1.3.3_2 = cp -f /usr/local/lib/libHermes.so.1 /usr/local/lib/compat/pkg being variable. 1.1 How makefiles get remade: http://uw713doc.sco.com/cgi-bin/info2html?(make.info)Remaking%2520Makefileslang=en 1.2 Including Other Makefiles: http://uw713doc.sco.com/cgi-bin/info2html?(make.info)Includelang=en 1.3 Computed Variable Names or recursive expandsion of variables: http://uw713doc.sco.com/cgi-bin/info2html?(make.info)Computed%2520Nameslang=en 1.4 The `shell' Function http://uw713doc.sco.com/cgi-bin/info2html?(make.info)Shell%2520Functionlang=en 1.5 Special characters in variable names such as dot, ., comma, ,, slash, /, and so on. 2. Does your solution account for cases when you're trying to install package a, which depends on package b, but because you built package b while trying to build a, and are at an intermediate package c (between a and b), the dependencies for a are only partially complete. Thus when you try to install direct or indirect dependences, the install fails? Some of answers are explain in the GNUmakefile. Anyway, ports+ categorizes all ports
Call for testers for yet another ports upgrade program, ports+
To Whom Slowness of Portupgrade Concerns a Lot: As I got tired of long waiting of portupgrade trying to resolve dependencies, I came up with yet another tool for upgrading FreeBSD ports system. Unlink other tools, it tries to maximize existing resource to maximize its performance. This program attempts to wrap around with another 'make' and expand use of FreeBSD ports system. The heart of ports+ is parsing INDEX and +CONTENTS files. The rest is handed to GNU make. I think it comes to a point where I seek wider audiences to test with it. Some short summaries of ports+ * Magnitude quicker dependency resolving compare to portupgrade. * Dependencies are resolved only once for all upgrades where portupgrade does before each 'make deinstall and make install' of ports. * Commands to execute for each ports are very flexible. * Allow concurrent builds of ports with -j option. (yet experimental) Some dry runs comparisons with about 500 ports insalled on Mobile Intel(R) Celeron(R) CPU 2.40GHz. Note that programs were run twice in a row after rebooting for each program. That is to kill disk-cache effect from the other program. The first run takes longer to read disk; however, the second run only accessed via file-cache. # time portupgrade -n -a -O 60.901u 36.208s 2:30.08 64.6% 147+13408k 4840+59io 77pf+0w # time portupgrade -n -a -O 83.581u 47.805s 2:51.34 76.6% 155+13197k 547+33io 0pf+0w # time gmake -n all 4.784u 1.982s 0:22.06 30.6% 258+2485k 1528+126io 9pf+0w # time gmake -n all 1.551u 0.205s 0:01.85 94.5% 179+8540k 0+0io 0pf+0w Three advises before trying: 1. Read the description in GNUmakefile and set PORTS+_IGNORE ports+.conf for ones you don't want to upgrade. 2. At least, backup /var/db/pkg so that you can try again even with other tools. 3. Do not 'make clean' so that you can try installing again. Ports+ requires: 1. GNU make, gmake. 2. GNU awk, gawk. 3. Up to date INDEX file. Known Issues: 1. Concurrent build with -j is still experimental although I always use it myself. There is a race condition when more than two ports updating the same +CONTENTS files at the same time. This could result +CONTENTS file out of sync. However, you could try at your risk, too. 2. If OPTIONS changes dependences of a port, ports+ won't handle it at this moment. 3. I don't know if I have to maintain any other file other than +CONTENTS. For these who want to read the description in the archive without getting one, it is copied and available at: http://uyota.asablo.jp/blog/2007/07/26/1681765 For these who want to give a shot wihtout reading further. Try these commands below. These two gmake examples are dry runs and won't harm your system. You can try with 'gmake -n' and once you are confident, you can take -n option out. # fetch http://www.ne.jp/asahi/export/uyota/src/ports+-1.59.tar.bz2 # tar xvf ports+-1.59.tar.bz2 # cd ports+1.59 # gmake test # gmake -n all Any feed backs are welcome including it works. I will applicate bug reports, suggestions on anything or specially solving +CONTENTS file race condition on parallel builds. I am indeed wondering if there is a way to queue or batch commands to run at even high load. Thanks, Hiro ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Speedup for make clean-depends (and thus make clean)
This is the fastest one. It is about O(1) and takes a couple of seconds to delete files whether millions or billions. 1.) Set WRKDIRPREFIX like WRKDIRPREFIX=/ports in /etc/make.conf 2.) Give a device to it. It could be via mdconfig as well. For example, mount /dev/ad0s2c /ports 3.) Build which ever you need but NEVER run clean during this time. 4.) Let's clean with umount /ports newfs -U /dev/ad0s2c mount /dev/ad0s2c /ports You can also play with this metnod with mdconfig -a -t vnode or mdconfig -a -t swap, too. If my way takes more than 10 sec., I will be very surprized how big disk/disks you have for building ports ;p Regards, Hiro PS If someone is really willing to speed this up, write this technique in the FreeBSD doc. so that people can take advantage of this, or let FreeBSD sets default WRKDIRPREFIX so that it builds everything under one directory (mounting a separate device is at user's wills.) On Sun, 20 May 2007 19:13:13 -0700 Jeremy Lea [EMAIL PROTECTED] wrote: Hi, On Sun, May 20, 2007 at 09:01:49AM +0200, Alexander Leidinger wrote: I could write such a new target, e.g. limited-clean, which could be used with update tools if there's some interest in something like this from the author of such a tool. The patch below should get you going on this. The first will save a stat and a make for each port, so it should reduce your time even more, and the second does the limited-clean. I've just added little targets to test them. One might want to use the logic that 'make clean' does a 'make clean limited-clean' if NOCLEANDEPENDS is not defined, and 'make clean' if it is, and leave 'make clean-depends' to do the full clean. One might want to make a switch like DEPENDS_CLEAN which did a clean before doing the install in dependency directories, so that stale port builds were not used. One might also want to document DEPENDS_CLEAN better so people know that it can be used like this 'cd /usr/ports/x11/xorg make -DNOCLEANDEPENDS -DDEPENDS_CLEAN install clean', and not only will it clean up like 'make install clean', it will use a lot less disk space... (Although DEPENDS_CLEAN is broken in the case of ports which overide DEPENDS_TARGET via the third depends arguement. It should be implemented directly in _INSTALL_DEPENDS) Regards, -Jeremy PS. ALL-DEPENDS-LIST has tabs set to 8 not 4 like bsd.port.mk should be. I don't think I'm responsible - it's been reimplemented a few times since I did the first rewrite... But maybe it was me ;-) ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to [EMAIL PROTECTED]
