FreeBSD-7.2-amd64 added to the build-farm

2009-08-24 Thread Kinkie
Hi all,
  I've set up and hooked to Hudson (for 3.X builds, 2.X will follow
unless someone beats me to it) a FreeBSD-7.2-amd64 vhost.
Builds are currently silent, and failing.
The system is accessible from eu as "diablo". The root password is the
same as vobsd, user hudson has no password.

-- 
/kinkie


Re: Alternate http repository for squid3

2009-08-24 Thread Robert Collins
On Tue, 2009-08-25 at 11:09 +1200, Amos Jeffries wrote:
> On Tue, 25 Aug 2009 04:20:31 +1000, Robert Collins
>  wrote:
> > On Fri, 2009-08-21 at 09:35 +0200, Kinkie wrote:
> >> On Fri, Aug 21, 2009 at 7:42 AM, Robert
> >> Collins wrote:
> >> > On Thu, 2009-08-20 at 14:07 +0200, Kinkie wrote:
> >> >> As part of the ongoing buildfarm work, I've published onto http the
> >> >> (read-only) repository mirror that's hosted on eu.
> >> >> It's available at http://www.eu.squid-cache.org/bzr/squid3/
> >> >
> >> > How often is this synced?
> >> 
> >> 6 hours. rsync-based cron job on eu, user "bzr".
> > 
> > If we're going to build test from it, it probably needs to be synced
> > on-demand.
> 
> Or at least hourly before the SCM polling happens.
> That frequency has worked so far for the SourceForge CVS copy.

The lower the latency the better. That allows change-test cycles to be
short, when dealing with build environments we don't locally have.

-Rob



signature.asc
Description: This is a digitally signed message part


Re: Alternate http repository for squid3

2009-08-24 Thread Amos Jeffries
On Tue, 25 Aug 2009 04:20:31 +1000, Robert Collins
 wrote:
> On Fri, 2009-08-21 at 09:35 +0200, Kinkie wrote:
>> On Fri, Aug 21, 2009 at 7:42 AM, Robert
>> Collins wrote:
>> > On Thu, 2009-08-20 at 14:07 +0200, Kinkie wrote:
>> >> As part of the ongoing buildfarm work, I've published onto http the
>> >> (read-only) repository mirror that's hosted on eu.
>> >> It's available at http://www.eu.squid-cache.org/bzr/squid3/
>> >
>> > How often is this synced?
>> 
>> 6 hours. rsync-based cron job on eu, user "bzr".
> 
> If we're going to build test from it, it probably needs to be synced
> on-demand.

Or at least hourly before the SCM polling happens.
That frequency has worked so far for the SourceForge CVS copy.

Amos



Re: Alternate http repository for squid3

2009-08-24 Thread Kinkie
Nah, I see it more as a way to hand-bootstrap a tree outside the US
(it' atrocious here in ITA) and as a disaster recovery thing :)

But it may make sense to advertise it

On 8/24/09, Robert Collins  wrote:
> On Fri, 2009-08-21 at 09:35 +0200, Kinkie wrote:
>> On Fri, Aug 21, 2009 at 7:42 AM, Robert
>> Collins wrote:
>> > On Thu, 2009-08-20 at 14:07 +0200, Kinkie wrote:
>> >> As part of the ongoing buildfarm work, I've published onto http the
>> >> (read-only) repository mirror that's hosted on eu.
>> >> It's available at http://www.eu.squid-cache.org/bzr/squid3/
>> >
>> > How often is this synced?
>>
>> 6 hours. rsync-based cron job on eu, user "bzr".
>
> If we're going to build test from it, it probably needs to be synced
> on-demand.
>
> -Rob
>


-- 
/kinkie


Re: Alternate http repository for squid3

2009-08-24 Thread Robert Collins
On Fri, 2009-08-21 at 09:35 +0200, Kinkie wrote:
> On Fri, Aug 21, 2009 at 7:42 AM, Robert
> Collins wrote:
> > On Thu, 2009-08-20 at 14:07 +0200, Kinkie wrote:
> >> As part of the ongoing buildfarm work, I've published onto http the
> >> (read-only) repository mirror that's hosted on eu.
> >> It's available at http://www.eu.squid-cache.org/bzr/squid3/
> >
> > How often is this synced?
> 
> 6 hours. rsync-based cron job on eu, user "bzr".

If we're going to build test from it, it probably needs to be synced
on-demand.

-Rob


signature.asc
Description: This is a digitally signed message part


[PATCH] DiskIO detection cleanup.

2009-08-24 Thread Amos Jeffries
This patch changes the interactions between --enable-disk-io 
--enable-storeio and also the related --with-aio and --with-pthreads 
options.


The old behaviour was very complex and incremental:
 - check some DiskIO dependencies
   - enable libraries (even if not needed)
 - scan *-storeio=
 - scan *-disk-io=
   - enable some diskio modules
 - scan *-disk-io=
   - strip any duplicates
 - scan some storeio dependencies
   - enable some more disk-io modules (but not the stores strangely)
 - scan *-storeio looking for required disk-io dependency
   - automatically enable needed diskio (despite explicit settings)
 - scan *-disk-io=
   - enable some more disk-io modules
 - scan some other disk-io dependencies
   - enable libraries (even if not needed)


This patch seeks to replace that with:
 - scan *-disk-io=
   - check for dependencies
   - auto-disable module if any dependencies are missing/unusable
 - scan *-store-io=
   - error if any depended diskio are disabled (saying which)

This both simplifies a complex situation, speeds up configure, and 
reduces the amount of useless libraries by two or more in some builds.


--with-async-io was a whole mess on its own with other dependent 
options. For now I've left it out of this change and it retains its old 
behaviour of being enabled even if not useful.



Users now have the choice:
1)  use simply "--enable-disk-io --enable-storeio" without explicit 
lists of modules to enable everything.


2) use the above _with_ an explicit set of modules.

3) disable all disk components with "--disable-disk-io 
--disnable-storeio" and use memory-only.



Theres a few bits of polish I know still needed:
 * some of the DiskIO dependencies I could not find clearly remain 
outside the scan block.
 * according to Robert the Windows Overlapping IO needs to be broken 
out into its own module. That is another cleanup sub-project.


And the build defaults needs to be assessed:

Current default is to enable only UFS with Blocking DiskIO

 * should we now make all disk modules default-enable?
   (to improve the experience of those building without these two options)

 * or, given that the default config is memory-only
   should we now make all disk modules default-disable?


Amos
=== modified file 'configure.in'
--- configure.in	2009-08-23 05:13:09 +
+++ configure.in	2009-08-24 13:58:37 +
@@ -414,25 +414,230 @@
 [Defines how many threads aufs uses for I/O])
 fi
 
-AC_ARG_WITH(pthreads,
-  AS_HELP_STRING([--with-pthreads],[Use POSIX Threads]))
-if test "$with_pthreads" = "yes"; then
-  AC_MSG_NOTICE([With pthreads])
-fi
-
-AC_ARG_WITH(aio,
-  AS_HELP_STRING([--with-aio],[Use POSIX AIO]))
-if test "$with_aio" = "yes"; then
-  AC_MSG_NOTICE([With aio])
-fi
-
-
 AC_ARG_WITH(dl,
   AS_HELP_STRING([--with-dl],[Use dynamic linking]))
 if test "$with_dl" = "yes"; then
   AC_MSG_NOTICE([With dl])
 fi
 
+AC_ARG_ENABLE(disk-io,
+  AS_HELP_STRING([--enable-disk-io="list of modules"],[Build support for the list of disk I/O modules.
+  If unset only the "Blocking" module will be built.
+  Set without a value all available modules will be built.
+  See src/DiskIO for a list of available modules, or
+  Programmers Guide section on DiskIO
+  for details on how to build your custom disk module]),
+[ case $enableval in
+  yes)
+	for dir in $srcdir/src/DiskIO/*; do
+	module="`basename $dir`"
+	if test -d "$dir" && test "$module" != CVS; then
+		AC_MSG_NOTICE([Autodetected $module DiskIO module])
+		MAYBE_DISK_MODULES="$MAYBE_DISK_MODULES $module"
+	fi
+	done
+	AC_DEFINE(USE_DISKIO,1,[DiskIO modules are expected to be available.])
+  	;;
+  no)
+	AC_DEFINE(USE_DISKIO,0,[DiskIO modules are expected to be available.])
+  	;;
+  *)
+	MAYBE_DISK_MODULES=" `echo $enableval| sed -e 's/,/ /g;s/  */ /g'` "
+	AC_DEFINE(USE_DISKIO,1,[DiskIO modules are expected to be available.])
+	;;
+  esac
+],
+[ if test -z "$MAYBE_DISK_MODULES"; then
+MAYBE_DISK_MODULES="Blocking"
+AC_MSG_NOTICE([Enabling Blocking DiskIO module (required default)])
+AC_DEFINE(USE_DISKIO,1,[DiskIO modules are expected to be available.])
+  fi
+])
+
+dnl Some autoconf.h defines we might enable later...
+AC_DEFINE(USE_DISKIO_AIO, 0, [Whether POSIX AIO support is needed. Automatic])
+USE_AIOPS_WIN32=0
+use_aio=
+
+dnl Setup the module paths etc.
+FOUND_DISKIO_AIO=
+FOUND_DISKIO_BLOCKING=
+FOUND_DISKIO_DISKDAEMON=
+FOUND_DISKIO_DISKTHREADS=
+DISK_LIBS=
+DISK_MODULES=
+DISK_LINKOBJS=
+for module in $MAYBE_DISK_MODULES none; do
+  if test "$module" = "none"; then
+	continue
+  fi
+  if ! test -d $srcdir/src/DiskIO/$module; then
+AC_MSG_ERROR(disk-io $module does not exist)
+  fi
+  case "$module" in
+DiskDaemon)
+	if test "$FOUND_DISKIO_DISKDAEMON" = "yes" ; then
+	  AC_MSG_ERROR([DiskIO DiskDaemon module listed twice.])
+	fi
+	FOUND_DISKIO_DISKDAEMON="yes"
+	AC_MSG_NOTICE(["Enabling DiskDaemon DiskIO module])
+	DISK_LIBS="$DISK_L