Bugs item #2774951, was opened at 2009-04-19 17:18 Message generated for change (Comment added) made by smansur You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=482468&aid=2774951&group_id=56967
Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Build Group: MonetDB Common CVS Head Status: Open Resolution: None Priority: 5 Private: No Submitted By: Syeed Mansur (smansur) Assigned to: Sjoerd Mullender (sjoerd) Summary: Build Fails on OpenSolaris 64-Bit Initial Comment: I am using SunOS opensolaris 5.11 64-bit, and my builds are failing. I downloaded all the sources from the CVS repository, and used bootstrap to setup the build. This produced a configure script in the MonetDB package directory, which when invoked produced the following error: configure: error: need GCC version >=3.X for 64 bits However, I am using gcc version 3.4.3: #gcc --version gcc (GCC) 3.4.3 So, to work-through the problem, I used aclocal and autoconf to rebuild the configure script, which produced the following errors: #aclocal /usr/share/aclocal/audiofile.m4:12: warning: underquoted definition of AM_PATH_AUDIOFILE /usr/share/aclocal/audiofile.m4:12: run info '(automake)Extending aclocal' /usr/share/aclocal/audiofile.m4:12: or see http://sources.redhat.com/automake/automake.html#Extending-aclocal configure.in:63: warning: macro `AM_MONETDB_XQ_VARS_1' not found in library configure.in:66: warning: macro `AM_MONETDB_DEFAULTS' not found in library configure.in:69: warning: macro `AM_MONETDB_COMPILER' not found in library configure.in:70: warning: macro `AM_MONETDB_TOOLS' not found in library configure.in:71: warning: macro `AM_MONETDB_OPTIONS' not found in library configure.in:72: warning: macro `AM_MONETDB_UTILS' not found in library configure.in:85: warning: macro `AM_MONETDB_LIBS' not found in library configure.in:103: warning: macro `AM_MONETDB_XQ_VARS_2' not found in library #autoconf -f configure.in:63: error: possibly undefined macro: AM_MONETDB_XQ_VARS_1 If this token and others are legitimate, please use m4_pattern_allow. See the Autoconf documentation. configure.in:66: error: possibly undefined macro: AM_MONETDB_DEFAULTS configure.in:69: error: possibly undefined macro: AM_MONETDB_COMPILER configure.in:70: error: possibly undefined macro: AM_MONETDB_TOOLS configure.in:71: error: possibly undefined macro: AM_MONETDB_OPTIONS configure.in:72: error: possibly undefined macro: AM_MONETDB_UTILS configure.in:85: error: possibly undefined macro: AM_MONETDB_LIBS configure.in:103: error: possibly undefined macro: AM_MONETDB_XQ_VARS_2 A configure script did get produced, and when I invoked this script, I received the following errors: #configure --prefix=/root/makemonet/bin --enable-bits=64 --enable-debug=no --enable-assert=no --enable-optimize=yes /root/sourcemonet/MonetDB/configure: line 19267: syntax error at line 19887: `>' unexpected I must be doing something wrong, but after several days of reviewing, I can't seem to figure out the problem. Any help would be greatly appreciated. Thanks, Sid ---------------------------------------------------------------------- Comment By: Syeed Mansur (smansur) Date: 2009-04-21 03:45 Message: The OpenSolaris and the Linux machine are identical (I installed MonetDB on the Linux machine last week, and then wiped Linux off that machine and installed OpenSolaris - my goal is to test the ZFS filesystem and see if I can get performance increases on Monetdb using the compression feature of ZFS). Anyway, the computer has 1 CPU, 2 cores, with 4GB of physical RAM and 4GB of swap. I was remote-logged in to my machine and it froze after I tried to copy the records into MonetDB. I will have to try the reload in single-threaded mode (by the way, does MonetDB automatically recognize the number of cores and spawn multiple threads to coincide with the cores?) after I go into the office and reboot the machine (probably in 5 hours). ---------------------------------------------------------------------- Comment By: Stefan Manegold (stmane) Date: 2009-04-21 03:33 Message: How much physical (RAM) and virtual (swap) memory do your OpenSolaris and Linux machines have? How many CPUs/cores do your machines have? If the OpenSolaris has more than 1 CPU/core, could you please try to start the mserver5 in single-threaded mode (--set gdk_nr_threads=1) for loading the data via copy into? (cf., https://sourceforge.net/tracker/?func=detail&aid=2771052&group_id=56967&atid=482468) In case the above doesn't work, instead of using the source tarball of the development trunk, could you please try the latest Stable release MonetDB-Feb2009-SuperBall-SP1.tar.{bz2,lzma} from http://monetdb.cwi.nl/downloads/sources/Feb2009-SP1/ ? ---------------------------------------------------------------------- Comment By: Syeed Mansur (smansur) Date: 2009-04-21 03:12 Message: Stefan, I did indeed try the pre-bootstrapped sourceball with 'gcc -m64'; monetdb-install.sh. That compiles fine, and then when I try to load my data (using COPY command for 1.5Billion rows), I get the following error: mclient -lsql --database=syscodb < /root/load_xactions.sql MAPI = mone...@localhost:50000 QUERY = copy 1500000000 records into transactions ERROR = !SQLException:importTable:failed to import table !ERROR: HEAPextend: failed to extend to 3000000000 for 20/2060tail !ERROR: TABLETcreate_bats: Failed to create bat of size 1500000000 !ERROR: TABLETload_file:could not allocate bats Note, this failure occurs on OpenSolaris, but this does not occur on Linux (Fedora 10). For reference, the results of 'mserver5 --version' is: MonetDB server v5.11.0 (64-bit), based on kernel v1.29.0 (64-bit oids) Copyright (c) 1993-July 2008 CWI Copyright (c) August 2008-2009 MonetDB B.V., all rights reserved Visit http://monetdb.cwi.nl/ for further information Configured for prefix: /root/MonetDB Libraries: libpcre: 7.4 2007-09-21 (compiled with 7.4) openssl: OpenSSL 0.9.8a 11 Oct 2005 (+ security patches to 2007-10-13) (compiled with OpenSSL 0.9.8a 11 Oct 2005 (+ security patches to 2007-10-13)) libxml2: 2.6.31 (compiled with 2.6.31) Compiled by: @opensolaris Compilation: gcc -m64 -m64 -I/usr/sfw/include -L/usr/sfw/lib -D__EXTENSIONS__ -std=c99 -O6 -fomit-frame-pointer -finline-functions -falign-loops=4 -falign-jumps=4 -falign-functions=4 -fexpensive-optimizations -funroll-loops -frerun-cse-after-loop -frerun-loop-opt Linking : /usr/ccs/bin/ld -m64 -L/usr/sfw/lib -R/usr/sfw/lib So, it looks like the 64-bit version of MonetDB has been picked up, so I'm not sure why my COPY is failing. In an attempt to work-through this issue from scratch, I figured I'd try to download the sources directly from CVS and build everything afresh. That led me to the errors which caused me to open this bug report. With respect to your other question below, I did recompile everything from scratch (I may have missed something, so I will re-do everything). I completely removed my build directory and install prefix directory. I'll let you know what happens. In the meantime, should I open a different bug report for the COPY failure above, or is that considered a part of this current bug report? Regards, Sid ---------------------------------------------------------------------- Comment By: Stefan Manegold (stmane) Date: 2009-04-21 02:07 Message: Syeed, after applying my patch, did you recompile everything from srcatch, i.e., remove your complete build director(y|ies) and install prefix, and then start with ./bootstrap in buildtools? If so, could you please provide us with the exact configure call you used as well as the complete output of ./bootstrap, configure, make & make install for buildtools & MonetDB? Alternatively, did you try the pre-bootstrapped super sourceball with export CC='gcc -m64' ; ./monetdb-install.sh ... ? Stefan ---------------------------------------------------------------------- Comment By: Syeed Mansur (smansur) Date: 2009-04-20 21:51 Message: Fabian & Stefan: Thanks for your quick responses. Fabian, the reason I'm not using the pre-bootstrapped install.sh command is that I've already tried it, and on OpenSolaris 64-bit it seems to produce binaries that are 32-bit, which is not good enough for me to load up a 530GB table. Stefan: I implemented your suggestion, and the build process got past the original error, but not it looks like its getting caught in a different problem. configure seems to complete successfully, and then when I issue the gmake command, the process finishes as follows: gmake[5]: Leaving directory `/root/makemonet/src/gdk' gmake[4]: Leaving directory `/root/makemonet/src/gdk' gmake[4]: Entering directory `/root/makemonet/src' gmake[4]: Nothing to be done for `all-am'. gmake[4]: Leaving directory `/root/makemonet/src' gmake[3]: Leaving directory `/root/makemonet/src' gmake[2]: Leaving directory `/root/makemonet/src' Making all in conf gmake[2]: Entering directory `/root/makemonet/conf' /usr/sfw/bin/gmake all-am gmake[3]: Entering directory `/root/makemonet/conf' gmake[3]: Nothing to be done for `all-am'. gmake[3]: Leaving directory `/root/makemonet/conf' gmake[2]: Leaving directory `/root/makemonet/conf' gmake[2]: Entering directory `/root/makemonet' gmake[2]: Leaving directory `/root/makemonet' gmake[1]: Leaving directory `/root/makemonet' I went ahead and invoked the "gmake install" command, and that (as expected) did not install any binaries. For reference, during the bootstrap, I recieved the following errors: Remember to add `AC_PROG_LIBTOOL' to `configure.in'. You should add the contents of `/usr/share/aclocal/libtool.m4' to `aclocal.m4'. Putting files in AC_CONFIG_AUX_DIR, `conf'. /usr/share/aclocal/audiofile.m4:12: warning: underquoted definition of AM_PATH_AUDIOFILE /usr/share/aclocal/audiofile.m4:12: run info '(automake)Extending aclocal' /usr/share/aclocal/audiofile.m4:12: or see http://sources.redhat.com/automake/automake.html#Extending-aclocal Looks like a unified context diff. Hunk #1 failed at line 6386. 1 out of 1 hunks failed: saving rejects to aclocal.m4.rej done configure.in:33: installing `conf/missing' configure.in:33: installing `conf/install-sh' By the way, the gcc -v command returns the following: l# gcc -v Reading specs from /usr/sfw/lib/gcc/i386-pc-solaris2.11/3.4.3/specs Configured with: /builds2/sfwnv-gate/usr/src/cmd/gcc/gcc-3.4.3/configure --prefix=/usr/sfw --with-as=/usr/sfw/bin/gas --with-gnu-as --with-ld=/usr/ccs/bin/ld --without-gnu-ld --enable-languages=c,c++,f77,objc --enable-shared Thread model: posix gcc version 3.4.3 (csl-sol210-3_4-20050802) ---------------------------------------------------------------------- Comment By: Stefan Manegold (stmane) Date: 2009-04-20 02:28 Message: Fabian: Internally (i.e., in buildtools/conf/MonetDB.m4), --enable-bits=<bits> is translated into CC="$CC -m<bits>"; hence, I don't see much of a difference... Syeed: Could you please undo/remove all your modification to checked-out and/or generated file, patch your checked-out buildtools/conf/MonetDB.m4 as follows: -------- Index: buildtools/conf/MonetDB.m4 =================================================================== RCS file: /cvsroot/monetdb/buildtools/conf/MonetDB.m4,v retrieving revision 1.108.2.2 diff -u -r1.108.2.2 MonetDB.m4 --- buildtools/conf/MonetDB.m4 14 Apr 2009 11:36:50 -0000 1.108.2.2 +++ buildtools/conf/MonetDB.m4 20 Apr 2009 06:24:04 -0000 @@ -586,7 +586,7 @@ *-*-*-*-$native_bits) ;; yes-*-solaris*-*-*) - case `$bits-$CC -v 2>&1` in + case "$bits-`$CC -v 2>&1`" in 32-*|*-*'gcc version '[[34]]'.'*) ;; *) AC_MSG_ERROR([need GCC version >=3.X for 64 bits]);; esac -------- and then try to compile & install buildtools & MonetDB from scratch, again, and report what happens? Also, could you please send us the output of `gcc -v` on your systems? ---------------------------------------------------------------------- Comment By: Fabian (mr-meltdown) Date: 2009-04-20 02:16 Message: Is it possible to use pre-bootstrapped sources, like e.g. the monetdb-install.sh script does? If you have to use CVS sources, does monetdb-install.sh --cvs produce the same autoconf errors? It seems to me you have an order problem. Last suggestion, don't use --enable-bits, but use export CC="gcc -m64" CXX="g++ -m64" instead. Be aware that the shipped compiler is quite old, and uses Sun ld though. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=482468&aid=2774951&group_id=56967 ------------------------------------------------------------------------------ Stay on top of everything new and different, both inside and around Java (TM) technology - register by April 22, and save $200 on the JavaOne (SM) conference, June 2-5, 2009, San Francisco. 300 plus technical and hands-on sessions. Register today. Use priority code J9JMT32. http://p.sf.net/sfu/p _______________________________________________ Monetdb-bugs mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/monetdb-bugs
