My 8 GB try worked too. This is good news. Thanks,
Gunnar On Tue, Mar 8, 2016 at 1:10 AM, Arvind Narain <[email protected]> wrote: > Thanks Gunnar for fixing the newline issue. > > My experiment was with 8gb env. > > Regards > Arvind > On Mar 7, 2016 11:59 PM, "Gunnar Tapper" <[email protected]> wrote: > >> Hi, >> >> Arvind: I already fixed the newline issue. You were just ahead of me. :) >> >> Nitin: My bad, I misunderstood. No matter, Steve's fix removed the >> dependency. >> >> I too was able to build successfully using a 12 GB machine. Trying 8 GB >> now. >> >> On Tue, Mar 8, 2016 at 12:42 AM, Arvind <[email protected]> wrote: >> >>> I tested the steps in >>> *https://cwiki.apache.org/confluence/display/TRAFODION/Create+Build+Environment* >>> <https://cwiki.apache.org/confluence/display/TRAFODION/Create+Build+Environment>. >>> Without installing qt-* and doing related qmake checks my setup and >>> build ( with -j 1 ) worked. >>> >>> I did have to edit my .bashrc to introduce new line after each “echo >>> -en” done as per the steps in the web page. Maybe we should just use “echo >>> -e ..... >> $HOME/.bashrc” since the lines get appended: >>> >>> echo -en "export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk.x86_64" >> >>> $HOME/.bashrc >>> >>> echo -en "export TOOLSDIR=$HOME/trafodion-build-tools" >> $HOME/.bashrc >>> >>> echo -en "PATH=\$PATH:\$TOOLSDIR/apache-maven-3.3.3/bin" >> $HOME/.bashrc >>> >>> results in: >>> >>> export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk.x86_64export >>> TOOLSDIR=/home/centos/trafodion-build-toolsPATH=$PATH:$TOOLSDIR/apache-maven-3.3.3/bin >>> >>> The last echo should have export ( export PATH=..). >>> >>> In the output of list of tools installed the following could be updated : >>> >>> >>> INFO: List of tools directory: >>> >>> apache-log4cxx-0.10.0 >>> >>> apache-maven-3.3.3 >>> >>> bison_3_linux >>> >>> dest-llvm-3.2 >>> >>> dest-mpich-3.0.4 >>> >>> icu4c_4.4 >>> >>> thrift-0.9.0 >>> >>> udis86-1.7.2 >>> >>> zookeeper-3.4.5 >>> >>> Regards >>> >>> Arvind >>> >>> -----Original Message----- >>> From: Gunnar Tapper [mailto:[email protected] >>> <[email protected]>] >>> Sent: Monday, March 7, 2016 10:05 PM >>> To: [email protected] >>> Subject: Re: Parallel Make Failures >>> >>> Thanks! >>> >>> The issue is related to installing qt-devel qt-config: >>> >>> src/thrift/qt/moc_TQTcpServer.cpp:14:2: error: #error "This file was >>> generated using the moc from 4.8.1. It" >>> >>> src/thrift/qt/moc_TQTcpServer.cpp:15:2: error: #error "cannot be used >>> with the include files from this version of Qt." >>> >>> src/thrift/qt/moc_TQTcpServer.cpp:16:2: error: #error "(The moc has >>> changed too much.)" >>> >>> src/thrift/qt/moc_TQTcpServer.cpp:47: error: no 'void >>> apache::thrift::async::TQTcpServer::qt_static_metacall(QObject*, >>> >>> QMetaObject::Call, int, void**)' member function declared in class >>> 'apache::thrift::async::TQTcpServer' >>> >>> I removed those two packages. bin and include build fine. >>> >>> Talk about catch-22 situation. >>> >>> On Mon, Mar 7, 2016 at 7:43 PM, Arvind <[email protected]> wrote: >>> >>> > Could you check on the output of traf_tools_setup.sh ? As per the code >>> >>> > if the bin directory is existing then the package is assumed to be >>> >>> > already installed. >>> >>> > >>> >>> > echo "INFO: Installing Thrift on $(date)" | tee -a >>> $LOGFILE >>> >>> > if [ -d $TOOLSDIR/thrift-0.9.0/bin ]; then >>> >>> > echo "INFO: Thrift is already installed, skipping to >>> >>> > next tool" | tee -a $LOGFILE >>> >>> > else >>> >>> > downloadSource >>> >>> > http://archive.apache.org/dist/thrift/0.9.0/thrift-0.9.0.tar.gz >>> >>> > thrift-0.9.0 >>> >>> > >>> >>> > And if $TOOLSDIR/thrift-0.9.0 is existing then the environment >>> >>> > variables mentioned by Roberta get set in sqenvcom.sh: >>> >>> > >>> >>> > # Common for all distros >>> >>> > if [[ -d $TOOLSDIR/thrift-${THRIFT_DEP_VER} ]]; then >>> >>> > # this is for a build environment, where we need >>> >>> > # thrift from TOOLSDIR >>> >>> > export >>> >>> > THRIFT_LIB_DIR=$TOOLSDIR/thrift-${THRIFT_DEP_VER}/lib >>> >>> > export >>> >>> > THRIFT_INC_DIR=$TOOLSDIR/thrift-${THRIFT_DEP_VER}/include >>> >>> > fi >>> >>> > >>> >>> > Regards >>> >>> > Arvind >>> >>> > >>> >>> > -----Original Message----- >>> >>> > From: Gunnar Tapper [mailto:[email protected] >>> <[email protected]>] >>> >>> > Sent: Monday, March 7, 2016 6:21 PM >>> >>> > To: [email protected] >>> >>> > Subject: Re: Parallel Make Failures >>> >>> > >>> >>> > My problems is that the include and lib directories are created under >>> >>> > ~/trafodion-build-tools/thrift-0.9.0/. So, I'd be pointing these >>> >>> > variables to non-existing directories. The effect is the same: >>> >>> > >>> >>> > [centos@gtdev incubator-trafodion]$ make all -j 1 Building all >>> >>> > Trafodion components cd core && make all >>> >>> > make[1]: Entering directory `/home/centos/incubator-trafodion/core' >>> >>> > ./bldenvchk.sh; >>> >>> > *** Warning: QT_TOOLKIT does not point to an existing directory. >>> >>> > *** Warning: SQL Compiler Debugger will not be built. >>> >>> > *** Error: THRIFT_LIB_DIR directory not found: >>> >>> > /home/centos/trafodion-build-tools/thrift-0.9.0/lib >>> >>> > *** Error: THRIFT_INC_DIR directory not found: >>> >>> > /home/centos/trafodion-build-tools/thrift-0.9.0/include >>> >>> > make[1]: *** [sqroot] Error 1 >>> >>> > make[1]: Leaving directory `/home/centos/incubator-trafodion/core' >>> >>> > make: *** [all] Error 2 >>> >>> > [centos@gtdev incubator-trafodion]$ env | grep THRIFT >>> >>> > THRIFT_LIB_DIR=/home/centos/trafodion-build-tools/thrift-0.9.0/lib >>> >>> > THRIFT_INC_DIR=/home/centos/trafodion-build-tools/thrift-0.9.0/include >>> >>> > THRIFT_DEP_VER=0.9.0 >>> >>> > >>> >>> > >>> >>> > On Mon, Mar 7, 2016 at 7:12 PM, Roberta Marton >>> >>> > <[email protected] <mailto:[email protected]> > >>> >>> > wrote: >>> >>> > >>> >>> > > I was wondering if you did setup them up before building if it fixed >>> >>> > > the build issue. >>> >>> > > >>> >>> > > Roberta >>> >>> > > >>> >>> > > -----Original Message----- >>> >>> > > From: Gunnar Tapper [mailto:[email protected] >>> <[email protected]>] >>> >>> > > Sent: Monday, March 7, 2016 6:07 PM >>> >>> > > To: [email protected] <mailto: >>> >>> > [email protected]> >>> >>> > > Subject: Re: Parallel Make Failures >>> >>> > > >>> >>> > > Do I set these variables before running the tools setup? >>> >>> > > >>> >>> > > On Mon, Mar 7, 2016 at 6:55 PM, Roberta Marton >>> >>> > > <[email protected] <mailto:[email protected]> > >>> >>> > > wrote: >>> >>> > > >>> >>> > > > When I was testing release 1.3 last year, I remember getting a >>> >>> > > > similar issue. Thrift was installed but something in the make was >>> >>> > > > not working correctly. >>> >>> > > > To get around the problem temporarily, I manually exported the >>> >>> > > > THRIFT_INC_DIR and THRIFT_LIB_DIR: >>> >>> > > > >>> >>> > > > export THRIFT_LIB_DIR=<location> >>> >>> > > > export THRIFT_INC_DIR=<location> >>> >>> > > > >>> >>> > > > The build then succeeded. >>> >>> > > > There were several later changes made in the setup files that >>> >>> > > > eventually did not require this workaround. >>> >>> > > > >>> >>> > > > If you manually set these envvars, does the build work? >>> >>> > > > >>> >>> > > > Roberta >>> >>> > > > >>> >>> > > > -----Original Message----- >>> >>> > > > From: Gunnar Tapper [mailto:[email protected] >>> <[email protected]>] >>> >>> > > > Sent: Monday, March 7, 2016 5:28 PM >>> >>> > > > To: [email protected] <mailto: >>> >>> > [email protected]> >>> >>> > > > Subject: Re: Parallel Make Failures >>> >>> > > > >>> >>> > > > I created a new VM and did a new git clone. As stated, include and >>> >>> > > > lib are not built for Thrift. Also, it seems to me that the qmake >>> >>> > > > change might be broken. See below. >>> >>> > > > >>> >>> > > > [centos@trafbuild ~]$ cd incubator-trafodion/ [centos@trafbuild >>> >>> > > > incubator-trafodion]$ source ./env.sh >>> >>> > > > >>> >>> > > > If you are ready to build Trafodion, perform one of the >>> >>> > > > following >>> >>> > > > options: >>> >>> > > > >>> >>> > > > make all (Build Trafodion, DCS, and REST) OR >>> >>> > > > make package (Build Trafodion, DCS, REST, and Client >>> drivers) >>> >>> > > > OR >>> >>> > > > make package-all (Build Trafodion, DCS, REST, Client >>> >>> > > > drivers, and >>> >>> > > > Tests) >>> >>> > > > >>> >>> > > > If Trafodion has been built and you want test: >>> >>> > > > >>> >>> > > > Execute the install_local_hadoop script which performs a >>> >>> > > > single node >>> >>> > > > install using a popular Hadoop distribution >>> >>> > > > >>> >>> > > > cd /home/centos/incubator-trafodion/core/sqf/sql/scripts >>> >>> > > > install_local_hadoop [-p <port option>] >>> >>> > > > install_traf_components >>> >>> > > > configure Trafodion and start the processes >>> >>> > > > HAVE FUN! >>> >>> > > > >>> >>> > > > You can also choose to install_local_hadoop before building >>> >>> > Trafodion. >>> >>> > > > >>> >>> > > > [centos@trafbuild incubator-trafodion]$ make all -j 1 Building >>> all >>> >>> > > > Trafodion components cd core && make all >>> >>> > > > make[1]: Entering directory `/home/centos/incubator-trafodion/core' >>> >>> > > > ./bldenvchk.sh; >>> >>> > > > *** Warning: QT_TOOLKIT does not point to an existing directory. >>> >>> > > > *** Warning: SQL Compiler Debugger will not be built. >>> >>> > > > *** Error: THRIFT_LIB_DIR directory not found: >>> >>> > > > //home/centos/trafodion-build-tools/thrift-0.9.0/lib >>> >>> > > > *** Error: THRIFT_INC_DIR directory not found: >>> >>> > > > //home/centos/trafodion-build-tools/thrift-0.9.0/include >>> >>> > > > make[1]: *** [sqroot] Error 1 >>> >>> > > > make[1]: Leaving directory `/home/centos/incubator-trafodion/core' >>> >>> > > > make: *** [all] Error 2 >>> >>> > > > >>> >>> > > > [centos@trafbuild incubator-trafodion]$ echo $QT_TOOLKIT >>> >>> > > > >>> >>> > > > [centos@trafbuild incubator-trafodion]$ ls >>> >>> > > > ~/trafodion-build-tools/thrift-0.9.0/ >>> >>> > > > bin >>> >>> > > > >>> >>> > > > >>> >>> > > > On Mon, Mar 7, 2016 at 3:16 PM, Steve Varnau >>> >>> > > > <[email protected] <mailto:[email protected]> > >>> >>> > > > wrote: >>> >>> > > > >>> >>> > > > > The traf_tools_setup.sh script has not changed in the in the >>> >>> > > > > last few days. >>> >>> > > > > I hope that the download location for thrift have not changed. >>> >>> > > > > The only thing I can see that has been changing is the wiki >>> >>> > > > > instructions for setting up the environment. >>> >>> > > > > >>> >>> > > > > I see at least one thing wrong on wiki. For instance, it says >>> >>> > > > > that CentOS >>> >>> > > > > 6.7 won't work. That is not correct. There was a mistaken >>> >>> > > > > impression that >>> >>> > > > > 6.7 has newer gcc. Not true. >>> >>> > > > > >>> >>> > > > > With the very latest code (as of this morning), the qt-devel and >>> >>> > > > > qt-config should no longer be needed. >>> >>> > > > > >>> >>> > > > > --Steve >>> >>> > > > > >>> >>> > > > > >>> >>> > > > > > -----Original Message----- >>> >>> > > > > > From: Gunnar Tapper [mailto:[email protected] >>> <[email protected]>] >>> >>> > > > > > Sent: Monday, March 7, 2016 2:03 PM >>> >>> > > > > > To: [email protected] <mailto: >>> >>> > [email protected]> >>> >>> > > > > > Subject: Re: Parallel Make Failures >>> >>> > > > > > >>> >>> > > > > > Alas, things are getting worse rather than better... :( >>> >>> > > > > > >>> >>> > > > > > I create a new VM, installed all required packages, did a git >>> >>> > > > > > clone of trafodion, and then I ran install/traf_tools_setup.sh. >>> >>> > > > > > Now, Thrift isn't building the include and lib folders. I run >>> >>> > > > > > the exact same command on the version that I downloaded last >>> >>> > > > > > Thursday and that works just fine. >>> >>> > > > > > >>> >>> > > > > > Did anything change that I should be aware of? >>> >>> > > > > > >>> >>> > > > > > Gunnar >>> >>> > > > > > >>> >>> > > > > > On Mon, Mar 7, 2016 at 12:17 PM, Gunnar Tapper >>> >>> > > > > > <[email protected] <mailto:[email protected]> > >>> >>> > > > > > wrote: >>> >>> > > > > > >>> >>> > > > > > > Well, today, the build doesn't work at all regardless of >>> >>> > > > > > > what -j or -l options I use. It constantly fails on: >>> >>> > > > > > > >>> >>> > > > > > > Generating C++ code from yacc file ../sqlci/sqlci_yacc.y >>> >>> > > > > > > ##(SQL) >>> >>> > > > > > > ../sqlci/sqlci_yacc.y: warning: 1 reduce/reduce conflict >>> >>> > > > > > > [-Wconflicts-rr] >>> >>> > > > > > > ##(SQL) >>> >>> > > > > > > /home/centos/traf-tools/bison_3_linux/bin/bison: >>> >>> > > > > > > >>> >>> > > /home/centos/trafodion//bison_3_linux/share/bison/m4sugar/m4sugar.m4: >>> >>> > > > > > > cannot open: No such file or directory ##(SQL) >>> >>> > > > > > > mv: cannot stat `sqlcilib/linux/64bit/debug/sqlci_yacc.cpp': >>> >>> > > > > > > No such file or directory ##(SQL) >>> >>> > > > > > > sed: can't read >>> sqlcilib/linux/64bit/debug/sqlci_yacc.cpp.tmp: >>> >>> > > > > > > No >>> >>> > > > such >>> >>> > > > > > > file or directory ##(SQL) >>> >>> > > > > > > sed: can't read sqlcilib/linux/64bit/debug/sqlci_yacc.hpp: >>> >>> > > > > > > No such file or >>> >>> > > > > > > directory ##(SQL) >>> >>> > > > > > > rm: cannot remove >>> `sqlcilib/linux/64bit/debug/sqlci_yacc.hpp': >>> >>> > > > > > > No >>> >>> > > > such >>> >>> > > > > > > file or directory ##(SQL) >>> >>> > > > > > > rm: cannot remove >>> >>> > `sqlcilib/linux/64bit/debug/sqlci_yacc.cpp.tmp': >>> >>> > > > > > > No such >>> >>> > > > > > > file or directory ##(SQL) >>> >>> > > > > > > make[4]: *** [sqlcilib/linux/64bit/debug/sqlci_yacc.h] Error >>> >>> > > > > > > 1 >>> >>> > > > > > > ##(SQL) >>> >>> > > > > > > make[4]: Leaving directory >>> >>> > > > > > > `/home/centos/incubator-trafodion/core/sql/nskgmake' ##(SQL) >>> >>> > > > > > > make[3]: *** [all] Error 2 ##(SQL) >>> >>> > > > > > > make[3]: Leaving directory >>> >>> > > > > > > `/home/centos/incubator-trafodion/core/sqf/sql' >>> >>> > > > > > > ##(SQL) >>> >>> > > > > > > make[2]: *** [make_sql] Error 2 >>> >>> > > > > > > make[2]: Leaving directory >>> >>> > > > `/home/centos/incubator-trafodion/core/sqf' >>> >>> > > > > > > make[1]: *** [foundation] Error 2 >>> >>> > > > > > > make[1]: Leaving directory >>> >>> > `/home/centos/incubator-trafodion/core' >>> >>> > > > > > > make: *** [all] Error 2 >>> >>> > > > > > > >>> >>> > > > > > > >>> >>> > > > > > > On Mon, Mar 7, 2016 at 12:05 PM, Amanda Moran >>> >>> > > > > > <[email protected] <mailto:[email protected]> > >>> >>> > > > > > > wrote: >>> >>> > > > > > > >>> >>> > > > > > >> On redhat 7.1 this command still works: >>> >>> > > > > > >> >>> >>> > > > > > >> [ec2-user@ip-10-0-0-175 ~]$ cat /etc/redhat-release Red >>> Hat >>> >>> > > > > > >> Enterprise Linux Server release 7.1 (Maipo) >>> >>> > > > > > >> >>> >>> > > > > > >> [ec2-user@ip-10-0-0-175 ~]$ grep processor /proc/cpuinfo | >>> >>> > > > > > >> wc -l >>> >>> > > > > > >> 4 >>> >>> > > > > > >> >>> >>> > > > > > >> >>> >>> > > > > > >> On Mon, Mar 7, 2016 at 9:56 AM, Steve Varnau >>> >>> > > > > > >> <[email protected] <mailto:[email protected]> > >>> >>> > > > > > >> wrote: >>> >>> > > > > > >> >>> >>> > > > > > >> > > It seems that the parallel make fails on 8 GB machines. >>> >>> > > > > > >> > >>> >>> > > > > > >> > I think your first sentence overstates the determinism of >>> >>> > > > > > >> > the >>> >>> > > > > problem >>> >>> > > > > > >> > a >>> >>> > > > > > >> > bit. >>> >>> > > > > > >> > I ran a normal, default build on 8GB machine last week >>> >>> > > > > > >> > and had no >>> >>> > > > > > >> problem. >>> >>> > > > > > >> > There must be an environmental problem, but I don't think >>> >>> > > > > > >> > we fully understand it yet. >>> >>> > > > > > >> > >>> >>> > > > > > >> > The aggressiveness of the make parallelism is set in >>> >>> > > > > > >> core/sqf/sqenvcom.sh. >>> >>> > > > > > >> > It sets the parallel factor based on how many CPUs are on >>> >>> > > > > > >> > your >>> >>> > > > > > >> > machine: >>> >>> > > > > > >> > >>> >>> > > > > > >> > # Set default build parallelism # Can be overridden on >>> >>> > > > > > >> > make commandline cpucnt=$(grep processor /proc/cpuinfo | >>> wc -l) >>> >>> > > > > > >> > # no number means unlimited, and will swamp the system >>> >>> > > > > > >> > export MAKEFLAGS="-j$cpucnt" >>> >>> > > > > > >> > >>> >>> > > > > > >> > If that calculation is wrong, maybe that could cause a >>> >>> > problem. >>> >>> > > > > > >> > >>> >>> > > > > > >> > --Steve >>> >>> > > > > > >> > >>> >>> > > > > > >> > >>> >>> > > > > > >> > > -----Original Message----- >>> >>> > > > > > >> > > From: Gunnar Tapper [mailto:[email protected] >>> <[email protected]>] >>> >>> > > > > > >> > > Sent: Monday, March 7, 2016 9:35 AM >>> >>> > > > > > >> > > To: [email protected] <mailto: >>> >>> > [email protected]> >>> >>> > > > > > >> > > Subject: Parallel Make Failures >>> >>> > > > > > >> > > >>> >>> > > > > > >> > > Hi, >>> >>> > > > > > >> > > >>> >>> > > > > > >> > > It seems that the parallel make fails on 8 GB machines. >>> >>> > > > > > >> > > At least, >>> >>> > > > > > >> Nitin >>> >>> > > > > > >> > > and >>> >>> > > > > > >> > > I both ran into make failures that did not appear when >>> >>> > > > > > >> > > running serial make. >>> >>> > > > > > >> > > I've also seen similar failures when building the code >>> >>> > > > > > >> > > on >>> >>> > > > > > >> > > 12 GB >>> >>> > > > > > >> machines. >>> >>> > > > > > >> > > >>> >>> > > > > > >> > > Based on previous discussions, the Trafodion >>> >>> > > > > > >> > > Contributor Guide >>> >>> > > > > > >> recommends >>> >>> > > > > > >> > > rerunning make a few times if running issues. >>> >>> > > > > > >> > > >>> >>> > > > > > >> > > I most wonder if there's a way to reduce the >>> >>> > > > > > >> > > aggressiveness of the >>> >>> > > > > > >> make >>> >>> > > > > > >> > in >>> >>> > > > > > >> > > general. Could we, for example, come up with a table >>> >>> > > > > > >> > > that correlates system size to define the -l option or >>> >>> > > > > > >> > > something similar? >>> >>> > > > > > >> > > >>> >>> > > > > > >> > > -- >>> >>> > > > > > >> > > Thanks, >>> >>> > > > > > >> > > >>> >>> > > > > > >> > > Gunnar >>> >>> > > > > > >> > > *If you think you can you can, if you think you can't >>> >>> > > > > > >> > > you're >>> >>> > > > > > >> > > right.* >>> >>> > > > > > >> > >>> >>> > > > > > >> >>> >>> > > > > > >> >>> >>> > > > > > >> >>> >>> > > > > > >> -- >>> >>> > > > > > >> Thanks, >>> >>> > > > > > >> >>> >>> > > > > > >> Amanda Moran >>> >>> > > > > > >> >>> >>> > > > > > > >>> >>> > > > > > > >>> >>> > > > > > > >>> >>> > > > > > > -- >>> >>> > > > > > > Thanks, >>> >>> > > > > > > >>> >>> > > > > > > Gunnar >>> >>> > > > > > > *If you think you can you can, if you think you can't you're >>> >>> > > > > > > right.* >>> >>> > > > > > > >>> >>> > > > > > >>> >>> > > > > > >>> >>> > > > > > >>> >>> > > > > > -- >>> >>> > > > > > Thanks, >>> >>> > > > > > >>> >>> > > > > > Gunnar >>> >>> > > > > > *If you think you can you can, if you think you can't you're >>> >>> > > > > > right.* >>> >>> > > > > >>> >>> > > > >>> >>> > > > >>> >>> > > > >>> >>> > > > -- >>> >>> > > > Thanks, >>> >>> > > > >>> >>> > > > Gunnar >>> >>> > > > *If you think you can you can, if you think you can't you're >>> >>> > > > right.* >>> >>> > > > >>> >>> > > >>> >>> > > >>> >>> > > >>> >>> > > -- >>> >>> > > Thanks, >>> >>> > > >>> >>> > > Gunnar >>> >>> > > *If you think you can you can, if you think you can't you're right.* >>> >>> > > >>> >>> > >>> >>> > >>> >>> > >>> >>> > -- >>> >>> > Thanks, >>> >>> > >>> >>> > Gunnar >>> >>> > *If you think you can you can, if you think you can't you're right.* >>> >>> > >>> >>> >>> -- >>> >>> Thanks, >>> >>> Gunnar >>> >>> *If you think you can you can, if you think you can't you're right.* >>> >> >> >> >> -- >> Thanks, >> >> Gunnar >> *If you think you can you can, if you think you can't you're right.* >> > -- Thanks, Gunnar *If you think you can you can, if you think you can't you're right.*
