Thanks Gunnar for fixing the newline issue. My experiment was with 8gb env.
Regards Arvind On Mar 7, 2016 11:59 PM, "Gunnar Tapper" <[email protected]> wrote: > Hi, > > Arvind: I already fixed the newline issue. You were just ahead of me. :) > > Nitin: My bad, I misunderstood. No matter, Steve's fix removed the > dependency. > > I too was able to build successfully using a 12 GB machine. Trying 8 GB > now. > > On Tue, Mar 8, 2016 at 12:42 AM, Arvind <[email protected]> wrote: > >> I tested the steps in >> *https://cwiki.apache.org/confluence/display/TRAFODION/Create+Build+Environment* >> <https://cwiki.apache.org/confluence/display/TRAFODION/Create+Build+Environment>. >> Without installing qt-* and doing related qmake checks my setup and >> build ( with -j 1 ) worked. >> >> I did have to edit my .bashrc to introduce new line after each “echo -en” >> done >> as per the steps in the web page. Maybe we should just use “echo -e ..... >> >> $HOME/.bashrc” since the lines get appended: >> >> echo -en "export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk.x86_64" >> >> $HOME/.bashrc >> >> echo -en "export TOOLSDIR=$HOME/trafodion-build-tools" >> $HOME/.bashrc >> >> echo -en "PATH=\$PATH:\$TOOLSDIR/apache-maven-3.3.3/bin" >> $HOME/.bashrc >> >> results in: >> >> export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk.x86_64export >> TOOLSDIR=/home/centos/trafodion-build-toolsPATH=$PATH:$TOOLSDIR/apache-maven-3.3.3/bin >> >> The last echo should have export ( export PATH=..). >> >> In the output of list of tools installed the following could be updated : >> >> >> INFO: List of tools directory: >> >> apache-log4cxx-0.10.0 >> >> apache-maven-3.3.3 >> >> bison_3_linux >> >> dest-llvm-3.2 >> >> dest-mpich-3.0.4 >> >> icu4c_4.4 >> >> thrift-0.9.0 >> >> udis86-1.7.2 >> >> zookeeper-3.4.5 >> >> Regards >> >> Arvind >> >> -----Original Message----- >> From: Gunnar Tapper [mailto:[email protected] >> <[email protected]>] >> Sent: Monday, March 7, 2016 10:05 PM >> To: [email protected] >> Subject: Re: Parallel Make Failures >> >> Thanks! >> >> The issue is related to installing qt-devel qt-config: >> >> src/thrift/qt/moc_TQTcpServer.cpp:14:2: error: #error "This file was >> generated using the moc from 4.8.1. It" >> >> src/thrift/qt/moc_TQTcpServer.cpp:15:2: error: #error "cannot be used >> with the include files from this version of Qt." >> >> src/thrift/qt/moc_TQTcpServer.cpp:16:2: error: #error "(The moc has >> changed too much.)" >> >> src/thrift/qt/moc_TQTcpServer.cpp:47: error: no 'void >> apache::thrift::async::TQTcpServer::qt_static_metacall(QObject*, >> >> QMetaObject::Call, int, void**)' member function declared in class >> 'apache::thrift::async::TQTcpServer' >> >> I removed those two packages. bin and include build fine. >> >> Talk about catch-22 situation. >> >> On Mon, Mar 7, 2016 at 7:43 PM, Arvind <[email protected]> wrote: >> >> > Could you check on the output of traf_tools_setup.sh ? As per the code >> >> > if the bin directory is existing then the package is assumed to be >> >> > already installed. >> >> > >> >> > echo "INFO: Installing Thrift on $(date)" | tee -a >> $LOGFILE >> >> > if [ -d $TOOLSDIR/thrift-0.9.0/bin ]; then >> >> > echo "INFO: Thrift is already installed, skipping to >> >> > next tool" | tee -a $LOGFILE >> >> > else >> >> > downloadSource >> >> > http://archive.apache.org/dist/thrift/0.9.0/thrift-0.9.0.tar.gz >> >> > thrift-0.9.0 >> >> > >> >> > And if $TOOLSDIR/thrift-0.9.0 is existing then the environment >> >> > variables mentioned by Roberta get set in sqenvcom.sh: >> >> > >> >> > # Common for all distros >> >> > if [[ -d $TOOLSDIR/thrift-${THRIFT_DEP_VER} ]]; then >> >> > # this is for a build environment, where we need >> >> > # thrift from TOOLSDIR >> >> > export >> >> > THRIFT_LIB_DIR=$TOOLSDIR/thrift-${THRIFT_DEP_VER}/lib >> >> > export >> >> > THRIFT_INC_DIR=$TOOLSDIR/thrift-${THRIFT_DEP_VER}/include >> >> > fi >> >> > >> >> > Regards >> >> > Arvind >> >> > >> >> > -----Original Message----- >> >> > From: Gunnar Tapper [mailto:[email protected] >> <[email protected]>] >> >> > Sent: Monday, March 7, 2016 6:21 PM >> >> > To: [email protected] >> >> > Subject: Re: Parallel Make Failures >> >> > >> >> > My problems is that the include and lib directories are created under >> >> > ~/trafodion-build-tools/thrift-0.9.0/. So, I'd be pointing these >> >> > variables to non-existing directories. The effect is the same: >> >> > >> >> > [centos@gtdev incubator-trafodion]$ make all -j 1 Building all >> >> > Trafodion components cd core && make all >> >> > make[1]: Entering directory `/home/centos/incubator-trafodion/core' >> >> > ./bldenvchk.sh; >> >> > *** Warning: QT_TOOLKIT does not point to an existing directory. >> >> > *** Warning: SQL Compiler Debugger will not be built. >> >> > *** Error: THRIFT_LIB_DIR directory not found: >> >> > /home/centos/trafodion-build-tools/thrift-0.9.0/lib >> >> > *** Error: THRIFT_INC_DIR directory not found: >> >> > /home/centos/trafodion-build-tools/thrift-0.9.0/include >> >> > make[1]: *** [sqroot] Error 1 >> >> > make[1]: Leaving directory `/home/centos/incubator-trafodion/core' >> >> > make: *** [all] Error 2 >> >> > [centos@gtdev incubator-trafodion]$ env | grep THRIFT >> >> > THRIFT_LIB_DIR=/home/centos/trafodion-build-tools/thrift-0.9.0/lib >> >> > THRIFT_INC_DIR=/home/centos/trafodion-build-tools/thrift-0.9.0/include >> >> > THRIFT_DEP_VER=0.9.0 >> >> > >> >> > >> >> > On Mon, Mar 7, 2016 at 7:12 PM, Roberta Marton >> >> > <[email protected] <mailto:[email protected]> > >> >> > wrote: >> >> > >> >> > > I was wondering if you did setup them up before building if it fixed >> >> > > the build issue. >> >> > > >> >> > > Roberta >> >> > > >> >> > > -----Original Message----- >> >> > > From: Gunnar Tapper [mailto:[email protected] >> <[email protected]>] >> >> > > Sent: Monday, March 7, 2016 6:07 PM >> >> > > To: [email protected] <mailto: >> >> > [email protected]> >> >> > > Subject: Re: Parallel Make Failures >> >> > > >> >> > > Do I set these variables before running the tools setup? >> >> > > >> >> > > On Mon, Mar 7, 2016 at 6:55 PM, Roberta Marton >> >> > > <[email protected] <mailto:[email protected]> > >> >> > > wrote: >> >> > > >> >> > > > When I was testing release 1.3 last year, I remember getting a >> >> > > > similar issue. Thrift was installed but something in the make was >> >> > > > not working correctly. >> >> > > > To get around the problem temporarily, I manually exported the >> >> > > > THRIFT_INC_DIR and THRIFT_LIB_DIR: >> >> > > > >> >> > > > export THRIFT_LIB_DIR=<location> >> >> > > > export THRIFT_INC_DIR=<location> >> >> > > > >> >> > > > The build then succeeded. >> >> > > > There were several later changes made in the setup files that >> >> > > > eventually did not require this workaround. >> >> > > > >> >> > > > If you manually set these envvars, does the build work? >> >> > > > >> >> > > > Roberta >> >> > > > >> >> > > > -----Original Message----- >> >> > > > From: Gunnar Tapper [mailto:[email protected] >> <[email protected]>] >> >> > > > Sent: Monday, March 7, 2016 5:28 PM >> >> > > > To: [email protected] <mailto: >> >> > [email protected]> >> >> > > > Subject: Re: Parallel Make Failures >> >> > > > >> >> > > > I created a new VM and did a new git clone. As stated, include and >> >> > > > lib are not built for Thrift. Also, it seems to me that the qmake >> >> > > > change might be broken. See below. >> >> > > > >> >> > > > [centos@trafbuild ~]$ cd incubator-trafodion/ [centos@trafbuild >> >> > > > incubator-trafodion]$ source ./env.sh >> >> > > > >> >> > > > If you are ready to build Trafodion, perform one of the >> >> > > > following >> >> > > > options: >> >> > > > >> >> > > > make all (Build Trafodion, DCS, and REST) OR >> >> > > > make package (Build Trafodion, DCS, REST, and Client >> drivers) >> >> > > > OR >> >> > > > make package-all (Build Trafodion, DCS, REST, Client >> >> > > > drivers, and >> >> > > > Tests) >> >> > > > >> >> > > > If Trafodion has been built and you want test: >> >> > > > >> >> > > > Execute the install_local_hadoop script which performs a >> >> > > > single node >> >> > > > install using a popular Hadoop distribution >> >> > > > >> >> > > > cd /home/centos/incubator-trafodion/core/sqf/sql/scripts >> >> > > > install_local_hadoop [-p <port option>] >> >> > > > install_traf_components >> >> > > > configure Trafodion and start the processes >> >> > > > HAVE FUN! >> >> > > > >> >> > > > You can also choose to install_local_hadoop before building >> >> > Trafodion. >> >> > > > >> >> > > > [centos@trafbuild incubator-trafodion]$ make all -j 1 Building all >> >> > > > Trafodion components cd core && make all >> >> > > > make[1]: Entering directory `/home/centos/incubator-trafodion/core' >> >> > > > ./bldenvchk.sh; >> >> > > > *** Warning: QT_TOOLKIT does not point to an existing directory. >> >> > > > *** Warning: SQL Compiler Debugger will not be built. >> >> > > > *** Error: THRIFT_LIB_DIR directory not found: >> >> > > > //home/centos/trafodion-build-tools/thrift-0.9.0/lib >> >> > > > *** Error: THRIFT_INC_DIR directory not found: >> >> > > > //home/centos/trafodion-build-tools/thrift-0.9.0/include >> >> > > > make[1]: *** [sqroot] Error 1 >> >> > > > make[1]: Leaving directory `/home/centos/incubator-trafodion/core' >> >> > > > make: *** [all] Error 2 >> >> > > > >> >> > > > [centos@trafbuild incubator-trafodion]$ echo $QT_TOOLKIT >> >> > > > >> >> > > > [centos@trafbuild incubator-trafodion]$ ls >> >> > > > ~/trafodion-build-tools/thrift-0.9.0/ >> >> > > > bin >> >> > > > >> >> > > > >> >> > > > On Mon, Mar 7, 2016 at 3:16 PM, Steve Varnau >> >> > > > <[email protected] <mailto:[email protected]> > >> >> > > > wrote: >> >> > > > >> >> > > > > The traf_tools_setup.sh script has not changed in the in the >> >> > > > > last few days. >> >> > > > > I hope that the download location for thrift have not changed. >> >> > > > > The only thing I can see that has been changing is the wiki >> >> > > > > instructions for setting up the environment. >> >> > > > > >> >> > > > > I see at least one thing wrong on wiki. For instance, it says >> >> > > > > that CentOS >> >> > > > > 6.7 won't work. That is not correct. There was a mistaken >> >> > > > > impression that >> >> > > > > 6.7 has newer gcc. Not true. >> >> > > > > >> >> > > > > With the very latest code (as of this morning), the qt-devel and >> >> > > > > qt-config should no longer be needed. >> >> > > > > >> >> > > > > --Steve >> >> > > > > >> >> > > > > >> >> > > > > > -----Original Message----- >> >> > > > > > From: Gunnar Tapper [mailto:[email protected] >> <[email protected]>] >> >> > > > > > Sent: Monday, March 7, 2016 2:03 PM >> >> > > > > > To: [email protected] <mailto: >> >> > [email protected]> >> >> > > > > > Subject: Re: Parallel Make Failures >> >> > > > > > >> >> > > > > > Alas, things are getting worse rather than better... :( >> >> > > > > > >> >> > > > > > I create a new VM, installed all required packages, did a git >> >> > > > > > clone of trafodion, and then I ran install/traf_tools_setup.sh. >> >> > > > > > Now, Thrift isn't building the include and lib folders. I run >> >> > > > > > the exact same command on the version that I downloaded last >> >> > > > > > Thursday and that works just fine. >> >> > > > > > >> >> > > > > > Did anything change that I should be aware of? >> >> > > > > > >> >> > > > > > Gunnar >> >> > > > > > >> >> > > > > > On Mon, Mar 7, 2016 at 12:17 PM, Gunnar Tapper >> >> > > > > > <[email protected] <mailto:[email protected]> > >> >> > > > > > wrote: >> >> > > > > > >> >> > > > > > > Well, today, the build doesn't work at all regardless of >> >> > > > > > > what -j or -l options I use. It constantly fails on: >> >> > > > > > > >> >> > > > > > > Generating C++ code from yacc file ../sqlci/sqlci_yacc.y >> >> > > > > > > ##(SQL) >> >> > > > > > > ../sqlci/sqlci_yacc.y: warning: 1 reduce/reduce conflict >> >> > > > > > > [-Wconflicts-rr] >> >> > > > > > > ##(SQL) >> >> > > > > > > /home/centos/traf-tools/bison_3_linux/bin/bison: >> >> > > > > > > >> >> > > /home/centos/trafodion//bison_3_linux/share/bison/m4sugar/m4sugar.m4: >> >> > > > > > > cannot open: No such file or directory ##(SQL) >> >> > > > > > > mv: cannot stat `sqlcilib/linux/64bit/debug/sqlci_yacc.cpp': >> >> > > > > > > No such file or directory ##(SQL) >> >> > > > > > > sed: can't read sqlcilib/linux/64bit/debug/sqlci_yacc.cpp.tmp: >> >> > > > > > > No >> >> > > > such >> >> > > > > > > file or directory ##(SQL) >> >> > > > > > > sed: can't read sqlcilib/linux/64bit/debug/sqlci_yacc.hpp: >> >> > > > > > > No such file or >> >> > > > > > > directory ##(SQL) >> >> > > > > > > rm: cannot remove `sqlcilib/linux/64bit/debug/sqlci_yacc.hpp': >> >> > > > > > > No >> >> > > > such >> >> > > > > > > file or directory ##(SQL) >> >> > > > > > > rm: cannot remove >> >> > `sqlcilib/linux/64bit/debug/sqlci_yacc.cpp.tmp': >> >> > > > > > > No such >> >> > > > > > > file or directory ##(SQL) >> >> > > > > > > make[4]: *** [sqlcilib/linux/64bit/debug/sqlci_yacc.h] Error >> >> > > > > > > 1 >> >> > > > > > > ##(SQL) >> >> > > > > > > make[4]: Leaving directory >> >> > > > > > > `/home/centos/incubator-trafodion/core/sql/nskgmake' ##(SQL) >> >> > > > > > > make[3]: *** [all] Error 2 ##(SQL) >> >> > > > > > > make[3]: Leaving directory >> >> > > > > > > `/home/centos/incubator-trafodion/core/sqf/sql' >> >> > > > > > > ##(SQL) >> >> > > > > > > make[2]: *** [make_sql] Error 2 >> >> > > > > > > make[2]: Leaving directory >> >> > > > `/home/centos/incubator-trafodion/core/sqf' >> >> > > > > > > make[1]: *** [foundation] Error 2 >> >> > > > > > > make[1]: Leaving directory >> >> > `/home/centos/incubator-trafodion/core' >> >> > > > > > > make: *** [all] Error 2 >> >> > > > > > > >> >> > > > > > > >> >> > > > > > > On Mon, Mar 7, 2016 at 12:05 PM, Amanda Moran >> >> > > > > > <[email protected] <mailto:[email protected]> > >> >> > > > > > > wrote: >> >> > > > > > > >> >> > > > > > >> On redhat 7.1 this command still works: >> >> > > > > > >> >> >> > > > > > >> [ec2-user@ip-10-0-0-175 ~]$ cat /etc/redhat-release Red Hat >> >> > > > > > >> Enterprise Linux Server release 7.1 (Maipo) >> >> > > > > > >> >> >> > > > > > >> [ec2-user@ip-10-0-0-175 ~]$ grep processor /proc/cpuinfo | >> >> > > > > > >> wc -l >> >> > > > > > >> 4 >> >> > > > > > >> >> >> > > > > > >> >> >> > > > > > >> On Mon, Mar 7, 2016 at 9:56 AM, Steve Varnau >> >> > > > > > >> <[email protected] <mailto:[email protected]> > >> >> > > > > > >> wrote: >> >> > > > > > >> >> >> > > > > > >> > > It seems that the parallel make fails on 8 GB machines. >> >> > > > > > >> > >> >> > > > > > >> > I think your first sentence overstates the determinism of >> >> > > > > > >> > the >> >> > > > > problem >> >> > > > > > >> > a >> >> > > > > > >> > bit. >> >> > > > > > >> > I ran a normal, default build on 8GB machine last week >> >> > > > > > >> > and had no >> >> > > > > > >> problem. >> >> > > > > > >> > There must be an environmental problem, but I don't think >> >> > > > > > >> > we fully understand it yet. >> >> > > > > > >> > >> >> > > > > > >> > The aggressiveness of the make parallelism is set in >> >> > > > > > >> core/sqf/sqenvcom.sh. >> >> > > > > > >> > It sets the parallel factor based on how many CPUs are on >> >> > > > > > >> > your >> >> > > > > > >> > machine: >> >> > > > > > >> > >> >> > > > > > >> > # Set default build parallelism # Can be overridden on >> >> > > > > > >> > make commandline cpucnt=$(grep processor /proc/cpuinfo | >> wc -l) >> >> > > > > > >> > # no number means unlimited, and will swamp the system >> >> > > > > > >> > export MAKEFLAGS="-j$cpucnt" >> >> > > > > > >> > >> >> > > > > > >> > If that calculation is wrong, maybe that could cause a >> >> > problem. >> >> > > > > > >> > >> >> > > > > > >> > --Steve >> >> > > > > > >> > >> >> > > > > > >> > >> >> > > > > > >> > > -----Original Message----- >> >> > > > > > >> > > From: Gunnar Tapper [mailto:[email protected] >> <[email protected]>] >> >> > > > > > >> > > Sent: Monday, March 7, 2016 9:35 AM >> >> > > > > > >> > > To: [email protected] <mailto: >> >> > [email protected]> >> >> > > > > > >> > > Subject: Parallel Make Failures >> >> > > > > > >> > > >> >> > > > > > >> > > Hi, >> >> > > > > > >> > > >> >> > > > > > >> > > It seems that the parallel make fails on 8 GB machines. >> >> > > > > > >> > > At least, >> >> > > > > > >> Nitin >> >> > > > > > >> > > and >> >> > > > > > >> > > I both ran into make failures that did not appear when >> >> > > > > > >> > > running serial make. >> >> > > > > > >> > > I've also seen similar failures when building the code >> >> > > > > > >> > > on >> >> > > > > > >> > > 12 GB >> >> > > > > > >> machines. >> >> > > > > > >> > > >> >> > > > > > >> > > Based on previous discussions, the Trafodion >> >> > > > > > >> > > Contributor Guide >> >> > > > > > >> recommends >> >> > > > > > >> > > rerunning make a few times if running issues. >> >> > > > > > >> > > >> >> > > > > > >> > > I most wonder if there's a way to reduce the >> >> > > > > > >> > > aggressiveness of the >> >> > > > > > >> make >> >> > > > > > >> > in >> >> > > > > > >> > > general. Could we, for example, come up with a table >> >> > > > > > >> > > that correlates system size to define the -l option or >> >> > > > > > >> > > something similar? >> >> > > > > > >> > > >> >> > > > > > >> > > -- >> >> > > > > > >> > > Thanks, >> >> > > > > > >> > > >> >> > > > > > >> > > Gunnar >> >> > > > > > >> > > *If you think you can you can, if you think you can't >> >> > > > > > >> > > you're >> >> > > > > > >> > > right.* >> >> > > > > > >> > >> >> > > > > > >> >> >> > > > > > >> >> >> > > > > > >> >> >> > > > > > >> -- >> >> > > > > > >> Thanks, >> >> > > > > > >> >> >> > > > > > >> Amanda Moran >> >> > > > > > >> >> >> > > > > > > >> >> > > > > > > >> >> > > > > > > >> >> > > > > > > -- >> >> > > > > > > Thanks, >> >> > > > > > > >> >> > > > > > > Gunnar >> >> > > > > > > *If you think you can you can, if you think you can't you're >> >> > > > > > > right.* >> >> > > > > > > >> >> > > > > > >> >> > > > > > >> >> > > > > > >> >> > > > > > -- >> >> > > > > > Thanks, >> >> > > > > > >> >> > > > > > Gunnar >> >> > > > > > *If you think you can you can, if you think you can't you're >> >> > > > > > right.* >> >> > > > > >> >> > > > >> >> > > > >> >> > > > >> >> > > > -- >> >> > > > Thanks, >> >> > > > >> >> > > > Gunnar >> >> > > > *If you think you can you can, if you think you can't you're >> >> > > > right.* >> >> > > > >> >> > > >> >> > > >> >> > > >> >> > > -- >> >> > > Thanks, >> >> > > >> >> > > Gunnar >> >> > > *If you think you can you can, if you think you can't you're right.* >> >> > > >> >> > >> >> > >> >> > >> >> > -- >> >> > Thanks, >> >> > >> >> > Gunnar >> >> > *If you think you can you can, if you think you can't you're right.* >> >> > >> >> >> -- >> >> Thanks, >> >> Gunnar >> >> *If you think you can you can, if you think you can't you're right.* >> > > > > -- > Thanks, > > Gunnar > *If you think you can you can, if you think you can't you're right.* >
