Hi,
Sorry I was mostly on vacation, so somewhat aloof. Now that I'm
visiting Pittsburgh for a month, I should be a bit more responsive.
The test has been redone to not depend on --help or --version. The
shell script passes a command line option. The Moses Jam script will
recognize this and decide based on version number. If the Moses jam
script doesn't work or the version number is too old, it will use the
provided version.
Kenneth
On 07/26/2012 03:51 AM, Tom Hoar wrote:
> We are moving beyond supporting only Ubuntu. We tested the bjam binary
> and bjam Bash script on Ubuntu 10.04 & 12.04 (LTS versions), Debian
> 6.0.4, Fedora 17 and openSuSE 12.1.
>
> In all cases, we installed the bjam binary and libboost from the native
> repositories on these newest Linux distros. It is true that Debian
> distros (including Ubuntu) place boost and bjam in different packages.
> Fedora 17 and openSuSE 12.1 install one package named "boost-jam". On
> all hosts, bypassing the bjam Bash script and using the bjam binary
> builds Moses without errors when given the complete command line.
>
> The bjam Bash script on all 5 hosts, however, fails to recognize the
> host's bjam binary and falls-back to the bjam version distributed with
> Moses. The bjam distributed with Moses worked every time. If this is the
> behavior you want, then the Bash script isn't broken.
>
> All failures of the Bash script to identify the host's bjam version come
> from the different command line interfaces (sometimes undocumented). As
> an example, the "stuff about boost-build.jam missing" message comes from
> bjam's failure to display usage information as defined in the manpage
> description of the --help argument. Instead, when launched from $HOME,
> it looks for something to build and then reports it didn't find
> anything. When launched from the Moses folder, it tries to build Moses
> without enough information to complete the job successfully. That
> doesn't mean bjam broken because it still builds Moses when given the
> complete command line.
>
> Here is how the different bjam binary options worked on the 5 hosts:
>
> --help : failed to display usage information on
> all five hosts. It is only in the manpage
> of Ubuntu 12.04 only.
>
> --version : failed on all hosts except Ubuntu 12.04.
> It is not documented in either the
> manpage or usage prompt of any version.
> Ubuntu 12.04 reports with format:
> Boost.Build V2 (Milestone 12)
> Boost.Jam 03.1.19
>
> -v : supported by all bjam tested versions and
> is in the manpage of all five versions.
> The report format is the same except the
> exact format of the version number on the
> first line of the output.
> Ubuntu 10.04 format:
> Boost.Jam Version 3.1.16. OS=LINUX.
> Ubuntu 12.04 format:
> Boost.Jam Version 3.1.19. OS=LINUX.
> Debian 6.0.4 format:
> Boost.Jam Version 3.1.16. OS=LINUX.
> Fedora 17 format:
> Boost.Jam Version 2011.4. OS=LINUX.
> openSuSE 12.1 format:
> Boost.Jam Version 3.1.19. OS=LINUX.
>
> You can reverse your changes to the Bash file if you like because we
> will not use them. Instead, we will removed the code to test the host's
> bjam altogether because the tests are unreliable, and only use the bjam
> distributed with Moses.
>
> Tom
>
>
>
> On Wed, 25 Jul 2012 21:29:28 -0400, Kenneth Heafield
> <[email protected]> wrote:
>> Hi,
>>
>> I'm trying to understand: is there a bug with the script as it
>> currently operates? Is there a broken version of bjam that it accepts
>> and uses?
>>
>> The grep command intentionally does not match your version string and
>> therefore uses the Moses internal version of bjam. It's not a bug,
>> it's a feature :-).
>>
>> Ubuntu put the bjam binary and build files in separate packages, but
>> the two are useless without each other. That's why you get the stuff
>> about boost-build.jam missing.
>>
>> Moreover, boost build is now distributed as part of the Boost tarball
>> and they stopped doing independent releases. But Ubuntu distributes
>> their last independent release.
>>
>> Kenneth
>>
>> On 07/25/12 20:36, Tom Hoar wrote:
>>> Hi Ken,
>>>
>>> I did a short test. I installed bjam on Ubuntu 10.04 using the
>>> repository (bjam version 3.1.16). According to the manpage, this bjam
>>> version does not support the --version and --help arguments at all.
>>> Here's the shorter usage output using the -h argument:
>>>
>>> $ /usr/bin/bjam -h
>>> Invalid option: -h
>>>
>>> usage: /usr/bin/bjam [ options ] targets...
>>>
>>> -a Build all targets, even if they are current.
>>> -dx Set the debug level to x (0-9).
>>> -fx Read x instead of Jambase.
>>> -jx Run up to x shell commands concurrently.
>>> -lx Limit actions to x number of seconds after which they are stopped.
>>> -n Don't actually execute the updating actions.
>>> -ox Write the updating actions to file x.
>>> -px x=0, pipes action stdout and stderr merged into action output.
>>> -q Quit quickly as soon as a target fails.
>>> -sx=y Set variable x=y, overriding environment.
>>> -tx Rebuild x, even if it is up-to-date.
>>> -v Print the version of jam and exit.
>>> --x Option is ignored.
>>>
>>>
>>> Also, Here's the output with --help, which according to the manpage of
>>> this version is not a valid argument:
>>>
>>> $ /usr/bin/bjam --help
>>> Unable to load Boost.Build: could not find "boost-build.jam"
>>> ---------------------------------------------------------------
>>> Attempted search from /home/tahoar/domy-2.5/src/mosesdecoder up to
>>> the root
>>> and in these directories from BOOST_BUILD_PATH and BOOST_ROOT:
>>> /usr/share/boost-build.
>>> Please consult the documentation at 'http://www.boost.org'.
>>>
>>>
>>> I used the -v argument with the results:
>>>
>>> $ /usr/bin/bjam -v
>>> Boost.Jam Version 3.1.16. OS=LINUX.
>>> Copyright 1993-2002 Christopher Seiwald and Perforce Software, Inc.
>>> Copyright 2001 David Turner.
>>> Copyright 2001-2004 David Abrahams.
>>> Copyright 2002-2005 Rene Rivera.
>>> Copyright 2003-2005 Vladimir Prus.
>>>
>>> The bjam Bash script's --version argument fails, but if it worked, the
>>> subsequent "grep" command would fail to find the output string you set
>>> in the Bash script.
>>>
>>>
>>> I went back to the version in 12.04 (bjam version 3.1.19). According to
>>> the manpage, it should support the --help argument, but in reality, it
>>> produces no help output and launches a full bjam session. The manpage
>>> does not list the --version argument, but in reality argument outputs
>>> yesterday's report. The -v argument gives a longer report as above, but
>>> with version 3.1.19.
>>>
>>>
>>> There are significant (& poorly documented) inconsistencies between the
>>> various bjam binary's command interfaces and their outputs (Debian
>>> distro versions 3.1.16 & 03.1.19, and your version). They make it
>>> difficult to use them with any confidence across different platforms.
>>> For example, reversing the lines in the Bash script only skips the
>>> system bjam on any Debian distro version for the last 2+ years (I
>>> searched on Debian.org) and forces the use of the moses copy of bjam.
>>>
>>> Would it be more reliable to remove references to the system bjam
>>> altogether from the Bash script?
>>>
>>>
>>>
>>>
>>> On Wed, 25 Jul 2012 18:39:30 -0400, Kenneth Heafield
>>> <[email protected]> wrote:
>>>> Hi,
>>>>
>>>> I've reversed the order of the checks, so that version goes first.
>>>> This way, your system bjam will fail the version check (as, apparently,
>>>> it should) before it gets to --help.
>>>>
>>>> Kenneth
>>>>
>>>> On 07/25/12 12:14, Tom Hoar wrote:
>>>>> I built a new machine with Ubuntu 12.04 Server. I learned the default
>>>>> setup installs bjam. (all my previous installs were on systems without
>>>>> bjam installs).
>>>>>
>>>>> ~$ bjam --version
>>>>> Boost.Build V2 (Milestone 12)
>>>>> Boost.Jam 03.1.19
>>>>>
>>>>> I'm using the last 20 July commit of Moses. My bjam command line is:
>>>>>
>>>>> src/mosesdecoder$ sudo ./bjam -j4 -a --debug-configuration
>>>>> --with-xmlrpc-c --prefix="/usr/local/lib/mosesdecoder"
>>>>> --install-scripts="/usr/local/lib/mosesdecoder/scripts"
>>>>> --with-irstlm=/usr/local --with-randlm=/usr/local
>>>>> --with-srilm=/usr/local
>>>>>
>>>>> The system hung with no stderr or stdout output. Top showed a single
>>>>> instance of cc1plus (not four per -j4) and it hung for about 30
>>>>> minutes
>>>>> before I hit Ctrl-C. I traced the problem to the moses bjam shell
>>>>> script. If I remove the two lines "${bjam}" --help & --version, the
>>>>> compile completes fine.
>>>>>
>>>>> if
>>>>> bjam="$(which bjam 2>/dev/null)" && #exists
>>>>> [ ${#bjam} != 0 ] && #paranoia about which printing nothing then
>>>>> returning true
>>>>> ! grep UFIHGUFIHBDJKNCFZXAEVA "${bjam}" </dev/null >/dev/null #bjam in
>>>>> path isn't this script
>>>>> then
>>>>> #Delegate to system bjam
>>>>> exec "${bjam}" "$@"
>>>>> fi
>>>>>
>>>>> In my case, "${bjam}" resolves to /usr/bin/bjam. More troubleshooting
>>>>> found:
>>>>>
>>>>> 1) the "/usr/bin/bjam --help" command ignores the --help argument and
>>>>> tries to process the Jamfiles in the default folder. When issuing the
>>>>> "/usr/bin/bjam --help" command from a default folder without any
>>>>> Jamfiles, the command terminates and reports "error: error: no Jamfile
>>>>> in current directory found, and no target references specified." From
>>>>> the Moses folder and with the Bash script redirecting both stderr and
>>>>> stdout to /dev/null, started processing without the command line
>>>>> scripts
>>>>> and system hung.
>>>>>
>>>>> 2) The --version output is a different format than the "grep" search
>>>>> expects and the bjam Bash script skips the system bjam installation
>>>>> and
>>>>> moves on to the bjam distributed with Moses.
>>>>>
>>>>> For my purposes, I will remove the two lines. For the main trunk, the
>>>>> --help command should be removed at a minimum. The grep line would
>>>>> have
>>>>> to be re-designed to handle the new report format if you really
>>>>> need to
>>>>> identify the Boost version.
>>>>>
>>>>> Tom
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Moses-support mailing list
>>>>> [email protected]
>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>> _______________________________________________
>>>> Moses-support mailing list
>>>> [email protected]
>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
