subject:"Re\: \[arts\-users\] 52% of CTEST fails during ARTS compilation"

Re: [arts-users] 52% of CTEST fails during ARTS compilation

2020-10-09 Thread Reno Choi

Hi Oliver,

Thanks for the link to the arts-xml-data. It certainly made a big
difference, let alone its size from 327 Mb (my 5 year old one) to 3.23 Gb
(the latest).

"make check" results reproduce exactly what you described - only one error
from "TestSpectroscopy.arts".

Now I can be sure that my compile is accomplished. Many thanks to your help.

Rgds,
Reno


On Fri, 9 Oct 2020 at 18:00, Oliver Lemke 
wrote:

> Hi Reno,
>
> > On 9 Oct 2020, at 09:56, Reno Choi  wrote:
> >
> > Hi Oliver,
> >
> > Many thanks for the prompt response. Sounds like it's CTEST, not a
> compile process.
> >
> > I followed your 1st test and showed 3 tests failed out of 71,
> >
> > 66 - arts.ctlfile.fast.artscomponents.helpers.TestRegridding (Failed)
> > 70 - arts.ctlfile.fast.artscomponents.wfuns.TestSpectroscopy (Failed)
> > 71 - arts.ctlfile.fast.artscomponents.cia.TestCIADerivs (Failed)
> >
> > Because of the following missing xml and data files, respectively.
> >
> > Cannot find input file:
> spectroscopy/cia/borysow/Borysow_CIA_JUICE_SWI.xml
> > Cannot find input file: spectroscopy/PartitionSums/TIPS/tips.xml
> > Cannot find input file:
> planets/Earth/ECMWF/ERA40/SurfaceAltitude_ERA40_1.0Degree
> >
> > Full screenshot is attached in "arts_make_check_results.txt".
>
> Since you're using the latest ARTS, you also have to make sure that you
> check out the latest arts-xml-data from our subversion repository.
> According to the path you specify in your cmake call, your still using
> arts-xml-data 2.2.5. You can check out the current arts-xml-data with:
>
> svn co https://arts.mi.uni-hamburg.de/svn/rt/arts-xml-data/trunk
> arts-xml-data
>
> > The 2nd test(make check-all) was carried out and 23 tests failed out of
> 225. This is far better than my earlier report and almost all were due to
> missing input files (Can you tell me where they are, btw?). The full
> screenshot is also attached herewith as "arts_make_check-all_results.txt".
> >
> > One thing that caught my eyes was at "arts_convert" during the "make
> check-all" process. It looks like arts' workspace methods are not well
> understood in converting into Python. Is my understanding correct? and are
> they to be examined before I use Pyarts?
>
> The conversion script currently just tries to convert every .arts
> controlfile it can find. The once that are failing are outdated and will be
> removed. It's safe to ignore these conversion failures for now. They have
> no impact on PyARTS.
>
> Cheers,
> Oliver
>
>

-- 
Reno K.-Y. Choi, PhD

M. (+82 0)10 4570 9066
O. (+82 0)64 780 6618
M. (+44 0)7946 459 108
___
arts_users.mi mailing list
arts_users.mi@lists.uni-hamburg.de
https://mailman.rrz.uni-hamburg.de/mailman/listinfo/arts_users.mi

Re: [arts-users] 52% of CTEST fails during ARTS compilation

2020-10-09 Thread Reno Choi

Hi Richard,

As mentioned, I've used Homebrew GCC ,
not OSX native LLVM/Clang compiler. It's not clear the GCC supports
"FASTWIGNER" option, but I went for it b'cause of OpenMP support.

I wish my report contains deeper investigations on well-known bug(s).
Hopefully possible in near future.

I've occasionally used ARTS since 6 years and I'm impressed (and
appreciated) with you and handful people's efforts to improve ARTS.

Rgds,
Reno
___
arts_users.mi mailing list
arts_users.mi@lists.uni-hamburg.de
https://mailman.rrz.uni-hamburg.de/mailman/listinfo/arts_users.mi

Re: [arts-users] 52% of CTEST fails during ARTS compilation

2020-10-09 Thread Oliver Lemke

Hi Reno,

> On 9 Oct 2020, at 09:56, Reno Choi  wrote:
> 
> Hi Oliver,
> 
> Many thanks for the prompt response. Sounds like it's CTEST, not a compile 
> process.
> 
> I followed your 1st test and showed 3 tests failed out of 71,
> 
> 66 - arts.ctlfile.fast.artscomponents.helpers.TestRegridding (Failed)
> 70 - arts.ctlfile.fast.artscomponents.wfuns.TestSpectroscopy (Failed)
> 71 - arts.ctlfile.fast.artscomponents.cia.TestCIADerivs (Failed)
> 
> Because of the following missing xml and data files, respectively.
> 
> Cannot find input file: spectroscopy/cia/borysow/Borysow_CIA_JUICE_SWI.xml
> Cannot find input file: spectroscopy/PartitionSums/TIPS/tips.xml
> Cannot find input file: 
> planets/Earth/ECMWF/ERA40/SurfaceAltitude_ERA40_1.0Degree
> 
> Full screenshot is attached in "arts_make_check_results.txt".

Since you're using the latest ARTS, you also have to make sure that you check 
out the latest arts-xml-data from our subversion repository. According to the 
path you specify in your cmake call, your still using arts-xml-data 2.2.5. You 
can check out the current arts-xml-data with:

svn co https://arts.mi.uni-hamburg.de/svn/rt/arts-xml-data/trunk arts-xml-data

> The 2nd test(make check-all) was carried out and 23 tests failed out of 225. 
> This is far better than my earlier report and almost all were due to missing 
> input files (Can you tell me where they are, btw?). The full screenshot is 
> also attached herewith as "arts_make_check-all_results.txt".
> 
> One thing that caught my eyes was at "arts_convert" during the "make 
> check-all" process. It looks like arts' workspace methods are not well 
> understood in converting into Python. Is my understanding correct? and are 
> they to be examined before I use Pyarts?

The conversion script currently just tries to convert every .arts controlfile 
it can find. The once that are failing are outdated and will be removed. It's 
safe to ignore these conversion failures for now. They have no impact on PyARTS.

Cheers,
Oliver



smime.p7s
Description: S/MIME cryptographic signature
___
arts_users.mi mailing list
arts_users.mi@lists.uni-hamburg.de
https://mailman.rrz.uni-hamburg.de/mailman/listinfo/arts_users.mi

Re: [arts-users] 52% of CTEST fails during ARTS compilation

2020-10-09 Thread Richard Larsson

Hi Reno,

A quick glance at your building procedure.  It seems like you are pointing
against a very old version of arts-xml-data.  Can you confirm that it is
the latest arts-xml-data from the dev-branch that you have got there?
Because the files should be there in the development branch.  Also, I am
not sure your compiler supports the "FASTWIGNER" option.  It relies on a
3rdparty library, and the developer specifically only supports GCC.  I know
clang and gcc are often similar, so it should work, but be aware of this.

You will probably still run into some issues, as Oliver states, because of
Mac being strange.  I added the spectroscopy derivative tests but I have no
Mac to test this on so I cannot fix the issue (it's the zero-crossings of
the derivatives, and we don't have a solution for making the comparison
reliable yet).

The tests that fail to convert to python are already on our bug tracker.
They are globbed into the python converter even though they are not run by
any test and therefore have absolutely no guarantee to work or even run.
There should be an update before we move off the development branch and
release a stable version of ARTS that removes or fixes all of these issues.

With hope,
//Richard

Den fre 9 okt. 2020 kl 09:57 skrev Reno Choi :

> Hi Oliver,
>
> Many thanks for the prompt response. Sounds like it's CTEST, not a compile
> process.
>
> I followed your 1st test and showed 3 tests failed out of 71,
>
> 66 - arts.ctlfile.fast.artscomponents.helpers.TestRegridding (Failed)
>> 70 - arts.ctlfile.fast.artscomponents.wfuns.TestSpectroscopy (Failed)
>> 71 - arts.ctlfile.fast.artscomponents.cia.TestCIADerivs (Failed)
>
>
> Because of the following missing xml and data files, respectively.
>
> Cannot find input file: spectroscopy/cia/borysow/Borysow_CIA_JUICE_SWI.xml
>> Cannot find input file: spectroscopy/PartitionSums/TIPS/tips.xml
>> Cannot find input file:
>> planets/Earth/ECMWF/ERA40/SurfaceAltitude_ERA40_1.0Degree
>
>
> Full screenshot is attached in "arts_make_check_results.txt".
>
> The 2nd test(make check-all) was carried out and 23 tests failed out of
> 225. This is far better than my earlier report and almost all were due to
> missing input files (Can you tell me where they are, btw?). The full
> screenshot is also attached herewith as "arts_make_check-all_results.txt".
>
> One thing that caught my eyes was at "arts_convert" during the "make
> check-all" process. It looks like arts' workspace methods are not well
> understood in converting into Python. Is my understanding correct? and are
> they to be examined before I use Pyarts?
>
> [100%] Converting *.arts controlfiles to Python
>> Running arts_convert ...
>> Error converting TestClouds_Venus.arts: abs_linesReadFromSplitArtscat is
>> not a known workspace method.
>> Error converting TestSpectro_core.arts: abs_linesReadFromHitran is not a
>> known workspace method.
>> Error converting TestClouds_Mars.arts: abs_linesReadFromSplitArtscat is
>> not a known workspace method.
>> Error converting TestReadCataloguePerrin.arts:
>> abs_linesReadFromSplitArtscat is not a known workspace method.
>> Error converting TestRadioLink.arts: iyRadioLink is not a known workspace
>> method.
>> Error converting TestRadioOccultation.arts: iyRadioLink is not a known
>> workspace method.
>> Error converting TestMolTau.arts: abs_linesReadFromArts is not a known
>> workspace method.
>> Error converting TestRelmat.arts: abs_lines_per_bandFromband_identifiers
>> is not a known workspace method.
>> Error converting odinsmr_544_absorption.arts: abs_linesReadFromArts is
>> not a known workspace method.
>> Error converting seviri_hitran.arts: abs_linesReadFromHitran is not a
>> known workspace method.
>> Error converting mviri_hitran.arts: abs_linesReadFromHitran is not a
>> known workspace method.
>> Error converting use-absOnTheFly.arts: abs_linesReadFromSplitArtscat is
>> not a known workspace method.
>> Error converting AbsLookup.arts: abs_linesReadFromSplitArtscat is not a
>> known workspace method.
>> Error converting AbsOnTheFly.arts: abs_linesReadFromSplitArtscat is not a
>> known workspace method.
>> Error converting make-and-use-absLUT_1Dfor3D.arts:
>> abs_linesReadFromSplitArtscat is not a known workspace method.
>> Error converting DemoScat_DOIT_1D.arts: iyInterpLinCloudboxField is not a
>> known workspace method.
>> Error converting DemoScat_FOS_1D.arts: iyFOS is not a known workspace
>> method.
>> Error converting SpecCat_Consistency_Perrin-vs-HITRAN.arts:
>> abs_linesReadFromHitran is not a known workspace method.
>> arts_convert done. Converted 230 out of 248 files.
>> [100%] Built target python_tests
>
>
>  Many thanks,
> Reno
>
>
> On Fri, 9 Oct 2020 at 15:06, Oliver Lemke 
> wrote:
>
>> Hi Reno,
>>
>> It looks like all the Python related tests are failing. This is currently
>> a flaw in our build setup when calling 'ctest' manually. It circumvents the
>> dependency tracking of 'make' and doesn't trigger the conversion of the
>>

Re: [arts-users] 52% of CTEST fails during ARTS compilation

2020-10-09 Thread Oliver Lemke

Hi Reno,

It looks like all the Python related tests are failing. This is currently a 
flaw in our build setup when calling 'ctest' manually. It circumvents the 
dependency tracking of 'make' and doesn't trigger the conversion of the 
controlfiles to Python. Try to use the provided check targets instead:

For basic controlfile checks:

make check

Now TestSpectroscopy should be the only test failing (due to differences in the 
math library on macOS, we're aware of this issue).

For all checks:

make check-all

To speed up the test execution, like you did by passing '-j4' to 'ctest', you 
can instead add the option '-DTEST_JOBS=4' to your 'cmake' command.

Hope this helps.

Cheers,
Oliver


> On 9 Oct 2020, at 05:39, Reno Choi  wrote:
> 
> Hello,
> 
> I've recently compiled the latest ARTS from GitHub and found many errors from 
> CTEST.
> 
> The ARTS are installed on my MacBook Pro, with the following environment,
>   • Catalina OSX v.10.15.7 with the latest Xcode 12.0.1 (12A7300)
>   • gcc/g++/gfortran 10.2.0 from Homebrew*
>   • Anaconda3
>   • MecTex
>   • and, all prerequisites shown in README.md
> * Add additional library and include path for the compiler to find the system 
> headers and libraries.
> >export 
> >LDFLAGS="-L/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib"
> >export 
> >CPPFLAGS="-I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include"
> 
> For cmake, 
> 
> > cmake -DARTS_XML_DATA_PATH=/Users/reno/Work/arts-xml-data-2.2.5 
> > -DWITH_HITRAN2008=0 -DENABLE_FORTRAN=1 
> > -DCMAKE_Fortran_COMPILER=/usr/local/Cellar/gcc/10.2.0.reinstall/bin/gfortran-10
> >  -DENABLE_TMATRIX_QUAD=0 
> > -DCMAKE_C_COMPILER=/usr/local/Cellar/gcc/10.2.0.reinstall/bin/gcc-10 
> > -DCMAKE_CXX_COMPILER=/usr/local/Cellar/gcc/10.2.0.reinstall/bin/g++-10 
> > -DENABLE_NETCDF=1 -DFASTWIGNER=1 ..
> 
> The cmake results are attached in 'arts_cmake_results.txt', which was 
> successful. Then, went for make,
> 
>  > make -j9
> 
> The screen capture are in 'arts_compile_results.txt'. The executable files 
> were generated, whie with a number of warnings (~100).
> 
> > ./arts -v
> arts-2.3.1287 (git: 60279fd2) (compiled Fri Oct  9 01:07:03 2020)
> Compiler: GNU 10.2.0 (/usr/local/Cellar/gcc/10.2.0/bin/g++-10)
> Compile flags:  -fvisibility=hidden -fvisibility-inlines-hidden 
> -ftemplate-depth-1024 -fopenmp  -W  -Wall  -Wshadow  -Wconversion  
> -Wno-sign-conversion  -Wno-unknown-pragmas  -Wno-strict-overflow   -O2 -g
> Features in this build:
>Numeric precision:double
>OpenMP support:   enabled
>Documentation server: enabled
>Zipped XML support:   enabled
>NetCDF support:   enabled
>Fortran support:  enabled (gfortran-10)
>Legacy Fortran Disort:disabled
>RT4 support:  enabled
>FASTEM support:   enabled
>OEM support:  enabled
>MPI support for OEM:  disabled
>Refice support:   enabled
>Tmatrix support:  enabled (double-precision)
>Hitran Xsec support:  enabled (experimental)
> Include search paths:
>.
>/Users/reno/Work/arts/controlfiles
> Data searchpaths:
>.
> 
> Now, I ran CTEST and results showed 108 tests failed out of 225. The 
> screenshot of CTEST results can be found in "arts_ctest_results.txt". I 
> carried out a single CTEST to see in detail,
> 
> (base) reno@Renos-MBP build % ctest -R TestForloop
> Test project /Users/reno/Work/arts/build
> Start 10: arts.ctlfile.fast.artscomponents.helpers.TestForloop
> 1/2 Test #10: arts.ctlfile.fast.artscomponents.helpers.TestForloop .. 
>   Passed0.04 sec
> Start 11: python.arts.ctlfile.fast.artscomponents.helpers.TestForloop
> 2/2 Test #11: python.arts.ctlfile.fast.artscomponents.helpers.TestForloop 
> ...***Failed0.03 sec
> 50% tests passed, 1 tests failed out of 2
> Total Test time (real) =   0.09 sec
> The following tests FAILED:
> 11 - python.arts.ctlfile.fast.artscomponents.helpers.TestForloop (Failed)
> Errors while running CTest
> 
> Nonetheless, I wondered if any example controlfiles were working. I tried 
> "TestAbs.art" and it seemed ok to me (please, see attached 
> "arts_TestAbs.arts_results.txt").
> 
> So far, I am curious if my compilation was successful or not. What does 52% 
> of CTEST fails mean? Am I missing anything at all?
> 
> I wonder if anyone in this community has similar experience or comments to 
> share.
> 
> Regards,
> Reno
> 
> 
> -- 
> Reno K.-Y. Choi, PhD
> 
> M. (+82 0)10 4570 9066
> O. (+82 0)64 780 6618
> M. (+44 0)7946 459 108
> ___
> arts_users.mi mailing list
> arts_users.mi@lists.uni-hamburg.de
> https://mailman.rrz.uni-hamburg.de/mailman/listinfo/arts_users.mi



smime.p7s
Description: S/MIME cryptographic signature
___
arts_users.mi mailing list
arts_users.mi@lists.uni-hamburg.de
https://mailman.rrz.uni-hamburg.de/mailman/listinfo/arts_users.mi

Re: [arts-users] 52% of CTEST fails during ARTS compilation

Re: [arts-users] 52% of CTEST fails during ARTS compilation

Re: [arts-users] 52% of CTEST fails during ARTS compilation

Re: [arts-users] 52% of CTEST fails during ARTS compilation

Re: [arts-users] 52% of CTEST fails during ARTS compilation

5 matches

Site Navigation

Mail list logo

Footer information