Re: [arts-users] ARTS 2.6.2 source code

[Richard Larsson]

Tyler,

2.6 onwards do not require a recompilation if you use the conda package.

All the binaries are included.

//Richard

On Fri, Mar 29, 2024, 10:25 Summers, Tyler (GSFC-613.0)[SCIENCE SYSTEMS AND
APPLICATIONS INC]  wrote:

> Hello,
>
>
>
> Regarding the new ARTS 2.6.2 release; I have installed the new version of
> pyARTS using “conda install -c rttools pyarts” as instructed. Do I also
> need to download a new version of the C++ source code and recompile? I
> noticed the installation instructions for the initial install listed on the
> github ahve changed since version 2.4.
>
>
>
> Thank you,
>
> Tyler
>
>
>
> *_*
>
> *Tyler Summers (he/him/his)*
>
> Scientific Programmer
>
>
>
> Science Systems and Applications Inc.
>
> NASA Goddard Climate and Radiation Lab
>
>
>

Re: [arts-users] ReadHITRAN problem

[Richard Larsson]

Hi Jiaan.

I recommend using the arts-cat-data package we provide.  It is available at
https://www.radiativetransfer.org/tools/ under the heading ARTS Catalog
Data.  I have added Zeeman effect and line mixing of O2 to those lines (the
exact changes are in my repository at https://github.com/atmtools/arts-cat).
They should be otherwise very similar as to what HITRAN provides.  It might
only work on a newer version of ARTS (which you can install easily if you
use conda-forge following our guide at
https://atmtools.github.io/arts-docs-master/installation.html).

//Richard

Den tors 7 dec. 2023 kl 07:33 skrev suifengbenpao2023 <
suifengbenpao2...@163.com>:

> Hi Richard,
>
> Thank you for your careful explanation. I do want to consider the Zeeman
> splitting effect (especially oxygen) and accelerate the calculation speed.
> According to your explanation, I need to consider quantum numbers. Is there
> a way for me to consider quantum numbers? Can we set the global quantum
> numbers and local quantum numbers for ReadHITRAN? I have tried this and
> found that the definition format of quantum numbers seems to be different
> from before? It seems that it cannot be set to：
> localquanta="J Omega F" upperglobalquanta="S 1/2 Lambda 1 parity 1
> kronigParity 102 v1 0 ElectronState 88" lowerglobalquanta="S 1/2 Lambda 1
> parity -1 kronigParity 102 v1 0 ElectronState 88"
> He reported an error saying that there cannot be numbers such as 2/1, 1,
> etc.
>
> Do I have any other options?
>
>
> Looking forward to your reply！Thank you!
>
> Best wishes,
>
> Jiaan He
>
>
>
> At 2023-12-07 13:16:42, "Richard Larsson"  wrote:
>
> Hi Jiaan,
>
> qns' and qns'' contain HITRAN quantum numbers (mostly based on VAMDC).
> There is some quantum number definition in the pure 150-char par-line, but
> we do not parse it as it is generally not possible with the information
> available (the paper-published format doesn't always agree with what is
> actually published in data).
>
> Your LBL calculations should not be affected by just reading or not
> reading the qns' and qns''.  If you want more accurate LBL results, say you
> want to activate Zeeman effect or need line mixing, you will need the
> quantum numbers to do that.  Likewise, if you want to speed up your
> calculations then having the quantum numbers also helps with that since it
> is easier to throw away bands.  So you are just limiting what you can do,
> not what the results of doing nothing extra will bring.
>
> //Richard
>
> Den tors 7 dec. 2023 kl 04:17 skrev suifengbenpao2023 <
> suifengbenpao2...@163.com>:
>
>> Hi Oliver，
>>
>> Thank you for your help! However, I have a doubt. I compared the
>> extracted data with the data in the directory/home/zc/arts XML
>> data-2.4/spectoscopy/Hitran. Their respective results are as follows:
>>
>> （1）> mirroringtype="None" populationtype="LTE" normalizationtype="None"
>> lineshapetype="VP" T0="296" cutofffreq="7.5e+11" linemixinglimit="-1"
>> localquanta="" upperglobalquanta="" lowerglobalquanta=""
>> broadeningspecies="SELF AIR" temperaturemodes="G0 T1 T1 D0 T5 T5">
>> 9021534521.6108 2.9232326101396e-22 3.16646203680995e-20 26 26 8.491e-11
>> nan nan 88761.6455958549 0.66 0 0 0 0.66 0 0 11834.8860794473 0.66 0 0 0
>> 0.66 0 0  (HITRAN2020)
>> （2）> mirroringtype="None" populationtype="LTE" normalizationtype="None"
>> lineshapetype="VP" T0="296" cutofffreq="0" linemixinglimit="-1"
>> localquanta="J Omega F" upperglobalquanta="S 1/2 Lambda 1 parity -1
>> kronigParity 101 v1 0 ElectronState 88" lowerglobalquanta="S 1/2 Lambda 1
>> parity 1 kronigParity 102 v1 0 ElectronState 88" broadeningspecies="SELF
>> AIR" temperaturemodes="G0 T1 T1 D0 T5 T5">
>> 9021534521.6108 2.9232326101396e-22 3.16646203680995e-20 26 26 8.491e-11
>> 0 0 11834.8860794473 0.66 0 0 0 0.66 0 0 11834.8860794473 0.66 0 0 0 0.66 0
>> 0 23/2 1/2 25/2 23/2 1/2 25/2
>>
>> In terms of labels and values, there are differences in their results, so
>> I would like to ask if the results I extracted in this way are correct?
>> Will it affect the line by line absorption calculation?
>>
>> Looking forward to your reply！Thank you!
>>
>> Best wishes,
>>
>> Jiaan He
>>
>> At 2023-12-05 14:52:23, "Lemke, Oliver"  wrote:
>> >Hi Jiaan,
>> >
>> >All parameters ARTS can us are included in the .par line (includ

Re: [arts-users] ReadHITRAN problem

[Richard Larsson]

Hi Jiaan,

qns' and qns'' contain HITRAN quantum numbers (mostly based on VAMDC).
There is some quantum number definition in the pure 150-char par-line, but
we do not parse it as it is generally not possible with the information
available (the paper-published format doesn't always agree with what is
actually published in data).

Your LBL calculations should not be affected by just reading or not reading
the qns' and qns''.  If you want more accurate LBL results, say you want to
activate Zeeman effect or need line mixing, you will need the quantum
numbers to do that.  Likewise, if you want to speed up your calculations
then having the quantum numbers also helps with that since it is easier to
throw away bands.  So you are just limiting what you can do, not what the
results of doing nothing extra will bring.

//Richard

Den tors 7 dec. 2023 kl 04:17 skrev suifengbenpao2023 <
suifengbenpao2...@163.com>:

> Hi Oliver，
>
> Thank you for your help! However, I have a doubt. I compared the extracted
> data with the data in the directory/home/zc/arts XML
> data-2.4/spectoscopy/Hitran. Their respective results are as follows:
>
> （1） mirroringtype="None" populationtype="LTE" normalizationtype="None"
> lineshapetype="VP" T0="296" cutofffreq="7.5e+11" linemixinglimit="-1"
> localquanta="" upperglobalquanta="" lowerglobalquanta=""
> broadeningspecies="SELF AIR" temperaturemodes="G0 T1 T1 D0 T5 T5">
> 9021534521.6108 2.9232326101396e-22 3.16646203680995e-20 26 26 8.491e-11
> nan nan 88761.6455958549 0.66 0 0 0 0.66 0 0 11834.8860794473 0.66 0 0 0
> 0.66 0 0  (HITRAN2020)
> （2） mirroringtype="None" populationtype="LTE" normalizationtype="None"
> lineshapetype="VP" T0="296" cutofffreq="0" linemixinglimit="-1"
> localquanta="J Omega F" upperglobalquanta="S 1/2 Lambda 1 parity -1
> kronigParity 101 v1 0 ElectronState 88" lowerglobalquanta="S 1/2 Lambda 1
> parity 1 kronigParity 102 v1 0 ElectronState 88" broadeningspecies="SELF
> AIR" temperaturemodes="G0 T1 T1 D0 T5 T5">
> 9021534521.6108 2.9232326101396e-22 3.16646203680995e-20 26 26 8.491e-11 0
> 0 11834.8860794473 0.66 0 0 0 0.66 0 0 11834.8860794473 0.66 0 0 0 0.66 0 0
> 23/2 1/2 25/2 23/2 1/2 25/2
>
> In terms of labels and values, there are differences in their results, so
> I would like to ask if the results I extracted in this way are correct?
> Will it affect the line by line absorption calculation?
>
> Looking forward to your reply！Thank you!
>
> Best wishes,
>
> Jiaan He
>
> At 2023-12-05 14:52:23, "Lemke, Oliver"  wrote:
> >Hi Jiaan,
> >
> >All parameters ARTS can us are included in the .par line (including 
> >broadening). ARTS will not utilize any other parameters besides what's 
> >contained in the .par line, qns' and qns''. Adding other parameters will 
> >also most likely break our reading routine.
> >
> >Cheers,
> >Oliver
> >
> >
> >> On 5. Dec 2023, at 02:30, suifengbenpao2023  
> >> wrote:
> >>
> >>
> >> Hi Oliver，
> >>
> >> Thank you for your help! Thank you for providing the online screenshot. 
> >> May I ask if you need to add any other parameters that can be obtained to 
> >> define the new output format according to your screenshot? For example, 
> >> self widening, external widening, etc.
> >>
> >> Looking forward to your reply！Thank you!
> >> Best wishes,
> >> Jiaan He
> >>
> >> At 2023-12-04 21:26:59, "Lemke, Oliver"  
> >> wrote:
> >> >Hi Jiaan,
> >> >
> >> >"Online" is the recommended way. You also need to define a custom output 
> >> >format to include the .par line and the quantum numbers qns' and qns''. 
> >> >Here is a screenshot with the configuration from the HITRAN online page:
> >> >
> >> >https://attachment.rrz.uni-hamburg.de/97c802f4/Screenshot-2023-12-04-at-14.04.57.png
> >> >
> >> >Cheers,
> >> >Oliver
> >> >
> >> >
> >> >> On 4. Dec 2023, at 13:33, suifengbenpao2023  
> >> >> wrote:
> >> >>
> >> >> Dear ARTS community,
> >> >>
> >> >> When I directly use the data downloaded from hitranonline (without 
> >> >> changing the data format), I can run the ReadHITRAN method to extract 
> >> >> lines data. I used the "Post2004" and "Online" methods, and the results 
> >> >> are the same. Is this correct?
> >> >>
> >> >> Looking forward to your reply！Thank you!
> >> >> Best wishes,
> >> >> Jiaan He
> >> >
> >
> >
> >
> >
> >
>
>

Re: [arts-users] **SPAM** Zeeman effect of complete oxygen models

[Richard Larsson]

Dear Shaofei,

No.

If you are only in the upper atmosphere, you need to use line-by-line when
considering the Zeeman Effect.

The full models fail there anyways because they don't consider Doppler
effects or Zeeman effects.

If your beam leaves the upper atmosphere, you need to do additional work.

With hope,
Richard




On Sun, Oct 8, 2023, 04:19 Shaofei Wang  wrote:

> Hi all,
>
> Recently, I've been doing some work on the BT simulations for 50-60 GHz.
>
> I have a question. Can I just use complete oxygen models provided by ARTS
> when considering the Zeeman effect for upper atmosphere? Or do I have to
> use the absorption line and the required line parameters to calculate the
> Zeeman effect?
>
> Best regards,
> Shaofei Wang
>
> 
>

Re: [arts-users] read in HITRAN par file

[Richard Larsson]

Dear Claudia,

There is no news.  It worked quite well recently in ARTS 2.5.X when last I
updated our redistributed catalog (at the arts-cat-data subversion server;
though this is not pure Hitran as we have some line mixing and line shapes
we select manually included).   I just confirmed that this still works by
manually downloading some data.

The box titled:

Available Parameters

contains all parameters.  It is a scroll box.  It is available after you
select to create a new format, so you should see it.  You have to scroll
down a bit to see the three fields I mentioned last year, but they are
still there.

With hope,
//Richard

Den mån 16 jan. 2023 kl 17:01 skrev Claudia Emde :

> Dear ARTS community,
>
> I have exactly the same problem that Pengwang has reported. Is there any
> news about reading HITRAN 2020 in ARTS?
>
> I tried to follow the instructions by Richard, but when I choose "create
> new output format" I do not see e.g. the ".par line" tag and the "qns' "
> tag. I only find a window where I can select available parameters to be
> written into the .par file.
>
> Best regards,
> Claudia
>
>
>
> On 2/8/22 11:24, Richard Larsson wrote:
> > Dear Pengwang,
> >
> > You should be able to read a Hitran file in 2.4 that you could read in
> > 2.3.  Now, it sounds like you have downloaded a new version of the
> > Hitran catalog.  Arts 2.4 does not directly support Hitran2020, neither
> > did 2.3 to be clear.  For water, I still think this should still be Ok
> > but I am not sure.
> >
> > If you want to give it a try, the 'modern' way of dealing with Hitran
> > data in Arts is to download it with the additional tags in the Hitran
> > download interface.  On the tab currently called '4. Select or Create
> > Output Format'*, you can create a new output format.  To create a new
> > format that works well with Arts, this format should contain at the top
> > the ".par line" tag, as the second option it should contain the " qns' "
> > tag, and as a last option it should have the " qns'' " tag.  The field
> > separator and line endings should be the default ([tab] ; (CR LF)).  If
> > you manage to download data like this from Hitran, you can give the
> > hitran_type="Online" option in ReadHITRAN.  This should give decent
> results.
> >
> > Again, remember that Arts 2.4 is from before Hitran2020 was released.
> > It is possible this will not work in 2.4.  What I describe above works
> > for the development version of Arts (v.2.5).
> >
> > With hope,
> > //Richard
> >
> > * From hitran.org <http://hitran.org>, Data Access -> Line-By-Line ->
> > select H2O and go forward -> go forward (or deselect some isotopologues
> > first) -> go forward (or select some Kayser range first) -> You are now
> > on the right tab to specify output format
> >
> > Den tors 3 feb. 2022 kl 16:06 skrev Pengwang Zhai  > <mailto:pwz...@umbc.edu>>:
> >
> > Hello, ARTS Community,
> >
> > I used arts 2.3 to generate gas absorption coefficient lookup table
> > for o2, co2, etc. One key function was: "abs_linesReadFromHitran”.
> > Now I need to revisit the calculation with HITRAN2020. I thought I
> > would use arts 2.4, a newer version. After successfully compiling
> > arts2.4, and run the same .arts file I used before (a modification
> > of TestAbs.arts), arts tells me that “abs_linesReadFromHitran” is
> > not available any more. I went over the documentation, and found
> > “READHITRAN”. I tried the following:
> >
> > ReadHITRAN( abs_lines,
> > “./h2o_61fbd3ce.par",
> >  1.3e+14,
> >  1.0e+15 )
> >
> > arts reports:
> >
> > "Run-time error in method: ReadHITRAN
> > Error parsing quantum number Sym”.
> >
> > Is there any way we can read in HITRAN par file and create a lookup
> > table? Do you suggest me going back to arts 2.3?
> >
> > Thanks for your advice,
> >
> > Pengwang
> >
> > 
> > Pengwang Zhai (he/his/him)
> > Graduate Program Director, Atmospheric Physics,
> > Associate Professor, Physics Department, UMBC
> > pwz...@umbc.edu <mailto:pwz...@umbc.edu>
> >
> >
> >
> >
> >
> >
> > ___
> > arts_users.mi mailing list
> > arts_users.mi@lists.uni-hamburg.de
> > <mailto:arts_users.mi@lists.uni-hamburg.de>
> > https://mailman.rrz.uni-hamburg.de/mailman/listinfo/arts_users.mi
> > <https://mailman.rrz.uni-hamburg.de/mailman/listinfo/arts_users.mi>
> >
> >
> > ___
> > arts_users.mi mailing list
> > arts_users.mi@lists.uni-hamburg.de
> > https://mailman.rrz.uni-hamburg.de/mailman/listinfo/arts_users.mi
>
> --
> Dr. Claudia Emde
> Fakultät für Physik, Meteorologisches Institut
> Ludwig-Maximilians-Universität
> Theresienstrasse 37, 80333 München
> Phone: +49 89 21804098, Fax: +49 89 2805508
>

Re: [arts-users] A (newbie) question about Pyarts

[Richard Larsson]

Dear Mattia,

What version are you running?  We found a memory leak in 2.5.6 that we
fixed in 2.5.8.  Just do "pyarts.version" in a python interpreter to see
your version string.

//Richard

Den mån 9 jan. 2023 kl 15:12 skrev Mattia Sabatini <
mattia.sabat...@artov.ismar.cnr.it>:

> Dear all,
>
> I wrote a Python script that uses Pyarts for executing batch calculations
> with ARTS.
>
> My task is to simulate top of atmosphere monochromatic radiance together
> with Jacobians of temperature, water vapour and surface temperature in the
> thermal infrared range using the Garand atmospheric profiles.
>
> Essentially the script consists of two nested loops:
>
> -
>
> from pyarts import workspace
>
> ws = workspace.Workspace()
>
>- for loop #1
>   - for loop #2
>  - ws.ybatchCalc()
>  - ws.WriteXML(ybatch filename)
>  - subprocess.run (gzip ybatch xml file)
>  - ws.WriteXML(ybatch_jacobians filename)
>  - subprocess.run (gzip ybatch_jacobians xml file)
>
> -
>
> The two for loops are used for calculations of spectral reflectivities,
> given that there are not emissivity models in ARTS suitable for the
> spectral range in exam.
>
> As the python execution goes on, the memory usage keeps increasing slowly
> but constantly, and eventually the process is killed. Even if I select only
> 4-5 profiles instead of the whole batch. The only way to make things work
> seems to be deactivating the Jacobians calculations for most of the
> variables (or all of them), or perhaps reducing the number of pressure
> levels onto which these calculations are performed.
>
> I was wondering why this is happening, given that both ybatch and
> ybatch_jacobians workspace variables are overwritten every time ybatchCalc
> is executed.
>
> Thanks,
>
> Mattia
>
>
>

Re: [arts-users] Issues using "scat_data" in ARTS dev version

[Richard Larsson]

Dear Renish,

This is not a real problem.  What it says is that we have no way to print a
SingleScatteringData type to screen.  You have a working
SingleScatteringData.  We realize that this is confusing so in the future
the __repr__ function you are calling (by just typing the .value in your
ipython terminal) will return the name of the variable by default.

You still have access to all the underlying data such as
ws.scat_data.value[0].ptype and ws.scat_data.value[0].description as you
can see here:
https://atmtools.github.io/arts-docs-master/pyarts/stubs/pyarts.arts.SingleScatteringData.html#pyarts.arts.SingleScatteringData
(though the information about the fields are a bit limited, they should
return the original information you loaded into them).  You can also
manipulate the content here, with all the risk that entails for future use
of the data.

With hope,
//Richard

Den fre 2 dec. 2022 kl 04:59 skrev Thomas,Renish <
renish.tho...@colostate.edu>:

> Dear ARTS Community,
>
> I am using the dev version of ARTS to use some of the newer workspace
> methods and was experiencing issues with "scat_data".
> After loading 'scat_data,' the values say: "SingleScatteringData: Output
> operator not implemented."
> As this was not an issue with the stable ARTS 2.4 version, I'd like to
> know if something has changed with the scattering data functions.
>
> Any help or hints are appreciated!
>
> Sample error Info:
> ws.scat_data.value
> Out[19]: [[SingleScatteringData: Output operator not implemented,
> SingleScatteringData: Output operator not implemented,
> SingleScatteringData: Output operator not implemented,
> SingleScatteringData: Output operator not implemented
> Thanks very much!
>
> Version: arts-2.5.7 (compiled 2022-11-22 13:20:32)
>
> Renish Thomas
> Microwave Systems Laboratory
> Colorado State University
> Fort Collins, CO
>
>

Re: [arts-users] Inquiry about arts-lectures on github

[Richard Larsson]

Hi Ryosuke,

I am not familiar with the arts-lecture package in general, but this issue
is because you lack the arts-xml-data folder in your path.  The
arts-xml-data can be downloaded using subversion or as a zip from:
https://www.radiativetransfer.org/tools/

You then need to set a system environment variable called ARTS_DATA_PATH to
point at the arts-xml-data main folder for the script to find the data.  If
you go into the arts-xml-data folder, you will see that it contains a path
to spectroscopy/Artscat/ where the necessary files are, and ARTS will look
there because of the environment variable.  If you do not want to set the
environment variable, just point basename to the same absolute path.  I
recommend the environment solution as there is a lot of helper data in
arts-xml-data and I suspect that the arts-lecture uses it extensively, as
you will also if you use ARTS for other things in the future.

Note that you need the version of the arts-xml-data that is for version 2.4
of ARTS as I have changed this a lot since then.

With hope,
//Richard

Den ons 9 nov. 2022 kl 03:39 skrev 田村 亮祐 :

> Dear ARTS members
>
>
>
> Nice to meet you.
>
> I’m Ryosuke Tamura working at Japanese Space Agency.
>
> Dr Y. Kasai invited me to arts mailing list.
>
>
>
> Now, I want to use arts-lecture on github.
>
>
> https://github.com/atmtools/arts-lectures/blob/master/exercises/01-rotational_spectra/absorption.ipynb
>
>
>
> To calculate absorption cross section, I have downloaded XML data style
> catalog from
>
> https://radiativetransfer.org/misc/download/stable/2.4/.
>
>
>
> Absorption_module.py program inside arts-lecture defines the data catalog
> path as follows.
>
> ws.abs_lines_per_speciesReadSpeciesSplitCatalog(
>
>basename="spectroscopy/Artscat/"
>
> )
>
> However, the downloaded XLM data (version2.4) doesn’t have the name of
> directory “Artscat”.
>
> To use the arts-lecture program, how to set basename=”” ?
>
>
>
> If latest XML data catalog doesn’t have Artscat,
>
> I’d appreciate it if you tell me the version and its URL of the catalog
> which has Artscat.
>
>
>
> Best regards
>
>
>
>
>
> 
>
> RYOSUKE Tamura
>
>   Sensor System Research Group
>
>   Japan Aerospace Exploration Agency (JAXA)
>
>   2-1-1, Sengen, Tsukuba, Ibaraki, 305-8505, Japan
>
>   tamura.ryohs...@jaxa.jp
>
> 
>

Re: [arts-users] Temperature Retrievals of Oxygen emission lines with Zeeman effect and line mixing

[Richard Larsson]

Dear Witali,

You can use my calculations for line mixing.  They are still not published
but available online in the arts-cat-data.  They have the disadvantage that
they are not published but the advantage that they are self-consistent.
You can download this data at https://www.radiativetransfer.org/tools/,
under the Arts Catalog Data header.

The format is simply x0 x1 x2 x3 from table 2.5 of the theory guide at the
correct positions by what the xml-tags say.

Note that the paper you found has been updated several times.  Two updates
are at these dois: http://dx.doi.org/10.1016/j.jqsrt.2013.02.019 and
https://doi.org/10.1016/j.jqsrt.2019.106798

Note especially that these correct their values in the first paper because
they found some theoretical errors.  Also note that my implementation in
Arts today differs from the one written in these papers because there are
some weird typos in there.

With hope,
--Richard

Den mån 12 sep. 2022 kl 13:30 skrev :

> Dear ARTS Team
>
>
> I am currently using the Zeeman module to retrieve stratospheric
> temperature profiles from oxygen emission lines from fine structure
> transitions at 52.542 and 53.067 GHz. I correct for the tropospheric
> influence using a simple tropospheric correction. However, I intend to
> improve the inversion algorithm by including line-mixing calculations
> instead.
>
> First, I am having trouble figuring out how to get line-mixing data. I
> found "Makarov, Tretyakov, and Rosenkranz, 2011" as a standard reference
> for line-mixing data in this frequency range. However, it seems to me that
> only the emission complex at 60 GHz is covered there. Is there any line
> mixing data for the emission lines at 52.542 and 53.067 GHz?
>
>
>
> Also, I would need some guidance on the implementation of the line mixing
> module. I guess the data set should be in a certain format, which I did not
> find in the ARTS manual or documentation (maybe I missed something?).
>
>
>
> With kind regards
>
> Witali Krochin
>
>

Re: [arts-users] Changing base parameter for abs_lines

[Richard Larsson]

Dear Eric,

The QI is not available for the data you are looking at.

It stands for QuantumIdentifier, which is the class in Arts that deals with
quantum numbers and gas species for the purpose of identifying a unique
absorption line.  In 2.4, you define this as something like "O3-666 TR UP J
1 LO J 2", for the gas species O3-666 with upper quantum number J=1 and
lower quantum number J=2.  I am not sure in 2.4 if you can simply define a
QuantumIdentifier in pyarts or if you are required to load it from a file.
You can see example files in the spectroscopy/zeeman/*.qid.xml files for
how an energy level is defined with this format (the EN tag makes the UP
and LO tags for different levels unnecessary).

Since the Perrin data does not provide this id-data, the lines are not
unique by Arts standards (or any standard really).  Thus, you cannot use
these methods to change any line parameters directly.  I'm afraid you have
to use more manual methods.  I will make a suggestion below.

You could define 'fake' quantum numbers for the lines that you are
interested in.  If you set localquanta="J" in the AbsorptionLines xml-tag,
and then append "0 0" for the first absorption line, "1 1" for the second,
and so on for all the absorption lines.  Now the lines that you are
interested in can be found using  "O3-666 TR UP J i LO J i", where i would
be the absorption line index.  This way you could perform the calculations
that you are after and use the built-in methods to change the base
parameters for the lines you identify as important.

Note though that if you go with this solutions and your interest is just to
compute the sensitivities, there are two Jacobian methods available for
that purpose - jacobianAddShapeCatalogParameter
and jacobianAddBasicCatalogParameter - that already analytically computes
what the partial derivative wrt the added catalog parameters will be.
Using them is the same as using any other type of Jacobian in Arts, so you
should be able to compute the sensitivities in a somewhat straight-forward
manner.  (The catalog_identity for these are the QI of the other
methods.).  You can add as many unique parameters (for multiple absorption
lines) you want and the output should just be OK.

With hope,
//Richard

Den tors 10 feb. 2022 kl 07:40 skrev :

> Dear ARTS community,
>
>
> In order to perform some sensitivity analysis on ozone retrievals, I would
> like to artificially modify some absorption lines parameters (e.g. line
> strength and frequencies) on the ozone line.
>
>
> From my understanding, this would be the goal of either ones the following
> workspace methods:
>
>
> abs_linesChangeBaseParameterForMatchingLines()
>
>
> abs_lines_per_speciesChangeBaseParameterForSpecies()
>
>
> which both require among a QI ("quantum identifier to match the line")
> parameter.
>
>
> If this is the case, I am not sure about what exactly is this QI and where
> I can find it (or set it) for a specific absorption line ?
>
>
> I am using ARTS 2.4 (through pyarts) and am defining the spectroscopy by
> reading the Perrin_newformat_speciessplit/o3-666.xml.gz located in the
> original ARTS-xml-data folder (2.4) and then uning the
> abs_lines_per_speciesCreateFromLines() method.
>
>
> Thanks for your help.
>
>
> Best,
>
> Eric
>
>
>
>
> ___
> arts_users.mi mailing list
> arts_users.mi@lists.uni-hamburg.de
> https://mailman.rrz.uni-hamburg.de/mailman/listinfo/arts_users.mi
>
___
arts_users.mi mailing list
arts_users.mi@lists.uni-hamburg.de
https://mailman.rrz.uni-hamburg.de/mailman/listinfo/arts_users.mi

Re: [arts-users] Cannot create abs_lookup with arts 2.5

[Richard Larsson]

Dear all,

The deprecation warning is there because we will remove
abs_xsec_per_speciesAddLines before Arts 3.0 is released.  It is just
warning you that some code will have to change in the future.  Ignore it
for the purpose of what you want to do here.

About the speed.  The new code should be comparable and often faster than
the old code (depending on your line-data inputs).

There are two caveats to that though.
  1)  Hitran2020 is much larger than previous copies.
  2)  You will not see parallelism unless you have enough temperature
perturbations.  There is a pending pull-request to fix this.

As Oliver points out, the recommended data to use is the
arts-cat-data/lines/ data.  This is based on Hitran2020 with a few
modifications we found necessary to improve the quality of the catalog.

//Richard

Den mån 14 feb. 2022 kl 13:49 skrev Pengwang Zhai :

> Hello, Mattia,
>
> Thanks for the update. I have to admit that I did not use the line.xml
> file. I used the H2O file downloaded from HITRAN 2020. Originally I said
> “it seems working”, today I still cannot definitely confirm whether it
> works from my side, as ARTS 2.5 did not finish the calculation after
> several days. Maybe I am trying to calculate too many frequency lines,
> though I believe ARTS 2.3 only took several hours to build the same lookup
> table.
>
> I did see the same warning that you saw regarding
> abs_xsec_per_speciesAddLines.
>
> Yours,
>
> Pengwang
>
> 
> Pengwang Zhai (he/his/him)
> Graduate Program Director, Atmospheric Physics,
> Associate Professor, Physics Department, UMBC
> pwz...@umbc.edu
>
>
>
>
>
>
> > On Feb 14, 2022, at 5:00 AM, Mattia Sabatini <
> mattia.sabat...@artov.ismar.cnr.it> wrote:
> >
> > Hello again Pengwang,
> >
> > I tested controlfile TestAbs.arts with your edit:
> >
> > AgendaSet( abs_xsec_agenda ){
> >abs_xsec_per_speciesInit
> >abs_xsec_per_speciesAddConts
> >lbl_checkedCalc
> >abs_xsec_per_speciesAddLines
> > }
> >
> > And with the following abs species:
> >
> > abs_speciesSet( species=[ "H2O", "H2O-PWR98", "O2-PWR93",
> "N2-SelfContStandardType" ] ).
> >
> > I thought that by adding abs_xsec_per_speciesAddLines in
> abs_xsec_agenda, just like Richard suggested me to do in my post (
> https://www.mail-archive.com/arts_users.mi@lists.uni-hamburg.de/msg00471.html),
> would solve the problem you were experiencing of having a lookup table
> filled with zeros. Unfortunately this is still an issue for me, even with
> the above settings: the H2O cross section values are all zeros, while
> values for "H2O-PWR98", "O2-PWR93", "N2-SelfContStandardType” are not.
> Also, during the execution I had this “Deprecated function warning”:
> >
> > abs_xsec_per_speciesAddLines is deprecated since 2021-07-13
> >
> > This function is no longer up to date.  It only exists to satisfy lookup
> table calculations before these are updated.
> > Once the lookup table calculations are up-to-date, this function is
> fully replaced with propmat_clearskyAddLines, with better functionality
> >
> > Apparently, abs_xsec_per_speciesAddConts is doing its job but
> abs_xsec_per_speciesAddLines does not. Did you had the chance to test it
> too?
> >
> > Mattia
> >
> >
> >
> >
> >
> >
> > Thanks, Oliver and Mattia, for you help.
> >
> > After studying Mattia and Richard's posts, I figured out the following:
> >
> > Comment out line 15 of TestAbs.arts:
> >
> > #Copy(abs_xsec_agenda, abs_xsec_agenda__noCIA)
> >
> > And insert the following to line 16:
> >
> > AgendaSet( abs_xsec_agenda ){
> >abs_xsec_per_speciesInit
> >abs_xsec_per_speciesAddConts
> >lbl_checkedCalc
> >abs_xsec_per_speciesAddLines
> > }
> >
> > Now it seems working, though I have not yet got the chance to check the
> outputs as it is still running.
> >
> > Yours
> >
> > Pengwang
> >
> >
> >
> >
> >
> >
> >
> > > On Feb 11, 2022, at 2:06 AM, Lemke, Oliver <
> oliver.le...@uni-hamburg.de> wrote:
> > >
> > > Hi Mattia, hi Pengwang,
> > >
> > > Thanks Mattia for helping out. :-)
> > >
> > > Since you were referring to Richard's earlier post, I'll take this
> opportunity to point out the searchable archive of this list, which might
> come in handy at times:
> > >
> > > https://www.mail-archive.com/arts_users.mi@lists.uni-hamburg.de/
> > >
> > > Here is the post Matti

Re: [arts-users] read in HITRAN par file

[Richard Larsson]

Dear Pengwang,

You should be able to read a Hitran file in 2.4 that you could read in
2.3.  Now, it sounds like you have downloaded a new version of the Hitran
catalog.  Arts 2.4 does not directly support Hitran2020, neither did 2.3 to
be clear.  For water, I still think this should still be Ok but I am not
sure.

If you want to give it a try, the 'modern' way of dealing with Hitran data
in Arts is to download it with the additional tags in the Hitran download
interface.  On the tab currently called '4. Select or Create Output
Format'*, you can create a new output format.  To create a new format that
works well with Arts, this format should contain at the top the ".par line"
tag, as the second option it should contain the " qns' " tag, and as a last
option it should have the " qns'' " tag.  The field separator and line
endings should be the default ([tab] ; (CR LF)).  If you manage to download
data like this from Hitran, you can give the hitran_type="Online" option in
ReadHITRAN.  This should give decent results.

Again, remember that Arts 2.4 is from before Hitran2020 was released.  It
is possible this will not work in 2.4.  What I describe above works for the
development version of Arts (v.2.5).

With hope,
//Richard

* From hitran.org, Data Access -> Line-By-Line -> select H2O and go forward
-> go forward (or deselect some isotopologues first) -> go forward (or
select some Kayser range first) -> You are now on the right tab to specify
output format

Den tors 3 feb. 2022 kl 16:06 skrev Pengwang Zhai :

> Hello, ARTS Community,
>
> I used arts 2.3 to generate gas absorption coefficient lookup table for
> o2, co2, etc. One key function was: "abs_linesReadFromHitran”. Now I need
> to revisit the calculation with HITRAN2020. I thought I would use arts 2.4,
> a newer version. After successfully compiling arts2.4, and run the same
> .arts file I used before (a modification of TestAbs.arts), arts tells me
> that “abs_linesReadFromHitran” is not available any more. I went over the
> documentation, and found “READHITRAN”. I tried the following:
>
> ReadHITRAN( abs_lines,
>“./h2o_61fbd3ce.par",
> 1.3e+14,
> 1.0e+15 )
>
> arts reports:
>
> "Run-time error in method: ReadHITRAN
> Error parsing quantum number Sym”.
>
> Is there any way we can read in HITRAN par file and create a lookup table?
> Do you suggest me going back to arts 2.3?
>
> Thanks for your advice,
>
> Pengwang
>
> 
> Pengwang Zhai (he/his/him)
> Graduate Program Director, Atmospheric Physics,
> Associate Professor, Physics Department, UMBC
> pwz...@umbc.edu
>
>
>
>
>
>
> ___
> arts_users.mi mailing list
> arts_users.mi@lists.uni-hamburg.de
> https://mailman.rrz.uni-hamburg.de/mailman/listinfo/arts_users.mi
>
___
arts_users.mi mailing list
arts_users.mi@lists.uni-hamburg.de
https://mailman.rrz.uni-hamburg.de/mailman/listinfo/arts_users.mi

Re: [arts-users] ARTS 2.5.0 - absorption

[Richard Larsson]

Dear Mattia,

It looks like you are removing line calculations from your lookup
calculations.

Could you try adding abs_xsec_per_speciesAddLines to your lookup
cross-section agenda?  I think this will solve the problem.

We are currently in a transition in the 2.5-branch of moving away from
abs_xsec_agenda entirely.  There are some lingering problems when using
lookup table calculations at this time.  Mainly, since the line
calculations should happen in propmat_clearsky_agenda now, they are not
part of any of the default cross-section agendas as in the past.  The
lookup generation interface is currently under active development, so this
interface might change in the near future.

With hope,
//Richard

Den fre 21 jan. 2022 kl 14:53 skrev Mattia Sabatini <
mattia.sabat...@artov.ismar.cnr.it>:

> Dear all,
>
>
>
> I have a question concerning the absorption calculation with ARTS 2.5.0.
> Here are some details about the controlfile I wrote:
>
>
>
> 1D atmosphere, clear sky, 4 IR channels
>
>
>
> VectorLinSpace( f_grid, 2.02456915e13, 4.2827494e13, 5e8 ) – i.e. from 7um
> to 14.8 um
>
>
>
> abs_species = "H2O, H2O-SelfContCKDMT100, H2O-ForeignContCKDMT100", "O3”,
> "CO2, CO2-CKDMT100", "N2O", "CH4", "CFC11-HXSEC", "CFC12-HXSEC",
> "HCFC22-HXSEC", "CFC113-HXSEC", "CFC114-HXSEC".
>
>
>
> abs_linesReadSpeciesSplitCatalog(abs_lines, "cat/") - from
> arts-xml-data-2.4/spectroscopy
>
> ReadXsecData(basename="coefficients_arts/") – from arts-crossfit-main/
>
>
>
> On-the-fly absorption set with
>
>
>
> AgendaSet( propmat_clearsky_agenda ){
>
>   Ignore(rtp_mag)
>
>   Ignore(rtp_los)
>
>   Ignore(rtp_nlte)
>
>   propmat_clearskyInit
>
>   propmat_clearskyAddXsecAgenda
>
>   propmat_clearskyAddLines
>
>   propmat_clearskyAddHitranXsec
>
> }
>
>
>
> AgendaSet( abs_xsec_agenda ){
>
>   abs_xsec_per_speciesInit
>
>   abs_xsec_per_speciesAddConts
>
> }
>
>
>
> Batch_atm_fields_compact from Eresmaa 137L, gases VMR from FASCOD except
> for CFC ones which are set as constant.
>
>
>
> Surface set with
>
>
>
> AgendaSet( surface_rtprop_agenda ){
>
>specular_losCalc
>
>Extract( surface_skin_t, t_surface_vector, ybatch_index )
>
>surfaceFlatScalarReflectivity
>
> }
>
> (Surface_vector contains the surface skin temperatures from the Eresmaa
> surface data. The correct skin temperature is extracted during ybatchCalc,
> accordingly to ybatch_index)
>
>
>
> This is my output for the first profile (channel BTs in Kelvin):
>
> 291.688825481684
>
> 288.903990987693
>
> 287.623301104291
>
> 284.402270055496
>
>
>
> I repeated the calculation by using a lookup table, editing the
> controlfile:
>
>
>
> AgendaSet( propmat_clearsky_agenda ){
>
> Ignore(rtp_mag)
>
>   Ignore(rtp_los)
>
>   Ignore(rtp_nlte)
>
>   propmat_clearskyInit
>
>   propmat_clearskyAddFromLookup
>
>   propmat_clearskyAddHitranXsec
>
> }
>
>
>
> abs_lookupSetupBatch
>
> abs_xsec_agenda_checkedCalc
>
> lbl_checkedCalc
>
> abs_lookupCalc
>
> WriteXML( "binary", abs_lookup )
>
> ReadXML( abs_lookup )
>
> abs_lookupAdapt
>
>
>
> The lookup calculation is done in few minutes, and the output I get this
> time is:
>
>
>
> 293.350716564285
>
> 291.452571755867
>
> 296.771709918824
>
> 292.567643470471
>
>
>
> Why are the two outputs for the same profile so different? Also, why is
> the lookup table calculation so fast compared to version 2.4 (a couple of
> hours with same f_grid and atmospheric profiles)? Where did I go wrong?
>
>
>
> Mattia
> ___
> arts_users.mi mailing list
> arts_users.mi@lists.uni-hamburg.de
> https://mailman.rrz.uni-hamburg.de/mailman/listinfo/arts_users.mi
>
___
arts_users.mi mailing list
arts_users.mi@lists.uni-hamburg.de
https://mailman.rrz.uni-hamburg.de/mailman/listinfo/arts_users.mi

Re: [arts-users] Calculated brightness temperature bias causes.

[Richard Larsson]

Hi,

Just by numbers:

RJBT at 300 K 183 GHz is 3.086705214957283e-15

Planck at 300 K 183 GHz is 3.0417434132511342e-15

This means you expect a 1.5 % difference, or about 4.5 K between them.

With hope,
//Richard

Den tis 20 apr. 2021 kl 13:22 skrev Thomas,Renish <
renish.tho...@colostate.edu>:

> Hi Stephan,
>
> I am using Rayleigh jeans. As I need to activate cloud box in some
> instances.
>
> I understand that RJBT instead of Planck can cause a dip in the brightness
> temperatures. Is this the only factor that can cause a bias, or does
> pressure levels, lat/lon grid resolution also cause a bias?
>
> Thanks,
> Renish
>
>
>  Original message 
> From: Stefan Buehler 
> Date: 4/20/21 6:11 AM (GMT-06:00)
> To: "Thomas,Renish" 
> Cc: "arts_users.mi@lists.uni-hamburg.de" <
> arts_users...@mailman.rrz.uni-hamburg.de>
> Subject: Re: [arts-users] Calculated brightness temperature bias causes.
>
> Dear Renish,
>
> do you use Planck or Rayleigh-Jeans brightness temperature? For Planck,
> you should indeed approach the ambient temperature if you go low enough.
>
> Cheers
>
> Stefan
>
> On 20 Apr 2021, at 12:46, Thomas,Renish wrote:
>
> > Hi Everyone,
> >
> > I had some questions about the calculated brightness temperature in
> > ARTS.
> >
> > When I calculate the brightness temperature for an atmospheric
> > scenario in "horizon looking mode" and in clearsky. I get a brightness
> > temperature at 183.31 GHz (Water vapor absorption line), which is
> > about 3 to 6 degrees lower than the ambient temperature.
> >
> > I would assume that at the water vapor absorption line and at low
> > altitudes (~2 km above sea level), I should measure very close to the
> > ambient temperature (Due to high absorption).
> >
> > So, my questions are:
> >
> > 1.)  Is this brightness temperature bias expected?, or can something
> > else cause this?
> >
> > Thanks,
> > Renish
> > ___
> > arts_users.mi mailing list
> > arts_users.mi@lists.uni-hamburg.de
> >
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmailman.rrz.uni-hamburg.de%2Fmailman%2Flistinfo%2Farts_users.midata=04%7C01%7CRenish.Thomas%40colostate.edu%7C8bb1cd94b52a4724954408d903ed19cb%7Cafb58802ff7a4bb1ab21367ff2ecfc8b%7C0%7C0%7C637545139171957024%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=7Qr43Du%2B54FAx%2FWFOIMnjdaHFegsWnBRslp2jGQYxb8%3Dreserved=0
> ___
> arts_users.mi mailing list
> arts_users.mi@lists.uni-hamburg.de
> https://mailman.rrz.uni-hamburg.de/mailman/listinfo/arts_users.mi
>
___
arts_users.mi mailing list
arts_users.mi@lists.uni-hamburg.de
https://mailman.rrz.uni-hamburg.de/mailman/listinfo/arts_users.mi

Re: [arts-users] sensor_time definition through PyARTS

[Richard Larsson]

Hi,

This is my fault.  Explanation if wanted:  sensor_time is now a different
type.  It now stores actual time stamps like "1970-01-01 01:00:01" (which
is what "1" means in time stamps if you are in CET).

I will fix it so you can set the time from a numpy array again in pyarts.
However, I don't understand the workspace interactions all that well.  The
solution you will have to use before that is fixed will look like this:

pyarts.classes.from_workspace(ws.sensor_time).data = np.array([1])
or
pyarts.classes.from_workspace(ws.sensor_time).data = [1]
or
pyarts.classes.from_workspace(ws.sensor_time).data = ["2021-02-16 16:19:00"]

I am committing code for this shortly and it should be updated and merged
soon.

Note that you have reading/writing routines for all ARTS variables via the
"pyarts.classes.from_workspace(x).savexml("filename.xml")" interface.  In
addition about the new type for time, using
pyarts.classes.Time.TimeGrid(pyarts.classes.from_workspace(ws.sensor_time)),
you will be able to use matplotlib's time-series plotting features if you
are so inclined.

I hope we can find a way so that when there are no explicit workspace
methods, then the fall-back is to use the pyarts.classes interface but my
python is not that good to fix this myself.

Anyways, please give it a day or so for the code to do the above to be
merged before updating the development branch and you should have
vector-interaction back.

With hope,
//Richard

Den tis 16 feb. 2021 kl 15:05 skrev Patrick Eriksson <
patrick.eriks...@chalmers.se>:

> Hi,
>
> There is the same/similar problem on the Matlab side (as there is no
> writing and reading of xml-files including the time group).
>
> So a general hint for all. If you just want to set sensor_time to some
> value, you can do this by these method calls
>
> timeNow
> ArrayOfTimeSetConstant( sensor_time, 1, time )
>
> Bye,
>
> Patrick
>
>
> On 2021-02-16 14:53, eric.sauvag...@iap.unibe.ch wrote:
> > Dear ARTS community,
> >
> >
> > In order to try a recent fix suggested and implemented (commit n.
> > 5951a72fbef6d7ac9a3de8d2c503b58ef5af7d17, fromTyphon mailing list) on
> > the development version of ARTS I came into another problem trying to
> > implement my ground-based retrievals through PyARTS. In fact, I can't
> > succeed anymore in setting the sensor_time variable through PyARTS,
> > which is then required to perform the retrievals.
> >
> >
> > I'm not sure if something got mixed maybe between the 2 ARTS version
> > installed on my laptop and the 2 attached PyARTS versions (each in a
> > separate conda env) or if this is really a bug from the development
> > version.
> >
> >
> > As a very short example, please find below a minimalist python script
> > that reproduces the problem. It seems to work fine if I use ARTS2.4 and
> > produces the following error with ARTS Dev version:
> >
> >
> >
> >
> ###
> > import os
> > import numpy as np
> > import xarray as xr
> > import matplotlib.pyplot as plt
> >
> > from pyarts.workspace import Workspace, arts_agenda
> > from dotenv import load_dotenv
> >
> >
> #load_dotenv('/home/eric/Documents/PhD/ARTS/arts-examples/.env.t490-arts2.5')
> >
> load_dotenv('/home/eric/Documents/PhD/ARTS/arts-examples/.env.t490-arts2.4')
> > ARTS_DATA_PATH = os.environ['ARTS_DATA_PATH']
> > ARTS_BUILD_PATH = os.environ['ARTS_BUILD_PATH']
> > ARTS_INCLUDE_PATH = os.environ['ARTS_INCLUDE_PATH']
> >
> > def test_sensor():
> >  # Initializing Workspace object
> >  ws = Workspace(verbosity=0, agenda_verbosity=0)
> >  ws.execute_controlfile("general/general.arts")
> >  ws.execute_controlfile("general/agendas.arts")
> >  ws.execute_controlfile("general/continua.arts")
> >  ws.execute_controlfile("general/planet_earth.arts")
> >
> >  ws.sensor_los = np.array([50])
> >  ws.sensor_pos = np.array([1e3])
> >  ws.sensor_time =  np.array([1])
> >
> >  print('sensor_time is :', ws.sensor_time.value)
> >
> > if __name__=="__main__":
> >  test_sensor()
> >
> >
> ###
> >
> >
> >
> > ___
> > arts_users.mi mailing list
> > arts_users.mi@lists.uni-hamburg.de
> > https://mailman.rrz.uni-hamburg.de/mailman/listinfo/arts_users.mi
> >
> ___
> arts_users.mi mailing list
> arts_users.mi@lists.uni-hamburg.de
> https://mailman.rrz.uni-hamburg.de/mailman/listinfo/arts_users.mi
>
___
arts_users.mi mailing list
arts_users.mi@lists.uni-hamburg.de
https://mailman.rrz.uni-hamburg.de/mailman/listinfo/arts_users.mi

Re: [arts-users] Zeeman effect

[Richard Larsson]

Hi Witali,

The line mixing does not matter yet.

OK, not 0/0 but you simply are not having any quantum numbers at all (these
are the the missing numbers)

The nan numbers will matter later.  You will see another error if you
fix the first problem.  I would suggest you just keep lines with v1=0
anyways, since you won't have any effect from the v1=1 band.

The problem with the missing quantum numbers at the end of the catalog is
there because you are reading a HITRAN .par file.  We deprecated quantum
number reading from the .par files because they were not reliable.  To
still read raw HITRAN, you can download it again from their
https://hitran.org/lbl/ webpage, selecting the species you want to use
(presumably only O2-66).  You then need to modify the parameter options so
that the first entry is the full .par, the second entry is qns', and the
last entry is qns''.  I attach a picture of how the webpage selection
looks to me.  This way, all species in HITRAN will have all their quantum
numbers read properly.  You can load these into ARTS using ReadHITRAN
method again, but setting hitran_type="Online".  This will load all the
quantum numbers into ARTS and also set the Zeeman effect from pure
calculations as far as I remember.  We maintain a curated version of HITRAN
in the arts-xml-data in case you do not wish to do this.  I was hoping to
add line mixing to it down the line but I couldn't find the mixing in
HITRAN online last I checked.

With hope,
//Richard

Ps. picture of HITRAN selection box:
[image: Screenshot from 2020-11-26 12-14-24.png]

Den ons 25 nov. 2020 kl 17:58 skrev :

> Dear Mr. Eriksson
>
>
> Thank you for the answer.
>
> Since i had my script in qpack, I just generated abs_lines in ARTS 2.4 in
> a short way to demonstrate my setup.
>
>
> The code was
>
>
> Arts2{
>   abs_speciesSet(species=["O2-Z-66-53.0469e9-53.0869e9","O2-66"])
>   VectorCreate(gs)
>   ArrayOfQuantumIdentifierCreate(qid)
>   ReadXML(gs, "/arts-xml-data-2.4.0/spectroscopy/zeeman/O2-66.g.xml")
>   ReadXML(qid, "/arts-xml-data-2.4.0/spectroscopy/zeeman/O2-66.qid.xml")
>   ReadHITRAN(abs_lines,"HITRAN2012.par")
>   abs_linesSetZeemanCoefficients(abs_lines, qid, gs)
>  WriteXML( "ascii", abs_lines, abs_lines.xml")
>
> It seems to have some 'nan' entries. I compared with your example from
> 18.11.2020 and an other  difference is that the last 4 numbers are missed.
>
> 52021456295.2042 1.82478855875538e-21 2.82227115907551e-20 61 63 5.622e-10
> 0.00213411290322581 -0.0645374497084855 9172.03671157167 0.71 0 0 0 0.71 0
> 0 9467.90886355786 0.71 0 0 0 0.71 0 0
>
> 52073500265.913 1.02264066638734e-24 5.88280179901669e-20 61 63 5.64e-10
> nan nan 9172.03671157167 0.71 0 0 0 0.71 0 0 9467.90886355786 0.71 0 0 0
> 0.71 0 0
>
> 52542435628.7166 4.0664650946183e-21 2.47600404813118e-20 57 59 5.79e-10
> 0.00241711264367816 -0.0689937941931067 9172.03671157167 0.71 0 0 0 0.71 0
> 0 9467.90886355786 0.71 0 0 0 0.71 0 0
>
> 52594539557.917 2.25794239624139e-24 5.54036634348622e-20 57 59 5.808e-10
> nan nan 9172.03671157167 0.71 0 0 0 0.71 0 0 9467.90886355786 0.71 0 0 0
> 0.71 0 0
>
> 53066952513.2334 8.54258889506482e-21 2.15218198757136e-20 53 55 5.967e-10
> 0.00276410582010582 -0.0741096869563658 9172.03671157167 0.71 0 0 0 0.71 0
> 0 9467.90886355786 0.71 0 0 0 0.71 0 0
>
> 53119086421.6796 4.69902633155893e-24 5.22012465293933e-20 53 55 5.982e-10
> nan nan 9172.03671157167 0.71 0 0 0 0.71 0 0 9467.90886355786 0.71 0 0 0
> 0.71 0 0
>
> 53595786409.1454 1.68833606335819e-20 1.85085761821127e-20 49 51 6.144e-10
> 0.00319595076923077 -0.0800431763685832 9172.03671157167 0.71 0 0 0 0.71 0
> 0 9467.90886355786 0.71 0 0 0 0.71 0 0
>
> Deos line mixing affect this part? I planed to care about that as soon
> as the quantum numbers are fixed.
>
> Best regards
> Witali
> ___
> arts_users.mi mailing list
> arts_users.mi@lists.uni-hamburg.de
> https://mailman.rrz.uni-hamburg.de/mailman/listinfo/arts_users.mi
>
___
arts_users.mi mailing list
arts_users.mi@lists.uni-hamburg.de
https://mailman.rrz.uni-hamburg.de/mailman/listinfo/arts_users.mi

Re: [arts-users] Zeeman effect

[Richard Larsson]

Hi Witali,

The error means that your lines' quantum numbers are defined in a way the
ARTS Zeeman module cannot understand (we need F or J to do these
computations).  If you print the line catalog, you will probably see
several "0/0" in there for the lines you have defined as using Zeeman
effect.  Is this correct?

If it is correct, the steps to fix this is to limit where you activate the
Zeeman effect, and to define the quantum numbers there correctly.  In your
"abs_species", however you have defined it, you should limit the Zeeman
species to just the few lines you wish to compute it for.  This is often
done by simply adding "-FLOW-FHIGH" at the end of a species, and then have
the species without the "-Z-" defined after the limited species in a
different entry.  As an example:

abs_speciesSet(species=["O2-Z-66-50e9-55e9", "O2-66"])

Now the abs_lines_per_species variable that lbl_checkedCalc takes should
have an entry with only the few lines between 50 and 55 GHz.  All of these
should already have their quantum numbers defined correctly in all standard
ways to define them.  If you still encounter this problem, you need to
manually add the quantum numbers because your original catalog is not
well-understood...

(Note that you must ensure that you have line mixing defined in your line
catalog for O2 16-16 or the above tags will anyways just produce nonsense
around the lines.)

If it is not correct that you see many 0/0's, please tell me what it looks
like and I will try to figure out the problem.

With hope,
//Richard

Den tis 24 nov. 2020 kl 12:43 skrev :

>
> Hi everyone,
>
>
> I am still working on the Zeeman effect in ARTS 2.4.
>
> I have set the quantum constants with "abs_linesSetZeemanCoefficients",
> but still get the error message:
>
>
> "Run-time error in method: lbl_checkedCalc
>
>   Bad upper state F(s) or J(s)."
>
> What can that mean?
>
> Best regards
> Witali Krochin
>
>
>
> ___
> arts_users.mi mailing list
> arts_users.mi@lists.uni-hamburg.de
> https://mailman.rrz.uni-hamburg.de/mailman/listinfo/arts_users.mi
>
___
arts_users.mi mailing list
arts_users.mi@lists.uni-hamburg.de
https://mailman.rrz.uni-hamburg.de/mailman/listinfo/arts_users.mi

Re: [arts-users] Zeeman Effect ARTS 2.4 (with qpack)

[Richard Larsson]

Hi Witali,

The functions in the README require 2 inputs.  You need to give them
an ArrayOfQuantumIdentifier, qid, and a Vector, gs.  The
xml-data/spec/zeeman/ contains these files.  Load O2-66.g.xml into a
Vector.  Load O2-66.qid.xml into an ArrayOfQuantumIdentifier.  You now call
either of the two provided functions with these variables and the data will
be loaded into your line-list.  To be sure that you have read them in
correctly, and that you have done the right thing, save the line-list and
look at your lines.

They should look something like this:
52021404431.109 1.83322270858126e-21 2.82226897398507e-20 61 63 5.647e-10
0.00213411290322581 -0.0645374497084855 10059.6531675302 0.72 0 0 0 0.72 0
0 10266.7636739206 0.72 0 0 0 0.72 0 0 30 31 31 31
52542394856.9423 4.08755046918299e-21 2.47600265761908e-20 57 59 5.82e-10
0.00241711264367816 -0.0689937941931067 10355.5253195164 0.72 0 0 0 0.72 0
0 10296.3508891192 0.72 0 0 0 0.72 0 0 28 29 29 29
53066912640.8365 8.58174744782781e-21 2.15218139163761e-20 53 55 5.994e-10
0.00276410582010582 -0.0741096869563658 10651.3974715026 0.72 0 0 0 0.72 0
0 10355.5253195164 0.72 0 0 0 0.72 0 0 26 27 27 27
53595757329.277 1.69616777391079e-20 1.8508572209221e-20 49 51 6.173e-10
0.00319595076923077 -0.0800431763685832 10947.2696234888 0.72 0 0 0 0.72 0
0 10414.6997499136 0.72 0 0 0 0.72 0 0 24 25 25 25

The numbers of interest are the two before the first 10k number.  Note that
you need to have the quantum numbers defined in your line catalog to make
this work.  In ARTS 2.2 this was optional, I think, but there is an
additional and strong J dependency on the Zeeman effect numbers as we
showed in a paper from last year: "Updated Zeeman effect splitting
coefficients for molecular oxygen in planetary applications".  We had to
change the method to read these or the code would simply be wrong.

Note that the lines above are not going to be directly usable by you.  You
need to add line mixing.  There are other methods in ARTS for that but
generally your original line database must be tuned for this to begin with
or your forward calculations will make no sense anyways.

With hope,
//Richard

Den mån 16 nov. 2020 kl 18:18 skrev :

>
> Hi everyone
>
> I am trying to initialize the Zeeman effect in ARTS 2.4 using the updated
> qpack version.
> The molecule of interest is oxygen, the frequency range is around
> 52-53GHz (the rotational emission lines around 52GHz and 53 GHz).
>
> I included the method "propmat clearskyAddZeeman" in the propmat clearsky
> Agenda.
>
> However I strugle with setting the quantum numbers and constants. There is
> a Zeeman file in the xml Data folder. For me it seems that the required
> constants are stored there. The README tell me to use the methods:
>
> - abs_lines_per_speciesSetZeemanCoefficients or
> - abs_linesSetZeemanCoefficients
>
> I checked the built in documentary, but still not understand how to use
> these methods.
> For example I don't know in which workspace variable the information
> should be feeded in.
>
> In Arts 2.2  I used readXML to read the file "Zeeman_constants.xml" (was
> included in xml data) and feeded the data in the Variable WSMS_AT_START.
>
> Furhtermore there was an Zeeman demo in the older atmlab version. That was
> very helpfull and seems to be missed now.
>
> Can anyone help me to set this constants? Just a few example lines would
> help.
>
> Best regards
> Witali
>
>
> ___
> arts_users.mi mailing list
> arts_users.mi@lists.uni-hamburg.de
> https://mailman.rrz.uni-hamburg.de/mailman/listinfo/arts_users.mi
>
___
arts_users.mi mailing list
arts_users.mi@lists.uni-hamburg.de
https://mailman.rrz.uni-hamburg.de/mailman/listinfo/arts_users.mi

Re: [arts-users] 52% of CTEST fails during ARTS compilation

[Richard Larsson]

Hi Reno,

A quick glance at your building procedure.  It seems like you are pointing
against a very old version of arts-xml-data.  Can you confirm that it is
the latest arts-xml-data from the dev-branch that you have got there?
Because the files should be there in the development branch.  Also, I am
not sure your compiler supports the "FASTWIGNER" option.  It relies on a
3rdparty library, and the developer specifically only supports GCC.  I know
clang and gcc are often similar, so it should work, but be aware of this.

You will probably still run into some issues, as Oliver states, because of
Mac being strange.  I added the spectroscopy derivative tests but I have no
Mac to test this on so I cannot fix the issue (it's the zero-crossings of
the derivatives, and we don't have a solution for making the comparison
reliable yet).

The tests that fail to convert to python are already on our bug tracker.
They are globbed into the python converter even though they are not run by
any test and therefore have absolutely no guarantee to work or even run.
There should be an update before we move off the development branch and
release a stable version of ARTS that removes or fixes all of these issues.

With hope,
//Richard

Den fre 9 okt. 2020 kl 09:57 skrev Reno Choi :

> Hi Oliver,
>
> Many thanks for the prompt response. Sounds like it's CTEST, not a compile
> process.
>
> I followed your 1st test and showed 3 tests failed out of 71,
>
> 66 - arts.ctlfile.fast.artscomponents.helpers.TestRegridding (Failed)
>> 70 - arts.ctlfile.fast.artscomponents.wfuns.TestSpectroscopy (Failed)
>> 71 - arts.ctlfile.fast.artscomponents.cia.TestCIADerivs (Failed)
>
>
> Because of the following missing xml and data files, respectively.
>
> Cannot find input file: spectroscopy/cia/borysow/Borysow_CIA_JUICE_SWI.xml
>> Cannot find input file: spectroscopy/PartitionSums/TIPS/tips.xml
>> Cannot find input file:
>> planets/Earth/ECMWF/ERA40/SurfaceAltitude_ERA40_1.0Degree
>
>
> Full screenshot is attached in "arts_make_check_results.txt".
>
> The 2nd test(make check-all) was carried out and 23 tests failed out of
> 225. This is far better than my earlier report and almost all were due to
> missing input files (Can you tell me where they are, btw?). The full
> screenshot is also attached herewith as "arts_make_check-all_results.txt".
>
> One thing that caught my eyes was at "arts_convert" during the "make
> check-all" process. It looks like arts' workspace methods are not well
> understood in converting into Python. Is my understanding correct? and are
> they to be examined before I use Pyarts?
>
> [100%] Converting *.arts controlfiles to Python
>> Running arts_convert ...
>> Error converting TestClouds_Venus.arts: abs_linesReadFromSplitArtscat is
>> not a known workspace method.
>> Error converting TestSpectro_core.arts: abs_linesReadFromHitran is not a
>> known workspace method.
>> Error converting TestClouds_Mars.arts: abs_linesReadFromSplitArtscat is
>> not a known workspace method.
>> Error converting TestReadCataloguePerrin.arts:
>> abs_linesReadFromSplitArtscat is not a known workspace method.
>> Error converting TestRadioLink.arts: iyRadioLink is not a known workspace
>> method.
>> Error converting TestRadioOccultation.arts: iyRadioLink is not a known
>> workspace method.
>> Error converting TestMolTau.arts: abs_linesReadFromArts is not a known
>> workspace method.
>> Error converting TestRelmat.arts: abs_lines_per_bandFromband_identifiers
>> is not a known workspace method.
>> Error converting odinsmr_544_absorption.arts: abs_linesReadFromArts is
>> not a known workspace method.
>> Error converting seviri_hitran.arts: abs_linesReadFromHitran is not a
>> known workspace method.
>> Error converting mviri_hitran.arts: abs_linesReadFromHitran is not a
>> known workspace method.
>> Error converting use-absOnTheFly.arts: abs_linesReadFromSplitArtscat is
>> not a known workspace method.
>> Error converting AbsLookup.arts: abs_linesReadFromSplitArtscat is not a
>> known workspace method.
>> Error converting AbsOnTheFly.arts: abs_linesReadFromSplitArtscat is not a
>> known workspace method.
>> Error converting make-and-use-absLUT_1Dfor3D.arts:
>> abs_linesReadFromSplitArtscat is not a known workspace method.
>> Error converting DemoScat_DOIT_1D.arts: iyInterpLinCloudboxField is not a
>> known workspace method.
>> Error converting DemoScat_FOS_1D.arts: iyFOS is not a known workspace
>> method.
>> Error converting SpecCat_Consistency_Perrin-vs-HITRAN.arts:
>> abs_linesReadFromHitran is not a known workspace method.
>> arts_convert done. Converted 230 out of 248 files.
>> [100%] Built target python_tests
>
>
>  Many thanks,
> Reno
>
>
> On Fri, 9 Oct 2020 at 15:06, Oliver Lemke 
> wrote:
>
>> Hi Reno,
>>
>> It looks like all the Python related tests are failing. This is currently
>> a flaw in our build setup when calling 'ctest' manually. It circumvents the
>> dependency tracking of 'make' and doesn't trigger the conversion of the
>> 

Re: [arts-users] O3 Jacobians

[Richard Larsson]

Hi again Rita,

The ARTS approach to all this comes from the idealistic view of a computer
model as a simple linear algebra system,

J · x = y,

where J is 'the model', or the Jacobian matrix, x is the model input, i.e.,
O3, and y is the model output, i.e., the simulated radiation.  From this,
the units in J must, when multiplied by x, give the unit of y.  You have
iy_unit in ARTS to set the radiation unit of J and y.  You can read about
iy_unit only in the individual *Standard iy-functions, like
iyEmissionStandard.  In your jacobianAddAbsSpecies, you set the unit of x
to be logrel.  So, the unit in your case of J will be [iy_unit /
logrel(O3)].

Note that if you add more components to the Jacobian, such as wind or
temperature or whatnot, then these additions will have different units in
their respective positions of J and x.

With hope,
//Richard

Den tors 4 apr. 2019 kl 12:46 skrev Rita Kajtar :

> Hello!
>
>
> Since it is hard to trust that the e-mails sent from my LTU account are
> reaching everybody in the ARTS users list, I will, from now on, use my
> Gmail account.
>
> Thank you to those who helped out with my first problem.
>
> Below is another short question I addressed to the community yesterday,
> but I am not sure who received it or if anyone received it at all. If
> someone did, my apologies for spamming again.
>
> This time my question is related to the calculation of Jacobians for O3.
>
> I am considering a no winds, no specific atmospheric background case and
> the jacobianAddAbsSpecies in my control file is set as it follows:
>
>
>
>
>
>
>
>
>
>
> *jacobianAddAbsSpecies(  g1=p_grid,  g2=gas1_jac_lat_grid,
> g3=gas1_jac_lon_grid,  species="O3",  method="analytical",  unit="logrel",
>   )*
>
> When looking at the change in Bt in my plots, in what units the change of
> O3 is being represented?
>
>
> Best regards,
>
> Rita Kajtar
> ___
> arts_users.mi mailing list
> arts_users.mi@lists.uni-hamburg.de
> https://mailman.rrz.uni-hamburg.de/mailman/listinfo/arts_users.mi
>
___
arts_users.mi mailing list
arts_users.mi@lists.uni-hamburg.de
https://mailman.rrz.uni-hamburg.de/mailman/listinfo/arts_users.mi

Re: [arts-users] Wind calculations (resent)

[Richard Larsson]

Hi Jana,

Den ons 3 apr. 2019 kl 17:08 skrev Jana Mendrok :

> Hi,
>
> thanks Richard for your feedback. I'm not sure, though, whether I really
> understand it (incl. wondering why you talk about jacobians here...).
> *scratches head*
>

I might have misunderstood the question slightly, but the easiest way to
understand this in the code is in relation to what a standard Jacobian
means, since it must relate to you the assumed internal signs and geometries


> we're trying to pin down where the wind is blowing in a 1D atm setup. 1D
> allows us to have vertical (w) and horizontal (v) winds that affect the
> signal. let's forget about the vertical, that one is clear. for the
> horizontal, there is two possible wind directions in a 1D case - a head
> wind (aka blowing in the observer's face) and a tail wind (aka blowing from
> the observer's back). which of these corresponds to a positive, which to a
> negative wind(_v) speed.
>

The documentation is too difficult for me to follow.  In the code, the
definition in the 1D case is the same as for the 3D fields.  Assuming
wind_u==0 is enforced somehow (I seldom use 1D geometry), a negative wind_v
value has a negative speed in the Doppler shift expression -- a blue shift
or a headwind -- and vice verse for a positive wind_v.


> starting from your "For the 'absolute' wind speed option, a positive
> retrieved sign is equivalent to a red-shift" and equating a red-shift with
> observer moving away from source (or source from observer...), a positive
> wind should mean a tail wind.
>

Yes.


> which actually seems indeed in agreement with info I dig from the ARTS
> documentation (not that easy to find and combine for a beginner...): wind_v
> doc says positive v-winds are winds blowing south to north (which alone
> isn't very helpful for 1D setups - there is no north in 1D...); sensor_los
> doc says for the 2D case (allowing positive and negative zenith angles - in
> order to distinguish two possible viewing directions) that positive angles
> are equivalent to viewing towards higher latitudes, ie towards north.
> assuming this is valid for 1D, too (ie in 1D the observer always looks
> towards north) means for the 1D case positive winds (blowing to north)
> equate tail winds (for an observer looking north), negative winds are head
> winds.
>

Indeed, the description seems coherent but difficult to follow.  Can you
point out where in the docs you found this?  It should be updated to
clearly read that a positive wind_v in 1D equates to a positive v in the
standard (1 - v/c) Doppler shift expression.

With hope,
//Richard

>
___
arts_users.mi mailing list
arts_users.mi@lists.uni-hamburg.de
https://mailman.rrz.uni-hamburg.de/mailman/listinfo/arts_users.mi

Re: [arts-users] Wind calculations (resent)

[Richard Larsson]

Hi Rita,

For the directional components, the sign of a retrieved component with the
ARTS Jacobian is positive along the axes themselves.  The direction the
sensor is looking only influences the scale of the Jacobian, not the sign
of the direction of the retrieval calculation.  If you look parallel or
anti-parallel to the field, the values inside the Jacobian switch signs
accordingly to work on the external field.  As an example, you should be
able to make measurements at several different directions and, if you
assume the field is uniform, retrieve just one field without any
transformation on the components themselves inside the calculations.

For the 'absolute' wind speed option, a positive retrieved sign is
equivalent to a red-shift, or a Doppler shift below unity if you wish.
This is much messier to work with during ARTS iterations, so make sure your
signal is easy to interpret if you use it.  This is of course closer to the
physics of what is going on with the signal itself, since all we really can
see of the wind is how much it is blue- or red-shifting the signal.

With hope,
//Richard

Ps. Oliver, please sent another email when it is fixed.



Den ons 3 apr. 2019 kl 13:36 skrev Oliver Lemke :

> Hi all,
>
> Sorry if you received the mail below already. We're currently
> investigating a mail delivery problem to Gmail adresses. If you've already
> received the mail from Rita yesterday please disregard this mail.
>
> Cheers,
> Oliver
>
>
> > From: Rita Edit Kajtar 
> > Subject: Wind calculations
> > Date: 2 April 2019 at 10:36:54 CEST
> > To: "arts_users.mi@lists.uni-hamburg.de" <
> arts_users.mi@lists.uni-hamburg.de>
> >
> >
> > Hello!
> >
> >
> > I have a short question regarding wind calculations.
> >
> > What is the general convention assumed for determining the sign of the
> wind speeds having the measuring instrument as the reference point? Are the
> winds blowing towards the instrument considered positive and those blowing
> away from the instrument negative or the other way around?
> >
> > The ARTS documentation provides information for how the sign should be
> considered regarding the four cardinal directions, but not relative to a
> measuring device.
> >
> >
> > Best regards,
> > Rita Kajtar
>
>
> ___
> arts_users.mi mailing list
> arts_users.mi@lists.uni-hamburg.de
> https://mailman.rrz.uni-hamburg.de/mailman/listinfo/arts_users.mi
>
___
arts_users.mi mailing list
arts_users.mi@lists.uni-hamburg.de
https://mailman.rrz.uni-hamburg.de/mailman/listinfo/arts_users.mi

Re: [arts-users] FASTEM in ARTS

[Richard Larsson]

Hi Stuart,

A development model problem.  I am not sure this will solve your issues
completely, but can you try adding SurfaceDummy at the start of your
iy_surface_agenda?

The new variables you are seeing are part of a method Patrick is working on
for calculating the Jacobian of surface features.  SurfaceDummy exists as a
method to ignore said feature and have ARTS work as before.

See line 180 in controlfiles/general/agendas.arts for an example.
Probably, ARTS-2.3.638 did not have said line because the feature is new
since ARTS-2-3-1010.

With hope,
//Richard

Den tors 8 nov. 2018 kl 17:40 skrev Fox, Stuart :

> I’m currently trying to use the ARTS version of FASTEM via iySurfaceFastem
> in iy_surface_agenda. This used to work, but on the latest version of the
> ARTS trunk (2.3.1146) it fails with:
>
> Run-time error in method: AgendaSet
>
> The agenda iy_surface_agenda must generate the output WSV
> dsurface_rmatrix_dx,
>
> but it does not. It only generates:
>
> diy_dx
>
> iy
>
> auto_iySurfaceFastem_gin3_fastem_version
>
>
>
> The section of my controlfile that sets the surface is:
>
> NumericCreate(wind_speed)
>
> NumericCreate(salinity)
>
> NumericCreate(wind_dir)
>
> ReadXML(wind_speed,"surface_wind_speed.xml")
>
> ReadXML(wind_dir, "surface_wind_dir.xml")
>
> ReadXML(salinity, "surface_salinity.xml")
>
> AgendaSet(iy_surface_agenda){
>
> iySurfaceFastem(salinity=salinity, wind_speed=wind_speed,
> wind_direction=wind_dir)
>
> }
>
>
>
> This used to work (certainly it worked at version 2.3.638), so I’m
> guessing something has changed (it looks like it’s related to calculating
> surface jacobians). Is this a bug, or do I need to add something extra to
> iy_surface_agenda to calculate the required quantities?
>
>
>
> Regards,
>
>
>
> Stuart
>
>
> ___
> arts_users.mi mailing list
> arts_users.mi@lists.uni-hamburg.de
> https://mailman.rrz.uni-hamburg.de/mailman/listinfo/arts_users.mi
>
___
arts_users.mi mailing list
arts_users.mi@lists.uni-hamburg.de
https://mailman.rrz.uni-hamburg.de/mailman/listinfo/arts_users.mi

Re: [arts-users] ARTS OEM

[Richard Larsson]

Hi Byungsuk,

We provide the TIPS as published by HITRAN via an XML file in:
arts-xml-data/spectroscopy/PartitionSums/TIPS/tips.xml.  The README
contains some more information.

About the 300 K limit today,  I do not know where it is from and the
differences between it and the TIPS data is not strongly defined.  The
builtin partition_functions are historical, and tests show that it should
be mostly reliable up to 300 K but not much more than that.

If you need a comparison for the temperature ranges you are interested in,
you can run "nlte_fieldSetLteInternalPartitionFunction" and
"nlte_fieldSetLteExternalPartitionFunction"
after having set partition_functions to tips.xml.  The NLTE field is a the
size of your atmosphere times the size of the energy levels you are
interested in.

With hope,
//Richard

2018-09-18 2:33 GMT+02:00 이병석 :

> Hello Richard,
> ​
> Thank you for your reply.
> ​
> May I ask which tips xml files you are referring to?
> ​
> Also a question: it seems that the bad partition function warning occurs
> for any input t_field temperature beyond the range between 150 K and 300 K.
> In reality the surface temperature of the Earth frequently exceeds 300 K
> (26.85 deg C). For input values such as 30 deg C or a little higher (e.g.
> 30-40 deg C, early 300's K), would the results still be bad if the warning
> is suppressed?
> ​
> Regards,
> ​
>
> Byungsuk Lee
>
> ARA Consulting & Technology
>
> 30 Songdomirae-ro D-1510, Yeonsu-gu, Incheon 21990, South Korea
>
> -Original Message-
> *From:* "Richard Larsson"
> *To:* "이병석";
> *Cc:* "Simon Pfreundschuh"; <
> arts_users...@mailman.rrz.uni-hamburg.de>;
> *Sent:* 2018-09-17 (월) 18:21:57
> *Subject:* Re: [arts-users] ARTS OEM
>
> Hi,
>
> Just as a warning, suppressing the bad partition function error is not
> good practice since it is known that they are, you know, bad.  Standard
> tests in ARTS fail by several tenths of Kelvin because the partition
> functions in ARTS are that bad.  Just run with the provided tips xml-files
> instead, because if your temperatures are lower than 70 K or higher than
> 3000 K you probably have another problem.
>
> With hope,
> //Richard
>
> 2018-09-17 10:31 GMT+02:00 이병석 :
>
> Hello Simon Pfreundschuh,
> ​
> Thank you for your reply.
> ​
> I have employed both of the suggestions and still get the irregular
> errors. From the same controlfile modified in the previous email, I have
> included and changed to the following lines:
> ​
> OEM(method="lm",
>max_iter=15,
>display_progress=1,
>lm_ga_settings=[100,5,2,1000,1,99])
> ​
> and
> ​
> AgendaSet( inversion_iterate_agenda ){
>
>  xClip(ijq = 0, limit_low = 150.0, limit_high = 350)
> ​
>  Ignore(inversion_iteration_counter)
>  # Map x to ARTS' variables
>  x2artsStandard
> ​
>  # To be safe, rerun checks dealing with the atmosphere
>  atmfields_checkedCalc
>  atmgeom_checkedCalc
> ​
>  # Calculate yf and Jacobian matching x.
>  yCalc( y=yf )
> ​
>  # Add baseline term
>  VectorAddVector( yf, yf, y_baseline )
> ​
>  # This method takes cares of some "fixes" that are needed to get the
> Jacobian
>  # right for iterative solutions. No need to call this WSM for linear
> inversions.
>  jacobianAdjustAndTransform
> }
> ​
> I included both of the suggestions, and from there
> increased the maximum number of iterations to 15
> and set the temperature limits to be between 150 K and 350 K.
> All other lines remain the same, including the input absorption line and
> atmospheric fields data.
> ​​​
> Now I irregularly get the errors during the OEM computation as before, of
> t_field having NaN values or of bad partition function warning (which
> happens for any t_field value outside between 150 K and 300 K). While the
> bad partition function warning could be suppressed in the code, it seems
> that the same problem as before persists.
> ​
> Could you please look into this issue a little more? I am not sure how to
> tackle the problem.
> ​
> Thank you for your service.
> ​
> Regards,
> ​
>
> Byungsuk Lee
>
> ARA Consulting & Technology
>
> 30 Songdomirae-ro D-1510, Yeonsu-gu, Incheon 21990, South Korea
>
>
>
> -Original Message-
> *From:* "Simon Pfreundschuh"
> *To:* "이병석"; "arts_users...@mailman.rrz.uni-hamburg.de"<
> arts_users...@mailman.rrz.uni-hamburg.de>;
> *Cc:*
> *Sent:* 2018-09-14 (금) 15:13:55
> *Subject:* Re: [arts-users] ARTS OEM
>
> Hi Byungsuk Lee,
>
> It is a bit surprising that these errors occur irregularly, since there is
> no randomness
> in the test 

Re: [arts-users] ARTS OEM

[Richard Larsson]

Hi,

Just as a warning, suppressing the bad partition function error is not good
practice since it is known that they are, you know, bad.  Standard tests in
ARTS fail by several tenths of Kelvin because the partition functions in
ARTS are that bad.  Just run with the provided tips xml-files instead,
because if your temperatures are lower than 70 K or higher than 3000 K you
probably have another problem.

With hope,
//Richard

2018-09-17 10:31 GMT+02:00 이병석 :

> Hello Simon Pfreundschuh,
> ​
> Thank you for your reply.
> ​
> I have employed both of the suggestions and still get the irregular
> errors. From the same controlfile modified in the previous email, I have
> included and changed to the following lines:
> ​
> OEM(method="lm",
>max_iter=15,
>display_progress=1,
>lm_ga_settings=[100,5,2,1000,1,99])
> ​
> and
> ​
> AgendaSet( inversion_iterate_agenda ){
>
>  xClip(ijq = 0, limit_low = 150.0, limit_high = 350)
> ​
>  Ignore(inversion_iteration_counter)
>  # Map x to ARTS' variables
>  x2artsStandard
> ​
>  # To be safe, rerun checks dealing with the atmosphere
>  atmfields_checkedCalc
>  atmgeom_checkedCalc
> ​
>  # Calculate yf and Jacobian matching x.
>  yCalc( y=yf )
> ​
>  # Add baseline term
>  VectorAddVector( yf, yf, y_baseline )
> ​
>  # This method takes cares of some "fixes" that are needed to get the
> Jacobian
>  # right for iterative solutions. No need to call this WSM for linear
> inversions.
>  jacobianAdjustAndTransform
> }
> ​
> I included both of the suggestions, and from there
> increased the maximum number of iterations to 15 and set the temperature 
> limits to be between 150
> K and 350 K. All other lines remain the same, including the input
> absorption line and atmospheric fields data.
> ​​​
> Now I irregularly get the errors during the OEM computation as before, of
> t_field having NaN values or of bad partition function warning (which
> happens for any t_field value outside between 150 K and 300 K). While the
> bad partition function warning could be suppressed in the code, it seems
> that the same problem as before persists.
> ​
> Could you please look into this issue a little more? I am not sure how to
> tackle the problem.
> ​
> Thank you for your service.
> ​
> Regards,
> ​
>
> Byungsuk Lee
>
> ARA Consulting & Technology
>
> 30 Songdomirae-ro D-1510, Yeonsu-gu, Incheon 21990, South Korea
>
>
>
> -Original Message-
> *From:* "Simon Pfreundschuh"
> *To:* "이병석"; "arts_users...@mailman.rrz.uni-hamburg.de"<
> arts_users...@mailman.rrz.uni-hamburg.de>;
> *Cc:*
> *Sent:* 2018-09-14 (금) 15:13:55
> *Subject:* Re: [arts-users] ARTS OEM
>
> Hi Byungsuk Lee,
>
> It is a bit surprising that these errors occur irregularly, since there is
> no randomness
> in the test setup. However, if you are running retrievals with external
> observation data
> it may well happen that the iteration ends up in a state that produces
> invalid values.
>
> For your temperature retrieval I would recommend one or both of the
> following solutions:
>
> 1. Use the Levenberg-Marquardt (LM) iteration scheme: Using the LM scheme
> with a high start
>value for the gamma parameter can help avoid reaching invalid values
> when the forward
>model is non-linear:
>
>   OEM(method="lm",
>   max_iter=5,
>   display_progress=1,
>   lm_ga_settings=[100,5,2,1000,1,99])
>
> 2. Clip the x-vector: Using the xClip WSM inside your inversion iterate
> agenda you can force the
>x vector to contain only temperatures within a certain range. If you
> use this approach you
>should check that none of the retrieved temperatures in your final
> state are clipped otherwise your
>results may not be valid.
>
> AgendaSet( inversion_iterate_agenda ){
>
>   xClip(ijq = 0, limit_low = 100.0)
>   Ignore(inversion_iteration_counter)
>   # Map x to ARTS' variables
>   x2artsStandard
>
>   # To be safe, rerun checks dealing with the atmosphere
>   atmfields_checkedCalc
>   atmgeom_checkedCalc
>
>   # Calculate yf and Jacobian matching x.
>   yCalc( y=yf )
>
>   # Add baseline term
>   VectorAddVector( yf, yf, y_baseline )
>
>   # This method takes cares of some "fixes" that are needed to get the
> Jacobian
>   # right for iterative solutions. No need to call this WSM for linear
> inversions.
>   jacobianAdjustAndTransform
> }
>
>
> Regards,
>
> Simon
>
> --
> *From:* arts_users.mi-boun...@mailman.rrz.uni-hamburg.de <
> arts_users.mi-boun...@mailman.rrz.uni-hamburg.de> on behalf of 이병석 <
> cb...@aracnt.com>
> *Sent:* Friday, September 14, 2018 2:31:25 AM
> *To:* arts_users...@mailman.rrz.uni-hamburg.de
> *Subject:* [arts-users] ARTS OEM
>
> Dear ARTS users,
> ​
> I am writing to address a question with the use of OEM in the ARTS
> development version.
> ​
> From the "TestOEM.arts" in "ARTS/controlfiles/artscomponents/oem", this
> time I changed the codes as follows.
> ​
> 1) From (line 

Re: [arts-users] calculate absorption coefficients from abs_coefCalc

[Richard Larsson]

Dear Pengwang Zhai,

I think we still have what you are looking for in ARTS.

The variable you are actually interested in is not abs_coef, but
abs_coef_per_species.

The way to create this is to first run abs_xsec_agenda and
then abs_coefCalcFromXsec.

I think your code should only change from:

abs_coefCalc

to

abs_xsec_agendaExecute
abs_coefCalcFromXsec

Now abs_coef_per_species is an array of matrices.  The array-size is the
species and the matrices are the size of f_grid and p_grid.  Save this and
do what you want with it.

This said, there is potentially a few other changes depending on what
version you are using and what species you are interested in.  You will
want to run jacobianOff, for instance, if you are using the dev-branch.

Cheers,
//Richard

2016-02-15 22:11 GMT+01:00 Pengwang Zhai :

> Thanks, Jana.
>
> > Note that for deriving abs_coef for this, you need to sum up over all
> species (i.e. over the 0th dimension of propmat_clearsky_field.
>
> This is exactly what I wanted to do within arts to get “abs_coef”.  If it
> is not possible with arts, I will use matlab to do this.
>
> The three dimensions are: [species, f_grid, p_grid]
>
> Cheers,
>
> PZ
>
>
> > On Feb 15, 2016, at 3:47 PM, Jana Mendrok 
> wrote:
> >
> > Hi,
> >
> > what do you intend to do with abs_coeff_user?
> > Do you really process this further within ARTS itself? only then it
> makes sense to process propmat_clearsky_field into a Tensor3*, i think.
> > Else, I strongly recommend to do any re-shaping / reduction outside of
> ARTS; other programming languages are much better suited for this kind of
> task (e.g. you can use the matlab-interface atmlab to have easy access to
> ARTS output. or the python interface typhon).
> >
> > abs_coef was a Matrix of dimension [f_grid, abs_p] (and
> abs_coef_per_species an Array holding one abs_coef matrix per defined
> abs_species).
> > propmat_clearsky_field is a Tensor7 of dimension [species, f_grid,
> stokes_dim, stokes_dim, p_grid, lat_grid, lon_grid].
> >
> > That is, for reducing propmat_clearsky_field to what was
> abs_coef_per_species before would be
> > propmat_clearsky_field[:, :, 0, 0, :, lat_index, lon_index] (in case of
> 0-indexed varibales; in case of a 1D calculation furthermore
> lat_index=lon_index=0). As far as I know, that's not possible (at least not
> easily possible) within ARTS itself.
> >
> > Note that for deriving abs_coef for this, you need to sum up over all
> species (i.e. over the 0th dimension of propmat_clearsky_field.
> >
> > Best wishes,
> > Jana
> >
> >
> > ps.
> > *by the way, what are the 3 dimensions? neither abs_coeff nor
> abs_coef_per_species is (or has been for a long time) a Tensor3. That is,
> you must have post-processed abs_coef(_per_species) anyways?
> >
> > On Mon, Feb 15, 2016 at 6:09 PM, Pengwang Zhai  wrote:
> > Thanks, Jana.  Too bad that the only arts feature that I am using is now
> obsolete.
> >
> > Now given propmat_clearsky_field, can I somehow obtain abs_coeff from it?
> >
> > I used the following commands to create abs_coeff_user, which is the
> absorption coefficients per specie,
> >
> > Tensor3Create(abs_coeff_user)
> > Reduce(abs_coeff_user,propmat_clearsky_field)
> >
> > can I add the coefficients per specie abs_coeff_user together to create
> abs_coeff within arts?
> >
> > I could do this with other tools but wish to know how to do in arts,
> which is more convenient.
> >
> > Thanks,
> >
> > Pengwang
> >
> >
> > > On Feb 15, 2016, at 12:00 PM, Jana Mendrok 
> wrote:
> > >
> > > Dear Pengwang,
> > >
> > > please take a look at the workspace variable propmat_clearsky_field
> (and the workspace method propmat_clearsky_fieldCalc, that calculates this
> variables). Could they probably provide the output you need?
> > >
> > > abs_coef and its *Calc method are obsolete and not supported anymore.
> > >
> > > Best wishes,
> > > Jana
> > >
> > >
> > > On Mon, Feb 15, 2016 at 4:30 PM, Pengwang Zhai 
> wrote:
> > > Hi,
> > >
> > > I used earlier version of ARTS to calculate the total absorption
> coefficients with:
> > >
> > > abs_coefCalc
> > >
> > > and found it is very useful.  Now with the new version, this does not
> work any more. After reading the change log I found that it is replaced with
> > >
> > > abs_coefCalcFromXsec
> > >
> > > However, after numerous attemps I could not make this method working
> for me.  Would you please provide a template arts file on the usage of this
> method?
> > >
> > > Thanks very much and hope all the best,
> > >
> > > Yours
> > >
> > > ___
> > > arts_users.mi mailing list
> > > arts_users.mi@lists.uni-hamburg.de
> > > https://mailman.rrz.uni-hamburg.de/mailman/listinfo/arts_users.mi
> > >
> > >
> > >
> > > --
> > > =
> > > Jana Mendrok, Ph.D. (Project Assistent)
> > > Chalmers University of