Re: Simon plans for scattering properties

[Richard Larsson]

Hi Stefan,

If we do something like this we have to start by removing abs_xsec_agenda.
There's no good way to mold the Zeeman effect into fitting with it.

I will inline the rest.

Den mån 1 feb. 2021 kl 17:29 skrev Stefan Buehler <
stefan.bueh...@uni-hamburg.de>:

> Hej Richard,
>
> to me the interesting point is that the specific inputs could be handled
> where the “Add..Habit” happens. In the case of absorption, a lot of
> difficulty comes from the dichotomy between the species tags and the
> methods that have to be added to the absorption agenda to actually do
> the calculations. Using Simons concept, one could potentially replace
> the tags by something like
>
> AbsorberAppendLBL(“H2O”)
> AbsorberAppendCIA(“N2-O2”)
> AbsorberAppendLBL(“O3”)
> AbsorberAppendXsec(“CFC-11”)
>
> So, instead of indentifying the model to be used by the tag name, we
> could identify it by the Add method used. The different Add methods
> would depend on different input variables, such as line catalog,
> tabulated cross-sections, etc.. That solution seems quite elegant to me.
>

I do like this idea.  It simplifies the interface significantly and users
would no longer have to deal with setting up the Agendas.  The "absorbers"
would do that for them.  This is similar to how you no longer have to deal
with line catalog settings.  It's a direction of development for ARTS that
I like.


> One problem is, though, that one absorption tag currently can be a
> combination of models for the same gas species. One way to solve this
> would be to have two types of methods, Add and Append, like this:
>
> AbsorberAppendLBL(“H2O”)
> AbsorberAddCont(“H2O”,”CKD-MT-3_7)
> AbsorberAppendCIA(“N2-O2”)
> AbsorberAppendLBL(“O2”)
> AbsorberAddCont(“O2”,”PWR20”)
> AbsorberAppendXsec(“CFC-11”)
>
> A strict interface for the output is no problem, since all absorption
> functions produce exactly the same output.
>

They do not have the same outputs.  You ignore abs_xsec_agenda, and it has
to go for this to work.  I cannot see this working with multiple levels of
absorption routines, since Zeeman and some others require polarization.
The rest I have to say depends on abs_xsec_agenda going away.

The tag system here is important.  You have to have a full line catalog
read when you start setting this up.  Otherwise the frequency limited tags
would not get their lines.  And perhaps even the non-frequency limited tags
would also get the frequency limited lines.  It is therefore very important
that the new "Absorber" class/list takes ownership of data from the line
catalog.  I guess having the data available ahead of time is important for
all types of "Absorber"s (like CIA data and Xsec data and line mixing data).

I think your way of "adding" many tags might become confusing.  The
Jacobian method requires an Array-tag of some kind for one of its ways to
compute the VMR derivatives.  From your water example, will that tag be
"H2O,H2O-CKD-MT-3_7", just "H2O", or will either of the two tags work for
that Jacobian VMR method?  I am not sure what to do here.


> For the (atmospheric) input it is potentially more problematic. For
> example, Zeeman absorption depends on the magnetic field, but none of
> the other absorption functions do.
>
> I don’t think there is so much overhead that one really gains a lot by
> for example doing all the LBL species in one go. One could basically use
> the present lower level functions, and just call them repeatedly for
> each gas in turn. When the absorption lookup table is generated, I think
> it is already done this way.
>

Well, the lower level functions of the absorption routines allocates quite
a lot of data.  I am not sure if repeating these operations is expensive or
not, but if you separate this more then they would be allocated not just
once per atmospheric level but once per species and atmospheric level.  As
an example, these variables are being allocated over-and-over again for a
call to the Zeeman routine:

In propmat_clearskyAddZeeman:
rtp_los (2 Numeric)

In zeeman_on_the_fly:
jacobian_quantities_positions (nq Index)
scratch (4 * nf + 2 * nq * nf + 4 + 2 *nq Complex)
sum (4 * nf + 2 * nq * nf + 4 + 2 *nq Complex)
B (nf Numeric if non-lte is active otherwise 0)
dBdT (nf Numeric if non-lte is active otherwise 0)
line_shape_vmr (nb)

Now I am not claiming that this is ideal, but for a standard run of 3
retrieval parameters and 65000 frequencies, without non-lte, that's
something like 3 Mb (?) of quite often repeated allocations.  At the very
least, if the scratch+sum values, which are necessary compute parameters,
would be kept as they are, this would be quite a lot less allocations
having to happen.  Note also how quickly this goes up if you try to compute
more for the derivatives.

Note that this is of course also the case for the absorption xsec agenda,
where "Init" allocates a significant amount of memory, but I am not sure if
these variables are kept around after an agenda closes down or not for 

Re: Simon plans for scattering properties

[Stefan Buehler]

Hej Richard,

to me the interesting point is that the specific inputs could be handled 
where the “Add..Habit” happens. In the case of absorption, a lot of 
difficulty comes from the dichotomy between the species tags and the 
methods that have to be added to the absorption agenda to actually do 
the calculations. Using Simons concept, one could potentially replace 
the tags by something like


AbsorberAppendLBL(“H2O”)
AbsorberAppendCIA(“N2-O2”)
AbsorberAppendLBL(“O3”)
AbsorberAppendXsec(“CFC-11”)

So, instead of indentifying the model to be used by the tag name, we 
could identify it by the Add method used. The different Add methods 
would depend on different input variables, such as line catalog, 
tabulated cross-sections, etc.. That solution seems quite elegant to me.


One problem is, though, that one absorption tag currently can be a 
combination of models for the same gas species. One way to solve this 
would be to have two types of methods, Add and Append, like this:


AbsorberAppendLBL(“H2O”)
AbsorberAddCont(“H2O”,”CKD-MT-3_7)
AbsorberAppendCIA(“N2-O2”)
AbsorberAppendLBL(“O2”)
AbsorberAddCont(“O2”,”PWR20”)
AbsorberAppendXsec(“CFC-11”)

A strict interface for the output is no problem, since all absorption 
functions produce exactly the same output.


For the (atmospheric) input it is potentially more problematic. For 
example, Zeeman absorption depends on the magnetic field, but none of 
the other absorption functions do.


I don’t think there is so much overhead that one really gains a lot by 
for example doing all the LBL species in one go. One could basically use 
the present lower level functions, and just call them repeatedly for 
each gas in turn. When the absorption lookup table is generated, I think 
it is already done this way.


AbsorberAppendLBL would depend on abs_lines_per_species, and just 
incorporate the lines for the species in questions to the local data of 
the AbsorberLBL class, inheriting the public interface from the general 
Absorber class.


So, I think in principle this structure could be built on top of the 
various absorption functions we have, replacing the current tag 
mechanism.


Ah, there probably are a lot of pitfalls.

One that I just noticed is that, if the species list is only established 
by these Append methods, then we have no good way to decide for which 
species we should read in the line catalogue data. (For that the list of 
species of interest would have to be defined before.) This could be 
solved by the Append method doing the catalog reading, but that would 
make the Append methods quite heavy. On the other hand it would make for 
very short and simple controlfiles, the call would be like:


AbsorberAppendLBL(,“H2O”)

I’m not sure whether I like that, though, it seems less flexible than 
what we have when it comes to dealing with the catalog.


All the best,

Stefan

On 1 Feb 2021, at 16:10, Richard Larsson wrote:


Simon, Stefan,

It seems to me that this is a proposal of using a modern "void *"
list/vector with some strict interface.  Have I misunderstood 
something
here?  I mean you might implement it with virtual classes, 
std::variant, or

some other reinterpreting interface but that seems about it.

I am not going to argue for the scattering implementation here because 
I
assume you guys know the current interface well enough and have ideas 
of
how to easily extend it using the new and old methods.  Perhaps the 
strict

interface helps here.

I would not mind having a "void *" list/vector in the LBL code but I 
am
very wary about the strict interface requirements here.  I would be 
much
more accepting if we could just get some kind of "const ClassImpl&" 
back
from the "void *" list.  Then the different methods can much more 
easily
know if they need to write to src or not, to a PropagationMatrix 
interface

or to an ArrayOfMatrix interface.

I don't like the idea of separating this by species in the LBL though. 
 It
would be much better to separate it by the types of calculations 
necessary
(the SpeciesTag-enum + the Absorption::PopulationType enum for LBL).  
That
way you could probably allow pre-allocations of all allocated 
variables
used by the algorithms taking the class without having to worry too 
much
about the memory footprint and leaving deallocation or reuse up to the 
user

(a cleanup method or more in the Agenda).

That said, I think it is quite a daunting task to change things this 
way.

And I am not sure how easy it is to take Simon's idea and use it with
value-semantics.

With hope,
//Richard

Den mån 1 feb. 2021 kl 14:14 skrev Stefan Buehler <
stefan.bueh...@uni-hamburg.de>:


Dear Simon,


The handling of the constant data is completely up to the
implementation of the specific scatterers.
For the ScatteringParticle class, for example, this is provided when
the object is constructed.
So when the ARTS user calls:

scattering_speciesAddScatteringHabit(...)

an object is created which will hold the scattering data. 

Re: Simon plans for scattering properties

[Richard Larsson]

Simon, Stefan,

It seems to me that this is a proposal of using a modern "void *"
list/vector with some strict interface.  Have I misunderstood something
here?  I mean you might implement it with virtual classes, std::variant, or
some other reinterpreting interface but that seems about it.

I am not going to argue for the scattering implementation here because I
assume you guys know the current interface well enough and have ideas of
how to easily extend it using the new and old methods.  Perhaps the strict
interface helps here.

I would not mind having a "void *" list/vector in the LBL code but I am
very wary about the strict interface requirements here.  I would be much
more accepting if we could just get some kind of "const ClassImpl&" back
from the "void *" list.  Then the different methods can much more easily
know if they need to write to src or not, to a PropagationMatrix interface
or to an ArrayOfMatrix interface.

I don't like the idea of separating this by species in the LBL though.  It
would be much better to separate it by the types of calculations necessary
(the SpeciesTag-enum + the Absorption::PopulationType enum for LBL).  That
way you could probably allow pre-allocations of all allocated variables
used by the algorithms taking the class without having to worry too much
about the memory footprint and leaving deallocation or reuse up to the user
(a cleanup method or more in the Agenda).

That said, I think it is quite a daunting task to change things this way.
And I am not sure how easy it is to take Simon's idea and use it with
value-semantics.

With hope,
//Richard

Den mån 1 feb. 2021 kl 14:14 skrev Stefan Buehler <
stefan.bueh...@uni-hamburg.de>:

> Dear Simon,
>
> > The handling of the constant data is completely up to the
> > implementation of the specific scatterers.
> > For the ScatteringParticle class, for example, this is provided when
> > the object is constructed.
> > So when the ARTS user calls:
> >
> > scattering_speciesAddScatteringHabit(...)
> >
> > an object is created which will hold the scattering data. It will then
> > use it when asked to
> > compute the bulk scattering properties. Since the
> > calculate_bulk_properties(...) method
> > is called on this object, it automatically has access to the required
> > data.
>
> Ah, yes. That is a great solution. Perhaps I could do the same for the
> absorbers. But I would have to break up the methods so that they are
> standalone for each absorption tag. Right now, the method that does LBL,
> for example, will calculate for all the LBL absorption tags, using a
> common workspace variable for the line catalog (which is internally
> split by species). All the housekeeping around this is a lot of trouble.
> Perhaps it would be better to always run it just for a single absorber,
> and use a technique similar to yours to set up the list, so that each
> absorber gets the line list he needs. (And similarly for the other types
> of absorption.)
>
> /Stefan
>

Re: Simon plans for scattering properties

[Stefan Buehler]

Dear Simon,

The handling of the constant data is completely up to the 
implementation of the specific scatterers.
For the ScatteringParticle class, for example, this is provided when 
the object is constructed.

So when the ARTS user calls:

scattering_speciesAddScatteringHabit(...)

an object is created which will hold the scattering data. It will then 
use it when asked to
compute the bulk scattering properties. Since the 
calculate_bulk_properties(...) method
is called on this object, it automatically has access to the required 
data.


Ah, yes. That is a great solution. Perhaps I could do the same for the 
absorbers. But I would have to break up the methods so that they are 
standalone for each absorption tag. Right now, the method that does LBL, 
for example, will calculate for all the LBL absorption tags, using a 
common workspace variable for the line catalog (which is internally 
split by species). All the housekeeping around this is a lot of trouble. 
Perhaps it would be better to always run it just for a single absorber, 
and use a technique similar to yours to set up the list, so that each 
absorber gets the line list he needs. (And similarly for the other types 
of absorption.)


/Stefan

Re: Simon plans for scattering properties

[Simon Pfreundschuh]

>From the perspective of computing scattering (or absorption) properties
for a given location in the atmosphere there are two types of required data:

- constant data describing the scatterer (single scattering data for particles, 
etc.)
- atmospheric input: These are quantities that vary with position (pressure, 
temperature, vmr, etc.)

The interface I described needs to handle only the second kind of data. This is 
important
because it's the key to decoupling the different scatterers from the RT methods 
that
use them.

The handling of the constant data is completely up to the implementation of the 
specific scatterers.
For the ScatteringParticle class, for example, this is provided when the object 
is constructed.
So when the ARTS user calls:

scattering_speciesAddScatteringHabit(...)

an object is created which will hold the scattering data. It will then use it 
when asked to
compute the bulk scattering properties. Since the 
calculate_bulk_properties(...) method
is called on this object, it automatically has access to the required data.

However, what this data is and how it is handled is completely up to the 
specific class.
For molecular scattering your would probably only need to know the atmospheric
composition.

The key here is the object oriented approach: By defining the interface through 
a class
hierarchy the interface needs to handle only the atmospheric input but can 
leave the
handling of ancillary data that doesn't vary to each class.

From: Stefan Buehler 
Sent: Monday, February 1, 2021 12:07:56 PM
To: Simon Pfreundschuh
Cc: ARTS Developers List
Subject: Re: Simon plans for scattering properties

Dear Simon,

how will you manage communication? I’m asking because there is a
related issue in computing absorption. It is easy to define a common
interface with respect to what the functions have to provide. But in
that case (and perhaps also in your case) a problem is that they require
quite different input, in addition to the atmosphere. (In the case of
absorption for example spectral line data for LBL functions, or a table
with measured cross-section data for other functions.)

Best wishes,

Stefan

On 31 Jan 2021, at 10:53, Simon Pfreundschuh wrote:

> Dear Stefan,
>
>
> If there's no user-configurable parameters required for molecular
> scattering then
>
> using it in a control file would be trivial:
>
>
> scattering_species_AddMolecularScattering()
>
>
> The iy method would then just need to check if the sun is on and
> calculate
>
> the bulk scattering properties for the first order scattering. In this
> way particles
>
> and molecules would be handle in a consistent way.
>
>
> What is required for this to work is of course a class that implements
> the interface
>
> defined by the ScatteringSpeciesImpl class (as indicated in the PDF).
>
>
> I also would really like to highlight the following, which you seem to
> have
> misunderstood:
>
>
> The interface that I described makes no assumptions whatsoever on the
> scattering
>
> species being either particles or a gas. It essentially already
> implements what
>
> you described but is even less restrictive. It only requires that a
> scattering species
>
> is able to provide bulk scattering data from an atmospheric state
> following the
>
> protocol defined by the ScatteringSpeciesImpl class.
>
> I am quite sure that with this system molecular and particle
> scattering can
> be handled consistently and the need for a molecular_scattering flag.
> However,
> I also agree that it would probably be inappropriate to  require Jon
> to use this
> system considering that he's just a Master's student and that this is
> still under
> development.
>
> Kind regards,
>
> Simon
>
>
>
> 
> From: arts_dev.mi-boun...@mailman.rrz.uni-hamburg.de
>  on behalf of Stefan
> Buehler 
> Sent: Friday, January 29, 2021 8:42:21 AM
> To: Simon Pfreundschuh
> Cc: ARTS Developers List
> Subject: Re: Simon plans for scattering properties
>
> Hej Simon,
>
> can you help me understand it then, please? I do want to find the best
> solution, not necessarily the quickest. (But simplicity is part of the
> criteria for “best”.)
>
> /Stefan
>
> On 29 Jan 2021, at 7:11, Simon Pfreundschuh wrote:
>
>> Dear Stefan,
>>
>>
>> I guess all I can say is that I am quite confident that you didn't
>> properly
>>
>> understand my proposal, which could quite easily handle the pressure
>>
>> dependency. Anyhow, it's quite clear that you are searching for a
>> quick
>>
>> solution and I won't be able to provide that. So let's just cut this
>> short.
>>
>>
>> /s
>>
>>
>>

Re: Simon plans for scattering properties

[Stefan Buehler]

Dear Simon,

how will you manage communication? I’m asking because there is a 
related issue in computing absorption. It is easy to define a common 
interface with respect to what the functions have to provide. But in 
that case (and perhaps also in your case) a problem is that they require 
quite different input, in addition to the atmosphere. (In the case of 
absorption for example spectral line data for LBL functions, or a table 
with measured cross-section data for other functions.)


Best wishes,

Stefan

On 31 Jan 2021, at 10:53, Simon Pfreundschuh wrote:


Dear Stefan,


If there's no user-configurable parameters required for molecular 
scattering then


using it in a control file would be trivial:


scattering_species_AddMolecularScattering()


The iy method would then just need to check if the sun is on and 
calculate


the bulk scattering properties for the first order scattering. In this 
way particles


and molecules would be handle in a consistent way.


What is required for this to work is of course a class that implements 
the interface


defined by the ScatteringSpeciesImpl class (as indicated in the PDF).


I also would really like to highlight the following, which you seem to 
have

misunderstood:


The interface that I described makes no assumptions whatsoever on the 
scattering


species being either particles or a gas. It essentially already 
implements what


you described but is even less restrictive. It only requires that a 
scattering species


is able to provide bulk scattering data from an atmospheric state 
following the


protocol defined by the ScatteringSpeciesImpl class.

I am quite sure that with this system molecular and particle 
scattering can
be handled consistently and the need for a molecular_scattering flag. 
However,
I also agree that it would probably be inappropriate to  require Jon 
to use this
system considering that he's just a Master's student and that this is 
still under

development.

Kind regards,

Simon




From: arts_dev.mi-boun...@mailman.rrz.uni-hamburg.de 
 on behalf of Stefan 
Buehler 

Sent: Friday, January 29, 2021 8:42:21 AM
To: Simon Pfreundschuh
Cc: ARTS Developers List
Subject: Re: Simon plans for scattering properties

Hej Simon,

can you help me understand it then, please? I do want to find the best
solution, not necessarily the quickest. (But simplicity is part of the
criteria for “best”.)

/Stefan

On 29 Jan 2021, at 7:11, Simon Pfreundschuh wrote:


Dear Stefan,


I guess all I can say is that I am quite confident that you didn't
properly

understand my proposal, which could quite easily handle the pressure

dependency. Anyhow, it's quite clear that you are searching for a
quick

solution and I won't be able to provide that. So let's just cut this
short.


/s




From: Stefan Buehler 
Sent: Thursday, January 28, 2021 5:02:28 PM
To: Simon Pfreundschuh
Cc: ARTS Developers List
Subject: Re: Simon plans for scattering properties

Dear Simon,

thanks for the summary!

I think I understand more or less how this works now. A bit
unfortunate
that so much information has to be moved around at the controlfile
level. (The copying of grids from one variable to the other, the
copying
of the scattering data themselves.) Is the idea that you would then 
in

practise store and load directly the ScatteringHabits, in order to
make
this easier?

On where the Rayleigh scattering fits in: Patrick suggested when we
talked yesterday that perhaps a more logical way to think about this
is
to distinguish between *gases* and *particles*, rather than between
*absorption* and *scattering*. (Including Rayleigh scattering with 
the

PND system really does not work, as it turns out, as it depends on
pressure.)

Perhaps we should move towards a *gas* / *particle* division in ARTS.
That would mean including the Liebe rain absorption in the particle
part. It would also affect your nomenclature, since one should then
preferably talk about *ParticleSpecies* rather than
*ScatteringSpecies*.
(And *gas_species* instead of *absorption_species*.)

All the best,

Stefan

On 26 Jan 2021, at 22:10, Simon Pfreundschuh wrote:


Dear Stefan,


I attach the description of the new scattering system that you
requested.

I tried to keep it general to avoid getting lost in technicalities.
If
you have

more specific questions please let me know.


Kind regards,


Simon


From: arts_dev.mi-boun...@mailman.rrz.uni-hamburg.de
 on behalf of Stefan
Buehler 
Sent: Monday, January 25, 2021 9:36:33 AM
To: ARTS Developers List
Subject: Simon plans for scattering properties

Dear all,

one thing I took home from the developer meeting on Friday is that I
would like to better understand Simon’s plans for the scattering
properties. We need this to make a good decision on how to proceed
with
the Rayleigh scattering (incorporate it into the existing scattering
data framework, or make a special case

Re: Simon plans for scattering properties

[Simon Pfreundschuh]

Dear Stefan,


If there's no user-configurable parameters required for molecular scattering 
then

using it in a control file would be trivial:


scattering_species_AddMolecularScattering()


The iy method would then just need to check if the sun is on and calculate

the bulk scattering properties for the first order scattering. In this way 
particles

and molecules would be handle in a consistent way.


What is required for this to work is of course a class that implements the 
interface

defined by the ScatteringSpeciesImpl class (as indicated in the PDF).


I also would really like to highlight the following, which you seem to have
misunderstood:


The interface that I described makes no assumptions whatsoever on the scattering

species being either particles or a gas. It essentially already implements what

you described but is even less restrictive. It only requires that a scattering 
species

is able to provide bulk scattering data from an atmospheric state following the

protocol defined by the ScatteringSpeciesImpl class.

I am quite sure that with this system molecular and particle scattering can
be handled consistently and the need for a molecular_scattering flag. However,
I also agree that it would probably be inappropriate to  require Jon to use this
system considering that he's just a Master's student and that this is still 
under
development.

Kind regards,

Simon




From: arts_dev.mi-boun...@mailman.rrz.uni-hamburg.de 
 on behalf of Stefan Buehler 

Sent: Friday, January 29, 2021 8:42:21 AM
To: Simon Pfreundschuh
Cc: ARTS Developers List
Subject: Re: Simon plans for scattering properties

Hej Simon,

can you help me understand it then, please? I do want to find the best
solution, not necessarily the quickest. (But simplicity is part of the
criteria for “best”.)

/Stefan

On 29 Jan 2021, at 7:11, Simon Pfreundschuh wrote:

> Dear Stefan,
>
>
> I guess all I can say is that I am quite confident that you didn't
> properly
>
> understand my proposal, which could quite easily handle the pressure
>
> dependency. Anyhow, it's quite clear that you are searching for a
> quick
>
> solution and I won't be able to provide that. So let's just cut this
> short.
>
>
> /s
>
>
>
> 
> From: Stefan Buehler 
> Sent: Thursday, January 28, 2021 5:02:28 PM
> To: Simon Pfreundschuh
> Cc: ARTS Developers List
> Subject: Re: Simon plans for scattering properties
>
> Dear Simon,
>
> thanks for the summary!
>
> I think I understand more or less how this works now. A bit
> unfortunate
> that so much information has to be moved around at the controlfile
> level. (The copying of grids from one variable to the other, the
> copying
> of the scattering data themselves.) Is the idea that you would then in
> practise store and load directly the ScatteringHabits, in order to
> make
> this easier?
>
> On where the Rayleigh scattering fits in: Patrick suggested when we
> talked yesterday that perhaps a more logical way to think about this
> is
> to distinguish between *gases* and *particles*, rather than between
> *absorption* and *scattering*. (Including Rayleigh scattering with the
> PND system really does not work, as it turns out, as it depends on
> pressure.)
>
> Perhaps we should move towards a *gas* / *particle* division in ARTS.
> That would mean including the Liebe rain absorption in the particle
> part. It would also affect your nomenclature, since one should then
> preferably talk about *ParticleSpecies* rather than
> *ScatteringSpecies*.
> (And *gas_species* instead of *absorption_species*.)
>
> All the best,
>
> Stefan
>
> On 26 Jan 2021, at 22:10, Simon Pfreundschuh wrote:
>
>> Dear Stefan,
>>
>>
>> I attach the description of the new scattering system that you
>> requested.
>>
>> I tried to keep it general to avoid getting lost in technicalities.
>> If
>> you have
>>
>> more specific questions please let me know.
>>
>>
>> Kind regards,
>>
>>
>> Simon
>>
>> 
>> From: arts_dev.mi-boun...@mailman.rrz.uni-hamburg.de
>>  on behalf of Stefan
>> Buehler 
>> Sent: Monday, January 25, 2021 9:36:33 AM
>> To: ARTS Developers List
>> Subject: Simon plans for scattering properties
>>
>> Dear all,
>>
>> one thing I took home from the developer meeting on Friday is that I
>> would like to better understand Simon’s plans for the scattering
>> properties. We need this to make a good decision on how to proceed
>> with
>> the Rayleigh scattering (incorporate it into the existing scattering
>> data framework, or make a special case for this analytical scattering
>> matrix). Simon, can you please summarize your concept briefly?
>>
>> All the best,
>>
>> Stefan

Re: Simon plans for scattering properties

[Stefan Buehler]

A controlfile example would perhaps help, as in the particle scattering 
example in your concept doc.


On 29 Jan 2021, at 8:42, Stefan Buehler wrote:


Hej Simon,

can you help me understand it then, please? I do want to find the best 
solution, not necessarily the quickest. (But simplicity is part of the 
criteria for “best”.)


/Stefan

On 29 Jan 2021, at 7:11, Simon Pfreundschuh wrote:


Dear Stefan,


I guess all I can say is that I am quite confident that you didn't 
properly


understand my proposal, which could quite easily handle the pressure

dependency. Anyhow, it's quite clear that you are searching for a 
quick


solution and I won't be able to provide that. So let's just cut this 
short.



/s




From: Stefan Buehler 
Sent: Thursday, January 28, 2021 5:02:28 PM
To: Simon Pfreundschuh
Cc: ARTS Developers List
Subject: Re: Simon plans for scattering properties

Dear Simon,

thanks for the summary!

I think I understand more or less how this works now. A bit 
unfortunate

that so much information has to be moved around at the controlfile
level. (The copying of grids from one variable to the other, the 
copying
of the scattering data themselves.) Is the idea that you would then 
in
practise store and load directly the ScatteringHabits, in order to 
make

this easier?

On where the Rayleigh scattering fits in: Patrick suggested when we
talked yesterday that perhaps a more logical way to think about this 
is

to distinguish between *gases* and *particles*, rather than between
*absorption* and *scattering*. (Including Rayleigh scattering with 
the

PND system really does not work, as it turns out, as it depends on
pressure.)

Perhaps we should move towards a *gas* / *particle* division in ARTS.
That would mean including the Liebe rain absorption in the particle
part. It would also affect your nomenclature, since one should then
preferably talk about *ParticleSpecies* rather than 
*ScatteringSpecies*.

(And *gas_species* instead of *absorption_species*.)

All the best,

Stefan

On 26 Jan 2021, at 22:10, Simon Pfreundschuh wrote:


Dear Stefan,


I attach the description of the new scattering system that you
requested.

I tried to keep it general to avoid getting lost in technicalities. 
If

you have

more specific questions please let me know.


Kind regards,


Simon


From: arts_dev.mi-boun...@mailman.rrz.uni-hamburg.de
 on behalf of Stefan
Buehler 
Sent: Monday, January 25, 2021 9:36:33 AM
To: ARTS Developers List
Subject: Simon plans for scattering properties

Dear all,

one thing I took home from the developer meeting on Friday is that I
would like to better understand Simon’s plans for the scattering
properties. We need this to make a good decision on how to proceed
with
the Rayleigh scattering (incorporate it into the existing scattering
data framework, or make a special case for this analytical 
scattering

matrix). Simon, can you please summarize your concept briefly?

All the best,

Stefan

Re: Simon plans for scattering properties

[Stefan Buehler]

Hej Simon,

can you help me understand it then, please? I do want to find the best 
solution, not necessarily the quickest. (But simplicity is part of the 
criteria for “best”.)


/Stefan

On 29 Jan 2021, at 7:11, Simon Pfreundschuh wrote:


Dear Stefan,


I guess all I can say is that I am quite confident that you didn't 
properly


understand my proposal, which could quite easily handle the pressure

dependency. Anyhow, it's quite clear that you are searching for a 
quick


solution and I won't be able to provide that. So let's just cut this 
short.



/s




From: Stefan Buehler 
Sent: Thursday, January 28, 2021 5:02:28 PM
To: Simon Pfreundschuh
Cc: ARTS Developers List
Subject: Re: Simon plans for scattering properties

Dear Simon,

thanks for the summary!

I think I understand more or less how this works now. A bit 
unfortunate

that so much information has to be moved around at the controlfile
level. (The copying of grids from one variable to the other, the 
copying

of the scattering data themselves.) Is the idea that you would then in
practise store and load directly the ScatteringHabits, in order to 
make

this easier?

On where the Rayleigh scattering fits in: Patrick suggested when we
talked yesterday that perhaps a more logical way to think about this 
is

to distinguish between *gases* and *particles*, rather than between
*absorption* and *scattering*. (Including Rayleigh scattering with the
PND system really does not work, as it turns out, as it depends on
pressure.)

Perhaps we should move towards a *gas* / *particle* division in ARTS.
That would mean including the Liebe rain absorption in the particle
part. It would also affect your nomenclature, since one should then
preferably talk about *ParticleSpecies* rather than 
*ScatteringSpecies*.

(And *gas_species* instead of *absorption_species*.)

All the best,

Stefan

On 26 Jan 2021, at 22:10, Simon Pfreundschuh wrote:


Dear Stefan,


I attach the description of the new scattering system that you
requested.

I tried to keep it general to avoid getting lost in technicalities. 
If

you have

more specific questions please let me know.


Kind regards,


Simon


From: arts_dev.mi-boun...@mailman.rrz.uni-hamburg.de
 on behalf of Stefan
Buehler 
Sent: Monday, January 25, 2021 9:36:33 AM
To: ARTS Developers List
Subject: Simon plans for scattering properties

Dear all,

one thing I took home from the developer meeting on Friday is that I
would like to better understand Simon’s plans for the scattering
properties. We need this to make a good decision on how to proceed
with
the Rayleigh scattering (incorporate it into the existing scattering
data framework, or make a special case for this analytical scattering
matrix). Simon, can you please summarize your concept briefly?

All the best,

Stefan

Re: Simon plans for scattering properties

[Simon Pfreundschuh]

Dear Stefan,


I guess all I can say is that I am quite confident that you didn't properly

understand my proposal, which could quite easily handle the pressure

dependency. Anyhow, it's quite clear that you are searching for a quick

solution and I won't be able to provide that. So let's just cut this short.


/s




From: Stefan Buehler 
Sent: Thursday, January 28, 2021 5:02:28 PM
To: Simon Pfreundschuh
Cc: ARTS Developers List
Subject: Re: Simon plans for scattering properties

Dear Simon,

thanks for the summary!

I think I understand more or less how this works now. A bit unfortunate
that so much information has to be moved around at the controlfile
level. (The copying of grids from one variable to the other, the copying
of the scattering data themselves.) Is the idea that you would then in
practise store and load directly the ScatteringHabits, in order to make
this easier?

On where the Rayleigh scattering fits in: Patrick suggested when we
talked yesterday that perhaps a more logical way to think about this is
to distinguish between *gases* and *particles*, rather than between
*absorption* and *scattering*. (Including Rayleigh scattering with the
PND system really does not work, as it turns out, as it depends on
pressure.)

Perhaps we should move towards a *gas* / *particle* division in ARTS.
That would mean including the Liebe rain absorption in the particle
part. It would also affect your nomenclature, since one should then
preferably talk about *ParticleSpecies* rather than *ScatteringSpecies*.
(And *gas_species* instead of *absorption_species*.)

All the best,

Stefan

On 26 Jan 2021, at 22:10, Simon Pfreundschuh wrote:

> Dear Stefan,
>
>
> I attach the description of the new scattering system that you
> requested.
>
> I tried to keep it general to avoid getting lost in technicalities. If
> you have
>
> more specific questions please let me know.
>
>
> Kind regards,
>
>
> Simon
>
> 
> From: arts_dev.mi-boun...@mailman.rrz.uni-hamburg.de
>  on behalf of Stefan
> Buehler 
> Sent: Monday, January 25, 2021 9:36:33 AM
> To: ARTS Developers List
> Subject: Simon plans for scattering properties
>
> Dear all,
>
> one thing I took home from the developer meeting on Friday is that I
> would like to better understand Simon’s plans for the scattering
> properties. We need this to make a good decision on how to proceed
> with
> the Rayleigh scattering (incorporate it into the existing scattering
> data framework, or make a special case for this analytical scattering
> matrix). Simon, can you please summarize your concept briefly?
>
> All the best,
>
> Stefan

Re: Simon plans for scattering properties

[Stefan Buehler]

Dear Simon,

thanks for the summary!

I think I understand more or less how this works now. A bit unfortunate 
that so much information has to be moved around at the controlfile 
level. (The copying of grids from one variable to the other, the copying 
of the scattering data themselves.) Is the idea that you would then in 
practise store and load directly the ScatteringHabits, in order to make 
this easier?


On where the Rayleigh scattering fits in: Patrick suggested when we 
talked yesterday that perhaps a more logical way to think about this is 
to distinguish between *gases* and *particles*, rather than between 
*absorption* and *scattering*. (Including Rayleigh scattering with the 
PND system really does not work, as it turns out, as it depends on 
pressure.)


Perhaps we should move towards a *gas* / *particle* division in ARTS. 
That would mean including the Liebe rain absorption in the particle 
part. It would also affect your nomenclature, since one should then 
preferably talk about *ParticleSpecies* rather than *ScatteringSpecies*. 
(And *gas_species* instead of *absorption_species*.)


All the best,

Stefan

On 26 Jan 2021, at 22:10, Simon Pfreundschuh wrote:


Dear Stefan,


I attach the description of the new scattering system that you 
requested.


I tried to keep it general to avoid getting lost in technicalities. If 
you have


more specific questions please let me know.


Kind regards,


Simon


From: arts_dev.mi-boun...@mailman.rrz.uni-hamburg.de 
 on behalf of Stefan 
Buehler 

Sent: Monday, January 25, 2021 9:36:33 AM
To: ARTS Developers List
Subject: Simon plans for scattering properties

Dear all,

one thing I took home from the developer meeting on Friday is that I
would like to better understand Simon’s plans for the scattering
properties. We need this to make a good decision on how to proceed 
with

the Rayleigh scattering (incorporate it into the existing scattering
data framework, or make a special case for this analytical scattering
matrix). Simon, can you please summarize your concept briefly?

All the best,

Stefan