Re: Simon plans for scattering properties
Hej Richard, to me the interesting point is that the specific inputs could be handled where the “Add..Habit” happens. In the case of absorption, a lot of difficulty comes from the dichotomy between the species tags and the methods that have to be added to the absorption agenda to actually do the calculations. Using Simons concept, one could potentially replace the tags by something like AbsorberAppendLBL(“H2O”) AbsorberAppendCIA(“N2-O2”) AbsorberAppendLBL(“O3”) AbsorberAppendXsec(“CFC-11”) So, instead of indentifying the model to be used by the tag name, we could identify it by the Add method used. The different Add methods would depend on different input variables, such as line catalog, tabulated cross-sections, etc.. That solution seems quite elegant to me. One problem is, though, that one absorption tag currently can be a combination of models for the same gas species. One way to solve this would be to have two types of methods, Add and Append, like this: AbsorberAppendLBL(“H2O”) AbsorberAddCont(“H2O”,”CKD-MT-3_7) AbsorberAppendCIA(“N2-O2”) AbsorberAppendLBL(“O2”) AbsorberAddCont(“O2”,”PWR20”) AbsorberAppendXsec(“CFC-11”) A strict interface for the output is no problem, since all absorption functions produce exactly the same output. For the (atmospheric) input it is potentially more problematic. For example, Zeeman absorption depends on the magnetic field, but none of the other absorption functions do. I don’t think there is so much overhead that one really gains a lot by for example doing all the LBL species in one go. One could basically use the present lower level functions, and just call them repeatedly for each gas in turn. When the absorption lookup table is generated, I think it is already done this way. AbsorberAppendLBL would depend on abs_lines_per_species, and just incorporate the lines for the species in questions to the local data of the AbsorberLBL class, inheriting the public interface from the general Absorber class. So, I think in principle this structure could be built on top of the various absorption functions we have, replacing the current tag mechanism. Ah, there probably are a lot of pitfalls. One that I just noticed is that, if the species list is only established by these Append methods, then we have no good way to decide for which species we should read in the line catalogue data. (For that the list of species of interest would have to be defined before.) This could be solved by the Append method doing the catalog reading, but that would make the Append methods quite heavy. On the other hand it would make for very short and simple controlfiles, the call would be like: AbsorberAppendLBL(,“H2O”) I’m not sure whether I like that, though, it seems less flexible than what we have when it comes to dealing with the catalog. All the best, Stefan On 1 Feb 2021, at 16:10, Richard Larsson wrote: Simon, Stefan, It seems to me that this is a proposal of using a modern "void *" list/vector with some strict interface. Have I misunderstood something here? I mean you might implement it with virtual classes, std::variant, or some other reinterpreting interface but that seems about it. I am not going to argue for the scattering implementation here because I assume you guys know the current interface well enough and have ideas of how to easily extend it using the new and old methods. Perhaps the strict interface helps here. I would not mind having a "void *" list/vector in the LBL code but I am very wary about the strict interface requirements here. I would be much more accepting if we could just get some kind of "const ClassImpl&" back from the "void *" list. Then the different methods can much more easily know if they need to write to src or not, to a PropagationMatrix interface or to an ArrayOfMatrix interface. I don't like the idea of separating this by species in the LBL though. It would be much better to separate it by the types of calculations necessary (the SpeciesTag-enum + the Absorption::PopulationType enum for LBL). That way you could probably allow pre-allocations of all allocated variables used by the algorithms taking the class without having to worry too much about the memory footprint and leaving deallocation or reuse up to the user (a cleanup method or more in the Agenda). That said, I think it is quite a daunting task to change things this way. And I am not sure how easy it is to take Simon's idea and use it with value-semantics. With hope, //Richard Den mån 1 feb. 2021 kl 14:14 skrev Stefan Buehler < stefan.bueh...@uni-hamburg.de>: Dear Simon, The handling of the constant data is completely up to the implementation of the specific scatterers. For the ScatteringParticle class, for example, this is provided when the object is constructed. So when the ARTS user calls: scattering_speciesAddScatteringHabit(...) an object is created which will hold the scattering data.
Re: Simon plans for scattering properties
Simon, Stefan, It seems to me that this is a proposal of using a modern "void *" list/vector with some strict interface. Have I misunderstood something here? I mean you might implement it with virtual classes, std::variant, or some other reinterpreting interface but that seems about it. I am not going to argue for the scattering implementation here because I assume you guys know the current interface well enough and have ideas of how to easily extend it using the new and old methods. Perhaps the strict interface helps here. I would not mind having a "void *" list/vector in the LBL code but I am very wary about the strict interface requirements here. I would be much more accepting if we could just get some kind of "const ClassImpl&" back from the "void *" list. Then the different methods can much more easily know if they need to write to src or not, to a PropagationMatrix interface or to an ArrayOfMatrix interface. I don't like the idea of separating this by species in the LBL though. It would be much better to separate it by the types of calculations necessary (the SpeciesTag-enum + the Absorption::PopulationType enum for LBL). That way you could probably allow pre-allocations of all allocated variables used by the algorithms taking the class without having to worry too much about the memory footprint and leaving deallocation or reuse up to the user (a cleanup method or more in the Agenda). That said, I think it is quite a daunting task to change things this way. And I am not sure how easy it is to take Simon's idea and use it with value-semantics. With hope, //Richard Den mån 1 feb. 2021 kl 14:14 skrev Stefan Buehler < stefan.bueh...@uni-hamburg.de>: > Dear Simon, > > > The handling of the constant data is completely up to the > > implementation of the specific scatterers. > > For the ScatteringParticle class, for example, this is provided when > > the object is constructed. > > So when the ARTS user calls: > > > > scattering_speciesAddScatteringHabit(...) > > > > an object is created which will hold the scattering data. It will then > > use it when asked to > > compute the bulk scattering properties. Since the > > calculate_bulk_properties(...) method > > is called on this object, it automatically has access to the required > > data. > > Ah, yes. That is a great solution. Perhaps I could do the same for the > absorbers. But I would have to break up the methods so that they are > standalone for each absorption tag. Right now, the method that does LBL, > for example, will calculate for all the LBL absorption tags, using a > common workspace variable for the line catalog (which is internally > split by species). All the housekeeping around this is a lot of trouble. > Perhaps it would be better to always run it just for a single absorber, > and use a technique similar to yours to set up the list, so that each > absorber gets the line list he needs. (And similarly for the other types > of absorption.) > > /Stefan >
Re: Simon plans for scattering properties
Dear Simon, The handling of the constant data is completely up to the implementation of the specific scatterers. For the ScatteringParticle class, for example, this is provided when the object is constructed. So when the ARTS user calls: scattering_speciesAddScatteringHabit(...) an object is created which will hold the scattering data. It will then use it when asked to compute the bulk scattering properties. Since the calculate_bulk_properties(...) method is called on this object, it automatically has access to the required data. Ah, yes. That is a great solution. Perhaps I could do the same for the absorbers. But I would have to break up the methods so that they are standalone for each absorption tag. Right now, the method that does LBL, for example, will calculate for all the LBL absorption tags, using a common workspace variable for the line catalog (which is internally split by species). All the housekeeping around this is a lot of trouble. Perhaps it would be better to always run it just for a single absorber, and use a technique similar to yours to set up the list, so that each absorber gets the line list he needs. (And similarly for the other types of absorption.) /Stefan
Re: Simon plans for scattering properties
>From the perspective of computing scattering (or absorption) properties for a given location in the atmosphere there are two types of required data: - constant data describing the scatterer (single scattering data for particles, etc.) - atmospheric input: These are quantities that vary with position (pressure, temperature, vmr, etc.) The interface I described needs to handle only the second kind of data. This is important because it's the key to decoupling the different scatterers from the RT methods that use them. The handling of the constant data is completely up to the implementation of the specific scatterers. For the ScatteringParticle class, for example, this is provided when the object is constructed. So when the ARTS user calls: scattering_speciesAddScatteringHabit(...) an object is created which will hold the scattering data. It will then use it when asked to compute the bulk scattering properties. Since the calculate_bulk_properties(...) method is called on this object, it automatically has access to the required data. However, what this data is and how it is handled is completely up to the specific class. For molecular scattering your would probably only need to know the atmospheric composition. The key here is the object oriented approach: By defining the interface through a class hierarchy the interface needs to handle only the atmospheric input but can leave the handling of ancillary data that doesn't vary to each class. From: Stefan Buehler Sent: Monday, February 1, 2021 12:07:56 PM To: Simon Pfreundschuh Cc: ARTS Developers List Subject: Re: Simon plans for scattering properties Dear Simon, how will you manage communication? I’m asking because there is a related issue in computing absorption. It is easy to define a common interface with respect to what the functions have to provide. But in that case (and perhaps also in your case) a problem is that they require quite different input, in addition to the atmosphere. (In the case of absorption for example spectral line data for LBL functions, or a table with measured cross-section data for other functions.) Best wishes, Stefan On 31 Jan 2021, at 10:53, Simon Pfreundschuh wrote: > Dear Stefan, > > > If there's no user-configurable parameters required for molecular > scattering then > > using it in a control file would be trivial: > > > scattering_species_AddMolecularScattering() > > > The iy method would then just need to check if the sun is on and > calculate > > the bulk scattering properties for the first order scattering. In this > way particles > > and molecules would be handle in a consistent way. > > > What is required for this to work is of course a class that implements > the interface > > defined by the ScatteringSpeciesImpl class (as indicated in the PDF). > > > I also would really like to highlight the following, which you seem to > have > misunderstood: > > > The interface that I described makes no assumptions whatsoever on the > scattering > > species being either particles or a gas. It essentially already > implements what > > you described but is even less restrictive. It only requires that a > scattering species > > is able to provide bulk scattering data from an atmospheric state > following the > > protocol defined by the ScatteringSpeciesImpl class. > > I am quite sure that with this system molecular and particle > scattering can > be handled consistently and the need for a molecular_scattering flag. > However, > I also agree that it would probably be inappropriate to require Jon > to use this > system considering that he's just a Master's student and that this is > still under > development. > > Kind regards, > > Simon > > > > > From: arts_dev.mi-boun...@mailman.rrz.uni-hamburg.de > on behalf of Stefan > Buehler > Sent: Friday, January 29, 2021 8:42:21 AM > To: Simon Pfreundschuh > Cc: ARTS Developers List > Subject: Re: Simon plans for scattering properties > > Hej Simon, > > can you help me understand it then, please? I do want to find the best > solution, not necessarily the quickest. (But simplicity is part of the > criteria for “best”.) > > /Stefan > > On 29 Jan 2021, at 7:11, Simon Pfreundschuh wrote: > >> Dear Stefan, >> >> >> I guess all I can say is that I am quite confident that you didn't >> properly >> >> understand my proposal, which could quite easily handle the pressure >> >> dependency. Anyhow, it's quite clear that you are searching for a >> quick >> >> solution and I won't be able to provide that. So let's just cut this >> short. >> >> >> /s >> >> >> >> >> From: Stefan Buehler >> Sent: Thursday, January 28, 2021 5:02:28 PM >> To: Simon Pfreundschuh >> Cc: ARTS Developers List >> Subject: Re: Simon plans for scattering properties >> >> Dear Simon, >> >> thanks for the summary! >> >> I think I understand more or less how this works now. A bit >> unfortunate >> that so much
Re: Simon plans for scattering properties
Dear Simon, how will you manage communication? I’m asking because there is a related issue in computing absorption. It is easy to define a common interface with respect to what the functions have to provide. But in that case (and perhaps also in your case) a problem is that they require quite different input, in addition to the atmosphere. (In the case of absorption for example spectral line data for LBL functions, or a table with measured cross-section data for other functions.) Best wishes, Stefan On 31 Jan 2021, at 10:53, Simon Pfreundschuh wrote: Dear Stefan, If there's no user-configurable parameters required for molecular scattering then using it in a control file would be trivial: scattering_species_AddMolecularScattering() The iy method would then just need to check if the sun is on and calculate the bulk scattering properties for the first order scattering. In this way particles and molecules would be handle in a consistent way. What is required for this to work is of course a class that implements the interface defined by the ScatteringSpeciesImpl class (as indicated in the PDF). I also would really like to highlight the following, which you seem to have misunderstood: The interface that I described makes no assumptions whatsoever on the scattering species being either particles or a gas. It essentially already implements what you described but is even less restrictive. It only requires that a scattering species is able to provide bulk scattering data from an atmospheric state following the protocol defined by the ScatteringSpeciesImpl class. I am quite sure that with this system molecular and particle scattering can be handled consistently and the need for a molecular_scattering flag. However, I also agree that it would probably be inappropriate to require Jon to use this system considering that he's just a Master's student and that this is still under development. Kind regards, Simon From: arts_dev.mi-boun...@mailman.rrz.uni-hamburg.de on behalf of Stefan Buehler Sent: Friday, January 29, 2021 8:42:21 AM To: Simon Pfreundschuh Cc: ARTS Developers List Subject: Re: Simon plans for scattering properties Hej Simon, can you help me understand it then, please? I do want to find the best solution, not necessarily the quickest. (But simplicity is part of the criteria for “best”.) /Stefan On 29 Jan 2021, at 7:11, Simon Pfreundschuh wrote: Dear Stefan, I guess all I can say is that I am quite confident that you didn't properly understand my proposal, which could quite easily handle the pressure dependency. Anyhow, it's quite clear that you are searching for a quick solution and I won't be able to provide that. So let's just cut this short. /s From: Stefan Buehler Sent: Thursday, January 28, 2021 5:02:28 PM To: Simon Pfreundschuh Cc: ARTS Developers List Subject: Re: Simon plans for scattering properties Dear Simon, thanks for the summary! I think I understand more or less how this works now. A bit unfortunate that so much information has to be moved around at the controlfile level. (The copying of grids from one variable to the other, the copying of the scattering data themselves.) Is the idea that you would then in practise store and load directly the ScatteringHabits, in order to make this easier? On where the Rayleigh scattering fits in: Patrick suggested when we talked yesterday that perhaps a more logical way to think about this is to distinguish between *gases* and *particles*, rather than between *absorption* and *scattering*. (Including Rayleigh scattering with the PND system really does not work, as it turns out, as it depends on pressure.) Perhaps we should move towards a *gas* / *particle* division in ARTS. That would mean including the Liebe rain absorption in the particle part. It would also affect your nomenclature, since one should then preferably talk about *ParticleSpecies* rather than *ScatteringSpecies*. (And *gas_species* instead of *absorption_species*.) All the best, Stefan On 26 Jan 2021, at 22:10, Simon Pfreundschuh wrote: Dear Stefan, I attach the description of the new scattering system that you requested. I tried to keep it general to avoid getting lost in technicalities. If you have more specific questions please let me know. Kind regards, Simon From: arts_dev.mi-boun...@mailman.rrz.uni-hamburg.de on behalf of Stefan Buehler Sent: Monday, January 25, 2021 9:36:33 AM To: ARTS Developers List Subject: Simon plans for scattering properties Dear all, one thing I took home from the developer meeting on Friday is that I would like to better understand Simon’s plans for the scattering properties. We need this to make a good decision on how to proceed with the Rayleigh scattering (incorporate it into the existing scattering data framework, or make a special case for this
