Re: [EXTERNAL] [BULK] 3D MC

2023-12-08 Thread Adams, Ian S {he, him, his} (GSFC-6120)
At one point, I was running a version of the single scattering radar code 
internally that used mc_antenna to sample a Gaussian beam. This could be 
applied similarly to the independent pixel approximation for the passive 
sensors, but the way I did it seemed very redundant. I never folded your 
updates in, which is the other danger of my approach, so that code has 
languished.


On 12/7/23, 2:29 AM, "Patrick Eriksson" mailto:patrick.eriks...@chalmers.se>> wrote:


CAUTION: This email originated from outside of NASA. Please take care when 
clicking links or opening attachments. Use the "Report Message" button to 
report suspicious messages to the NASA SOC.








Ian,


Thanks for the input. Great that you have stress-tested MC. Too bad that
it revealed a limitation.


Good suggestion about iyMC. Today it would not be possible to do the
random sampling from yCalc, it would require information on the sensor
not at hand inside yCalc today. But we are planning to redesign the way
the sensor is described, and this should be considered.


Not totally sure exactly what you mean with using MC sampled antenna
pattern more broadly, but I tend to agree. It would be good if there
would be mechanisms to give monochromatic pencil beam calculations some
width in frequency and space. It would speed up simulations of
observations. As example, I have been playing around with a scheme to
locally average the surface emissivity around the point you hit the
surface, to make simulations in coastal areas faster.


And yes, the most tricky part is finding the time for the work.


Bye,


Patrick




On 2023-12-06 16:09, Adams, Ian S {he, him, his} (GSFC-6120) wrote:
> Hi Stefan,
>
> I have been contemplating changes to the MC codes. One thing we have found is 
> that MCGeneral breaks down when Q starts to get large. We see unrealistic 
> results at 684 GHz when using horizontally aligned particles with high aspect 
> ratio. Yuli Liu, who is working with us now, did comprehensive analysis, and 
> we believe that the issue is the way the backwards algorithm is using 
> importance sampling to avoid the issue of inverting the extinction matrix; 
> however, this approach neglects the mixing of I and Q. I believe this is a 
> simple fix.
>
> The other issue is that MCGeneral is not very ARTS-like. Looking at the way 
> it is structured, I think a better approach would be to have an iyMC that 
> traces a single "photon," and yMC would integrate these individual results. 
> Random sampling of both the antenna pattern and the bandwidth could be 
> performed at this level. I also think that the MC sampled antenna pattern 
> could be more widely useful across ARTS.
>
> These papers provide an interesting curveball. The ARTS MC codes are 
> particularly slow, and they are not optimized for optically thin or extremely 
> optically thick atmospheres. We could look at using these libraries, or at 
> least techniques, but I'm not sure how intensive such a restructuring of the 
> code would be.
>
> Of course, the tricky piece here is finding someone with the time to do this 
> work. But, I think these changes would make the codes significantly more 
> usable, and hopefully therefore used.
>
> Cheers,
> Ian
>
> On 11/29/23, 11:22 AM, "arts_dev.mi on behalf of Stefan Buehler" 
>  <mailto:arts_dev.mi-boun...@lists.uni-hamburg.de> 
> <mailto:arts_dev.mi-boun...@lists.uni-hamburg.de 
> <mailto:arts_dev.mi-boun...@lists.uni-hamburg.de>> on behalf of 
> stefan.bueh...@uni-hamburg.de <mailto:stefan.bueh...@uni-hamburg.de> 
> <mailto:stefan.bueh...@uni-hamburg.de 
> <mailto:stefan.bueh...@uni-hamburg.de>>> wrote:
>
>
> Dear all,
>
>
> I stumbled accross this interesting paper on an open C library for 
> particularly efficient MC calculations. Could this be the basis of ARTS 3D MC 
> flux and heating rate calculations? Using MC sampling also for the spectral 
> dimension, to be efficient, as in the second paper, which is also impressive, 
> I think. They use MC sampling even for the spectral lines, if I got it right! 
> (Basically treating each transition as if it were its own absorption species.)
>
>
> /Stefan
>
>
> https://www.dropbox.com/scl/fi/smsisfgc2it3sx4gov970/J-Adv-Model-Earth-Syst-2019-Villefranque-A-Path-E2-80-90Tracing-Monte-Carlo-Library-for-3-E2-80-90D-Radiative-Transfer-in-Highly.pdf?rlkey=v5yvrm64fnljaf739j4ssllux=0
>  
> <https://www.dropbox.com/scl/fi/smsisfgc2it3sx4gov970/J-Adv-Model-Earth-Syst-2019-Villefranque-A-Path-E2-80-90Tracing-Monte-Carlo-Library-for-3-E2-80-90D-Radiative-Transfer-in-Highly.pdf?rlkey=v5yvrm64fnljaf739j4ssllux=0>
>  
> <https://www.dropbox.com/scl/fi/smsisfgc2it3sx4gov970/J-Adv-Model-Earth-Syst-2019-Villefranque-A-Path-E2-80-90Tracing-Mont

Re: [EXTERNAL] [BULK] 3D MC

2023-12-07 Thread Stefan Buehler
Dear Ian,

I also agree. In fact, I hope that we could use MC much more widely in some 
future version of ARTS, perhaps all the way down to MC sampling individual 
spectral lines.

/Stefan

On 7 Dec 2023, at 8:27, Patrick Eriksson wrote:

> Ian,
>
> Thanks for the input. Great that you have stress-tested MC. Too bad that it 
> revealed a limitation.
>
> Good suggestion about iyMC. Today it would not be possible to do the random 
> sampling from yCalc, it would require information on the sensor not at hand 
> inside yCalc today. But we are planning to redesign the way the sensor is 
> described, and this should be considered.
>
> Not totally sure exactly what you mean with using MC sampled antenna pattern 
> more broadly, but I tend to agree. It would be good if there would be 
> mechanisms to give monochromatic pencil beam calculations some width in 
> frequency and space. It would speed up simulations of observations. As 
> example, I have been playing around with a scheme to locally average the 
> surface emissivity around the point you hit the surface, to make simulations 
> in coastal areas faster.
>
> And yes, the most tricky part is finding the time for the work.
>
> Bye,
>
> Patrick
>
>
> On 2023-12-06 16:09, Adams, Ian S {he, him, his} (GSFC-6120) wrote:
>> Hi Stefan,
>>
>> I have been contemplating changes to the MC codes. One thing we have found 
>> is that MCGeneral breaks down when Q starts to get large. We see unrealistic 
>> results at 684 GHz when using horizontally aligned particles with high 
>> aspect ratio. Yuli Liu, who is working with us now, did comprehensive 
>> analysis, and we believe that the issue is the way the backwards algorithm 
>> is using importance sampling to avoid the issue of inverting the extinction 
>> matrix; however, this approach neglects the mixing of I and Q. I believe 
>> this is a simple fix.
>>
>> The other issue is that MCGeneral is not very ARTS-like. Looking at the way 
>> it is structured, I think a better approach would be to have an iyMC that 
>> traces a single "photon," and yMC would integrate these individual results. 
>> Random sampling of both the antenna pattern and the bandwidth could be 
>> performed at this level. I also think that the MC sampled antenna pattern 
>> could be more widely useful across ARTS.
>>
>> These papers provide an interesting curveball. The ARTS MC codes are 
>> particularly slow, and they are not optimized for optically thin or 
>> extremely optically thick atmospheres. We could look at using these 
>> libraries, or at least techniques, but I'm not sure how intensive such a 
>> restructuring of the code would be.
>>
>> Of course, the tricky piece here is finding someone with the time to do this 
>> work. But, I think these changes would make the codes significantly more 
>> usable, and hopefully therefore used.
>>
>> Cheers,
>> Ian
>>
>> On 11/29/23, 11:22 AM, "arts_dev.mi on behalf of Stefan Buehler" 
>> > <mailto:arts_dev.mi-boun...@lists.uni-hamburg.de> on behalf of 
>> stefan.bueh...@uni-hamburg.de <mailto:stefan.bueh...@uni-hamburg.de>> wrote:
>>
>>
>> Dear all,
>>
>>
>> I stumbled accross this interesting paper on an open C library for 
>> particularly efficient MC calculations. Could this be the basis of ARTS 3D 
>> MC flux and heating rate calculations? Using MC sampling also for the 
>> spectral dimension, to be efficient, as in the second paper, which is also 
>> impressive, I think. They use MC sampling even for the spectral lines, if I 
>> got it right! (Basically treating each transition as if it were its own 
>> absorption species.)
>>
>>
>> /Stefan
>>
>>
>> https://www.dropbox.com/scl/fi/smsisfgc2it3sx4gov970/J-Adv-Model-Earth-Syst-2019-Villefranque-A-Path-E2-80-90Tracing-Monte-Carlo-Library-for-3-E2-80-90D-Radiative-Transfer-in-Highly.pdf?rlkey=v5yvrm64fnljaf739j4ssllux=0
>>  
>> <https://www.dropbox.com/scl/fi/smsisfgc2it3sx4gov970/J-Adv-Model-Earth-Syst-2019-Villefranque-A-Path-E2-80-90Tracing-Monte-Carlo-Library-for-3-E2-80-90D-Radiative-Transfer-in-Highly.pdf?rlkey=v5yvrm64fnljaf739j4sslluxdl=0>
>>
>>
>> https://www.dropbox.com/scl/fi/r1tm3jdzx57kb85nowmt0/Yaniss_ea_PNAS_2023_smi.pdf?rlkey=8d4a7rb4u8pehckawbfk08c9f=0
>>  
>> <https://www.dropbox.com/scl/fi/r1tm3jdzx57kb85nowmt0/Yaniss_ea_PNAS_2023_smi.pdf?rlkey=8d4a7rb4u8pehckawbfk08c9fdl=0>
>>

smime.p7s
Description: S/MIME digital signature


Re: [EXTERNAL] [BULK] 3D MC

2023-12-06 Thread Patrick Eriksson

Ian,

Thanks for the input. Great that you have stress-tested MC. Too bad that 
it revealed a limitation.


Good suggestion about iyMC. Today it would not be possible to do the 
random sampling from yCalc, it would require information on the sensor 
not at hand inside yCalc today. But we are planning to redesign the way 
the sensor is described, and this should be considered.


Not totally sure exactly what you mean with using MC sampled antenna 
pattern more broadly, but I tend to agree. It would be good if there 
would be mechanisms to give monochromatic pencil beam calculations some 
width in frequency and space. It would speed up simulations of 
observations. As example, I have been playing around with a scheme to 
locally average the surface emissivity around the point you hit the 
surface, to make simulations in coastal areas faster.


And yes, the most tricky part is finding the time for the work.

Bye,

Patrick


On 2023-12-06 16:09, Adams, Ian S {he, him, his} (GSFC-6120) wrote:

Hi Stefan,

I have been contemplating changes to the MC codes. One thing we have found is 
that MCGeneral breaks down when Q starts to get large. We see unrealistic 
results at 684 GHz when using horizontally aligned particles with high aspect 
ratio. Yuli Liu, who is working with us now, did comprehensive analysis, and we 
believe that the issue is the way the backwards algorithm is using importance 
sampling to avoid the issue of inverting the extinction matrix; however, this 
approach neglects the mixing of I and Q. I believe this is a simple fix.

The other issue is that MCGeneral is not very ARTS-like. Looking at the way it is 
structured, I think a better approach would be to have an iyMC that traces a single 
"photon," and yMC would integrate these individual results. Random sampling of 
both the antenna pattern and the bandwidth could be performed at this level. I also think 
that the MC sampled antenna pattern could be more widely useful across ARTS.

These papers provide an interesting curveball. The ARTS MC codes are 
particularly slow, and they are not optimized for optically thin or extremely 
optically thick atmospheres. We could look at using these libraries, or at 
least techniques, but I'm not sure how intensive such a restructuring of the 
code would be.

Of course, the tricky piece here is finding someone with the time to do this 
work. But, I think these changes would make the codes significantly more 
usable, and hopefully therefore used.

Cheers,
Ian

On 11/29/23, 11:22 AM, "arts_dev.mi on behalf of Stefan Buehler" 
mailto:arts_dev.mi-boun...@lists.uni-hamburg.de> on 
behalf of stefan.bueh...@uni-hamburg.de <mailto:stefan.bueh...@uni-hamburg.de>> wrote:


Dear all,


I stumbled accross this interesting paper on an open C library for particularly 
efficient MC calculations. Could this be the basis of ARTS 3D MC flux and 
heating rate calculations? Using MC sampling also for the spectral dimension, 
to be efficient, as in the second paper, which is also impressive, I think. 
They use MC sampling even for the spectral lines, if I got it right! (Basically 
treating each transition as if it were its own absorption species.)


/Stefan


https://www.dropbox.com/scl/fi/smsisfgc2it3sx4gov970/J-Adv-Model-Earth-Syst-2019-Villefranque-A-Path-E2-80-90Tracing-Monte-Carlo-Library-for-3-E2-80-90D-Radiative-Transfer-in-Highly.pdf?rlkey=v5yvrm64fnljaf739j4ssllux=0
 
<https://www.dropbox.com/scl/fi/smsisfgc2it3sx4gov970/J-Adv-Model-Earth-Syst-2019-Villefranque-A-Path-E2-80-90Tracing-Monte-Carlo-Library-for-3-E2-80-90D-Radiative-Transfer-in-Highly.pdf?rlkey=v5yvrm64fnljaf739j4sslluxdl=0>


https://www.dropbox.com/scl/fi/r1tm3jdzx57kb85nowmt0/Yaniss_ea_PNAS_2023_smi.pdf?rlkey=8d4a7rb4u8pehckawbfk08c9f=0
 
<https://www.dropbox.com/scl/fi/r1tm3jdzx57kb85nowmt0/Yaniss_ea_PNAS_2023_smi.pdf?rlkey=8d4a7rb4u8pehckawbfk08c9fdl=0>



Re: [EXTERNAL] [BULK] 3D MC

2023-12-06 Thread Adams, Ian S {he, him, his} (GSFC-6120)
Hi Stefan,

I have been contemplating changes to the MC codes. One thing we have found is 
that MCGeneral breaks down when Q starts to get large. We see unrealistic 
results at 684 GHz when using horizontally aligned particles with high aspect 
ratio. Yuli Liu, who is working with us now, did comprehensive analysis, and we 
believe that the issue is the way the backwards algorithm is using importance 
sampling to avoid the issue of inverting the extinction matrix; however, this 
approach neglects the mixing of I and Q. I believe this is a simple fix.

The other issue is that MCGeneral is not very ARTS-like. Looking at the way it 
is structured, I think a better approach would be to have an iyMC that traces a 
single "photon," and yMC would integrate these individual results. Random 
sampling of both the antenna pattern and the bandwidth could be performed at 
this level. I also think that the MC sampled antenna pattern could be more 
widely useful across ARTS.

These papers provide an interesting curveball. The ARTS MC codes are 
particularly slow, and they are not optimized for optically thin or extremely 
optically thick atmospheres. We could look at using these libraries, or at 
least techniques, but I'm not sure how intensive such a restructuring of the 
code would be.

Of course, the tricky piece here is finding someone with the time to do this 
work. But, I think these changes would make the codes significantly more 
usable, and hopefully therefore used.

Cheers,
Ian

On 11/29/23, 11:22 AM, "arts_dev.mi on behalf of Stefan Buehler" 
mailto:arts_dev.mi-boun...@lists.uni-hamburg.de> on behalf of 
stefan.bueh...@uni-hamburg.de <mailto:stefan.bueh...@uni-hamburg.de>> wrote:


Dear all,


I stumbled accross this interesting paper on an open C library for particularly 
efficient MC calculations. Could this be the basis of ARTS 3D MC flux and 
heating rate calculations? Using MC sampling also for the spectral dimension, 
to be efficient, as in the second paper, which is also impressive, I think. 
They use MC sampling even for the spectral lines, if I got it right! (Basically 
treating each transition as if it were its own absorption species.)


/Stefan


https://www.dropbox.com/scl/fi/smsisfgc2it3sx4gov970/J-Adv-Model-Earth-Syst-2019-Villefranque-A-Path-E2-80-90Tracing-Monte-Carlo-Library-for-3-E2-80-90D-Radiative-Transfer-in-Highly.pdf?rlkey=v5yvrm64fnljaf739j4ssllux=0
 
<https://www.dropbox.com/scl/fi/smsisfgc2it3sx4gov970/J-Adv-Model-Earth-Syst-2019-Villefranque-A-Path-E2-80-90Tracing-Monte-Carlo-Library-for-3-E2-80-90D-Radiative-Transfer-in-Highly.pdf?rlkey=v5yvrm64fnljaf739j4sslluxdl=0>


https://www.dropbox.com/scl/fi/r1tm3jdzx57kb85nowmt0/Yaniss_ea_PNAS_2023_smi.pdf?rlkey=8d4a7rb4u8pehckawbfk08c9f=0
 
<https://www.dropbox.com/scl/fi/r1tm3jdzx57kb85nowmt0/Yaniss_ea_PNAS_2023_smi.pdf?rlkey=8d4a7rb4u8pehckawbfk08c9fdl=0>



3D MC

2023-11-29 Thread Stefan Buehler
Dear all,

I stumbled accross this interesting paper on an open C library for particularly 
efficient MC calculations. Could this be the basis of ARTS 3D MC flux and 
heating rate calculations? Using MC sampling also for the spectral dimension, 
to be efficient, as in the second paper, which is also impressive, I think. 
They use MC sampling even for the spectral lines, if I got it right! (Basically 
treating each transition as if it were its own absorption species.)

/Stefan

https://www.dropbox.com/scl/fi/smsisfgc2it3sx4gov970/J-Adv-Model-Earth-Syst-2019-Villefranque-A-Path-E2-80-90Tracing-Monte-Carlo-Library-for-3-E2-80-90D-Radiative-Transfer-in-Highly.pdf?rlkey=v5yvrm64fnljaf739j4ssllux=0

https://www.dropbox.com/scl/fi/r1tm3jdzx57kb85nowmt0/Yaniss_ea_PNAS_2023_smi.pdf?rlkey=8d4a7rb4u8pehckawbfk08c9f=0

smime.p7s
Description: S/MIME digital signature