date:20200122

Re: [Ifeffit] smoothing XAS data

2020-01-22 Thread Carlo Segre



I agree with all of the comments.  If I take data using step scans, then I 
never smooth.  However, in continuous scan mode where the data is 
oversampled, a boxcar average in the EXAFS region is OK.  The alternative 
is that the fitting software interpolates using an algorithm which 
effectively averages over the delta k = 0.05 bin.  By using a boxcar 
average the statistics of the measurement are all included.  This is not 
really smoothing though.


Carlo

On Thu, 23 Jan 2020, Will Bennett wrote:


I entirely agree with Mike and Scott.

I never smooth data because it is a slippery slope to start on! How much should you smooth? Should 
you only smooth data if it looks "bad"? What does "bad data" look like anyway?

If the only point of smoothing is to improve the visual look of data, then you are doing 
it for the wrong reason - it is better to present the true data and let the reader decide 
if your interpretation is robust. In my view, smoothing data is somewhat scientifically 
dishonest unless you have a strong justification that extends beyond trying to make the 
data look "better"...

As someone who works with environmental samples at low concentrations, I'm very 
used to seeing noisy data. You just have to get used to long count times and 
lots of replicate scans!

Cheers,

Will


Dr William W. Bennett

Senior Lecturer
Environmental Futures Research Institute
School of Environment and Science
Griffith University
Gold Coast, Queensland, Australia
M +61 401 186 488
w.benn...@griffith.edu.au

Assistant Professor
Nordcee, Department of Biology
University of Southern Denmark
Odense, Denmark
M +45 8193 8111
wbenn...@biology.sdu.dk


From: Ifeffit  on behalf of Mike Massey 

Sent: Thursday, 23 January 2020 2:50 PM
To: Carlo Segre ; XAFS Analysis using Ifeffit 

Subject: Re: [Ifeffit] smoothing XAS data

At this juncture, I'd like to bring up a "rule" I've made for myself regarding 
data quality and smoothing: I'd rather walk away from a beam run with one good spectrum 
than a hundred poor ones.

Meaning, for me anyway, if the data aren't smooth enough on their own, I'd 
rather spend more precious time counting than move on. I'd count for three days 
on a sample if I had to (I've never had to).

While it has been proven that, contrary to the popular saying, one _can_ polish 
a turd, I'd personally rather not try.

I realize this approach was more difficult when I was a grad student and everyone 
in the research group was clamoring for data, but even in that situation, good 
data > not-good data.

Apologies for the tangent, but I hope someone out there might find it useful.


Cheers,



Mike






On Jan 22, 2020, at 5:44 PM, Carlo Segre  wrote:


Hi Daria:

I smooth (or boxcar average) the data which it is oversampled, such as in a 
continuous scan where the data in the exafs region is spaced closer than the 
usual delta k of 0.05.

I generally don't like to smooth XANES data since smoothing does tend to 
attenuate sharp features such as the ones which exist near the absorption edge.

Carlo


On Wed, 22 Jan 2020, Daria Boglaienko wrote:

Hello,

In what case smoothing of the data is OK? My data have a lot of noise and
smoothing really helps (visually). I compared fit done in Artemis for
smoothed and non-smoothed data sets and the result is very similar, however
when I searched about it online, it looks like it is not recommended.
What is a good way to justify it?

Thank you!

-Daria



--
Carlo U. Segre -- Duchossois Leadership Professor of Physics
Directory, Center for Synchrotron Radiation Research and Instrumentation
Illinois Institute of Technology
Voice: 312.567.3498 Fax: 312.567.3494
se...@iit.edu http://phys.iit.edu/~segre 
se...@debian.org
___
Ifeffit mailing list
Ifeffit@millenia.cars.aps.anl.gov
http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit
Unsubscribe: 
http://millenia.cars.aps.anl.gov/mailman/options/ifeffit


___
Ifeffit mailing list
Ifeffit@millenia.cars.aps.anl.gov
http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit
Unsubscribe: 
http://millenia.cars.aps.anl.gov/mailman/options/ifeffit



--
Carlo U. Segre -- Duchossois Leadership Professor of Physics
Directory, Center for Synchrotron Radiation Research and Instrumentation
Illinois Institute of Technology
Voice: 312.567.3498Fax: 312.567.3494
se...@iit.edu   http://phys.iit.edu/~segre   se...@debian.org___
Ifeffit mailing list
Ifeffit@millenia.cars.aps.anl.gov
http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit
Unsubscribe: http://millenia.cars.aps.anl.gov/mailman/options/ifeffit

Re: [Ifeffit] Ifeffit Digest, Vol 203, Issue 5

2020-01-22 Thread Christopher Chantler

Dear all
I agree with Mike and Scott too.
I might suggest Schalken, M. J. and Chantler, C. T. "Preservation of 
uncertainty in experiment: structures of Ni (II) coordination complexes" (2018) 
J. Synchrotron Rad., 25, 920-934 as a partial discussion around this topic if 
people are interested.

It could be read as 'don't smooth data if you can avoid it'.

Very best wishes to all
Chris

--
Christopher Chantler, Professor, FAIP, Fellow American Physical Society
Editor-in-Chief, Radiation Physics and Chemistry
Chair, International IUCr Commission on XAFS; CIT, CCN
IPP, International Radiation Physics Society
School of Physics, University of Melbourne
Parkville Victoria 3010 Australia
+61-3-83445437 FAX +61-3-93474783
chant...@unimelb.edu.au<https://owa.unimelb.edu.au/owa/redir.aspx?C=c7BoS0kVVkC1_S95-9x9l5cBu6YTjdAITgSrfUpfDAUV5oUH1LFYBcz08w8xvHMJoosZRdagfQM.&URL=mailto%3achantler%40unimelb.edu.au>
 
chant...@me.com<https://owa.unimelb.edu.au/owa/redir.aspx?C=c7BoS0kVVkC1_S95-9x9l5cBu6YTjdAITgSrfUpfDAUV5oUH1LFYBcz08w8xvHMJoosZRdagfQM.&URL=mailto%3achantler%40me.com>
http://optics.ph.unimelb.edu.au/~chantler/xrayopt/xrayopt.html<https://owa.unimelb.edu.au/owa/redir.aspx?C=c7BoS0kVVkC1_S95-9x9l5cBu6YTjdAITgSrfUpfDAUV5oUH1LFYBcz08w8xvHMJoosZRdagfQM.&URL=http%3a%2f%2foptics.ph.unimelb.edu.au%2f%7echantler%2fxrayopt%2fxrayopt.html>
http://optics.ph.unimelb.edu.au/~chantler/home.html<https://owa.unimelb.edu.au/owa/redir.aspx?C=c7BoS0kVVkC1_S95-9x9l5cBu6YTjdAITgSrfUpfDAUV5oUH1LFYBcz08w8xvHMJoosZRdagfQM.&URL=http%3a%2f%2foptics.ph.unimelb.edu.au%2f%7echantler%2fhome.html>


From: Ifeffit  on behalf of 
ifeffit-requ...@millenia.cars.aps.anl.gov 

Sent: Thursday, 23 January 2020 4:05 PM
To: ifeffit@millenia.cars.aps.anl.gov 
Subject: Ifeffit Digest, Vol 203, Issue 5

Send Ifeffit mailing list submissions to
ifeffit@millenia.cars.aps.anl.gov

To subscribe or unsubscribe via the World Wide Web, visit
http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit
or, via email, send a message with subject or body 'help' to
ifeffit-requ...@millenia.cars.aps.anl.gov

You can reach the person managing the list at
ifeffit-ow...@millenia.cars.aps.anl.gov

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Ifeffit digest..."


Today's Topics:

   1. smoothing XAS data (Daria Boglaienko)
   2. Re: smoothing XAS data (Carlo Segre)
   3. Re: smoothing XAS data (Scott Calvin)
   4. Re: smoothing XAS data (Mike Massey)
   5. Re: smoothing XAS data (Will Bennett)


--

Message: 1
Date: Wed, 22 Jan 2020 17:38:43 -0800
From: Daria Boglaienko 
To: ifeffit@millenia.cars.aps.anl.gov
Subject: [Ifeffit] smoothing XAS data
Message-ID:

Content-Type: text/plain; charset="utf-8"

Hello,

In what case smoothing of the data is OK? My data have a lot of noise and
smoothing really helps (visually). I compared fit done in Artemis for
smoothed and non-smoothed data sets and the result is very similar, however
when I searched about it online, it looks like it is not recommended.
What is a good way to justify it?

Thank you!

-Daria
-- next part --
An HTML attachment was scrubbed...
URL: 
<http://millenia.cars.aps.anl.gov/pipermail/ifeffit/attachments/20200122/f5449543/attachment-0001.html>

--

Message: 2
Date: Wed, 22 Jan 2020 19:44:22 -0600 (CST)
From: Carlo Segre 
To: XAFS Analysis using Ifeffit 
Subject: Re: [Ifeffit] smoothing XAS data
Message-ID:

Content-Type: text/plain; charset=US-ASCII; format=flowed


Hi Daria:

I smooth (or boxcar average) the data which it is oversampled, such as in
a continuous scan where the data in the exafs region is spaced closer than
the usual delta k of 0.05.

I generally don't like to smooth XANES data since smoothing does tend to
attenuate sharp features such as the ones which exist near the absorption
edge.

Carlo

On Wed, 22 Jan 2020, Daria Boglaienko wrote:

> Hello,
>
> In what case smoothing of the data is OK? My data have a lot of noise and
> smoothing really helps (visually). I compared fit done in Artemis for
> smoothed and non-smoothed data sets and the result is very similar, however
> when I searched about it online, it looks like it is not recommended.
> What is a good way to justify it?
>
> Thank you!
>
> -Daria
>

--
Carlo U. Segre -- Duchossois Leadership Professor of Physics
Directory, Center for Synchrotron Radiation Research and Instrumentation
Illinois Institute of Technology
Voice: 312.567.3498Fax: 312.567.3494
se...@iit.edu   http://phys.iit.edu/~segre   se...@debian.org


--

Message: 3
Date: W

Re: [Ifeffit] smoothing XAS data

2020-01-22 Thread Will Bennett

I entirely agree with Mike and Scott.

I never smooth data because it is a slippery slope to start on! How much should 
you smooth? Should you only smooth data if it looks "bad"? What does "bad data" 
look like anyway?

If the only point of smoothing is to improve the visual look of data, then you 
are doing it for the wrong reason - it is better to present the true data and 
let the reader decide if your interpretation is robust. In my view, smoothing 
data is somewhat scientifically dishonest unless you have a strong 
justification that extends beyond trying to make the data look "better"...

As someone who works with environmental samples at low concentrations, I'm very 
used to seeing noisy data. You just have to get used to long count times and 
lots of replicate scans!

Cheers,

Will

Dr William W. Bennett

Senior Lecturer
Environmental Futures Research Institute
School of Environment and Science
Griffith University
Gold Coast, Queensland, Australia
M +61 401 186 488
w.benn...@griffith.edu.au

Assistant Professor
Nordcee, Department of Biology
University of Southern Denmark
Odense, Denmark
M +45 8193 8111
wbenn...@biology.sdu.dk

From: Ifeffit  on behalf of Mike 
Massey 
Sent: Thursday, 23 January 2020 2:50 PM
To: Carlo Segre ; XAFS Analysis using Ifeffit 

Subject: Re: [Ifeffit] smoothing XAS data

At this juncture, I'd like to bring up a "rule" I've made for myself regarding 
data quality and smoothing: I'd rather walk away from a beam run with one good 
spectrum than a hundred poor ones.

Meaning, for me anyway, if the data aren't smooth enough on their own, I'd 
rather spend more precious time counting than move on. I'd count for three days 
on a sample if I had to (I've never had to).

While it has been proven that, contrary to the popular saying, one _can_ polish 
a turd, I'd personally rather not try.

I realize this approach was more difficult when I was a grad student and 
everyone in the research group was clamoring for data, but even in that 
situation, good data > not-good data.

Apologies for the tangent, but I hope someone out there might find it useful.

Cheers,

Mike

> On Jan 22, 2020, at 5:44 PM, Carlo Segre  wrote:
>
> 
> Hi Daria:
>
> I smooth (or boxcar average) the data which it is oversampled, such as in a 
> continuous scan where the data in the exafs region is spaced closer than the 
> usual delta k of 0.05.
>
> I generally don't like to smooth XANES data since smoothing does tend to 
> attenuate sharp features such as the ones which exist near the absorption 
> edge.
>
> Carlo
>
>> On Wed, 22 Jan 2020, Daria Boglaienko wrote:
>>
>> Hello,
>>
>> In what case smoothing of the data is OK? My data have a lot of noise and
>> smoothing really helps (visually). I compared fit done in Artemis for
>> smoothed and non-smoothed data sets and the result is very similar, however
>> when I searched about it online, it looks like it is not recommended.
>> What is a good way to justify it?
>>
>> Thank you!
>>
>> -Daria
>>
>
> --
> Carlo U. Segre -- Duchossois Leadership Professor of Physics
> Directory, Center for Synchrotron Radiation Research and Instrumentation
> Illinois Institute of Technology
> Voice: 312.567.3498 Fax: 312.567.3494
> se...@iit.edu http://phys.iit.edu/~segre 
> se...@debian.org
> ___
> Ifeffit mailing list
> Ifeffit@millenia.cars.aps.anl.gov
> http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit
> Unsubscribe: 
> http://millenia.cars.aps.anl.gov/mailman/options/ifeffit

___
Ifeffit mailing list
Ifeffit@millenia.cars.aps.anl.gov
http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit
Unsubscribe: 
http://millenia.cars.aps.anl.gov/mailman/options/ifeffit
___
Ifeffit mailing list
Ifeffit@millenia.cars.aps.anl.gov
http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit
Unsubscribe: http://millenia.cars.aps.anl.gov/mailman/options/ifeffit

Re: [Ifeffit] smoothing XAS data

2020-01-22 Thread Mike Massey

At this juncture, I'd like to bring up a "rule" I've made for myself regarding 
data quality and smoothing: I'd rather walk away from a beam run with one good 
spectrum than a hundred poor ones.

Meaning, for me anyway, if the data aren't smooth enough on their own, I'd 
rather spend more precious time counting than move on. I'd count for three days 
on a sample if I had to (I've never had to).

While it has been proven that, contrary to the popular saying, one _can_ polish 
a turd, I'd personally rather not try.

I realize this approach was more difficult when I was a grad student and 
everyone in the research group was clamoring for data, but even in that 
situation, good data > not-good data.

Apologies for the tangent, but I hope someone out there might find it useful.

Cheers,

Mike

> On Jan 22, 2020, at 5:44 PM, Carlo Segre  wrote:
> 
> 
> Hi Daria:
> 
> I smooth (or boxcar average) the data which it is oversampled, such as in a 
> continuous scan where the data in the exafs region is spaced closer than the 
> usual delta k of 0.05.
> 
> I generally don't like to smooth XANES data since smoothing does tend to 
> attenuate sharp features such as the ones which exist near the absorption 
> edge.
> 
> Carlo
> 
>> On Wed, 22 Jan 2020, Daria Boglaienko wrote:
>> 
>> Hello,
>> 
>> In what case smoothing of the data is OK? My data have a lot of noise and
>> smoothing really helps (visually). I compared fit done in Artemis for
>> smoothed and non-smoothed data sets and the result is very similar, however
>> when I searched about it online, it looks like it is not recommended.
>> What is a good way to justify it?
>> 
>> Thank you!
>> 
>> -Daria
>> 
> 
> -- 
> Carlo U. Segre -- Duchossois Leadership Professor of Physics
> Directory, Center for Synchrotron Radiation Research and Instrumentation
> Illinois Institute of Technology
> Voice: 312.567.3498Fax: 312.567.3494
> se...@iit.edu   http://phys.iit.edu/~segre   se...@debian.org
> ___
> Ifeffit mailing list
> Ifeffit@millenia.cars.aps.anl.gov
> http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit
> Unsubscribe: http://millenia.cars.aps.anl.gov/mailman/options/ifeffit

___
Ifeffit mailing list
Ifeffit@millenia.cars.aps.anl.gov
http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit
Unsubscribe: http://millenia.cars.aps.anl.gov/mailman/options/ifeffit

Re: [Ifeffit] smoothing XAS data

2020-01-22 Thread Scott Calvin

Not only can smoothing XANES data attenuate sharp features, it can also shift 
them, since many XANES features, particularly in the “pre-edge” region, are 
significantly asymmetric about their peak. That can make analysis quite 
confusing, since, e.g., the size of the boxcar ends up affecting the position 
of the peak!

Best,
Scott Calvin
Lehman College of the City University of New York

> On Jan 22, 2020, at 8:44 PM, Carlo Segre  wrote:
> 
> I generally don't like to smooth XANES data since smoothing does tend to 
> attenuate sharp features such as the ones which exist near the absorption 
> edge.

___
Ifeffit mailing list
Ifeffit@millenia.cars.aps.anl.gov
http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit
Unsubscribe: http://millenia.cars.aps.anl.gov/mailman/options/ifeffit

Re: [Ifeffit] smoothing XAS data

2020-01-22 Thread Carlo Segre




Hi Daria:

I smooth (or boxcar average) the data which it is oversampled, such as in 
a continuous scan where the data in the exafs region is spaced closer than 
the usual delta k of 0.05.


I generally don't like to smooth XANES data since smoothing does tend to 
attenuate sharp features such as the ones which exist near the absorption 
edge.


Carlo

On Wed, 22 Jan 2020, Daria Boglaienko wrote:


Hello,

In what case smoothing of the data is OK? My data have a lot of noise and
smoothing really helps (visually). I compared fit done in Artemis for
smoothed and non-smoothed data sets and the result is very similar, however
when I searched about it online, it looks like it is not recommended.
What is a good way to justify it?

Thank you!

-Daria



--
Carlo U. Segre -- Duchossois Leadership Professor of Physics
Directory, Center for Synchrotron Radiation Research and Instrumentation
Illinois Institute of Technology
Voice: 312.567.3498Fax: 312.567.3494
se...@iit.edu   http://phys.iit.edu/~segre   se...@debian.org
___
Ifeffit mailing list
Ifeffit@millenia.cars.aps.anl.gov
http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit
Unsubscribe: http://millenia.cars.aps.anl.gov/mailman/options/ifeffit

[Ifeffit] smoothing XAS data

2020-01-22 Thread Daria Boglaienko

Hello,

In what case smoothing of the data is OK? My data have a lot of noise and
smoothing really helps (visually). I compared fit done in Artemis for
smoothed and non-smoothed data sets and the result is very similar, however
when I searched about it online, it looks like it is not recommended.
What is a good way to justify it?

Thank you!

-Daria
___
Ifeffit mailing list
Ifeffit@millenia.cars.aps.anl.gov
http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit
Unsubscribe: http://millenia.cars.aps.anl.gov/mailman/options/ifeffit

Re: [Ifeffit] smoothing XAS data

Re: [Ifeffit] Ifeffit Digest, Vol 203, Issue 5

Re: [Ifeffit] smoothing XAS data

Re: [Ifeffit] smoothing XAS data

Re: [Ifeffit] smoothing XAS data

Re: [Ifeffit] smoothing XAS data

[Ifeffit] smoothing XAS data

7 matches

Site Navigation

Mail list logo

Footer information