Re: [Therion] Declination handling imprecise?

[Olly Betts via Therion]

On Sat, Feb 18, 2017 at 09:24:42PM +, Olly Betts via Therion wrote:
> On Sat, Feb 18, 2017 at 10:20:54AM +0200, Benedikt Hallinger via Therion 
> wrote:
> > For the date-observation, indeed my conclusion came from the summary in the
> > log. Thank you for claryfying this. Good to hear, therion uses fractions
> > here, and 1/12 is perfectly good.
> 
> To be clear, therion calculates the fractional year taking into account the
> day of the month - it just doesn't assign quite equal lengths to every day of
> the year, but instead puts the start of each month 1/12 of a year apart, and
> then splits up each 1/12 into 28, 29, 30 or 31 equal pieces.  Really it
> should split the year into 365 or 366 equal pieces.

FYI, I submitted a patch to address this which Stacho has merged, so the
next release should calculate this using even sized days (leap years aside):

https://github.com/therion/therion/pull/52

> > However i have seen that it looks like the calculated declination is maybe
> > rounded to full degrees? Or is this just an display thing in aven viewer?
> 
> Aven currently only shows a whole number of degrees, which is unhelpful for
> this sort of thing.  It probably ought to show one decimal place.

The next release of Survex will show one decimal place.

Cheers,
Olly
___
Therion mailing list
Therion@speleo.sk
https://mailman.speleo.sk/listinfo/therion

Re: [Therion] Declination handling imprecise?

[Olly Betts via Therion]

On Sun, Feb 19, 2017 at 09:00:52PM +, Olly Betts via Therion wrote:
> On Sun, Feb 19, 2017 at 01:50:45PM +0200, Benedikt Hallinger via Therion 
> wrote:
> > Is the centerline shown in aven correct and only the displayed angles
> > rounded (i.e. the calculated coordinates are valid)?
> > How about loch?
> 
> Not sure about loch, but (as I mentioned in the message you replied to)
> released versions of aven indeed show the bearing of the measuring line
> as an integer (probably actually truncated to the integer <= the value
> rather than rounded).

Someone smarter than me just pointed out that you're actually asking if the
angles in the displayed centreline are also rounded.

The only rounding of the centreline is that coordinates in the 3d file are
stored in centimetres, which should have a negligible effect on the ~1000m
long legs in your testcase.

Cheers,
Olly
___
Therion mailing list
Therion@speleo.sk
https://mailman.speleo.sk/listinfo/therion

Re: [Therion] Declination handling imprecise?

[Olly Betts via Therion]

On Sun, Feb 19, 2017 at 01:50:45PM +0200, Benedikt Hallinger via Therion wrote:
> Thank you everybode for those insights!
> It is good to hear therion is this accurate, however i need to learn what i
> understand wrong in the test data.
> 
> Is the centerline shown in aven correct and only the displayed angles
> rounded (i.e. the calculated coordinates are valid)?
> How about loch?

Not sure about loch, but (as I mentioned in the message you replied to)
released versions of aven indeed show the bearing of the measuring line
as an integer (probably actually truncated to the integer <= the value
rather than rounded).

I've already pushed a change to show one decimal place, as this thread
reminded me that I'd been meaning to sort this out, and it was a simple
issue to address.

Cheers,
Olly
___
Therion mailing list
Therion@speleo.sk
https://mailman.speleo.sk/listinfo/therion

Re: [Therion] Declination handling imprecise?

[Benedikt Hallinger via Therion]

Thank you everybode for those insights!
It is good to hear therion is this accurate, however i need to learn what i 
understand wrong in the test data.


Is the centerline shown in aven correct and only the displayed angles 
rounded (i.e. the calculated coordinates are valid)?

How about loch?



Am 2017-02-18 23:24, schrieb Olly Betts:
On Sat, Feb 18, 2017 at 10:20:54AM +0200, Benedikt Hallinger via Therion 
wrote:
If therion calculates the average position of all fixed points this is 
fine.

the cave spans about 5km w/e and about 3km n/s.


Ah, the 100km is total passage length not extent then.

For a few km the variation is probably fairly small for most of the world
compared to the errors from the instrument readings, though the instrument
errors are random and the declination discrepenacy is systematic
- systematic errors are worse because they don't lessen when you combine a
lot of readings (for random errors the error of the sum increases as the
square root of the number of readings - e.g. for 100 readings, the random
error only increases 10 times).

The grid convergence may well be larger (IIRC therion calculates that at
the same average fixed point location; I know that Survex calculates it at
the same coordinates you give in "*declination auto").  To quantify these
errors, for the coordinates in your test file, 5km E means about 0.015
degrees change in declination and about 0.049 degrees change in grid
convergence (in opposite directions).

For the date-observation, indeed my conclusion came from the summary in 
the

log. Thank you for claryfying this. Good to hear, therion uses fractions
here, and 1/12 is perfectly good.


To be clear, therion calculates the fractional year taking into account 
the day
of the month - it just doesn't assign quite equal lengths to every day of 
the
year, but instead puts the start of each month 1/12 of a year apart, and 
then
splits up each 1/12 into 28, 29, 30 or 31 equal pieces.  Really it should 
split

the year into 365 or 366 equal pieces.

However i have seen that it looks like the calculated declination is 
maybe

rounded to full degrees? Or is this just an display thing in aven viewer?


Aven currently only shows a whole number of degrees, which is unhelpful 
for

this sort of thing.  It probably ought to show one decimal place.

Based on what you have said, i would assume that the problem source lies 
in

therion itself summarizing all centreline dates into one survey average
date, and not just by centerline as i would assume, when each centerline 
has

a date specification.


I think you need Stacho or Martin to comment on whether that's happening.

Cheers,
Olly


___
Therion mailing list
Therion@speleo.sk
https://mailman.speleo.sk/listinfo/therion

Re: [Therion] Declination handling imprecise?

[Olly Betts via Therion]

On Sat, Feb 18, 2017 at 10:20:54AM +0200, Benedikt Hallinger via Therion wrote:
> If therion calculates the average position of all fixed points this is fine.
> the cave spans about 5km w/e and about 3km n/s.

Ah, the 100km is total passage length not extent then.

For a few km the variation is probably fairly small for most of the world
compared to the errors from the instrument readings, though the instrument
errors are random and the declination discrepenacy is systematic
- systematic errors are worse because they don't lessen when you combine a
lot of readings (for random errors the error of the sum increases as the
square root of the number of readings - e.g. for 100 readings, the random
error only increases 10 times).

The grid convergence may well be larger (IIRC therion calculates that at
the same average fixed point location; I know that Survex calculates it at
the same coordinates you give in "*declination auto").  To quantify these
errors, for the coordinates in your test file, 5km E means about 0.015
degrees change in declination and about 0.049 degrees change in grid
convergence (in opposite directions).

> For the date-observation, indeed my conclusion came from the summary in the
> log. Thank you for claryfying this. Good to hear, therion uses fractions
> here, and 1/12 is perfectly good.

To be clear, therion calculates the fractional year taking into account the day
of the month - it just doesn't assign quite equal lengths to every day of the
year, but instead puts the start of each month 1/12 of a year apart, and then
splits up each 1/12 into 28, 29, 30 or 31 equal pieces.  Really it should split
the year into 365 or 366 equal pieces.

> However i have seen that it looks like the calculated declination is maybe
> rounded to full degrees? Or is this just an display thing in aven viewer?

Aven currently only shows a whole number of degrees, which is unhelpful for
this sort of thing.  It probably ought to show one decimal place.

> Based on what you have said, i would assume that the problem source lies in
> therion itself summarizing all centreline dates into one survey average
> date, and not just by centerline as i would assume, when each centerline has
> a date specification.

I think you need Stacho or Martin to comment on whether that's happening.

Cheers,
Olly
___
Therion mailing list
Therion@speleo.sk
https://mailman.speleo.sk/listinfo/therion

Re: [Therion] Declination handling imprecise?

[Martin Budaj via Therion]

On Sat, Feb 18, 2017 at 9:20 AM, Benedikt Hallinger via Therion
 wrote:
> If therion calculates the average position of all fixed points this is fine.
> the cave spans about 5km w/e and about 3km n/s.

The change in declination over 5 km distance in your area is about 1'
which is completely negligible for compass surveys.

> I strongly dont think this is the error source as it is also visible in the
> test file.

The problem with your test file is that the declination on 2013-12-31
should be 3.08 degrees, not 3.8. If you use 3.08 (or more precisely
3.08), than the error in the test file is less than 20 cm on 2 km
legs. You should also take into account that the declination value at
GFZ Potsdam is rounded to the nearest minute.

> Also, calculating the igrf for each station should not be
> neccessary unless there are some strong amd small local anomalys which is
> not the case here.

IGRF model does not take local anomalies into account anyway.

> However i have seen that it looks like the calculated declination is maybe
> rounded to full degrees? Or is this just an display thing in aven viewer?

No rounding in therion here.

> Based on what you have said, i would assume that the problem source lies in
> therion itself summarizing all centreline dates into one survey average
> date, and not just by centerline as i would assume, when each centerline has
> a date specification.

The automatic declination is bound to centreline. You can check it
also in your example if you change the date of the first centerline to
e.g. 1950.

Maybe there is some other source of error in the data?

Martin

> Am 2017-02-17 23:26, schrieb Olly Betts:
>>
>> On Fri, Feb 17, 2017 at 07:08:48PM +0200, Benedikt Hallinger via Therion
>> wrote:
>>>
>>> i found out that the declination handling seems to be somewhat
>>> inaccurate.
>>> The produced error is marginal and negligible with small caves, however i
>>> am
>>> currently working on a system with >100km and this produces errors of
>>> >10m
>>> there.
>>
>>
>> Therion calculates all declination values at the same location (the
>> average
>> location of all the fixed points), which is potentially problematic if
>> your
>> survey spans that sort of distance.  I wonder if this might actually be
>> the
>> source of your problems.
>>
>> Survex requires specifying the location to calculate declination values
>> at,
>> and you can use different locations for different parts of the survey
>> hierarchy.  So keeping your centreline data in Survex format and feeding
>> therion a 3d file would be one solution if this is the issue you're
>> hitting.
>>
>> (Using the actual location of the station the reading was taken at is hard
>> as you need the declination value to calculate that location.  It would
>> also be
>> fairly slow to evaluate the IGRF model at every station where a compass
>> was
>> read.)
>>
>>> What i observe is the following:
>>> - Therion seems to use the first day of any given year referenced by
>>> "date"
>>> command.
>>
>>
>> If that's from looking at therion's log file, the "geomag declinations
>> (deg)"
>> table there is just a summary.  It indeed shows the values on January 1st
>> of
>> each "interesting" year, but doesn't reflect the dates actually being used
>> to
>> calculate declinations for surveys.
>>
>> Or was that from looking at the source code?  The get_start_year() and
>> get_end_year() methods actually return a fractional year value, not just
>> the
>> year of the survey.  The fraction is calculated assuming each month is
>> 1/12 of
>> the year - not quite right, but probably not a significant error in
>> practice.
>>
>>> - If a survey was taken at the end of a year it seems that the 1.1 of the
>>> following year will be used (i assume this happens at mid-year).
>>> - This approach looses at most half a years magnetic drift.
>>> This alone is acceptable as the data by itself is already not that
>>> accurate.
>>>
>>> However if you have several centerlines in the survey, each with
>>> different
>>> date-commands, therion seems to calculate the average of all dates and
>>> then
>>> uses the calculated declination for that date (applying the rule above,
>>> so
>>> taking the "closest 1.1.").
>>> This is then applied to all shots of the whole survey.
>>
>>
>> Not sure - I don't know therion's code that well.
>>
>>> A possible solution to this seems, to get the correct declination for
>>> each
>>> date and explicitely write suitable "declination" commands that override
>>> the
>>> "date"-calculated values. This then gives the proper results.
>>
>>
>> This approach is less than ideal, and not just because it's a significant
>> effort.
>>
>> Each generation of the IGRF model is necessarily predictive for dates
>> after it
>> was issued and only declared to be definitive for dates more than 5 years
>> before it was issued.  So each new generation of the model can change the
>> declination figures reported for the last 10 years compared to the

Re: [Therion] Declination handling imprecise?

[Martin Sluka via Therion]

> 18. 2. 2017 v 9:20, Benedikt Hallinger via Therion :
> 
> Based on what you have said, i would assume that the problem source lies in 
> therion itself summarizing all centreline dates into one survey average date, 
> and not just by centerline as i would assume, when each centerline has a date 
> specification.
> 
> The main issue here is, that we have surveys which were taken in the 1950s 
> and extended with an additional centerline in 2000s.

date and date - each survey has calculated its own declination according the 
date it was surveyed. 

The position of point for which the declination is calculated is averaged.

m.s.
___
Therion mailing list
Therion@speleo.sk
https://mailman.speleo.sk/listinfo/therion

Re: [Therion] Declination handling imprecise?

[Benedikt Hallinger via Therion]

If therion calculates the average position of all fixed points this is fine. 
the cave spans about 5km w/e and about 3km n/s.
I strongly dont think this is the error source as it is also visible in the 
test file. Also, calculating the igrf for each station should not be 
neccessary unless there are some strong amd small local anomalys which is not 
the case here.


For the date-observation, indeed my conclusion came from the summary in the 
log. Thank you for claryfying this. Good to hear, therion uses fractions 
here, and 1/12 is perfectly good.


However i have seen that it looks like the calculated declination is maybe 
rounded to full degrees? Or is this just an display thing in aven viewer?



Based on what you have said, i would assume that the problem source lies in 
therion itself summarizing all centreline dates into one survey average date, 
and not just by centerline as i would assume, when each centerline has a date 
specification.


The main issue here is, that we have surveys which were taken in the 1950s 
and extended with an additional centerline in 2000s.



Am 2017-02-17 23:26, schrieb Olly Betts:
On Fri, Feb 17, 2017 at 07:08:48PM +0200, Benedikt Hallinger via Therion 
wrote:
i found out that the declination handling seems to be somewhat 
inaccurate.
The produced error is marginal and negligible with small caves, however i 
am
currently working on a system with >100km and this produces errors of 
>10m

there.


Therion calculates all declination values at the same location (the 
average
location of all the fixed points), which is potentially problematic if 
your
survey spans that sort of distance.  I wonder if this might actually be 
the

source of your problems.

Survex requires specifying the location to calculate declination values 
at,

and you can use different locations for different parts of the survey
hierarchy.  So keeping your centreline data in Survex format and feeding
therion a 3d file would be one solution if this is the issue you're 
hitting.


(Using the actual location of the station the reading was taken at is hard
as you need the declination value to calculate that location.  It would 
also be
fairly slow to evaluate the IGRF model at every station where a compass 
was

read.)


What i observe is the following:
- Therion seems to use the first day of any given year referenced by 
"date"

command.


If that's from looking at therion's log file, the "geomag declinations 
(deg)"
table there is just a summary.  It indeed shows the values on January 1st 
of
each "interesting" year, but doesn't reflect the dates actually being used 
to

calculate declinations for surveys.

Or was that from looking at the source code?  The get_start_year() and
get_end_year() methods actually return a fractional year value, not just 
the
year of the survey.  The fraction is calculated assuming each month is 
1/12 of
the year - not quite right, but probably not a significant error in 
practice.



- If a survey was taken at the end of a year it seems that the 1.1 of the
following year will be used (i assume this happens at mid-year).
- This approach looses at most half a years magnetic drift.
This alone is acceptable as the data by itself is already not that 
accurate.


However if you have several centerlines in the survey, each with 
different
date-commands, therion seems to calculate the average of all dates and 
then
uses the calculated declination for that date (applying the rule above, 
so

taking the "closest 1.1.").
This is then applied to all shots of the whole survey.


Not sure - I don't know therion's code that well.

A possible solution to this seems, to get the correct declination for 
each
date and explicitely write suitable "declination" commands that override 
the

"date"-calculated values. This then gives the proper results.


This approach is less than ideal, and not just because it's a significant
effort.

Each generation of the IGRF model is necessarily predictive for dates 
after it

was issued and only declared to be definitive for dates more than 5 years
before it was issued.  So each new generation of the model can change the
declination figures reported for the last 10 years compared to the 
previous

generation.

In simple terms, when IGRF-13 comes out in late 2019 it'll potentially 
give
different answers for all dates in 2010 and later (not hugely different 
one

would hope, but your motivation here is to avoid introducing unnecessary
errors).

See table 1 here for the date information for each IGRF model generation:


http://earth-planets-space.springeropen.com/articles/10.1186/s40623-015-0228-9

So unless you're dealing with historic data, you really want your software
to calculate the declination automatically so that you get the benefit of
improvements when the IGRF model is updated.

Cheers,
Olly


___
Therion mailing list
Therion@speleo.sk
https://mailman.speleo.sk/listinfo/therion

Re: [Therion] Declination handling imprecise?

[Olly Betts via Therion]

On Fri, Feb 17, 2017 at 07:08:48PM +0200, Benedikt Hallinger via Therion wrote:
> i found out that the declination handling seems to be somewhat inaccurate.
> The produced error is marginal and negligible with small caves, however i am
> currently working on a system with >100km and this produces errors of >10m
> there.

Therion calculates all declination values at the same location (the average
location of all the fixed points), which is potentially problematic if your
survey spans that sort of distance.  I wonder if this might actually be the
source of your problems.

Survex requires specifying the location to calculate declination values at,
and you can use different locations for different parts of the survey
hierarchy.  So keeping your centreline data in Survex format and feeding
therion a 3d file would be one solution if this is the issue you're hitting.

(Using the actual location of the station the reading was taken at is hard
as you need the declination value to calculate that location.  It would also be
fairly slow to evaluate the IGRF model at every station where a compass was
read.)

> What i observe is the following:
> - Therion seems to use the first day of any given year referenced by "date"
> command.

If that's from looking at therion's log file, the "geomag declinations (deg)"
table there is just a summary.  It indeed shows the values on January 1st of
each "interesting" year, but doesn't reflect the dates actually being used to
calculate declinations for surveys.

Or was that from looking at the source code?  The get_start_year() and
get_end_year() methods actually return a fractional year value, not just the
year of the survey.  The fraction is calculated assuming each month is 1/12 of
the year - not quite right, but probably not a significant error in practice.

> - If a survey was taken at the end of a year it seems that the 1.1 of the
> following year will be used (i assume this happens at mid-year).
> - This approach looses at most half a years magnetic drift.
> This alone is acceptable as the data by itself is already not that accurate.
> 
> However if you have several centerlines in the survey, each with different
> date-commands, therion seems to calculate the average of all dates and then
> uses the calculated declination for that date (applying the rule above, so
> taking the "closest 1.1.").
> This is then applied to all shots of the whole survey.

Not sure - I don't know therion's code that well.

> A possible solution to this seems, to get the correct declination for each
> date and explicitely write suitable "declination" commands that override the
> "date"-calculated values. This then gives the proper results.

This approach is less than ideal, and not just because it's a significant
effort.

Each generation of the IGRF model is necessarily predictive for dates after it
was issued and only declared to be definitive for dates more than 5 years
before it was issued.  So each new generation of the model can change the
declination figures reported for the last 10 years compared to the previous
generation.

In simple terms, when IGRF-13 comes out in late 2019 it'll potentially give
different answers for all dates in 2010 and later (not hugely different one
would hope, but your motivation here is to avoid introducing unnecessary
errors).

See table 1 here for the date information for each IGRF model generation:

http://earth-planets-space.springeropen.com/articles/10.1186/s40623-015-0228-9

So unless you're dealing with historic data, you really want your software
to calculate the declination automatically so that you get the benefit of
improvements when the IGRF model is updated.

Cheers,
Olly
___
Therion mailing list
Therion@speleo.sk
https://mailman.speleo.sk/listinfo/therion