Re: [Talk-us] Low-quality NHD imports

[Andy Townsend]

On 14/10/2017 03:56, Clifford Snow wrote:


Unnamed streams are helpful to people hiking in the forest areas by 
giving a landmark for navigation.


I'm in the UK so I'm not familiar with NHD, but here I am familiar with 
streams here traced from NPE (old maps, very inaccurate by modern 
standards).  NPE-traced streams can be useful in that they can tell you 
that there's a stream here somewhere, but you absolutely can't use them 
to tell which side of the water you're supposed to be on (or reliably 
count crossing streams before a path junction).


However it is possible to use the tags on them to make it clear to 
someone looking at the data where it came from.  For Garmin maps I 
append "(NPE)" to the name of NPE-traced streams and you could certainly 
do something similar with NHD ones.


OSM has always proceeded by evolution - stuff I added 9 years ago is 
considerably less accurate than stuff I add now because there were fewer 
sources available to compare - and EGNOS wasn't available then (at least 
not to me).  I'm sure that what I add now will appear "similarly 
inaccurate" in 9 years time!


Best Regards,

Andy


___
Talk-us mailing list
Talk-us@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] Low-quality NHD imports

[Ben Discoe]

I've probably done the most NHD cleanup so far (at least some degree
of fixing on the entire state of NC, most of IL, northern MI, parts of
OK/TX/UT/CO, and a lot of CA), many hundreds of hours of manual work.

Just to chime in with agreement on what everyone has said, yes to:
1. NHD has lots of issues
2. a lot of it could have been imported better
3. there are some NHD tags (like 'nhd:com_id') which are of little use
and just get in the way
4. a lot is badly out of date
5. a lot is badly overnoded (like a perfectly straight ditch using 200
noisy nodes)
6. some is badly tagged (like waterway=canal for ditches in the western USA)
7. it's still very valuable and in some places, of surprising quality
and completeness

I never just remove useless tags for the sake of removing.  They only
get touched as part of manual cleanup, which goes through a lot of
steps.

I really should write a Diary post on the steps I go through to clean
up NHD (part of these steps I already covered in
http://www.openstreetmap.org/user/bdiscoe/diary/37421)

1. Select all the waterways ("type:way AND (natural=water OR
natural=wetland OR waterway OR (child natural=water) OR (child
natural=wetland) OR (child waterway)) AND allindownloadedarea ")
2. Use JOSM validator fix-it, which will solve topology problems like
dupe vertices.
3. If the region is badly overnoded, simplify to an appropriate value
(like 80cm), then follow up with a manual cleanup/alignment.
4. Check the "waterway ends without a connection" warnings; some can
be manually fixed
5. For at least major crossings, add the bridges and culverts (I just
JOSM scripts I wrote to make this faster, but it's still very manual,
one at a time)
6. For major features that are out of date (like streams or wetlands
that were destroyed and are now shopping malls), delete or re-align
them.
7. Look over the data and fix anything else that looks very wrong

So, to answer Frederik's original question, "Is there any systematic
(or even sporadic) effort of cleaning up these old imports?"

Answer: Yes.  In addition to all the people doing great work in their
local areas, there is me.

___
Talk-us mailing list
Talk-us@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] Low-quality NHD imports

[Michael Patrick]

> Maybe we can get someone to host NHD data for others to trace in. If
anyone
is reading this and is willing to host the data, I'm willing to help with
the design and conversions.

The USGS provides several WMS endpoints for the NHD
 , most likely these will connect with
JOSM ( and qGIS ) for tracing or import.

Depending when imports were made will affect the quality - NHD is a moving
target, and as high resolution aerial Lidar and Satellite SAR elevation
data is made available, the resolution improves dramatically. Also, there
are different datasets available for different scales. See
https://nhd.usgs.gov/NHDPlus_HR.html

> (Winters can be miserable.)

I miss Chinooks! https://en.wikipedia.org/wiki/Chinook_wind

Michael Patrick
Data Ferret
___
Talk-us mailing list
Talk-us@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] Low-quality NHD imports

[Mike Thompson]

On Fri, Oct 13, 2017 at 8:56 PM, Clifford Snow 
wrote:

>
>
>>
> Unnamed streams are helpful to people hiking in the forest areas by giving
> a landmark for navigation. From areas I'm familiar with, there are
> thousands of unnamed streams. They are unnamed because civilization just
> hasn't reached it. For example, we have Logan Creek nearby. If it was in a
> national forest it would most likely be unnamed.
>
Agree, and they may provide a source of drinking water (with treatment)
when backpacking or on extended hikes in the back country. Please do not
remove!

>
>
___
Talk-us mailing list
Talk-us@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] Low-quality NHD imports

[Clifford Snow]

On Mon, Oct 16, 2017 at 3:56 PM, Wolfgang Zenker 
wrote:

>
> the problem here is that at least those NHD imports I have seen in
> Montana have only some of the existing streams. I don't know if this is
> because NHD does not have more or because the import used not all
> available streams. "Existing streams" here means both streams seen on
> USGS Topo maps and stream beds seen on imagery.
>
>
Wolfgang,
Not sure how the NHD data for Montana got into OSM. In my own county in
Washington, it's far from complete, at least compared to NHD data. I've
been slowly adding them by tracing in streams from current, and much better
resolution NHD data and aerial imagery. My county is relatively small,
especially to Montana. It usually takes me two or three days to drive from
west to east. I can be out of my county in 3 hours. Of course in Montana,
it's mostly the foothills and mountain ranges that have the bulk of the
steams to edit.

Maybe we can get someone to host NHD data for others to trace in. If anyone
is reading this and is willing to host the data, I'm willing to help with
the design and conversions.

BTW - I lived in Montana for a number of years and will be back there after
the State of the Map US conference to do some hiking and sightseeing. I
remember fishing in the Bitterroot. Nothing like fresh caught trout. It's
one of the nicest states to visit. (Winters can be miserable.)

Clifford
-- 
@osm_seattle
osm_seattle.snowandsnow.us
OpenStreetMap: Maps with a human touch
___
Talk-us mailing list
Talk-us@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] Low-quality NHD imports

[Wolfgang Zenker]

Hi,

* Charlotte Wolter  [171016 23:50]:
>  Clifford makes some very good points. In the West, particularly,
> those little intermittent streams are important landmarks. Particularly
> when hiking in a featureless area, such as pinyon-juniper forest, a
> trail direction may say something like, "turn right after crossing the third
> drainage."
> [..]

the problem here is that at least those NHD imports I have seen in
Montana have only some of the existing streams. I don't know if this is
because NHD does not have more or because the import used not all
available streams. "Existing streams" here means both streams seen on
USGS Topo maps and stream beds seen on imagery.

Wolfgang

___
Talk-us mailing list
Talk-us@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] Low-quality NHD imports

[OSM Volunteer stevea]

Echoing (+1-ing) what Charlotte (and Clifford) said:  waterway=creek on a 
downward-flowing way (and an intermittent=yes tag, if true) are extensively 
used and helpful, certainly where I am.  While hiking, they are relatively 
unambiguous wayfinding attributes, especially when/as they are 
numbered/sequenced in an order and/or you are keeping count of them between 
here and there.  This is a simple method to aid navigation:   name= tags of 
"Creek 1" through "Creek 9" (up to n, depending) along a directed path 
(intersection of path and creek) does not seem a terrible faux pas to me.  Of 
course, if the creeks are truly named, use those values in name= tags instead.

I repeat an important aspect of "stream tagging:"  please assure the way's 
direction "points downstream."  It truly helps to get this correct!  Also, 
where a confluence occurs (two waterway ways join), assure the ways connect 
with a node, this makes a more correct waterway network.

I welcome any or all of these as MapRoulette waterway improvement challenges!

SteveA
California

___
Talk-us mailing list
Talk-us@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] Low-quality NHD imports

[Charlotte Wolter]

Hello, all,

Clifford makes some very good points. In the West, particularly,
those little intermittent streams are important landmarks. Particularly
when hiking in a featureless area, such as pinyon-juniper forest, a
trail direction may say something like, "turn right after crossing the third
drainage."
And, during the summer monsoon, you want to know where they
are because they might flash flood.
The arid West has many intermittent drainages. Whatever they
are named (arroyo, stream, creek, etc.), it is important to include them.

Charlotte


At 07:56 PM 10/13/2017, you wrote:




On Fri, Oct 13, 2017 at 10:34 AM, Christoph Hormann 
<o...@imagico.de> wrote:

On Friday 13 October 2017, Kevin Kenny wrote:
>
> I remain unconvinced that importing or not importing has had any
> significant impact on whether people improve the map manually.


There are a number of possible measures that could be considered for
improving old NHD imports:

* removal of unnecessary tags to reduce the baggage mappers would have
to deal with when working on the data.
* removal of small unnamed streams which are not necessary for the
overall river network connectivity in areas where the geometric
accuracy is poor by current standards (and it is therefore usually
easier for mappers to newly trace those streams instead of trying to
improve the inaccurate data)



Unnamed streams are helpful to people hiking in the forest areas by 
giving a landmark for navigation. From areas I'm familiar with, 
there are thousands of unnamed streams. They are unnamed because 
civilization just hasn't reached it. For example, we have Logan 
Creek nearby. If it was in a national forest it would most likely be unnamed.



* creating maproulette challenges for fixing inaccurate waterway
classifications - in particular waterways tagged 'waterway=stream' but
with a name containing 'Creek' or 'River' will often qualify as
waterway=river. Same for artificial waterways with 'waterway=ditch'
but names containing 'Canal' or ther other way round.


When I see creek in the name, it implies stream, at least in areas 
I'm familiar with, then again that's where I usually map. I'm not 
sure where you are from but I never consider telling you how to 
classify something just by the name. Maproulette could be handy if 
we had NHD classification differences between what's tagged in OSM and NHD.


* creating maproulette challenges for unconnected waterways.


+1

* adding missing 'intermittent=yes' to waterways in imports where this
was not properly set based on the feature codes.


+1


--
@osm_seattle
osm_seattle.snowandsnow.us
OpenStreetMap: Maps with a human touch
___
Talk-us mailing list
Talk-us@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-us


Charlotte Wolter
927 18th Street Suite A
Santa Monica, California
90403
+1-310-597-4040
Mobile: 310-663-3699
techl...@techlady.com
Skype: thetechlady

___
Talk-us mailing list
Talk-us@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] Low-quality NHD imports

[Clifford Snow]

On Fri, Oct 13, 2017 at 10:34 AM, Christoph Hormann  wrote:

> On Friday 13 October 2017, Kevin Kenny wrote:
> >
> > I remain unconvinced that importing or not importing has had any
> > significant impact on whether people improve the map manually.
>
>
> There are a number of possible measures that could be considered for
> improving old NHD imports:
>
> * removal of unnecessary tags to reduce the baggage mappers would have
> to deal with when working on the data.
> * removal of small unnamed streams which are not necessary for the
> overall river network connectivity in areas where the geometric
> accuracy is poor by current standards (and it is therefore usually
> easier for mappers to newly trace those streams instead of trying to
> improve the inaccurate data)
>


Unnamed streams are helpful to people hiking in the forest areas by giving
a landmark for navigation. From areas I'm familiar with, there are
thousands of unnamed streams. They are unnamed because civilization just
hasn't reached it. For example, we have Logan Creek nearby. If it was in a
national forest it would most likely be unnamed.


* creating maproulette challenges for fixing inaccurate waterway
> classifications - in particular waterways tagged 'waterway=stream' but
> with a name containing 'Creek' or 'River' will often qualify as
> waterway=river.  Same for artificial waterways with 'waterway=ditch'
> but names containing 'Canal' or ther other way round.
>

When I see creek in the name, it implies stream, at least in areas I'm
familiar with, then again that's where I usually map. I'm not sure where
you are from but I never consider telling you how to classify something
just by the name. A Maproulette could be handy if we had NHD classification
differences between what't tagged in OSM and NHD.

* creating maproulette challenges for unconnected waterways.
>

+1


> * adding missing 'intermittent=yes' to waterways in imports where this
> was not properly set based on the feature codes.
>

+1


-- 
@osm_seattle
osm_seattle.snowandsnow.us
OpenStreetMap: Maps with a human touch
___
Talk-us mailing list
Talk-us@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] Low-quality NHD imports

[Kevin Kenny]

On Fri, Oct 13, 2017 at 1:34 PM, Christoph Hormann  wrote:
> There are a number of possible measures that could be considered for
> improving old NHD imports:
>
> * removal of unnecessary tags to reduce the baggage mappers would have
> to deal with when working on the data.
> * removal of small unnamed streams which are not necessary for the
> overall river network connectivity in areas where the geometric
> accuracy is poor by current standards (and it is therefore usually
> easier for mappers to newly trace those streams instead of trying to
> improve the inaccurate data)
> * creating maproulette challenges for fixing inaccurate waterway
> classifications - in particular waterways tagged 'waterway=stream' but
> with a name containing 'Creek' or 'River' will often qualify as
> waterway=river.  Same for artificial waterways with 'waterway=ditch'
> but names containing 'Canal' or ther other way round.
> * creating maproulette challenges for unconnected waterways.
> * adding missing 'intermittent=yes' to waterways in imports where this
> was not properly set based on the feature codes.


I've no argument with any of these (but please keep reachcode!).  They
all seem to be worthy projects. I'm of two minds about maproulette
challenges, only because I've seen some pretty low-quality results
coming out of them.

I have issues only with 'newly trace the streams' in areas with dense
tree cover - even some significant streams bordering on
'waterway=river' can be virtually invisible on the aerial images
around here. A lot of the time if I am tracing a stream, I cross-check
with the 1/3-arc-second (or 1/9-arc-second where I can get it) radar
altimetry and make sure that the streambed appears to be following the
'v's in the contour lines, and I'm not sure I'm actually doing any
better than NHD. Since I'm in an area where NHD is pretty good, I
usually start by importing individual watercourses and then try to
improve the data. (One of my projects for when the snow starts flying
is to make sure that I've brought in enough that the streams that I
added for some specific large-scale rendered maps actually reach
the rivers.)

A definite +1 on the inaccurate waterway classifications. In
particular, anything with an artificial flowline and a drawn riverbank
is a 'river,' whatever the name or the FCode say.  (I've had one
discussion in the past with someone who wanted to downgrade the
Schoharie Creek to 'stream' - apparently the guy couldn't quite grasp
that it has dams, reservoirs, power generation stations, catastrophic
floods from time to time. Despite 'Creek' in the name, it's a
third-order river.)

Oh, and a side question: should we be mapping artificial connections,
and if so, how? It is known, for instance (by accidental
contamination) that the water from one small lake locally flows
underground and exits through caves in a cliff a few km distant, but
not by what route it flows. NHD has an artificial 'connector' flowline
for that purpose, that I've never seen fit to map, having absolutely
no idea how to map it. In other cases of sinks and springs around
here, the connections may not even be known. (Ah, the joys of
glacio-karst terrain.)

For that matter, I also have No Clue what to do about
rapids. "waterway=rapids" is deprecated, and I'm not a canoeist or
kayaker - the "Whitewater Sports" page on the wiki says that if you
don't know the practice on the river in question, don't map it. (I
would have thought that "rapids here" had some utility even in the
absence of further information, but apparently the community
disagrees?)

Then again, that 'whitewater sports' page has some other misleading
stuff: "All rivers with width more than 5 meters has to be draw by
waterway=riverbank or natural=water+water=river in addition to
waterway=river.  I'm sorry, if I can't see the bank in aerials, but
can follow the flow line on elevation maps, I'm still going to map the
stream. Incomplete information is better than a blank map. If we
demand perfection from mappers when information is first being
gathered, that'll scare even more away. I'll agree with "it is
desirable to show the riverbank on rivers more than XX metres wide",
and '5' is probably too small a value for XX - I don't get that sort
of repeatability from GPS, and I'm surely not going to do a plane
table survey of the banks of my local streams!

In short: We agree about good ways to move forward, even if I reserve
judgment on the reasons.

___
Talk-us mailing list
Talk-us@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] Low-quality NHD imports

[Christoph Hormann]

On Friday 13 October 2017, Kevin Kenny wrote:
>
> I remain unconvinced that importing or not importing has had any
> significant impact on whether people improve the map manually.

In case of NHD imports in the US there are certainly significant parts 
of the country where no NHD data has been imported and there is also no 
manual waterbody mapping worth mentioning - which would allow you to 
conclude there are likely also areas of NHD imports where at least so 
far this did not have a negative effect.  But in areas of significant 
interest, in particular popular outdoor destinations, i am pretty sure 
you can observe this - i showed an example from southern Montana where 
the limit of the detailed and accurate manual mapping clearly matches 
the limit of the previous NHD import or in other words:  The mapper 
mapping that clearly did not feel like bringing the NHD data to the 
same level of quality as the manually mapped area.  This is just one 
case and you cannot simple conclude it is the same everywhere else but 
i would not simply dismiss the idea that there is a discouraging effect 
of imports on manual mapping in some cases.

There are a number of possible measures that could be considered for 
improving old NHD imports:

* removal of unnecessary tags to reduce the baggage mappers would have 
to deal with when working on the data.
* removal of small unnamed streams which are not necessary for the 
overall river network connectivity in areas where the geometric 
accuracy is poor by current standards (and it is therefore usually 
easier for mappers to newly trace those streams instead of trying to 
improve the inaccurate data)
* creating maproulette challenges for fixing inaccurate waterway 
classifications - in particular waterways tagged 'waterway=stream' but 
with a name containing 'Creek' or 'River' will often qualify as 
waterway=river.  Same for artificial waterways with 'waterway=ditch' 
but names containing 'Canal' or ther other way round.
* creating maproulette challenges for unconnected waterways.
* adding missing 'intermittent=yes' to waterways in imports where this 
was not properly set based on the feature codes. 

-- 
Christoph Hormann
http://www.imagico.de/

___
Talk-us mailing list
Talk-us@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] Low-quality NHD imports

[Kevin Kenny]

On Fri, Oct 13, 2017 at 10:06 AM, Frederik Ramm  wrote:
> I only posted that on the talk list and not here, so for those on
> talk-us who don't read talk and who are familiar with the "imports are
> always bad for the community" discussion, you might want to have a look
> at a scientific paper recently written by Abhishek Nagaraj (UC
> Berkeley-Haas) which finds that:
>
> "... a higher level of information seeding significantly lowered
> follow-on knowledge production and contributor activity on OpenStreetMap
> and was also associated with lower levels of long-term quality."
>
> The paper can be freely downloaded here
> https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3044581 and there
> has been a little discussion about it over on the talk list, here:
> https://lists.openstreetmap.org/pipermail/talk/2017-October/079116.html

Interesting. The interesting take-away is that there may be an ideal
level of 'information seeding' so that potential contributors are
initially attracted to the platform (if there's nothing there, there's
no reason to believe that your contribution will ever have value)
and not driven off by a sense that the job is done. The
'inverted U' effect is hypothesized several times in the
paper.

I'm a post-TIGER mapper, so the TIGER import has always
been part of my environment. I know that when I was a beginner,
one thing that quite put me off trying to improve it was
all the noise tags that came with the import. Since at the
time, I had no idea what the purpose of that information
was, much less that it serves no purpose whatsoever, I
was reluctant to edit TIGER ways for fear of breaking
something. In the environment we live in now, we certainly
need some way to encourage new mappers to go ahead and
break things. (I know, this argument is anecdotal, but
I can't even think of how I could set out to test it.)

I also think the balance shifts some in the places where
there simply are never going to be enough mappers.
I could, in theory, lawfully get out in the field and
survey the boundaries of the big parks, for instance.
But the boundaries don't move much, and there aren't
enough people like me with the necessary motivation
and skills. But people relate to parks, and react
badly to an argument that they shouldn't be
on the map unless someone has devoted that
sort of effort. (Someone has. The state surveyor. And
even the state survey may reblaze the bounds only
once or twice a century.) Remember that in my part of
the word, there are even indefinite county boundaries.
County lines that have never been surveyed and
monumented. (It messes up everyone's idea of topology
not to draw lines on the maps, so people do.)

I think the ideal level of imports is greater than zero and less
than TIGER. We'll go on arguing about where the maximum
value is along that curve, and that's a good thing. In the
meantime, we all try to improve the map, and to recruit
mappers - which, we see, are sometimes conflicting goals.

___
Talk-us mailing list
Talk-us@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] Low-quality NHD imports

[Clifford Snow]

I live in the west coast of the US where manually surveying waterways is
not only difficult, but almost impossible. I can't quantify how much as
been cleaned up, but I do know of efforts to fix problems. For example,
most of the waterways in the Olympic Peninsula were reversed. That's been
fix. (I did a few, but another mapper did just about everything else.) I
regularly improve waterways and waterbodies around me. Just recently I
added names to lakes and lagoons in Island County.

The difficulty in physically mapping waterways is so expensive that our
state and counties have dropped most efforts to map them, instead they use
NHD data. The counties do mapping of salmon stream but doing so requires
hiring companies to fly drones equipped with LIDAR to map the stream. My
own county is even looking at OSM data because we have been improving the
waterways from imagery.

Frederik, I'm not sure what you are suggesting. Too many tags on NHD
imported data, that we should import data, or not enough love is being
given to streams and lakes? I certainly hope you are not suggesting we
revert all the steams and waterbodies.

One study on imports is inciteful, but hardly the last word. I hope to
discuss the report with the author later this month. I have some questions
about his findings and how much we can infer from them.

Best,
Clifford


-- 
@osm_seattle
osm_seattle.snowandsnow.us
OpenStreetMap: Maps with a human touch
___
Talk-us mailing list
Talk-us@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] Low-quality NHD imports

[Kevin Kenny]

On Fri, Oct 13, 2017 at 4:53 AM, Christoph Hormann  wrote:
> I think this is probably a good example for imports discouraging manual
> mapping.  If this data was not there mappers would probably meanwhile
> have added at least the larger rivers but with the dense network of NHD
> geometries with a lot of cryptic tags and all flatly tagged as
> waterway=stream it is quite hard for mappers to identify the larger
> rivers and improve mapping there.  Like here (NHD import on the left,
> newer manual mapping on the right):

What *should* have been done: artificial flowlines with shoreline should have
been 'river', most other should have been 'stream'. That would have worked
in most cases. But that's all water under the bridge (and people complain
that in initial mapping, I just cross roads over streams, and go back and
add the bridges and culverts later as time permits - usually it doesn't).

I remain unconvinced that importing or not importing has had any
significant impact on whether people improve the map manually.

NHD was never imported around here. I miss it. Only the most significant
rivers are in OSM. The lack of an import didn't motivate manual mapping,
except that at one point there was a project to map ponds (from a file of
point data). The result was a bunch of pond outlines that are pretty rough,
and confuse floating and emergent vegetation (common in our shallow
waters) with land. For that specific set, I find NHD more reliable.

Across the state line, NHD was imported in some neighbouring states.
There, I at least have the waterways in OSM. Are they any better or
worse than the few manually-mapped ones I have here? On the whole,
they appear to be better. There are some specific instances were
they're very bad. I don't see much maintenance of the data on
either side of the border.

I just inhabit a place where detailed mapping is always going to be
haphazard. Too many land, not enough people, not nearly enough
network connectivity.

___
Talk-us mailing list
Talk-us@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] Low-quality NHD imports

[Frederik Ramm]

Hi,

On 10/13/17 15:52, Kevin Kenny wrote:
> (I ignore the arguments that
> are based solely on contentions that "imports are always bad for the
> community," or else I'd never import anything.)

I only posted that on the talk list and not here, so for those on
talk-us who don't read talk and who are familiar with the "imports are
always bad for the community" discussion, you might want to have a look
at a scientific paper recently written by Abhishek Nagaraj (UC
Berkeley-Haas) which finds that:

"... a higher level of information seeding significantly lowered
follow-on knowledge production and contributor activity on OpenStreetMap
and was also associated with lower levels of long-term quality."

The paper can be freely downloaded here
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3044581 and there
has been a little discussion about it over on the talk list, here:
https://lists.openstreetmap.org/pipermail/talk/2017-October/079116.html

Bye
Frederik

-- 
Frederik Ramm  ##  eMail frede...@remote.org  ##  N49°00'09" E008°23'33"

___
Talk-us mailing list
Talk-us@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] Low-quality NHD imports

[Kevin Kenny]

 On 10/13/2017 02:06 AM, Frederik Ramm wrote:

Hi,

   there's a LOT of NHD:* (and nhd:*) tags on OSM objects, see
https://taginfo.openstreetmap.org/search?q=NHD%3A

- 1.9 million NHD:FCode, but also 188k "NHD:Permanent_" (note the
underscore), 10k "NHD:WBAreaComI", or 1.5m "NHD:Resolution" just to grab
a few.

I haven't researched who added them and when, but they would certainly
not clear the quality standards we have for imports today. Most of this
information can be properly modelled in usual OSM tags, and where it
cannot, it probably shouldn't be in OSM in the first place.

Is there any systematic (or even sporadic) effort of cleaning up these
old imports? Is there reason to believe that the neglect extends to more
than just the tags - do geometry and topology usually work well on
these, or are the funny tags a huge "this whole area hasn't had any love
in a long time" sign?

ON IRRELEVANT TAGGING:

I, at least, ordinarily do not make a specific effort to ferret out
irrelevant tags. For the most part, they're harmless to me. If some
random object on the map haapens to have 'zqx3:identifier=2718281828'
among its tags, the only real damage is the diffuse cost of shipping
the data around.

That said, you're quite right that such tags might indeed be a symptom
of a neglected import, or one that was originally done with processes
that wouldn't clear today's bar. Even that has only some bearing on
the data that are meaningful.

ON IMPORTING NHD:

In the specific case of NHD, data quality varies by region. As Dave
correctly notes, Alaska is uniformly atrocious. (There really are no
good mapping data for Alaska. The technical challenge of acquiring
high-quality data for much of the state simply is greater than the
perceived value of the data.)

Where I am, on the other had, NHD is actually quite good - in the
maps that I render, which are almost all in rural areas, I use it.
I most often use it in combination with OSM and with other data
sources (USFWS national wetland inventory, Adirondack Park Authority
wetland inventory, NYSDOT, ...) which give the rendered maps a
somewhat 'cubist' appearance, but I find that appearance helpful -
it's an indication of data variability, and gives me an idea how much
uncertainty to expect in the field.

The fact that NHD is often quite 'stale' does not bother me at all
locally. I live in a heavily glaciated area, and the cities have been
settled for quite a long time by US standards. Out in the countryside,
the streams run typically in deep ravines, disproportionate to the
size of the streams. They aren't moving anywhere. They most likely
haven't moved significantly since the Wisconsinan glaciation, 14000
years ago.

In the valleys, the detailed course of the streams does shift a bit,
but in the cities and towns, the streams are engineered, and
elsewhere, the terrain tends to be beaver swamp, and the streams shift
with every move of the rodents or every major storm. I never expect
the track of a watercourse within a wetland to be accurate, on any
map, ever.

NHD's topology is audited before it is released, so it's at least
consistent (and likely correct).

It's certainly hypothetically possible to map the streams using
'hand-crafted' methods - and I have done so for a few, when I've
happened to follow them in wilderness travel. (I occasionally go
hiking off-trail.) But the OSM community is never going to be able to
do that for the great many watercourses that flow over my extremely
well-watered area. There simply is too much land inhabited by too few
people, most of whom are not well enough connected nor technologically
literate enough to become OSM mappers. (Seriously, in some of these
communities, there is no cell service and only a quarter of the houses
have any sort of network connectivity. It's effectively working with
Third World infrastructure.)

It's virtually impossible to map most of these watercourses as an
'armchair mapper.' Our 'old second growth' timber gives rise to
extraordinarily dense tree cover - denser than true 'old growth'
forest. Even some fairly major watercourses - major enough that I
wouldn't attempt to ford in springtime - are difficult or impossible
to see in aerials.

There has been an OSM project to map lakes and ponds in New York
State, starting from point features giving their names. I've preserved
these tracings in OSM, because I don't replace mappers' work with
imports, ever. Nevertheless, I find them to be uniformly worse than
NHD. They're usually quite rough, and in a great many of them, the
mappers treated mats of floating or emergent vegetation as the
shoreline, making shallow ponds much smaller than they are.

For all these reasons, NHD is what I have in my area. I've never done
a large-scale NHD import, and nobody else has done one around me.  If
I need a stream for a rendered map, and don't want the 'cubist' data,
I sometimes import it as a single object from NHD. Where else will I
get it?

(That's pretty much my guideline on when 

Re: [Talk-us] Low-quality NHD imports

[Christoph Hormann]

On Friday 13 October 2017, Frederik Ramm wrote:
>
> I haven't researched who added them and when, but they would
> certainly not clear the quality standards we have for imports today.
> Most of this information can be properly modelled in usual OSM tags,
> and where it cannot, it probably shouldn't be in OSM in the first
> place.

Those are several regional imports made by different people, see 

https://taginfo.openstreetmap.org/keys/NHD%3AFCode#map

to get an idea of the distribution.

In general the NHD water line features are next to impossible to 
properly model in OSM since the natural waterway lines (feature codes 
46000-46007) and the artificial waterway lines (feature codes 
33600-33603) include all sizes of waterways.  They were usually all 
imported as waterway=stream and waterway=ditch which is incorrect in 
many cases.

The geometries in the NHD source data are usually fine (at least in 
recent NHD versions) - though conflation is often poor, lacking 
connectivity to pre-existing data - like here:

https://www.openstreetmap.org/#map=16/45.8945/-111.3596=D

And there are of course also obvious errors like this:

https://www.openstreetmap.org/way/39263506

or this:

https://www.openstreetmap.org/#map=13/38.9733/-108.4790

> Is there any systematic (or even sporadic) effort of cleaning up
> these old imports? Is there reason to believe that the neglect
> extends to more than just the tags - do geometry and topology usually
> work well on these, or are the funny tags a huge "this whole area
> hasn't had any love in a long time" sign?

I think this is probably a good example for imports discouraging manual 
mapping.  If this data was not there mappers would probably meanwhile 
have added at least the larger rivers but with the dense network of NHD 
geometries with a lot of cryptic tags and all flatly tagged as 
waterway=stream it is quite hard for mappers to identify the larger 
rivers and improve mapping there.  Like here (NHD import on the left, 
newer manual mapping on the right):

https://www.openstreetmap.org/#map=13/45.2453/-110.1321

-- 
Christoph Hormann
http://www.imagico.de/

___
Talk-us mailing list
Talk-us@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] Low-quality NHD imports

[Dave Swarthout]

Sometimes I think it might have been better if OSM had never imported Tiger
data. It is simply pitiful, almost worse than nothing, in many areas of
Alaska. Same with the coastlines and NHD water bodies. I know they
represent a first approximation and without any coastlines we couldn't have
a map, period, but the sheer amount of editing required to clean up the
horrendous data in Alaska is daunting indeed. Same for the riverbanks. I
usually delete them and start over fresh because it's easier to create new
ones than to try to adjust and align the bad stuff already in place.

Given the actual total length of coastline in Alaska, I would venture to
say it will never be cleaned up. Here is an example of a relation
(id:2057975) "glacier" imported from NHD that is actually many named
glaciers all rolled into one. I have been working hard for the past year to
add individual named glaciers in Alaska but I swear, looking at something
like this monster containing 542 members makes me think I'll never get it
done.

Also, if you want to see something else that is really horrendous, take a
look at the Canvec imports for Canada; rivers and streams composed of many
tiny sections, water and woods where there isn't any. Sheesh!

Sorry, couldn't resist.



On Fri, Oct 13, 2017 at 2:18 PM, Wolfgang Zenker 
wrote:

> Hi,
>
> * Frederik Ramm  [171013 08:06]:
> >there's a LOT of NHD:* (and nhd:*) tags on OSM objects, see
>
> > https://taginfo.openstreetmap.org/search?q=NHD%3A
>
> > - 1.9 million NHD:FCode, but also 188k "NHD:Permanent_" (note the
> > underscore), 10k "NHD:WBAreaComI", or 1.5m "NHD:Resolution" just to grab
> > a few.
>
> > I haven't researched who added them and when, but they would certainly
> > not clear the quality standards we have for imports today. Most of this
> > information can be properly modelled in usual OSM tags, and where it
> > cannot, it probably shouldn't be in OSM in the first place.
>
> > Is there any systematic (or even sporadic) effort of cleaning up these
> > old imports? Is there reason to believe that the neglect extends to more
> > than just the tags - do geometry and topology usually work well on
> > these, or are the funny tags a huge "this whole area hasn't had any love
> > in a long time" sign?
>
> the NHD imports that I have encountered so far have numerous problems:
> The data is several decades old, the so-called "medium resolution" is
> pretty bad, and the data was basically just dumped into the OSM database
> without any conflation happening. And larger rivers where often imported
> as monstrous riverbank polygons without the river itself as a flowline.
>
> The worst junk like lakes covering motorways has been mostly cleaned up
> by now, but it is still easy to see where NHD data has been imported by
> looking a KeepRights display of broken highway/waterway crossings.
>
> I clean up the imports in areas where I'm doing TIGER reviews, but I
> have to admit that a few times I have decided to work on different areas
> instead because the huge riverbank polygons where almost impossible to
> edit.
>
> Wolfgang
>
> ___
> Talk-us mailing list
> Talk-us@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk-us
>



-- 
Dave Swarthout
Homer, Alaska
Chiang Mai, Thailand
Travel Blog at http://dswarthout.blogspot.com
___
Talk-us mailing list
Talk-us@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] Low-quality NHD imports

[Wolfgang Zenker]

Hi,

* Frederik Ramm  [171013 08:06]:
>there's a LOT of NHD:* (and nhd:*) tags on OSM objects, see

> https://taginfo.openstreetmap.org/search?q=NHD%3A

> - 1.9 million NHD:FCode, but also 188k "NHD:Permanent_" (note the
> underscore), 10k "NHD:WBAreaComI", or 1.5m "NHD:Resolution" just to grab
> a few.

> I haven't researched who added them and when, but they would certainly
> not clear the quality standards we have for imports today. Most of this
> information can be properly modelled in usual OSM tags, and where it
> cannot, it probably shouldn't be in OSM in the first place.

> Is there any systematic (or even sporadic) effort of cleaning up these
> old imports? Is there reason to believe that the neglect extends to more
> than just the tags - do geometry and topology usually work well on
> these, or are the funny tags a huge "this whole area hasn't had any love
> in a long time" sign?

the NHD imports that I have encountered so far have numerous problems:
The data is several decades old, the so-called "medium resolution" is
pretty bad, and the data was basically just dumped into the OSM database
without any conflation happening. And larger rivers where often imported
as monstrous riverbank polygons without the river itself as a flowline.

The worst junk like lakes covering motorways has been mostly cleaned up
by now, but it is still easy to see where NHD data has been imported by
looking a KeepRights display of broken highway/waterway crossings.

I clean up the imports in areas where I'm doing TIGER reviews, but I
have to admit that a few times I have decided to work on different areas
instead because the huge riverbank polygons where almost impossible to
edit.

Wolfgang

___
Talk-us mailing list
Talk-us@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-us

[Talk-us] Low-quality NHD imports

[Frederik Ramm]

Hi,

   there's a LOT of NHD:* (and nhd:*) tags on OSM objects, see

https://taginfo.openstreetmap.org/search?q=NHD%3A

- 1.9 million NHD:FCode, but also 188k "NHD:Permanent_" (note the
underscore), 10k "NHD:WBAreaComI", or 1.5m "NHD:Resolution" just to grab
a few.

I haven't researched who added them and when, but they would certainly
not clear the quality standards we have for imports today. Most of this
information can be properly modelled in usual OSM tags, and where it
cannot, it probably shouldn't be in OSM in the first place.

Is there any systematic (or even sporadic) effort of cleaning up these
old imports? Is there reason to believe that the neglect extends to more
than just the tags - do geometry and topology usually work well on
these, or are the funny tags a huge "this whole area hasn't had any love
in a long time" sign?

Bye
Frederik

-- 
Frederik Ramm  ##  eMail frede...@remote.org  ##  N49°00'09" E008°23'33"

___
Talk-us mailing list
Talk-us@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-us