Hi John,
I guess that there is a clear point that you
mentioned:"a validator's role is more to ensure that the major requirements of the project are met and guidelines followed."
The issue of the huts size is a tricky one, and as you say, it depends on how quickly the maps are needed by the relief agencies. For simply the number of huts,inaccurate sizes may be acceptable
for a fast
delivery but if populations are to be estimated from the size of the huts, data
would then
been lost. For the validator to either invalidate or remap, is problem, but I guess that this relies on the mapper understanding the task descriptions. Should not the task description explicitly say
something like "correct size and shape of the dwellings to be
interpreted", for example. The other issue here is the scale at which the images are interpreted. If someone maps at a too smaller scale (perhaps for a fast delivery), then potential accuracy could be lost.
Thanks for highlighting these two links:
http://wiki.openstreetmap.org/wiki/Highway_Tag_Africa
<http://wiki.openstreetmap.org/wiki/Highway_Tag_Africa>
http://wiki.openstreetmap.org/wiki/Map_Features
Validating as the project goes along from your point of view seems the good way. This is particularly
relevant with mentoring, and
valuable feed back to mappers at the time. Without feedback, new mapper might lose interest, as they want to know if they have done a good job or not, or if they have a question, they need an answer. As you say, this also gets the projects done faster.
Getting more validators, you suggest some form of recognition, this is whyI thought for that "badge" idea. Still, perhaps someone else can think of a better idea.
My final idea might be, if the
HOT team knew of interested and capable validators, perhaps they could invite them to support some projects
as
validators. Being asked for support due to one's skills, is often a reward in itself.
Thanks for your views and ideas on this subject, it has opened my eyes to the
issues and challenges of validating, as well as other issues of mapping in
general.
Regards,
Graham
On 6/4/2016 19:55, john whelan wrote:
I tend to think in shades of grey rather than black and white. In OSM
there are different mappers, each mapper interprets things slightly
differently so two mappers will rarely give exactly the same result
with the same inputs.
For example one might like to add large buildings, the other will
ignore them. Both are correct but are different interpretations.
Which to my mind means a validator's role is more to ensure that the
major requirements of the project are met and guidelines followed.
One difficulty is the size of huts. For many mappers <crtl>c and
<crtl>v is a quick way to map these. However there are ways to analyse
the size of the hut then estimate how many people are living in it.
Do we expect validators to check the size of the hut mapped is the
same as the size in the image? Then do we invalidate or remap?
A particular project may have a deadline to meet, NGO staff flying out
for example. In which case they may wish to reduce the information
requested to the bare essential minimum in order to get the project
completed before the deadline. However many NGOs etc will make use of
the map at some point in time in the future so additional information
may be useful and in any event there will be some economic advantages
to the locals in having the area mapped.
We use different standards for mapping in different parts of the world
and whilst local knowledge helps it is not essential for HOT mapping.
What is essential is following the local agreed standards ie
http://wiki.openstreetmap.org/wiki/Highway_Tag_Africa and some very
experienced mappers may not be aware of them.
JOSM / iD / Potlatch / OSMand amongst others are all valid methods for
entering map data. JOSM does some validation which catches a fair
number of basic errors before uploading and its building plugin is
unbeaten for quickly and accurately drawing in buildings. It is my
editor of choice. However it needs to be installed and that includes
JAVA. JAVA has been listed in the past as being a security hole by US
government so for some corporate machines installing it is not an
option. Also there can be some issues with Apple computers and JAVA.
Additionally installing it is more complex than just opening up a
browser and using iD. So in the same way that some people freeze when
asked to add two numbers together so some have problems even thinking
about installing JOSM. JOSM is also very rich and I don’t use all its
features so when you train with JOSM you need to train people just to
use the features they need. If its just highways and settlements
these are fairly minimal. It does have the advantage of being able to
tag anything so you can point them back to
http://wiki.openstreetmap.org/wiki/Map_Features which helps with
consistency and standard ways of tagging.
Different people think in different ways about validation. To me in
order to reduce the number of errors being made in the first place I
prefer to validate as the project is done. This is a form of coaching
the mappers. Others take the view of map it all first then validate.
Fine except if you’re coming in to validate two months after the
mapping its not really worthwhile giving feedback to a mapper about
something they did two months ago.
The worst thing I’ve seen is a new mapper validating other’s work,
select a task at random would occasionally suggest this but then you
really don’t know if the mapping is good or not. In Nepal it was so
bad we just invalidated everything in sight and revalidated.
I often work with others validating and if its working against time
then we delegate, check this etc and any queries bug me. I’ll go in
and pull in much of the project into JOSM afterwards and give it a
more through check. Are the correct tags being used? A common
problem building=yes being used for a group of buildings.
Highway=pedestrian and there are a few others. <crtl>i in JOSM can
give you a fair bit of information about the item.
How do we get more validators? I’m not sure, in general most mappers
with 300 plus edits are ready to validate. However I’ve seen some
mappers when I validate their work consistently miss 5% of
settlements. I’ve seen others some of whom have a background in GIS
and I’m happy if they validate after seeing them map a dozen tiles. I
think we need to value them more and give them some sort of
recognition for the work they do. On a couple of projects now we’ve
validated as the mapping was done and the project gets completed very
quickly. You’ve probably noticed that a project gets about two weeks
in the limelight and if it hasn’t made good progress in those two
weeks it lingers on unfinished. Get a validator in there early and
you get more mapping with the feedback and you get more of a team
spirit to complete the project.
To my mind all projects could benefit from a validator but we don’t
have enough and coaxing them onto new projects is the most effective
use we can make of the ones we have.
Cheerio John
On 5 April 2016 at 22:07, graham <[email protected]
<mailto:[email protected]>> wrote:
Thanks for you feedback John and Blake,
I have tried to pull out some of the points raised here, for my
understanding and perhaps for others:
There are different standards of mapping and to some extent
different standards of validation needed, depending on the project.
In-country knowledge of the country can effect the quality of
mapping/validation.
A good mapper does not mean that their are a good validator,
different skill sets are needed.
The capacity to map/validate is also dependant to some degree on
the the quality of the video system and screen one is using.
There seems to be a difference of quality of mapping depending on
the software used: JOSM vs. iD. Is this an issue to be addressed?
Was it suggested that coaching is a good way for people to become
good validators?
Certain projects could do with a certified validator but naturally
to have a "badge" is not for everybody.
A link to validating guidelines
http://wiki.openstreetmap.org/wiki/OSM_Tasking_Manager/Validating_data
Please say if I missed and misrepresented your points.
Regards,
Graham
On 5/4/2016 21:49, john whelan wrote:
I think the point was that that there are different standards of
mapping and to some extent validation as well. In Nepal we had
time and resource constraints and I must confess I took some
short cuts and didn't do a through a job as could have been
done. Sometimes it's a judgement call and I think as you and I
have discussed sometimes some validators do an excellent job but
check rather more than either of us would when validating.
I've even heard a whisper of a validator taking one look at
someone's work and zapping it and just remapping as it was
quicker but of course that never happens in OSM and would never
be documented.
Cheerio John
On 5 April 2016 at 09:36, Blake Girardot <[email protected]
<mailto:[email protected]>> wrote:
On 4/5/2016 3:15 PM, john whelan wrote:
Blake thought my methods of validation in Nepal left much
to be
desired.
That sounds a little strong :) I forget the question I had or
the comment, but after talking with you I was good with it
and agreed is all I remember.
>I agreed but given that 70% of the mappers were new even the
basic validation I did improved the data quality quickly.
This I 100% agree with.
Cheers,
Blake
I also had a
couple of mappers who were visually checking tiles and
finding 30% more
buildings sometimes. Which comes back to the quality of
the video
system and screen you’re using. “Why does your laptop
show the image in
JOSM better than mine in iD?” The lap top I was using
was an old Dell
professional grade one and my desktop screen at home
shows an even
clearer image. So the equipment the validator has
available might be an
important factor on the quality of the validation.
To me validation is a form of coaching being good at
something doesn’t
mean you make a good coach. To me Maperthons are a
source of a dozen
new mappers and really questionable data. The faster we
can get in and
give feedback the better. When you need to add 50
settlements to a tile
it takes resources and to do this I’ve used sensible
mappers with a
month’s experience and delegated. If its just
highway=unclassified and
landuse=residential that’s fine. I’ve also seen mappers
with a thousand
edits to their name who don’t make good validators, the
project asks for
settlements and connecting highways, they like to map all
the tracks as
well. I’ve seen tiles invalidated for missing things
that were not
requested in the project instructions.
It also needs tact, a European mapper who has been
mapping locally will
almost certainly use the wrong tags in Africa for
highways. They’re
high quality mappers of the type Africa needs but
invalidating the tile
because the tags are wrong may not help with the
retention rates.
http://wiki.openstreetmap.org/wiki/OSM_Tasking_Manager/Validating_data
is a start, I’m not certain I agree with every statement
but it’s a good
start.
There is a problem with iD mappers they tend to have more
crossing ways
and highways that almost meet than others. Probably
because there is no
easy way to check for these in iD. So whilst I would
comment JOSM
validation detected six crossing ways normally, if I know
they’re an iD
editor I just correct and don’t comment.
If it’s a more complex project, map and tag everything in
sight I don’t
even bother validating these days. I’ll let someone else
with more
experience than I go and do it. I only have 8,000+ edits
to my name.
These projects certainly could do with a certified
validator and to be
honest I have no interest in getting a badge.
Cheerio John
On 5 April 2016 at 05:37, graham <[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>> wrote:
Dear Hi,____
__ __
With the subject of validators in that last few
emails, and past emails thread discussing how to know
when some one can start validating or not. How much
experience does one need? One may think that they are
good, but other may not etc..
____
__ __
So, I thought to just suggest an idea. I am not sure
what people might think about it, maybe it has already
been discussed before.
Maybe the issue has been solved already.____
__ __
The idea:____
I think that maybe if there was a process to become
officially recognised as an"Validator", then it would be
a "position" worth achieving, it would make it a
challenge. For now, I do not think that there is any
solid pre-requisite to be a validator?____
__ __
Below I am suggesting a potential avenue to become an
official validator:____
__ __
1) To complete a minimum number of tiles, as proof of
commitment to HOT and direct experience with HOT mapping
(regardless of one's professional career), then 2) to
take a short test of proficiency.____
__ __
Details:____
1) Can members the HOT team determine the number of
tiles each user has completed? I know that more that one
user can complete one tile, but perhaps if one user
completes a high percentage of a tile (of all the points,
line, and polygons), that would be considered a "complete
tile" for this purpose. As for another users then
reworking the tile, this is another discussion point.____
__ __
2)The test might be to review a number of tiles from
previous projects and they need to comment on the quality
of the digitalisation and complete any reworking that is
needed. A minimum number of tiles need to be correct, (to
a certain standard), in order to pass the test. In each
test, a random subset of tiles for a complete selection
of tiles could be used, so that not cheating could occur.____
__ __
__ __
Additionally, these official validators could have a
symbol next to their users names. This way, when a mapper
asks for advise regarding their mapping, the mapper would
know if a "validator" is responding or not.____
__ __
Just an idea...____
__ __
Regards,____
__ __
Graham____
_______________________________________________
HOT mailing list
[email protected] <mailto:[email protected]>
<mailto:[email protected] <mailto:[email protected]>>
https://lists.openstreetmap.org/listinfo/hot
_______________________________________________
HOT mailing list
[email protected] <mailto:[email protected]>
https://lists.openstreetmap.org/listinfo/hot
_______________________________________________
HOT mailing list
[email protected]
https://lists.openstreetmap.org/listinfo/hot
