Hi,
I passed the three last months to add support of mapcss validation into
Osmose.
I rewrite an antlr4 grammar for the mapccs and produce a python parser.
Then I use the parser to generate python code for selectors,
declarations and mapcss functions. The result is some kind of
"compilation" of mapcss in python. The generated code use one of the
Osmose checking framework to do the Job.
Osmose have many validator engines. The simplest only check tags of one
object at once (no geometries involved nor object relations). I only
generate code for rules able to be lunched on this engine (it's possible
to support the others, but to do).
The result of running the official JOSM and contrib rules for the whole
world is now available on Osmose, from at least a week. We refresh the
result every two days. (Technically the whole world is not yet computed,
but it's a matter of finishing to update all the Osmose backends around
the world).
So, show me the data!
http://osmose.openstreetmap.fr/en/errors/?limit=0&item=9xxx&useDevItem=true
http://osmose.openstreetmap.fr/en/map/#item=9xxx&useDevItem=true&zoom=12&lat=49.4805&lon=6.3461&level=1
(play with the severity selector if nothing displayed)
This page take a while to load, you can sort on column by clicking on
the header (and wait).
All this content is on Osmose but not yet publicly displayed.
All this checks have generated so far 20.1 millions of issues. I
excluded one rule of official JOSM validator because just this rule
raise 4 millions of issues ("Unnamed unclassified highway").
The goal of supporting mapcss in Osmose is to try to state a common
format for data validation. The next step is to move a part of Osmose
validation to mapcss and make them available into JOSM as contrib.
During doing all this stuff we encountered some problems.
- Osmose issue level VS JOSM throw level: Osmose report only issues
where there is a high probability of something go wrong, with level mean
the impact on data usability. But JOSM have an orthogonal level with
Error, Warning, Other. For now, I simply map throwError to Osmose level1
and so on.
- Translation. Because we use official MapCSS, we also reuse translation
too. The only way to get it is via bzr ?
- Harmonisation of throw level in contrib: contrib mapcss validator have
a very different use of throw level and sometime the inside() selector
is missing. Not yet done, but I plan to blacklist in Osmose lot of rules
from "Brazilian-Specific" and lower some other ("transport"). I added
the ability in Osmose to limit a mapccs validator file to a country,
whatever the inside() selector are used or not.
- I generate the selectors and the declarations in python but also I
generate the tests. And I'm able to run all the included tests. Except
one from the contrib de_openrailwaymap, in short the test try to
validate that "5" != "05". As the content are numbers, my implementation
check 5 != 5, and it fails.
About port of Osmose hand writen python rules to Mapcss, we should add
some extension into mappcss declarations (like "osmose-level" or
"osmose-item"). And I plan to add this kind of declaration into some
existing mapcss validation rule files to avoid duplicating the report to
user, one from JOSM ruleset, one from Osmose ruleset.
The grammar and the mapcss2osmose code:
https://github.com/osm-fr/osmose-backend/tree/master/mapcss
The Osmose analysers produced from mapcss code (all MapCss* files):
https://github.com/osm-fr/osmose-backend/tree/master/plugins
I'm widely open to comment.
Reagrds.
Frédéric.
