Re: [OSM-dev] Comparing quality of different geocoders?

2020-08-26 Thread Mateusz Konieczny via dev



Aug 26, 2020, 11:04 by [email protected]:

> Hi,
>
> On Wed, Aug 26, 2020 at 09:11:44AM +0200, Mateusz Konieczny via dev wrote:
>
>> Is anyone aware about some test suite comparing different geocoders?
>>
>> Something with query + expected result pairs, and listing where Nominatim / 
>> Photon / etc succeeded/failed?
>>
>> I want something like that, tried to find one and failed.
>>
>> I want to ask whatever I missed something like that before I will make it.
>>
>
> There are a couple of test suits out there. The two that come
> immediately to mind are:
> https://github.com/geocoders/geocoder-tester
> https://github.com/pelias/fuzzy-tester
>
Thanks!

I will need to look how tightly pelias/fuzzy-tester is
integrated with just pelias and is it viable to expand/use
it to support testing REST apis.

Maybe it will be easier to write something small from scratch.

> The issue with all these is that you have to define what you
> actually want to compare. Depending on what kind of queries you
> run and how you define the results you get, you'll get very different
> results.
>
Oh, I know!

For example cases such as "Elementary school number 12, Streetname 19"
where address and object are at different locations and expected result
completely depends on what user wants.

Or slang/local/vulgar names where some would prefer not finding them
and not listing them over finding them, even if naming is real.
___
dev mailing list
[email protected]
https://lists.openstreetmap.org/listinfo/dev


Re: [OSM-dev] Comparing quality of different geocoders?

2020-08-26 Thread Sarah Hoffmann
Hi,

On Wed, Aug 26, 2020 at 09:11:44AM +0200, Mateusz Konieczny via dev wrote:
> Is anyone aware about some test suite comparing different geocoders?
> 
> Something with query + expected result pairs, and listing where Nominatim / 
> Photon / etc succeeded/failed?
> 
> I want something like that, tried to find one and failed.
> 
> I want to ask whatever I missed something like that before I will make it.

There are a couple of test suits out there. The two that come
immediately to mind are:
https://github.com/geocoders/geocoder-tester
https://github.com/pelias/fuzzy-tester

The issue with all these is that you have to define what you
actually want to compare. Depending on what kind of queries you
run and how you define the results you get, you'll get very different
results.

Sarah


___
dev mailing list
[email protected]
https://lists.openstreetmap.org/listinfo/dev