Re: [OSM-talk] OSM data, how can we contribute to keep it to a reasonable size?

[Martin Koppenhoefer]

Since we found out that our growth isn't exponential but linear, I'm not
too worried about the requirements. See for example here:
https://wiki.openstreetmap.org/wiki/File:Osmdbstats1_log.png
Also this: https://wiki.openstreetmap.org/wiki/File:Osmdbstats2.png

Cheers,
Martin
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] OSM data, how can we contribute to keep it to a reasonable size?

[Bryan Housel]

Milo's point on database growth is actually very important, and I worry that it 
might be missed.  We shouldn’t be concerned about shaving a few bytes off of 
the tag values to save space, or the power consumption of different processors.

OSM needs a growth plan.

We should already have pretty solid metrics on how fast the database is 
growing, and roughly when we’ll outgrow the current database architecture.  I’d 
be very happy to see OSM grow by 10x, 100x, or more as we continue to map 
everything the world, and we should be prepared for that.  

Curious if anyone else is thinking about this?

Bryan




> On Jan 22, 2018, at 3:57 AM, Milo van der Linden  wrote:
> 
> In my personal opinion, the database will grow and keep growing. So
> instead of looking at means to make things smaller, I think we should
> look at enabling even more storage. Couldn't geographical
> "shards/segments" be a solution? Like, keeping a database for Europe,
> Asia, North-America and so on that are glued together in a smart way,
> but allow for downloading partials straight from the main databases?
> And might it be smart to create a more distributed database? When I
> look at the hardware, there are currently 3 database; karm, ramoth and
> katla and it seems all 3 are in the UK.
> 
> Not intending to start an off topic, but this is just my opinion.
> 
> 2018-01-21 12:05 GMT+01:00 Oleksiy Muzalyev :
>> On 21.01.18 10:04, Martin Koppenhoefer wrote:
>> 
>> 
>> On 21. Jan 2018, at 09:21, Oleksiy Muzalyev 
>> wrote:
>> 
>> For example, Intel i7-8700K 3.7 GHz 8th generation processor consumes 95 W
>> [2], add to this the fans, hard disks, etc., it comes to 400 W power supply
>> unit. 
>> 
>> For comparison, the Raspeberry Pi 3 single-board computer requires only 10 W
>> power supply [4], 40 times less.
>> 


___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] OSM data, how can we contribute to keep it to a reasonable size?

[Milo van der Linden]

In my personal opinion, the database will grow and keep growing. So
instead of looking at means to make things smaller, I think we should
look at enabling even more storage. Couldn't geographical
"shards/segments" be a solution? Like, keeping a database for Europe,
Asia, North-America and so on that are glued together in a smart way,
but allow for downloading partials straight from the main databases?
And might it be smart to create a more distributed database? When I
look at the hardware, there are currently 3 database; karm, ramoth and
katla and it seems all 3 are in the UK.

Not intending to start an off topic, but this is just my opinion.

2018-01-21 12:05 GMT+01:00 Oleksiy Muzalyev :
> On 21.01.18 10:04, Martin Koppenhoefer wrote:
>
>
> On 21. Jan 2018, at 09:21, Oleksiy Muzalyev 
> wrote:
>
> For example, Intel i7-8700K 3.7 GHz 8th generation processor consumes 95 W
> [2], add to this the fans, hard disks, etc., it comes to 400 W power supply
> unit. 
>
> For comparison, the Raspeberry Pi 3 single-board computer requires only 10 W
> power supply [4], 40 times less.
>
>
>
> 10W? Look at this baby, the whole system is running on solar power and
> doesn’t even need external power sources:
> https://www.galeria-kaufhof.de/p/casio-taschenrechner-fx-85ms/1002764080
>
> Pricing is also very accessible.
>
> Seriously, you can’t compare a high end CPU with a raspberry pi looking only
> at the power consumption ;-)
>
>
> cheers,
> Martin
>
> Modern motherboards are capable to reduce the CPU power consumption to 12 W
> [1], when there are no computation intensive tasks. So improving data
> quality at the source would benefit the high end systems too.
>
> [1] https://www.asus.com/Motherboards/PRIME-Z270-A/ (Energy efficient
> design)
>
> Best regards,
>
> Oleksiy
>
>
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk
>



-- 
Milo van der Linden
web: dogodigi
tel: +31-6-16598808

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] OSM data, how can we contribute to keep it to a reasonable size?

[Oleksiy Muzalyev]

On 21.01.18 10:04, Martin Koppenhoefer wrote:


On 21. Jan 2018, at 09:21, Oleksiy Muzalyev 
> wrote:


For example, Intel i7-8700K 3.7 GHz 8th generation processor consumes 
95 W [2], add to this the fans, hard disks, etc., it comes to 400 W 
power supply unit. 


For comparison, the Raspeberry Pi 3 single-board computer requires 
only 10 W power supply [4], 40 times less.



10W? Look at this baby, the whole system is running on solar power and 
doesn’t even need external power sources: 
https://www.galeria-kaufhof.de/p/casio-taschenrechner-fx-85ms/1002764080


Pricing is also very accessible.

Seriously, you can’t compare a high end CPU with a raspberry pi 
looking only at the power consumption ;-)



cheers,
Martin


Modern motherboards are capable to reduce the CPU power consumption to 
12 W [1], when there are no computation intensive tasks. So improving 
data quality at the source would benefit the high end systems too.


[1] https://www.asus.com/Motherboards/PRIME-Z270-A/ (Energy efficient 
design)


Best regards,

Oleksiy

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] OSM data, how can we contribute to keep it to a reasonable size?

[Martin Koppenhoefer]

sent from a phone

> On 21. Jan 2018, at 09:21, Oleksiy Muzalyev  
> wrote:
> 
> For example, Intel i7-8700K 3.7 GHz 8th generation processor consumes 95 W 
> [2], add to this the fans, hard disks, etc., it comes to 400 W power supply 
> unit. 
> 
> For comparison, the Raspeberry Pi 3 single-board computer requires only 10 W 
> power supply [4], 40 times less.


10W? Look at this baby, the whole system is running on solar power and doesn’t 
even need external power sources: 
https://www.galeria-kaufhof.de/p/casio-taschenrechner-fx-85ms/1002764080

Pricing is also very accessible.

Seriously, you can’t compare a high end CPU with a raspberry pi looking only at 
the power consumption ;-)


cheers,
Martin ___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] OSM data, how can we contribute to keep it to a reasonable size?

[Oleksiy Muzalyev]

On 19.01.18 01:53, Paul Norman wrote:
Speaking as a developer and frequent consumer of OSM data, don't do 
any of these things to save space.


Instead, worry about ease of editing. If a few meter long way has 
hundreds of nodes, that's a problem for editing, and should be fixed; 
a mass of unnecessary import-sourced tags confuses people, don't use 
them; and overlapping landuse with lots of multipolygons is difficult 
to edit, so should be avoided. Following these behaviors will slightly 
reduce data size, but the point is keeping the map maintainable.


It's also difficult to say what will affect size without a detailed 
understanding of the format and how it's processed. Size is also not 
the only indicator of time to process - for various reasons, relations 
are much slower to work with than ways with most data consumption 
workflows.


I've met multipolygons which are hard to understand. It is similar here 
with the programming code, it should be first of all readable, otherwise 
if the developer disappears due to a bus factor [1], it would be hard to 
maintain.


Certainly, throwing hardware at the problem is a perfectly respectable 
solution. However, if we look at the power consumption of modern 
processors it ceases to look perfect. For example, Intel i7-8700K 3.7 
GHz 8th generation processor consumes 95 W [2], add to this the fans, 
hard disks, etc., it comes to 400 W power supply unit. And it did not 
change from, for instance, 4th generation processors, which also had 
power consumption of 60 - 90 W [3].


For comparison, the Raspeberry Pi 3 single-board computer requires only 
10 W power supply [4], 40 times less. And this single-board computer 
with the Raspbian Lite GUI-less OS is still capable to run an Apache2 & 
database web-server for a simple site with a light traffic.


So keeping the map maintainable, correcting polygon or tagging errors, 
avoiding unnecessary import-sourced tags and nodes functions as a 
precalculation, which is done only once and by this reduces waste 
multiple times.


[1] https://en.wikipedia.org/wiki/Bus_factor

[2] https://www.brack.ch/intel-cpu-core-i7-8700k-3-616011

[3] https://www.brack.ch/intel-cpu-core-i7-4790k-4-307846

[4] https://www.raspberrypi.org/products/raspberry-pi-3-model-b/

Best regards,

Oleksiy


___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] OSM data, how can we contribute to keep it to a reasonable size?

[Paul Norman]

On 1/17/2018 9:14 PM, Oleksiy Muzalyev wrote:


What can I as a map editor do to keep these data files to a reasonable 
size without compromising  data quality? I mean in the sense, - take 
care of the pennies and the pounds will take care of themselves?


I could think of the following three approaches so far:

- using as short an URL as possible, website=http://somewebsite.com 
instead of website=http://www.somewebsite.com , three characters less; 
[2]


- correct phone number ISO format, phone=+12 345 678 90 12 instead of 
phone=+12 (345) 678 90 12 , two characters less;  [3]


- deleting unnecessary nodes from a way (Shift-Y in JOSM) with 
consequent verification of its geometry;


What else, if anything, could be done? 


Speaking as a developer and frequent consumer of OSM data, don't do any 
of these things to save space.


Instead, worry about ease of editing. If a few meter long way has 
hundreds of nodes, that's a problem for editing, and should be fixed; a 
mass of unnecessary import-sourced tags confuses people, don't use them; 
and overlapping landuse with lots of multipolygons is difficult to edit, 
so should be avoided. Following these behaviors will slightly reduce 
data size, but the point is keeping the map maintainable.


It's also difficult to say what will affect size without a detailed 
understanding of the format and how it's processed. Size is also not the 
only indicator of time to process - for various reasons, relations are 
much slower to work with than ways with most data consumption workflows.


___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] OSM data, how can we contribute to keep it to a reasonable size?

[Andrew Hain]

Or a mass change from Name=Untitled Polygon (wasteful but not wrong) to 
name=Untitled Polygon.

--
Andrew

From: Mark Wagner <mark+...@carnildo.com>
Sent: 18 January 2018 19:07:20
To: talk@openstreetmap.org
Subject: Re: [OSM-talk] OSM data, how can we contribute to keep it to a 
reasonable size?

On Thu, 18 Jan 2018 19:44:47 +0100
Oleksiy Muzalyev <oleksiy.muzal...@bluewin.ch> wrote:

> Imre,
>
> It is very good and surprising idea.
>
> I discovered on the page "Error categories" a tool
> https://www.keepright.at/ with the help of which I found already
> dozens obviously misspelled tags. It functions quite intuitively,
> just select "misspelled tags" check box and move the map to an area
> of interest.

But make sure they're really misspelled.  I recently saw a change that
fixed a dozen instances where the key "brand" was misspelled as "band"
-- except that one of the "band" tags was correct, describing the fact
that a radio antenna operated in the two-meter band.

--
Mark

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] OSM data, how can we contribute to keep it to a reasonable size?

[Oleksiy Muzalyev]

On 18.01.2018 20:07, Mark Wagner wrote:

On Thu, 18 Jan 2018 19:44:47 +0100
Oleksiy Muzalyev  wrote:


Imre,

It is very good and surprising idea.

I discovered on the page "Error categories" a tool
https://www.keepright.at/ with the help of which I found already
dozens obviously misspelled tags. It functions quite intuitively,
just select "misspelled tags" check box and move the map to an area
of interest.

But make sure they're really misspelled.  I recently saw a change that
fixed a dozen instances where the key "brand" was misspelled as "band"
-- except that one of the "band" tags was correct, describing the fact
that a radio antenna operated in the two-meter band.

Indeed, I see already that there could be from time to time 
false-positives. It makes sense to open a node in a separate browser tab 
and to check a tag info at the OSM wiki before correction.



___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] OSM data, how can we contribute to keep it to a reasonable size?

[Mark Wagner]

On Thu, 18 Jan 2018 19:44:47 +0100
Oleksiy Muzalyev  wrote:

> Imre,
> 
> It is very good and surprising idea.
> 
> I discovered on the page "Error categories" a tool 
> https://www.keepright.at/ with the help of which I found already
> dozens obviously misspelled tags. It functions quite intuitively,
> just select "misspelled tags" check box and move the map to an area
> of interest.

But make sure they're really misspelled.  I recently saw a change that
fixed a dozen instances where the key "brand" was misspelled as "band"
-- except that one of the "band" tags was correct, describing the fact
that a radio antenna operated in the two-meter band.

-- 
Mark

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] OSM data, how can we contribute to keep it to a reasonable size?

[Oleksiy Muzalyev]

Imre,

It is very good and surprising idea.

I discovered on the page "Error categories" a tool 
https://www.keepright.at/ with the help of which I found already dozens 
obviously misspelled tags. It functions quite intuitively, just select 
"misspelled tags" check box and move the map to an area of interest.


Best regards,
Oleksiy

On 18.01.2018 15:48, Imre Samu wrote:
>What can I as a map editor do to keep these data files to a 
reasonable size without compromising  data quality?


According to the "Lean thinking" ( 
https://en.wikipedia.org/wiki/Lean_thinking ) we should focus on 
" eliminating waste"

Waste is:
*  Any polygon or  tagging errors ( because we can't use this 
information, and need lot of space or processing resources )  or any 
from this: https://wiki.openstreetmap.org/wiki/Error_categories ;  etc
*  or any mapping errors ( bad street names ;  routing problems: waste 
for users   )
*  or any not "UpToDate" data/information  ( old phone numbers -   it 
is useless,  so it is waste )


some examples:
http://area.jochentopf.com/stats/
- Errors: Intersections
- Errors: Duplicate nodes
- Errors: Duplicate segments (*~160.000*)
- Errors: Open rings  ( ~9.000)
- Errors: Inner rings with same tags as outer rings
- Errors: Wrong role ( *~ 700.000 *)

some key problems:  ( unused/bad keys is a waste )
- https://taginfo.openstreetmap.org/reports/characters_in_keys#problem 
( Keys with possibly problematic characters )
- https://taginfo.openstreetmap.org/reports/characters_in_keys#space 
 ( Keys with whitespace )

- or my favorite:
--- https://taginfo.openstreetmap.org/keys/latitude#values
--- https://taginfo.openstreetmap.org/keys/LAT#values

And we have lot of low quality imports we should fix.

>What can I as a map editor

imho:
Any quality assurance work helps a lot: 
https://wiki.openstreetmap.org/wiki/Quality_assurance
so fixing data problems in your area helps "eliminating waste"    and  
less waste is good for data size



Imre



2018-01-18 6:14 GMT+01:00 Oleksiy Muzalyev 
>:


Good morning,

I started to experiment with the OSM data [1] on a local computer,
and I begin to realize how big these data files are. It takes
quite a while to load into the local database just the data for
one country.

What can I as a map editor do to keep these data files to a
reasonable size without compromising  data quality? I mean in the
sense, - take care of the pennies and the pounds will take care of
themselves?

I could think of the following three approaches so far:

- using as short an URL as possible,
website=http://somewebsite.com instead of
website=http://www.somewebsite.com  ,
three characters less; [2]

- correct phone number ISO format, phone=+12 345 678 90 12 instead
of phone=+12 (345) 678 90 12 , two characters less; [3]

- deleting unnecessary nodes from a way (Shift-Y in JOSM) with
consequent verification of its geometry;

What else, if anything, could be done?

[1] https://wiki.openstreetmap.org/wiki/Downloading_data

[2] https://wiki.openstreetmap.org/wiki/Key:website

[3] https://wiki.openstreetmap.org/wiki/Key:phone


With best regards,
Oleksiy
osm: Alex-7

___
talk mailing list
talk@openstreetmap.org 
https://lists.openstreetmap.org/listinfo/talk





___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] OSM data, how can we contribute to keep it to a reasonable size?

[Imre Samu]

>What can I as a map editor do to keep these data files to a reasonable
size without compromising  data quality?

According to the "Lean thinking" (
https://en.wikipedia.org/wiki/Lean_thinking ) we should focus on
" eliminating waste"
Waste is:
*  Any polygon or  tagging errors ( because we can't use this information,
and need lot of space or processing resources )   or any from this:
https://wiki.openstreetmap.org/wiki/Error_categories ;  etc
*  or any mapping errors ( bad street names ;  routing problems: waste for
users   )
*  or any not "UpToDate" data/information  ( old phone numbers -   it is
useless,  so it is waste )

some examples:
http://area.jochentopf.com/stats/
- Errors: Intersections
- Errors: Duplicate nodes
- Errors: Duplicate segments (* ~160.000*)
- Errors: Open rings  ( ~9.000)
- Errors: Inner rings with same tags as outer rings
- Errors: Wrong role (  *~ 700.000 *)

some key problems:  ( unused/bad keys is a waste )
- https://taginfo.openstreetmap.org/reports/characters_in_keys#problem
( Keys with possibly problematic characters )
- https://taginfo.openstreetmap.org/reports/characters_in_keys#space
 ( Keys with whitespace )
- or my favorite:
---  https://taginfo.openstreetmap.org/keys/latitude#values
---  https://taginfo.openstreetmap.org/keys/LAT#values

And we have lot of low quality imports we should fix.

>What can I as a map editor

imho:
Any quality assurance work helps a lot:
https://wiki.openstreetmap.org/wiki/Quality_assurance
so fixing data problems in your area helps "eliminating waste"and  less
waste is good for data size


Imre



2018-01-18 6:14 GMT+01:00 Oleksiy Muzalyev :

> Good morning,
>
> I started to experiment with the OSM data [1] on a local computer, and I
> begin to realize how big these data files are. It takes quite a while to
> load into the local database just the data for one country.
>
> What can I as a map editor do to keep these data files to a reasonable
> size without compromising  data quality? I mean in the sense, - take care
> of the pennies and the pounds will take care of themselves?
>
> I could think of the following three approaches so far:
>
> - using as short an URL as possible, website=http://somewebsite.com
> instead of website=http://www.somewebsite.com , three characters less; [2]
>
> - correct phone number ISO format, phone=+12 345 678 90 12 instead of
> phone=+12 (345) 678 90 12 , two characters less;  [3]
>
> - deleting unnecessary nodes from a way (Shift-Y in JOSM) with consequent
> verification of its geometry;
>
> What else, if anything, could be done?
>
> [1] https://wiki.openstreetmap.org/wiki/Downloading_data
> [2] https://wiki.openstreetmap.org/wiki/Key:website
> [3] https://wiki.openstreetmap.org/wiki/Key:phone
>
> With best regards,
> Oleksiy
> osm: Alex-7
>
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] OSM data, how can we contribute to keep it to a reasonable size?

[john whelan]

source=Bing ad source=bing aren't the same when it comes to data
compression.  So getting a little more consistency in the tags would help.
Not so much the .osm format but certainly the compressed formats.

Cheerio John

On 18 January 2018 at 06:28, Martin Koppenhoefer 
wrote:

> 2018-01-18 12:01 GMT+01:00 Oleksiy Muzalyev :
>
>>
>>
>> You make sound 1.7% as not much, however, in say civil aviation or
>> maritime industries, consistent 1.7% of fuel efficiency increase would be a
>> breakthrough.
>>
>> I do not suggest, speaking figuratively, not breathing to save air. On
>> the contrary, I think the mapping just begins, at least in some areas. So
>> there will be much more, perhaps, times more data.
>>
>> At the same time, it would not harm to be mindful of data size issue and
>> to follow simple best practices, as the shortest URL possible, the ISO
>> phone format, etc.
>
>
>
>
> do not forget that computers get more powerful very quickly, compared to
> airplane efficiency for example. Cost of RAM and Diskspace is constantly
> falling, as is raising transfer speed of our internet connections. Reducing
> detail to save on disk space isn't what I would recommend. With some
> patience, you can still import the whole planet on a 6 years old, lower end
> computer with slow HDD (takes approx. 2 weeks), if you need more speed,
> invest in RAM and an SSD RAID array (all not so expensive any more). For a
> country extract, don't expect it to import within a few minutes, but
> besides the biggest countries, a few hours should suffice.
>
> Cheers,
> Martin
>
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk
>
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] OSM data, how can we contribute to keep it to a reasonable size?

[Martin Koppenhoefer]

2018-01-18 12:01 GMT+01:00 Oleksiy Muzalyev :

>
>
> You make sound 1.7% as not much, however, in say civil aviation or
> maritime industries, consistent 1.7% of fuel efficiency increase would be a
> breakthrough.
>
> I do not suggest, speaking figuratively, not breathing to save air. On the
> contrary, I think the mapping just begins, at least in some areas. So there
> will be much more, perhaps, times more data.
>
> At the same time, it would not harm to be mindful of data size issue and
> to follow simple best practices, as the shortest URL possible, the ISO
> phone format, etc.




do not forget that computers get more powerful very quickly, compared to
airplane efficiency for example. Cost of RAM and Diskspace is constantly
falling, as is raising transfer speed of our internet connections. Reducing
detail to save on disk space isn't what I would recommend. With some
patience, you can still import the whole planet on a 6 years old, lower end
computer with slow HDD (takes approx. 2 weeks), if you need more speed,
invest in RAM and an SSD RAID array (all not so expensive any more). For a
country extract, don't expect it to import within a few minutes, but
besides the biggest countries, a few hours should suffice.

Cheers,
Martin
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] OSM data, how can we contribute to keep it to a reasonable size?

[Komяpa]

Hi,

Please don't try to save on keeping data in text format, this is a path to
removing source= tags and starting to use abbreviations in name for the
sake of file size, ignoring the fact that each object has much longer
iso8601 timestamp attached to it.

Use proper format, like .osm.pbf, that employs text strings compression,
binary representation of everything numeric and fixes the issue of 'spaces
around country code' and 'www.' by GZIPing the contents.

чт, 18 янв. 2018 г. в 14:03, Oleksiy Muzalyev :

> On 18.01.18 06:58, Mark Wagner wrote:
> > On Thu, 18 Jan 2018 06:14:17 +0100
> > Oleksiy Muzalyev  wrote:
> >
> >> Good morning,
> >>
> >> I started to experiment with the OSM data [1] on a local computer,
> >> and I begin to realize how big these data files are. It takes quite a
> >> while to load into the local database just the data for one country.
> >>
> >> What can I as a map editor do to keep these data files to a
> >> reasonable size without compromising  data quality? I mean in the
> >> sense, - take care of the pennies and the pounds will take care of
> >> themselves?
> >>
> >> I could think of the following three approaches so far:
> >>
> >> - using as short an URL as possible, website=http://somewebsite.com
> >> instead of website=http://www.somewebsite.com , three characters
> >> less; [2]
> >>
> >> - correct phone number ISO format, phone=+12 345 678 90 12 instead of
> >> phone=+12 (345) 678 90 12 , two characters less;  [3]
> >>
> >> - deleting unnecessary nodes from a way (Shift-Y in JOSM) with
> >> consequent verification of its geometry;
> >>
> >> What else, if anything, could be done?
> > Honestly, the best thing you can do is not worry about it.  The world
> > is a big, complicated place.  The three bytes you save from shortening
> > a URL?  You'd need to repeat it 400 times to fit one extra gas station
> > on the map.
> >
> > As an experiment, I tried applying various data-saving
> > options to a park I've mapped.  I managed to reduce the size from
> > 1240354 bytes to 1218966 bytes for a savings of 1.7% -- savings that
> > will vanish almost instantly this spring once I resume mapping
> > seasonal wetlands.
> >
> Mark,
>
> You make sound 1.7% as not much, however, in say civil aviation or
> maritime industries, consistent 1.7% of fuel efficiency increase would
> be a breakthrough.
>
> I do not suggest, speaking figuratively, not breathing to save air. On
> the contrary, I think the mapping just begins, at least in some areas.
> So there will be much more, perhaps, times more data.
>
> At the same time, it would not harm to be mindful of data size issue and
> to follow simple best practices, as the shortest URL possible, the ISO
> phone format, etc.
>
> Best regards,
> Oleksiy
> osm: Alex-7
>
>
>
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] OSM data, how can we contribute to keep it to a reasonable size?

[Oleksiy Muzalyev]

On 18.01.18 06:58, Mark Wagner wrote:

On Thu, 18 Jan 2018 06:14:17 +0100
Oleksiy Muzalyev  wrote:


Good morning,

I started to experiment with the OSM data [1] on a local computer,
and I begin to realize how big these data files are. It takes quite a
while to load into the local database just the data for one country.

What can I as a map editor do to keep these data files to a
reasonable size without compromising  data quality? I mean in the
sense, - take care of the pennies and the pounds will take care of
themselves?

I could think of the following three approaches so far:

- using as short an URL as possible, website=http://somewebsite.com
instead of website=http://www.somewebsite.com , three characters
less; [2]

- correct phone number ISO format, phone=+12 345 678 90 12 instead of
phone=+12 (345) 678 90 12 , two characters less;  [3]

- deleting unnecessary nodes from a way (Shift-Y in JOSM) with
consequent verification of its geometry;

What else, if anything, could be done?

Honestly, the best thing you can do is not worry about it.  The world
is a big, complicated place.  The three bytes you save from shortening
a URL?  You'd need to repeat it 400 times to fit one extra gas station
on the map.

As an experiment, I tried applying various data-saving
options to a park I've mapped.  I managed to reduce the size from
1240354 bytes to 1218966 bytes for a savings of 1.7% -- savings that
will vanish almost instantly this spring once I resume mapping
seasonal wetlands.


Mark,

You make sound 1.7% as not much, however, in say civil aviation or 
maritime industries, consistent 1.7% of fuel efficiency increase would 
be a breakthrough.


I do not suggest, speaking figuratively, not breathing to save air. On 
the contrary, I think the mapping just begins, at least in some areas. 
So there will be much more, perhaps, times more data.


At the same time, it would not harm to be mindful of data size issue and 
to follow simple best practices, as the shortest URL possible, the ISO 
phone format, etc.


Best regards,
Oleksiy
osm: Alex-7



___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] OSM data, how can we contribute to keep it to a reasonable size?

[Oleksiy Muzalyev]

On 18.01.18 10:01, Martin Koppenhoefer wrote:
2018-01-18 6:14 GMT+01:00 Oleksiy Muzalyev 
>:



- deleting unnecessary nodes from a way (Shift-Y in JOSM) with
consequent verification of its geometry;




please do not use SHIFT+y (simplify way / DouglasPeucker) because it 
doesn't know about the actual geometry, in particular sharp bends or 
curves in general will get worse, it will reduce overall accuracy 
(within the distance limit that you set/default, but ignoring angles). 
You will achieve better results by adjusting the ways manually. IMHO 
applying this algorithm to geometries other than those you added 
yourself is an automated edit and should generally be discouraged. 
There are some sensible usecases (serious "overnoding", particularly 
intermediate nodes in straight lines), but for every longer way (more 
than a screenfull in reasonable zoomlevel) you will typically not see 
all the effects, hence it's an automated edit.


Cheers,
Martin

Martin,

You are absolutely right. The Simplify way, Shift-Y, in JOSM, 
(Douglas-Peucker simplification) should be used only for own edits or 
for undeniable over-noding, i.e. a short direct line consisting from 
hundreds of nodes (probably from imports or kind of OCR editing).


Otherwise the lines and shapes will look angular.

I had to mention it myself. Thank you.

Best regards,
Oleksiy
osm: Alex-7


___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] OSM data, how can we contribute to keep it to a reasonable size?

[Martin Koppenhoefer]

2018-01-18 6:14 GMT+01:00 Oleksiy Muzalyev :

>
> - deleting unnecessary nodes from a way (Shift-Y in JOSM) with consequent
> verification of its geometry;




please do not use SHIFT+y (simplify way / DouglasPeucker) because it
doesn't know about the actual geometry, in particular sharp bends or curves
in general will get worse, it will reduce overall accuracy (within the
distance limit that you set/default, but ignoring angles). You will achieve
better results by adjusting the ways manually. IMHO applying this algorithm
to geometries other than those you added yourself is an automated edit and
should generally be discouraged. There are some sensible usecases (serious
"overnoding", particularly intermediate nodes in straight lines), but for
every longer way (more than a screenfull in reasonable zoomlevel) you will
typically not see all the effects, hence it's an automated edit.

Cheers,
Martin
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] OSM data, how can we contribute to keep it to a reasonable size?

[Mark Wagner]

On Thu, 18 Jan 2018 06:14:17 +0100
Oleksiy Muzalyev  wrote:

> Good morning,
> 
> I started to experiment with the OSM data [1] on a local computer,
> and I begin to realize how big these data files are. It takes quite a
> while to load into the local database just the data for one country.
> 
> What can I as a map editor do to keep these data files to a
> reasonable size without compromising  data quality? I mean in the
> sense, - take care of the pennies and the pounds will take care of
> themselves?
> 
> I could think of the following three approaches so far:
> 
> - using as short an URL as possible, website=http://somewebsite.com 
> instead of website=http://www.somewebsite.com , three characters
> less; [2]
> 
> - correct phone number ISO format, phone=+12 345 678 90 12 instead of 
> phone=+12 (345) 678 90 12 , two characters less;  [3]
> 
> - deleting unnecessary nodes from a way (Shift-Y in JOSM) with 
> consequent verification of its geometry;
> 
> What else, if anything, could be done?

Honestly, the best thing you can do is not worry about it.  The world
is a big, complicated place.  The three bytes you save from shortening
a URL?  You'd need to repeat it 400 times to fit one extra gas station
on the map.

As an experiment, I tried applying various data-saving
options to a park I've mapped.  I managed to reduce the size from
1240354 bytes to 1218966 bytes for a savings of 1.7% -- savings that
will vanish almost instantly this spring once I resume mapping
seasonal wetlands.

-- 
Mark

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

[OSM-talk] OSM data, how can we contribute to keep it to a reasonable size?

[Oleksiy Muzalyev]

Good morning,

I started to experiment with the OSM data [1] on a local computer, and I 
begin to realize how big these data files are. It takes quite a while to 
load into the local database just the data for one country.


What can I as a map editor do to keep these data files to a reasonable 
size without compromising  data quality? I mean in the sense, - take 
care of the pennies and the pounds will take care of themselves?


I could think of the following three approaches so far:

- using as short an URL as possible, website=http://somewebsite.com 
instead of website=http://www.somewebsite.com , three characters less; [2]


- correct phone number ISO format, phone=+12 345 678 90 12 instead of 
phone=+12 (345) 678 90 12 , two characters less;  [3]


- deleting unnecessary nodes from a way (Shift-Y in JOSM) with 
consequent verification of its geometry;


What else, if anything, could be done?

[1] https://wiki.openstreetmap.org/wiki/Downloading_data
[2] https://wiki.openstreetmap.org/wiki/Key:website
[3] https://wiki.openstreetmap.org/wiki/Key:phone

With best regards,
Oleksiy
osm: Alex-7

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk