Re: [OSM-dev] Static maps from “pafciu17” no longer work?

[Michal Migurski]

Wow, the whole Zend framework is in there.

What would be a better way to implement this to let it run for the foreseeable 
future? Python + Gunicorn?

It's something I use pretty regularly and would be happy to bring up to date, 
though perhaps just a portion of the API at first.

-mike.

On Feb 8, 2013, at 6:22 PM, Grant Slater wrote:

> I've restored it for the moment, but I really want to see it replaced
> with something better. With a few scrapers calling complex maps it
> easily overloads the dev server:
> http://wiki.osm.org/wiki/Servers/errol
> 
> The code is here: https://code.google.com/p/gsoc-os-static-maps-api/
> but suspect it is a little out of date.
> It was abandoned around 2009/2010.
> 
> / Grant
> 
> On 9 February 2013 01:47, Ian Dees  wrote:
>> Grant turned it off today because it causes a very spikey and high load on
>> errol (the dev server).
>> 
>> The code could probably be cleaned up and hosted elsewhere once it was
>> hardened against scrapers and high load.
>> 
>> 
>> On Fri, Feb 8, 2013 at 7:39 PM, Michal Migurski  wrote:
>>> 
>>> Hi,
>>> 
>>> I'm curious why this no longer returns static maps:
>>>http://pafciu17.dev.openstreetmap.org/
>>> 
>>> According to the linked wiki page, it should still be up:
>>>http://wiki.openstreetmap.org/wiki/Static_map_images
>>> 
>>> -mike.
>>> 
>>> 
>>> michal migurski- contact info and pgp key:
>>> sf/cahttp://mike.teczno.com/contact.html
>>> 
>>> 
>>> 
>>> 
>>> 
>>> ___
>>> dev mailing list
>>> dev@openstreetmap.org
>>> http://lists.openstreetmap.org/listinfo/dev
>> 
>> 
>> 
>> ___
>> dev mailing list
>> dev@openstreetmap.org
>> http://lists.openstreetmap.org/listinfo/dev
>> 
> 


michal migurski- contact info and pgp key:
sf/cahttp://mike.teczno.com/contact.html





___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Static maps from “pafciu17” no longer work?

[Grant Slater]

I've restored it for the moment, but I really want to see it replaced
with something better. With a few scrapers calling complex maps it
easily overloads the dev server:
http://wiki.osm.org/wiki/Servers/errol

The code is here: https://code.google.com/p/gsoc-os-static-maps-api/
but suspect it is a little out of date.
It was abandoned around 2009/2010.

/ Grant

On 9 February 2013 01:47, Ian Dees  wrote:
> Grant turned it off today because it causes a very spikey and high load on
> errol (the dev server).
>
> The code could probably be cleaned up and hosted elsewhere once it was
> hardened against scrapers and high load.
>
>
> On Fri, Feb 8, 2013 at 7:39 PM, Michal Migurski  wrote:
>>
>> Hi,
>>
>> I'm curious why this no longer returns static maps:
>> http://pafciu17.dev.openstreetmap.org/
>>
>> According to the linked wiki page, it should still be up:
>> http://wiki.openstreetmap.org/wiki/Static_map_images
>>
>> -mike.
>>
>> 
>> michal migurski- contact info and pgp key:
>> sf/cahttp://mike.teczno.com/contact.html
>>
>>
>>
>>
>>
>> ___
>> dev mailing list
>> dev@openstreetmap.org
>> http://lists.openstreetmap.org/listinfo/dev
>
>
>
> ___
> dev mailing list
> dev@openstreetmap.org
> http://lists.openstreetmap.org/listinfo/dev
>

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Static maps from “pafciu17” no longer work?

[Michal Migurski]

Well that's a total bummer.

Where does the code live? Could it be protected from high load and scrapers at 
the Nginx/Apache/CDN level?

-mike.

On Feb 8, 2013, at 5:47 PM, Ian Dees wrote:

> Grant turned it off today because it causes a very spikey and high load on 
> errol (the dev server).
> 
> The code could probably be cleaned up and hosted elsewhere once it was 
> hardened against scrapers and high load.
> 
> 
> On Fri, Feb 8, 2013 at 7:39 PM, Michal Migurski  wrote:
> Hi,
> 
> I'm curious why this no longer returns static maps:
> http://pafciu17.dev.openstreetmap.org/
> 
> According to the linked wiki page, it should still be up:
> http://wiki.openstreetmap.org/wiki/Static_map_images
> 
> -mike.
> 
> 
> michal migurski- contact info and pgp key:
> sf/cahttp://mike.teczno.com/contact.html
> 
> 
> 
> 
> 
> ___
> dev mailing list
> dev@openstreetmap.org
> http://lists.openstreetmap.org/listinfo/dev
> 


michal migurski- contact info and pgp key:
sf/cahttp://mike.teczno.com/contact.html





___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Static maps from “pafciu17” no longer work?

[Ian Dees]

Grant turned it off today because it causes a very spikey and high load on
errol (the dev server).

The code could probably be cleaned up and hosted elsewhere once it was
hardened against scrapers and high load.


On Fri, Feb 8, 2013 at 7:39 PM, Michal Migurski  wrote:

> Hi,
>
> I'm curious why this no longer returns static maps:
> http://pafciu17.dev.openstreetmap.org/
>
> According to the linked wiki page, it should still be up:
> http://wiki.openstreetmap.org/wiki/Static_map_images
>
> -mike.
>
> 
> michal migurski- contact info and pgp key:
> sf/cahttp://mike.teczno.com/contact.html
>
>
>
>
>
> ___
> dev mailing list
> dev@openstreetmap.org
> http://lists.openstreetmap.org/listinfo/dev
>
___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

[OSM-dev] Static maps from “pafciu17” no longer work?

[Michal Migurski]

Hi,

I'm curious why this no longer returns static maps:
http://pafciu17.dev.openstreetmap.org/

According to the linked wiki page, it should still be up:
http://wiki.openstreetmap.org/wiki/Static_map_images

-mike.


michal migurski- contact info and pgp key:
sf/cahttp://mike.teczno.com/contact.html





___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Reminder: Node 32-bit exhaustion

[Dominik Perpeet]

A while ago I announced that I have newly compiled windows versions of
osm2pgsql (Intel C++ Compiler XE 13.0, MS Visual Studio 2010). After my
last build someone added the no_fork option (thanks!) which made my life
easier. Compared to the master version,  I basically only needed to
remove the flat-nodes option due to file-sync issues. I guess it could
be made to work, but I didn't want to invest the time. Remember that
windows has no fork, therefore this binary version will run sequentially
only.
All other build options should be at their default values.

I still haven't had the time to properly document what I did. Since
32bit seems to be a bit of an issue though, I've decided to share the
binaries anyway. You can get them at

http://customdebug.com/osm/osm2pgsql.zip

I didn't have the time to properly test them, but I've used previous
builds for full planet imports and rendering maps with mapnik (as well
as some postgis queries for other things). The zip file includes 32 and
64 bit versions of 32 and 64 bit node id each (yes, four binaries). I've
included all dependencies as well as I could. Please mind the licenses!

Please test before using in any kind of production manner (seriously,
who wouldn't?). In my environment I've only really used non-slim and
in-memory modes with osm-planet, since I don't have access to a large
enough ssd space (ironically more ram than ssd).

If people are interested, I can share the entire chain: script to create
and prepare the postgis db, import planet and a small qt app to render
tiles using mapnik (2.0 currently)

Feedback and bug reports welcome. I will share the source and build
files soon.

-Dominik


my "release notes":
--
Compiled by Dominik Perpeet
Feb 06 2013

git rev af61cae663 Dec 17, 2012

no fork (not supported in windows)
no flatnodes support (node-persistent-cache)

64 bit node id: osm2pgsql.exe
32 bit node id: osm2pgsql_32bitId.exe

Please keep in mind licenses of all parts:

osm2pgsql (http://wiki.openstreetmap.org/wiki/Osm2pgsql)

versions of libraries:
bzip2 1.0.6
geos 3.3.5
libxml 2.9.0
pgsql 9.2.1
proj 4.8.0
proto_buf_c 0.1.5 (2.4.1)
zlib 1.2.7
--

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Reminder: Node 32-bit exhaustion

[Stephan Knauss]

On 08.02.2013 06:59, Ilya Zverev wrote:

Though I fail to understand what would be broken if you continue to use
32-bit osm2pgsql. Indices, maybe.


don't be confused by the term 32-bit. It is required that the software 
can handle 64-bit data types for processing the node id. In postgresql 
it will be stored in a column declared as BIGINT.


Data types often are signed. So one of the 32 bits is used to store the 
sign. The remaining 31 bits to store the ID are exhausted now.


Software can be compiled as a 32-bit or 64-bit executable. If you can, 
you should use a 64-bit executable as this can access over 4GB of 
memory, giving you the chance to speed up the import a lot.



What happens when you continue to use osm2pgsql with 32 bit ids (int4) 
instead of int8?


Have a look at osmtypes.h. You'll see that osmid_t is declared as 
int32_t which is signed:

typedef signed long int int32_t

So the value will overflow. In the best case osm2pgsql will abort with 
an error. In the worst case your database content will be corrupted. 
Your database structure still is only int4, again signed. So it all 
needs to be converted into int8.


Probably easiest is to re-import into a fresh DB with 64-bit IDs. Most 
people have done this step during the re-import after license change.


Stephan



___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Latest full-history planet

[Michal Migurski]

Great!

How often do you think the full history might be recreated?

-mike.

On Feb 8, 2013, at 10:25 AM, Matt Amos wrote:

> now available from:
> 
> http://planet.openstreetmap.org/planet/full-history/
> 
> get it while it's hot :-) and please let me know if you find any errors with 
> it.
> 
> cheers,
> 
> matt
> 
> ___
> dev mailing list
> dev@openstreetmap.org
> http://lists.openstreetmap.org/listinfo/dev
> 


michal migurski- contact info and pgp key:
sf/cahttp://mike.teczno.com/contact.html





___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Latest full-history planet

[Paweł Paprota]

Great, thanks! Been waiting for that bugger this week!

On Fri, Feb 8, 2013, at 19:25, Matt Amos wrote:
> now available from:
> 
> http://planet.openstreetmap.org/planet/full-history/
> 
> get it while it's hot :-) and please let me know if you find any errors
> with it.
> 
> cheers,
> 
> matt
> 
> ___
> dev mailing list
> dev@openstreetmap.org
> http://lists.openstreetmap.org/listinfo/dev

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

[OSM-dev] Latest full-history planet

[Matt Amos]

now available from:

http://planet.openstreetmap.org/planet/full-history/

get it while it's hot :-) and please let me know if you find any errors with it.

cheers,

matt

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Is there any API to get data from OSM as geoJson?

[Paweł Paprota]

Hi Ander,

Probably not exactly what you are looking for but at least it's OSM data 
in GeoJSON format :-) - OwL API serves GeoJSON for changeset data:


http://owl.osm.org/api/0.1/changesets/15/18076/11129/18080/11131.geojson

In the future (whether near or far, hard to say) there should be 
additional API for serving "vector tiles" worldwide - that would 
possibly power browser-side map rendering (KothicJS and such).


Paweł

On 02/07/2013 05:58 PM, Ander Pijoan wrote:

Hi everyone

I am currently working with different technologies for showing OSM data
in slippy maps and I noticed that some of those maps would work better
with GeoJSON data format.

I use Overpass API for downloading data but it doesn't export in GeoJSON
format. The nearest approach would be to ask for it as OSM-JSON, store
all the nodes ids and locations, store all the ways ids and node refs
and then build the relations geometries but it's quite a big pain for
browsers.

I've been looking for some info in OSM wiki but I haven't been able to
find nothing. Does anyone know if there exists something?

Thank you very much.

--
Ander Pijoan Lamas
Research Assistant, Deustotech
Computer Science Engineer
University of Deusto

E-mail: ander.pij...@deusto.es 
Phone: +34 664471228
in: http://www.linkedin.com/profile/view?id=162888312


___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev




___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Reminder: Node 32-bit exhaustion

[Sven Geggus]

Andrew M. Bishop  wrote:

> If using the "standard" toolchain of osm2psql, postgresql and mapnik
> what is the minimum software versions that are needed to continue
> creating maps after ids reach 2^31-1?

osm2pgsql supports 64bit id space already for a long time. But since this is
a build-time option the default has been changed to 64bit id space not that
long ago )see svn log for exact date).

As far as mapnik is concerned 2.1 did already work well using osm2psql in 64
bit id mode.

Sven

-- 
"Whenever there is a conflict between human rights and property
rights, human rights must prevail." (Abraham Lincoln)

/me is giggls@ircnet, http://sven.gegg.us/ on the Web

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev