Re: [datameet] How does OSM store its data

[Nikhil VJ]

Thanks Devdatta and Sajjad,

I did send the query to osm list too. But my follow-up queries will be more
about data management than maps.

Momentarily keeping maps in mind, has anybody worked on Cartaro in Drupal?
http://cartaro.org/


NoSQL/Object Store database :
It seems MongoDB is a big name in the NoSQL world. Anyone who's worked on
it?

Is it possible to implement this in a CMS ?

I spotted this in my bigrock web hosting PHP setting, so getting hopes that
I can get something running there: screenshot



I'll lay out the use case scenario:

1. I log in.
2. I click to create a new item (analogous to a node in osm)
3. I enter a key and then a value. Dropdowns come as i type to suggest keys
that others have entered and values that others have entered for that key.
4. I press a plus button to add another key and value.




5. I also have a way of making a 'relational' key and choosing a
pre-existing item.
6. When i'm done, I click a button to publish my item.

7. I see a listing of many such items published.
8. I build and run a query like its done in Overpass Turbo.




9. The query returns results not just from data I have submitted, but from
data submitted by others on the platform too. I download the results I get
in Json, XML, CSV or other formats.

10. I can click on one of the items to go into it.
11. There, i can enter a 'suggestions' mode and add new key-value pairs. If
i have the permission, i can propose changes to existing ones or mark some
for deletion.
12. I can click to submit my proposed changes and exit to the main listing.


Where I want to apply this : Towards open data gathering of course! If
anyone's interested in exploring this together, please mail in.

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [datameet] How does OSM store its data

[Sajjad Anwar]

Hey Nikhil,

OSM uses a PostgreSQL database as the main data store, along with PostGIS
extension for geometries. The tags are stored in key and value columns and
uses hstore, which makes it easy to add new ones without having to modify
the table schema. You can read about the database schema and infrastructure
here https://wiki.openstreetmap.org/wiki/Database

Overpass is independent of the above infrastructure. It reads from the
replication files and builds a binary on-disk database. More about Overpass
here https://wiki.openstreetmap.org/wiki/Overpass_API

Also, you should join the OSM Talk-in mailing list if you haven't!
https://lists.openstreetmap.org/listinfo/talk-in

Sajjad

On Mon, Feb 5, 2018 at 11:12 AM, Nikhil VJ  wrote:

> Hi,
>
> I want to learn how OpenStreetmap stores and manages its data, plus how
> Overpass Turbo extracts it.
>
> I find the key-value pair system fascinating (variable fields, organic
> building up of data) and want to replicate that elsewhere. Just the data
> storing part.. don't want to generate map tiles etc.
>
> Any pointers? Any place I should ask?
>
> For those interested, here's a link to the project I have in mind:
> Crowdsourcing-Map-based-data
> 
>
> --
> Cheers,
> Nikhil VJ
> Pune, India
>
> --
> Datameet is a community of Data Science enthusiasts in India. Know more
> about us by visiting http://datameet.org
> ---
> You received this message because you are subscribed to the Google Groups
> "datameet" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to datameet+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [datameet] How does OSM store its data

[Devdatta Tengshe]

Hi Nikhil,

The Database design has been documented here:
https://wiki.openstreetmap.org/wiki/Database

This is, at its essence, a modified EAV Model(
https://en.wikipedia.org/wiki/Entity–attribute–value_model). A good place
to start researching for these kinds of database design, is to look at
various documents on EAV, its shortcomings & Strengths.

You should also evaluate if a NoSQL/Object Store database might be a better
fit for your purpose, than designing an EAV Data model to be stored in a
relational Database.


Regards,
Devdatta

On Mon, Feb 5, 2018 at 11:12 AM, Nikhil VJ  wrote:

> Hi,
>
> I want to learn how OpenStreetmap stores and manages its data, plus how
> Overpass Turbo extracts it.
>
> I find the key-value pair system fascinating (variable fields, organic
> building up of data) and want to replicate that elsewhere. Just the data
> storing part.. don't want to generate map tiles etc.
>
> Any pointers? Any place I should ask?
>
> For those interested, here's a link to the project I have in mind:
> Crowdsourcing-Map-based-data
> 
>
> --
> Cheers,
> Nikhil VJ
> Pune, India
>
> --
> Datameet is a community of Data Science enthusiasts in India. Know more
> about us by visiting http://datameet.org
> ---
> You received this message because you are subscribed to the Google Groups
> "datameet" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to datameet+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[datameet] How does OSM store its data

[Nikhil VJ]

Hi,

I want to learn how OpenStreetmap stores and manages its data, plus how
Overpass Turbo extracts it.

I find the key-value pair system fascinating (variable fields, organic
building up of data) and want to replicate that elsewhere. Just the data
storing part.. don't want to generate map tiles etc.

Any pointers? Any place I should ask?

For those interested, here's a link to the project I have in mind:
Crowdsourcing-Map-based-data


--
Cheers,
Nikhil VJ
Pune, India

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.