Remi, sounds like you're close to finding a pathway that matches your needs.
there are always several ways to solve every problem. It seems yours is about
storage and fast access to the bits that are relevant. You mention circa 1
thousand tiles.
The only reason I'm pursuing this topic is because I'm sensing a broader
solution to an emerging set of challenges. Particularly with the "internet of
things", where we will collect seriously BIG spatio-temporal data - eg from
sensors and we can't afford to pull everything into PostGIS.
iOT requires solutions that can deal with datasets many orders of magnitude
larger, but can also scale down to suit small projects.
> using Big Data /noSQL solutions to do some of the simple heavy lifting
> mongo db has got spatial capabilities And you can use the mongo FDW to return
> query results into PG.
Mongo db when I last looked at this only has 2d spatial indexing. Not ideal,
but still useful for point cloud storage... I also think the attached article
is handy because the author lays out the business problem and how he's tackled
it.
http://rs.tudelft.nl/~rlindenbergh/workshop/BoehmIQmulus.pdf
I'm not advocating that people jump on the NoSQL bandwagon - I love Postgis and
prefer to do as much of my work in postgis - but it's good to know each DBs
respective strengths and why..
IMHO, the quicker we become conversant inPolyglot DB design, the better.
If we think every problem is a nail and therefore postgis is the hammer /
answer, then innovation ( which is about using and combining existing
technologies in new ways) will leave us behind.
Ok, time to park NoSQL and apply some left field thinking..
http://rs.tudelft.nl/~rlindenbergh/workshop/BoehmIQmulus.pdf
I'm not advocating that people jump on the NoSQL bandwagon - I love Postgis and
prefer to do as much of my work in postgis - but it's good to know each DBs
respective strengths and why..
> IMHO, the quicker we become conversant inPolyglot DB design, the better.
>
> If we think every problem is a nail and therefore postgis is the hammer /
> answer, then innovation ( which is about using and combining existing
> technologies in new ways) will leave us behind.
Ok, time to park NoSQL and apply some left field thinking..
take one of Paul's recent innovations...
https://github.com/pramsey/pgsql-ogr-fdw
What if we could read straight from file... Or "document" into PG...? And I'm
not talking every document - just the ones relevant to us... On an as needs
basis.
This is what the mongo_fdw does... In fact we can write back too.
FDW's go to heart of polyglot design.
The common theme for IOT is the need to leverage a cheap distributed file /
document storage system to enable fast search and retrieval... With a basic
level of spatial awareness that can inform and feed downstream applications
incl, PostGIS.
The idea is that stuff not needed or of no business value downstream gets left
behind on the cheapest h/w, s/w before discarding.
I'm keen to hear from anyone about how they are using postgis in combination
with NoSQL DBs for their big data pipelines ?
Or if there's another mailing list that covers spatial IT / spatial - temporal
big data processing, I'd love to hear about it.
Thanks
_______________________________________________
postgis-users mailing list
[email protected]
http://lists.osgeo.org/cgi-bin/mailman/listinfo/postgis-users
