Re: [postgis-users] Partitionning using geometry

2015-04-04 Thread Rémi Cura
Hey, I explore the possibility to store thousands of Billions of points into postgres (without partitionning, 10's billion, with partitionning, 1000's billions). MongoDB is actively researched for this [1] and [2], with exciting perspective, and several downfall as well. I think you may mixing

Re: [postgis-users] Partitionning using geometry

2015-04-03 Thread Rémi Cura
Hey, thanks for the answer Mark (I saw your pg_routing mail, I never used pg_routing, so I asked a co_worker and I'm waiting his answer)! What I want to achieve is slightly different. It is not a processing issue (one big table, cutting it in pieces to process it faster trough pl/R, pl/python,

Re: [postgis-users] Partitionning using geometry

2015-04-03 Thread Arnaud Lesauvage
Hi Rémi, have you taken a look at ST_GeoHash ? It might be a good candidate for your constraint exclusion check. There are restrictions though. Coordinates must be in lat/lon for instance. http://postgis.net/docs/ST_GeoHash.html Arnaud Le 3/04/2015 10:49, Rémi Cura a écrit : Now postgres

Re: [postgis-users] Partitionning using geometry

2015-04-03 Thread Rémi Cura
Hey Arnaud, thanks for this good idea. I hadn't thought of that! I really like the concept of reading more or less data to get more or less precision, I used it for point cloud Level of Details. Indeed it could be used in partiton check. We would still have the limit that the geohash used to

Re: [postgis-users] Partitionning using geometry

2015-04-03 Thread Mark Wynter
I always keep open mind. I hate marketing hype of noSQL vendors, plus GIS vendors too! But when a colleague who is one of leading data engineers in Visual Effects (vfx) industry combines mongo db and postgis, then I take note. mongo db has got spatial capabilities And you can use the mongo

Re: [postgis-users] Partitionning using geometry

2015-04-03 Thread Mark Wynter
Remi, sounds like you're close to finding a pathway that matches your needs. there are always several ways to solve every problem. It seems yours is about storage and fast access to the bits that are relevant. You mention circa 1 thousand tiles. The only reason I'm pursuing this topic is

Re: [postgis-users] Partitionning using geometry

2015-04-02 Thread Mark Wynter
Hi Remi I might be off the mark with what you are trying to achieve. One thing I've experimented with, which is allied to vector tiling, is to assign tile IDs to features, based on various spatial relationships, and to use the tile id to index and subset the tables prior to doing other stuff.