Re: [OSGeo-Discuss] any spatial databases for high performance geo-computing
Hi Shuai, Everyone It's worth a look at a few projects @ LocationTech too. There's a nice community growing around these. GeoTrellis http://www.locationtech.org/projects/technology.geotrellis, Apache v2 License, Intro Article http://www.eclipse.org/community/eclipse_newsletter/2014/march/article4.php, Video http://youtu.be/aS8BAmu9daU, Discussion list https://www.locationtech.org/mailman/listinfo/geotrellis-dev GeoMesa http://www.locationtech.org/projects/technology.geomesa, Apache v2 License, Intro Article http://www.eclipse.org/community/eclipse_newsletter/2014/march/article3.php, Video http://youtu.be/JsQiOuGGWds, Discussion list https://www.locationtech.org/mailman/listinfo/geomesa-users GeoJinni http://www.locationtech.org/proposals/geojinni (formerly Spatial Hadoop), Video http://www.youtube.com/watch?v=A3Q2XlGvYcUfeature=share, discussion list coming soon Glad to help connect anyone who might be interested. Kind regards, Andrew On 19/05/14 18:04, Zhang, Shuai wrote: Hi All, sorry for asking, but what do you think is a good choice of spatial database for high performance geo-computing? In some high performance computing scenarios, data size tends to be huge, and a bunch of computer clusters work together with high throughput and tense computation. sometimes we use parallel filesystems like lustre, gfs, hdfs to handle specific problems but what if a spatial database? I explored some of postgresql cluster solutions, such as streaming replication, pgpool, slony and so on. I think most of them are designed for failover, and they might not be able to stand up with the huge data size and high performance demands. the case is quite alike in oracle and db2 spatial, i think. so any suggestions for projects aiming to build a distributed and parallel spatial database running on a cluster? Thanks, shuai ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] any spatial databases for high performance geo-computing
you might look at rasdaman running on top of PostgreSQL (www.rasdaman.org). Supports n-D arrays + an SQL-style array query language. -Peter On 05/20/2014 12:04 AM, Zhang, Shuai wrote: Hi All, sorry for asking, but what do you think is a good choice of spatial database for high performance geo-computing? In some high performance computing scenarios, data size tends to be huge, and a bunch of computer clusters work together with high throughput and tense computation. sometimes we use parallel filesystems like lustre, gfs, hdfs to handle specific problems but what if a spatial database? I explored some of postgresql cluster solutions, such as streaming replication, pgpool, slony and so on. I think most of them are designed for failover, and they might not be able to stand up with the huge data size and high performance demands. the case is quite alike in oracle and db2 spatial, i think. so any suggestions for projects aiming to build a distributed and parallel spatial database running on a cluster? Thanks, shuai ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss -- Dr. Peter Baumann - Professor of Computer Science, Jacobs University Bremen www.faculty.jacobs-university.de/pbaumann mail: p.baum...@jacobs-university.de tel: +49-421-200-3178, fax: +49-421-200-493178 - Executive Director, rasdaman GmbH Bremen (HRB 26793) www.rasdaman.com, mail: baum...@rasdaman.com tel: 0800-rasdaman, fax: 0800-rasdafax, mobile: +49-173-5837882 Si forte in alienas manus oberraverit hec peregrina epistola incertis ventis dimissa, sed Deo commendata, precamur ut ei reddatur cui soli destinata, nec preripiat quisquam non sibi parata. (mail disclaimer, AD 1083) ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
[OSGeo-Discuss] any spatial databases for high performance geo-computing
Hi All, sorry for asking, but what do you think is a good choice of spatial database for high performance geo-computing? In some high performance computing scenarios, data size tends to be huge, and a bunch of computer clusters work together with high throughput and tense computation. sometimes we use parallel filesystems like lustre, gfs, hdfs to handle specific problems but what if a spatial database? I explored some of postgresql cluster solutions, such as streaming replication, pgpool, slony and so on. I think most of them are designed for failover, and they might not be able to stand up with the huge data size and high performance demands. the case is quite alike in oracle and db2 spatial, i think. so any suggestions for projects aiming to build a distributed and parallel spatial database running on a cluster? Thanks, shuai ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] any spatial databases for high performance geo-computing
On 05/19/2014 03:04 PM, Zhang, Shuai wrote: Hi All, sorry for asking, but what do you think is a good choice of spatial database for high performance geo-computing? In some high performance computing scenarios, data size tends to be huge, and a bunch of computer clusters work together with high throughput and tense computation. sometimes we use parallel filesystems like lustre, gfs, hdfs to handle specific problems but what if a spatial database? I explored some of postgresql cluster solutions, such as streaming replication, pgpool, slony and so on. I think most of them are designed for failover, and they might not be able to stand up with the huge data size and high performance demands. the case is quite alike in oracle and db2 spatial, i think. so any suggestions for projects aiming to build a distributed and parallel spatial database running on a cluster? Thanks, shuai I haven't used but have a seen a few Hadoop implementations. If you do research on Sharding that's the kind of db where the data is split across nodes not redundantly. http://www.nathankerr.com/projects/parallel-gis-processing/alternative_approaches_to_parallel_gis_processing.html http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3814183/ Then of course depending on your needs there are plenty of MPI compatible libraries in various languages. Thanks, Alex ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] any spatial databases for high performance geo-computing
Tools like Hadoop, Neteeza Teradata support very large databases are fully spatially aware, generally through custom re-implementations of FOSS tools like GDAL. http://www10.giscafe.com/nbc/articles/view_article.php?articleid=590803 http://www.teradata.com.au/products-and-services/teradata-geospatial or in the FOSS arena: http://spatialhadoop.cs.umn.edu/ Brent Wood On 05/19/2014 03:04 PM, Zhang, Shuai wrote: Hi All, sorry for asking, but what do you think is a good choice of spatial database for high performance geo-computing? In some high performance computing scenarios, data size tends to be huge, and a bunch of computer clusters work together with high throughput and tense computation. sometimes we use parallel filesystems like lustre, gfs, hdfs to handle specific problems but what if a spatial database? I explored some of postgresql cluster solutions, such as streaming replication, pgpool, slony and so on. I think most of them are designed for failover, and they might not be able to stand up with the huge data size and high performance demands. the case is quite alike in oracle and db2 spatial, i think. so any suggestions for projects aiming to build a distributed and parallel spatial database running on a cluster? Thanks, shuai___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] any spatial databases for high performance geo-computing
Two options for distributed spatial databasing, based on Solr's spatial types: ElasticSearch: http://www.elasticsearchtutorial.com/spatial-search-tutorial.html Riak: http://www.christopherbiscardi.com/2014/02/07/geospatial-indexing-with-riak-search-2-0-yokozunasolr/ I've had great success with ElasticSearch and spatial search, but not in a clustered configuration. Riak is designed from the ground up to be a distributed database, but its Solr support is quite new and still in beta. Neither one support particularly complex geometric operations AFAIK, so YMMV. Good luck! SDE On May 19, 2014, at 3:04 PM, Zhang, Shuai sh...@illinois.edu wrote: Hi All, sorry for asking, but what do you think is a good choice of spatial database for high performance geo-computing? In some high performance computing scenarios, data size tends to be huge, and a bunch of computer clusters work together with high throughput and tense computation. sometimes we use parallel filesystems like lustre, gfs, hdfs to handle specific problems but what if a spatial database? I explored some of postgresql cluster solutions, such as streaming replication, pgpool, slony and so on. I think most of them are designed for failover, and they might not be able to stand up with the huge data size and high performance demands. the case is quite alike in oracle and db2 spatial, i think. so any suggestions for projects aiming to build a distributed and parallel spatial database running on a cluster? Thanks, shuai ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
