On 23/11/20 07:39, Lars Aksel Opsahl wrote:
Hi At NIBIO we have got Postgis Topology to perform quite ok with more than 25 million edges that represents land, water, roads, field types and more. We use topology.TopoGeo_addLinestring. To get this to work we had to use content based grids (https://github.com/larsop/content_balanced_grid) and work inside each grid until each single cell are done and then start to merge cells together and the end. The process of merging cells is more time consuming related to too each edge, but the number of edges are also limited because we only have to work with edges that cross cell borders. Using content based grids has advantages like * You can safely wOn 23/11/20 07:39, Lars Aksel Opsahl wrote:ork in parallel * It performance good when building up new large topology datasets. In the case below we have more than 25 million edges that we split up into around 7000 cells and we see the number off cells handled pr hour below are not decreasing, when running 20 threads in parallel. (The number of edges are not equal pr cell, but limited to a max number of polygons pr. cell so the idea is to vary the size of cell to get the workload pr cell more equal) 852 1113 840 563 461 541 583 704 705 Before we started to use content based grids we had the same problems as you describe here, the performance decreased when starting to work with big datasets. You find the code I used here https://github.com/larsop/resolve-overlap-and-gap <https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Flarsop%2Fresolve-overlap-and-gap&data=04%7C01%7Camsilva%40infoportugal.impresa.pt%7C259b72e4e3de463a867608d88f82f7ec%7Cd227b2e71c404f63b5132f3665c334e6%7C0%7C0%7C637417139959964915%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=JLx0AJGgL%2FkcRCbngapI4musohbE6s32M0kOQgaUqVc%3D&reserved=0> if you want more info. Lars _______________________________________________ postgis-users mailing list postgis-users@lists.osgeo.org https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.osgeo.org%2Fmailman%2Flistinfo%2Fpostgis-users&data=04%7C01%7Camsilva%40infoportugal.impresa.pt%7C259b72e4e3de463a867608d88f82f7ec%7Cd227b2e71c404f63b5132f3665c334e6%7C0%7C0%7C637417139959984905%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=giHwlsDnMLC7q4NmPZjM%2FCnbFLaW5Xl0ZPKD3BaBCtw%3D&reserved=0
Hi and thanks for your input, I saw your foss4g presentation and your approach was one that I was considering use, but I hadn't find the source code yet outside of the slides. Do the repo approach have the same ruleset as the presentation? I have the need of processing only cells when there is no line falling outside that cell or the ordering may not be respected, so the cell merging is also a point that I have to ensure. Thanks, Alexandre Silva [http://newsletter.impresapublishing.pt/i/barra_ip.jpg] _______________________________________________ postgis-users mailing list postgis-users@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/postgis-users
