On Tue, 21 Mar 2006 00:06:12 +1100, Shoaib Burq <[EMAIL PROTECTED]> wrote:
>Hi > >I have a question which I am have difficulty putting in words but I'll >try anyway... > >Can anyone suggest how large data vendors organise their raster and >vector data with the view of having the flexibility to update, >reorganise, manage and search/query. How can this process be automated >using the various OS GIS components (e.g. PostGIS, Mapserver, etc). >Are data sets divide into categories. How does one deal with temporal >variability over the same extents? Is their an ideal way of organising >the data on disk? How should the various mapserver components like >Mapfiles be organised internally and on disk? Understandably it could >get very ugly as data increases. At Bureau of Regional Sciences we are currently building a site for drought assessment which involved a large number of data sets. Data sets are updated at different intervals depending on the type of data and are stored in PostGIS and on the file system depending on the type and putpose of the data. We have ended up building an application database (as separate the the data sets) to drive the many layers and the time series data which has given us alot of flexibility on where and how we store the data. A simple database structure that links products->layers->datasets means that we can have many datasets that make up a layer and can also give us the location of the dataset at runtime so we can alter the data acording to date parameters entered by the user. This format also means that we can use the dataset table as a reference to the actual datasets stored in the database and how to perform data queries on them without knowing in advance what columns they have. We use the same table for the file system. Each time we get a new dataset then a new entry goes in the dataset table with a relative path to the data on the file system or a reference to the data in a table. The categorisation has been treated as a 1->many link to the layers table as we use categories to help display the layers in some sort of meanful context but that is all it does. Cheers, Simon Haddon >[snipped] > >Then having some way of mapping their spatial locations and locations >on disk along with metadata info into PostGIS tables. > >I'd like to get some feedback from anyone who has had similar experience > >thanks in advance >shoaib
