Re: [OSGeo-Discuss] Image Management in an RDBMS...(was OS Spatial environment 'sizing')
[EMAIL PROTECTED] wrote: IMO: [...] I am probably too vector oriented to understand the problems involved here but my experience is that there should be no issue if you have your services configured all right. I don't agree. But then my requirements are for spatial data that covers a large geographic area and is used by many people in a corporate environment. SOA, while it has definite advantages does not provide all of the answers. Just for the records. I said that services are cool for all kinds of things, I did *not* say that SOA is *the* solution. Background: The term SOA is currently being (mis)used by minor IT corporations (IBM, SUN, Microsoft, SAP, Oracle and the like) to enhance their very private vendor-lock-in strategies. Therefore beware whenever someone says SOA and try to question their motivation. Regards, Arnulf. (c) by Arnulf's Corporate Paranoia (tm) ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] Image Management in an RDBMS...(was OS Spatial environment 'sizing')
Randy George wrote: Hi Bruce, What approaches are people using with large Lidar datasets? You might take a look at the WeoGeo group. They are a commercial operation, not FOSS, BEEP! The opposite to FOSS is proprietary! Some related information can be found here: http://wiki.osgeo.org/wiki/Commercial_Services and here: http://wiki.osgeo.org/wiki/Category:Advocacy Regards, Arnulf. (sorry to be picky, but its my job) but they are throwing dedicated AWS instances at the issue of lidar file serving. The dedicated instance, I gather, is for the sole use of the download client with any clip, re-project, or re-format processing requested. Paul Bisset would be happy to clarify. http://www.weogeo.com/ Some more details here: http://www.cadmaps.com/gisblog/?p=25 The lidar itself is I assume a raw file, probably split into a set of S3 objects which are stream concatenated and run through the processing instance and out to the requestor. Since the data sets are very large the dedicated assignment of a temporary cpu or two is justified. Beyond that I would like to see at least some lidar sets treated like the JPL srtm which is made accessible via WMS with pixel coded elevation as grayscale. Paul Ramsey's inimitable advice would work well for lidar too. The end client can turn a WMS request on a lidar image pyramid behind say Geoserver into whatever is desired, in my case I would turn it into 3D xaml meshes. I imagine this could be done in a more standards compliant manner through the WCS spec. I believe that the JPL srtm was made public before an implementation of WCS was accessible. http://onearth.jpl.nasa.gov/index.html The default styles for the elevation layers are a version of the elevation maps scaled to 8 bit. The full elevation values can be retrieved from this server by requesting the short_int or feet_short_int styles in combination with the image/png image/geotiff or image/tiff value for the format argument. The result of such a request will be an image where the signed short integer values contained in the image file for each pixel are the elevation of the respective point on the map, in either meters or feet. The base data is in meters. The us_ned layer base data is floating point real numbers in meters, data which can be retried in tiff or geotiff format when using the real or feet_real as styles. I like the interchange of comments on this subject . In the end I'm inclined to stick with the simplicity of file storage for imagery. Thanks Randy From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of [EMAIL PROTECTED] Sent: Thursday, February 21, 2008 6:25 PM To: OSGeo Discussions Subject: Re: [OSGeo-Discuss] Image Management in an RDBMS...(was OS Spatial environment 'sizing') IMO: Paul, On Feb 21, 2008, at 4:19 PM, [EMAIL PROTECTED] wrote: What it comes down to is what is appropriate for your use case. Indeed! However, there seem to be vanishingly few use cases for which raster-in-database is actually the more appropriate solution. ;-) I beg to differ. (BTW, point-in-time recovery, a nice example of a place where database semantics have an upper hand. Although more modern file systems and enterprise backup systems are pretty competitive now... even a relatively simple hack like the OS/X Time Machine feature solves that problem for-all-practical-purposes.) Trying to manage very large regional datasets via a file based solution is problematic as described earlier with tile based approach to vector data in particular. Again for my use case the DB is better. Just to throw in another related issue: Lidar systems are throwing out an enormous amount of data. I had one dataset of only around 17 million odd records several years ago (of course stored in our corporate db ;-) ) that we could not handle with ArcGIS Desktop (v9.1). From memory it was a 32bit issue. What approaches are people using with large Lidar datasets? Bruce Notice: This email and any attachments may contain information that is personal, confidential, legally privileged and/or copyright. No part of it should be reproduced, adapted or communicated without the prior written consent of the copyright owner. It is the responsibility of the recipient to check for and remove viruses. If you have received this email in error, please notify the sender by return email, delete it from your system and destroy any copies. You are not authorised to use, communicate or rely on the information contained in this email. Please consider the environment before printing this email. ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman
Re: [OSGeo-Discuss] Image Management in an RDBMS...(was OS Spatial environment 'sizing')
On Fri, 2008-02-22 at 10:49 +0900, Tim Bowden wrote: Bruce, Note: I've never tried this, and I'm making it up as I go. Others more informed than me may well kill the idea (Paul?). Ok, kill this one. Tried it and it doesn't work. Tim ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] Image Management in an RDBMS...(was OS Spatial environment 'sizing')
IMO: Hi Tyler, Thanks for the reply. Am I correct in believing that the two things people desire with images in an RDB, is having an abstract 1) storage framework (tables) and 2) a common access language (SQL) for managing the framework. You could have the most complex storage set up behind the scenes, but as long as the access interface plays well, the complexity could be minimised by good UI design. At least I think so, but haven't done it before. I can't speak for other people's needs, but only give my own opinion. My experience with storing spatial data in a database is mainly limited to ESRI's solutions based on ArcSDE/Oracle (~7 years). I have had a cursory look at Oracle Spatial and PostGres/PostGIS and intend looking a lot closer at both. I've also used a number of spatial tools over the years from a number of vendors. I've implemented and managed a number of ArcSDE instances over the years. As skeptical as I am about the decision to rename the product as ArcGIS Server basic (or whatever its called now), ESRI have done a great job with the product. Particularly with its integration with ArcGIS Desktop as the primary user interface for adding, maintaining and managing spatial data from a GUI. You don't need to use SQL, but it also has its advantages (when appropriate). I've found a number of benefits with managing spatial data in a corporate database environment. These comments apply to both vector and image data. I'm sure that these comments are equally pertinent to most RDBMS maintained spatial data. Some examples are: - Within a large organisation, it is possible to get rid of most of the islands of data that are hidden in a wide variety of departments. If implemented right, people come to see the database as the authoritive source of their data and respect it as such. - This can remove the situation where you get multiple copies of the same dataset around your organisation, with different people making their own independent edits to the data and expecting someone to reconcile the edits with the authoritative data set at a later time (if you're lucky). - It can also remove the situation where someone takes a copy of a critical data set and does not update it for several years, leaving business people making critical decisions on suspect data. - You can start managing your data for a given geographic phenomena as a single entity covering a large geographic region, without having to resort to tiles and all the related edge matching problems that we had in the past (e.g. mismatching pixels, lines, polygons that just end at the tile boundary or have an incorrect attibute on the matching sheet etc). - Some of the biggest advantages though, come from the corporate IT support that you come to rely on, e.g. regular backups, large disk capacity on fast SAN devices, secure access to data by authorised custodians, redundant databases for disaster recovery, point in time restoration of data etc. To date, I have not found a suitable solution for managing imagery that includes multi and hyper spectral data in a database. But I'm looking. Ideally the solution will use open data formats for storage and delivery. I'm getting sick of having to redevelop corporate applications with the same functionality because a vendor has decided to change their technology and data formats. This results in a lot of wasted time and money that would be better used implementing effective decision support tools that allow businesses to better understand and exploit their data. Some more recent raster in db discussion here: http://spatialgalaxy.net/2008/02/15/rasters-in-the-database-why-bother/ Thanks for the heads-up Tyler. I obviously don't know what I'm talking about ;-) (eh Tim?). Bruce Notice: This email and any attachments may contain information that is personal, confidential, legally privileged and/or copyright.No part of it should be reproduced, adapted or communicated without the prior written consent of the copyright owner. It is the responsibility of the recipient to check for and remove viruses. If you have received this email in error, please notify the sender by return email, delete it from your system and destroy any copies. You are not authorised to use, communicate or rely on the information contained in this email. Please consider the environment before printing this email. ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] Image Management in an RDBMS...(was OS Spatial environment 'sizing')
[EMAIL PROTECTED] wrote: IMO: Hi Tyler, Thanks for the reply. Am I correct in believing that the two things people desire with images in an RDB, is having an abstract 1) storage framework (tables) and 2) a common access language (SQL) for managing the framework. You could have the most complex storage set up behind the scenes, but as long as the access interface plays well, the complexity could be minimised by good UI design. At least I think so, but haven't done it before. I can't speak for other people's needs, but only give my own opinion. Hi, sorry to bother again but I still see the need to clarify that there are two issues here. One is pragmatic experience with raster storage in a proprietary set of software. The other is whether there is a *need* to store rasters in a database at all. We have a customer who loves her SDE/Oracle and would never switch back to file based access because she feels it is falling back into stone age. I tell her that she has been brain washed by certain creative minds who sell things (instead of develop software) into thinking that files based systems are stone age. She hates me for it and curses her vendors at the same time now... There are arguments for file based approaches, one of them is that hardware is still developing really fast. Even disk read head caches are part of the spatial data infrastructure. It is just a question on whether you make use of them. Whether they are accessed by an operating system directly or by a database that sits on top of the operating system will surely make a difference in performance. My experience with storing spatial data in a database is mainly limited to ESRI's solutions based on ArcSDE/Oracle (~7 years). I have had a cursory look at Oracle Spatial and PostGres/PostGIS and intend looking a lot closer at both. I've also used a number of spatial tools over the years from a number of vendors. I've implemented and managed a number of ArcSDE instances over the years. As skeptical as I am about the decision to rename the product as ArcGIS Server basic (or whatever its called now), ESRI have done a great job with the product. Particularly with its integration with ArcGIS Desktop as the primary user interface for adding, maintaining and managing spatial data from a GUI. You don't need to use SQL, but it also has its advantages (when appropriate). This should probably rather read with ArcGIS Desktop as the *only* user interface - which makes you depend on that vendor. I would amend the second part to read Unfortunately you can't use SQL,... but thats also just a personal opinion. I've found a number of benefits with managing spatial data in a corporate database environment. These comments apply to both vector and image data. I'm sure that these comments are equally pertinent to most RDBMS maintained spatial data. Some examples are: Just to make sure: corporate database environment refers to any database software, not just Oracle? SDE is not a corporate database environment or do you see it as a part of it? I can follow and underline the arguments related to holding vector data in a database but still fail to understand the need for rasters (which is probably mainly due to my ignorance). - Within a large organisation, it is possible to get rid of most of the islands of data that are hidden in a wide variety of departments. If implemented right, people come to see the database as the authoritive source of their data and respect it as such. Yes. But you would not want to have people talk to the database (ugh - SQL) but rather to a service. Give people a link to a service instead of a file to store on their own machine. There is no explicit need for a database, just encapsulate whatever you have behind a service. Users don't care what is behind the service, it can be a database or a lump of files on a SAN. - This can remove the situation where you get multiple copies of the same dataset around your organisation, with different people making their own independent edits to the data and expecting someone to reconcile the edits with the authoritative data set at a later time (if you're lucky). Absolutely. But I fail to see the need to store rasters in a database arise from this argument. - It can also remove the situation where someone takes a copy of a critical data set and does not update it for several years, leaving business people making critical decisions on suspect data. - You can start managing your data for a given geographic phenomena as a single entity covering a large geographic region, without having to resort to tiles and all the related edge matching problems that we had in the past (e.g. mismatching pixels, lines, polygons that just end at the tile boundary or have an incorrect attibute on the matching sheet etc). I am probably too vector oriented to understand the problems involved here but my experience is that there should be no issue if you have
Re: [OSGeo-Discuss] Image Management in an RDBMS...(was OS Spatial environment 'sizing')
All too often, the benefits touted for raster-in-database have nothing to do with the database, and everything to do with the data preparation tools that the vendor is including with their raster-in- database solution. To store rasters in a database, you need a set of tools that will (a) chop the inputs into something small enough to fit on a database page (tiling), (b) pyramid the source data up so you don't have to run re- sampling operations on the fly, and (c) handle mosaicking operations at the edges of source input files. Here's how to do all of this stuff without the database: On Feb 20, 2008, at 4:29 PM, [EMAIL PROTECTED] wrote: - store a relatively large amount imagery and utilise it as a single entity (e.g. a layer). Use a web service to expose the data, such as WMS. You can view your WMS in ArcMap, in Google Earth, in a web browser typing in a URL with your toes. Far more interoperable than raster-in-database client code, because it is far simpler. Using Mapserver, and a tile-indexed raster layer, you can publish a seamless view of as many source files as you like. (Chris Hodgson did it for over 1000 source files, see http://mapserver.gis.umn.edu/community/conferences/MUM3/present/session2/hodgson/view) - only retrieve the records (tiles) required for the geographic area being viewed. That is we did not need to load the entire mosaic into memory, just stream the records required. Exactly what the tile-indexed raster collection does. When the rasters that are indexed are themselves internally tiled (using the TIFF internal tiling scheme) the effect is identical to the chunked database approach. - only utilised an appropriate image sample for the viewing scale utilised via the pyramid layers (a common technique used by RS products). For the large scale tile-indexed approach a combination of pyramiding the source files, and creating a pyramided master mosaic for super- overviews achieves this. - if required, add additional data to the mosaic. Add new file to directory, re-run master mosaic process. - take advantage of corporate data management techniques as discussed previously. Why people think backing up databases is easier than backing up directories is beyond me. Doesn't your IT department back up files? I know personally I would rather back up a huge file system, than a huge database table space. There are way more tool options and way less complexity. Anyhow, the tools for loading and pyramiding that the raster-in- database vendors provide are indubitably helpful in preparing the data, but the database itself adds nothing, zip, bupkus, to the value equation. And the same stuff the vendor tools do can be done with GDAL and Mapserver and little else. And I hope someone wants to have a performance run-off. Go ahead, punk. Make my day. :) P. -- Paul Ramsey [EMAIL PROTECTED] +1 250 885 0632 ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] Image Management in an RDBMS...(was OS Spatial environment 'sizing')
On Thu, 2008-02-21 at 16:24 -0800, Paul Ramsey wrote: On Feb 21, 2008, at 4:19 PM, [EMAIL PROTECTED] wrote: What it comes down to is what is appropriate for your use case. Indeed! However, there seem to be vanishingly few use cases for which raster-in-database is actually the more appropriate solution. (BTW, point-in-time recovery, a nice example of a place where database semantics have an upper hand. Although more modern file systems and enterprise backup systems are pretty competitive now... even a relatively simple hack like the OS/X Time Machine feature solves that problem for-all-practical-purposes.) Completely off the wall thought, but what about using git? Tim P. ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] Image Management in an RDBMS...(was OS Spatial environment 'sizing')
IMO: Paul, On Feb 21, 2008, at 4:19 PM, [EMAIL PROTECTED] wrote: What it comes down to is what is appropriate for your use case. Indeed! However, there seem to be vanishingly few use cases for which raster-in-database is actually the more appropriate solution. ;-) I beg to differ. (BTW, point-in-time recovery, a nice example of a place where database semantics have an upper hand. Although more modern file systems and enterprise backup systems are pretty competitive now... even a relatively simple hack like the OS/X Time Machine feature solves that problem for-all-practical-purposes.) Trying to manage very large regional datasets via a file based solution is problematic as described earlier with tile based approach to vector data in particular. Again for my use case the DB is better. Just to throw in another related issue: Lidar systems are throwing out an enormous amount of data. I had one dataset of only around 17 million odd records several years ago (of course stored in our corporate db ;-) ) that we could not handle with ArcGIS Desktop (v9.1). From memory it was a 32bit issue. What approaches are people using with large Lidar datasets? Bruce Notice: This email and any attachments may contain information that is personal, confidential, legally privileged and/or copyright.No part of it should be reproduced, adapted or communicated without the prior written consent of the copyright owner. It is the responsibility of the recipient to check for and remove viruses. If you have received this email in error, please notify the sender by return email, delete it from your system and destroy any copies. You are not authorised to use, communicate or rely on the information contained in this email. Please consider the environment before printing this email. ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] Image Management in an RDBMS...(was OS Spatial environment 'sizing')
IMO: Tim, Would you like to expand on this? Bruce Completely off the wall thought, but what about using git? Tim Notice: This email and any attachments may contain information that is personal, confidential, legally privileged and/or copyright.No part of it should be reproduced, adapted or communicated without the prior written consent of the copyright owner. It is the responsibility of the recipient to check for and remove viruses. If you have received this email in error, please notify the sender by return email, delete it from your system and destroy any copies. You are not authorised to use, communicate or rely on the information contained in this email. Please consider the environment before printing this email. ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
RE: [OSGeo-Discuss] Image Management in an RDBMS...(was OS Spatial environment 'sizing')
Hi Bruce, What approaches are people using with large Lidar datasets? You might take a look at the WeoGeo group. They are a commercial operation, not FOSS, but they are throwing dedicated AWS instances at the issue of lidar file serving. The dedicated instance, I gather, is for the sole use of the download client with any clip, re-project, or re-format processing requested. Paul Bisset would be happy to clarify. http://www.weogeo.com/ Some more details here: http://www.cadmaps.com/gisblog/?p=25 The lidar itself is I assume a raw file, probably split into a set of S3 objects which are stream concatenated and run through the processing instance and out to the requestor. Since the data sets are very large the dedicated assignment of a temporary cpu or two is justified. Beyond that I would like to see at least some lidar sets treated like the JPL srtm which is made accessible via WMS with pixel coded elevation as grayscale. Paul Ramsey's inimitable advice would work well for lidar too. The end client can turn a WMS request on a lidar image pyramid behind say Geoserver into whatever is desired, in my case I would turn it into 3D xaml meshes. I imagine this could be done in a more standards compliant manner through the WCS spec. I believe that the JPL srtm was made public before an implementation of WCS was accessible. http://onearth.jpl.nasa.gov/index.html The default styles for the elevation layers are a version of the elevation maps scaled to 8 bit. The full elevation values can be retrieved from this server by requesting the short_int or feet_short_int styles in combination with the image/png image/geotiff or image/tiff value for the format argument. The result of such a request will be an image where the signed short integer values contained in the image file for each pixel are the elevation of the respective point on the map, in either meters or feet. The base data is in meters. The us_ned layer base data is floating point real numbers in meters, data which can be retried in tiff or geotiff format when using the real or feet_real as styles. I like the interchange of comments on this subject . In the end I'm inclined to stick with the simplicity of file storage for imagery. Thanks Randy From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of [EMAIL PROTECTED] Sent: Thursday, February 21, 2008 6:25 PM To: OSGeo Discussions Subject: Re: [OSGeo-Discuss] Image Management in an RDBMS...(was OS Spatial environment 'sizing') IMO: Paul, On Feb 21, 2008, at 4:19 PM, [EMAIL PROTECTED] wrote: What it comes down to is what is appropriate for your use case. Indeed! However, there seem to be vanishingly few use cases for which raster-in-database is actually the more appropriate solution. ;-) I beg to differ. (BTW, point-in-time recovery, a nice example of a place where database semantics have an upper hand. Although more modern file systems and enterprise backup systems are pretty competitive now... even a relatively simple hack like the OS/X Time Machine feature solves that problem for-all-practical-purposes.) Trying to manage very large regional datasets via a file based solution is problematic as described earlier with tile based approach to vector data in particular. Again for my use case the DB is better. Just to throw in another related issue: Lidar systems are throwing out an enormous amount of data. I had one dataset of only around 17 million odd records several years ago (of course stored in our corporate db ;-) ) that we could not handle with ArcGIS Desktop (v9.1). From memory it was a 32bit issue. What approaches are people using with large Lidar datasets? Bruce Notice: This email and any attachments may contain information that is personal, confidential, legally privileged and/or copyright. No part of it should be reproduced, adapted or communicated without the prior written consent of the copyright owner. It is the responsibility of the recipient to check for and remove viruses. If you have received this email in error, please notify the sender by return email, delete it from your system and destroy any copies. You are not authorised to use, communicate or rely on the information contained in this email. Please consider the environment before printing this email. ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] Image Management in an RDBMS...(was OS Spatial environment 'sizing')
IMO 12 million records is teensy. Stuff it into PostGIS. It's the billion- point LIDAR sets that leave me queasy, but I can't begin to think of a reasonable architecture for that without learning more about how the points are actually USED, which I really am not clear on at the moment. Paul, Agreed. Generation of TINs or surfaces of roughness over that number of points will challenge any data management solution. However, the time is coming / has come when people will want to do it. It is perhaps a good candidate for Grid architectures and high performance computing. Bruce Notice: This email and any attachments may contain information that is personal, confidential, legally privileged and/or copyright.No part of it should be reproduced, adapted or communicated without the prior written consent of the copyright owner. It is the responsibility of the recipient to check for and remove viruses. If you have received this email in error, please notify the sender by return email, delete it from your system and destroy any copies. You are not authorised to use, communicate or rely on the information contained in this email. Please consider the environment before printing this email. ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
RE: [OSGeo-Discuss] Image Management in an RDBMS...(was OS Spatial environment 'sizing')
IMO: Hi Randy, Ivan and Arnulf, I seem to have spawned another thread, moving away from my original post and Randy's excellent response. Sorry. I've renamed this thread accordingly. more below... [EMAIL PROTECTED] wrote on 21/02/2008 02:38:13 AM: Hi Ivan and Bruce, I am curious to know what advantage an arcSDE/Oracle stack would provide on image storage. I had understood imagery was simply stored as large blob fields and streamed in and out of the DB where it is processed/viewed etc. The original state I had understood was unchanged (lossy, wavelet, pk or otherwise happening outside the DB), just residing in the DB directory rather than the disk hierarchy. Other than possible table corruption issues I imagined that the overhead for streaming a blob into an image object was the only real concern on DB storage. The ArcSDE storage of imagery solution that I described in my earlier post was at a previous place of employment. They still utilise the solution effectively. While the storage of imagery using ArcSDE can technically utilise multiple bands of radiometric data, it is mainly using a set of blob records as Randy identified. This limits the usefulness of the product when you want a flexible tool to manage multi or hyper spectral data. This is also one of the reasons that I'm looking for alternate RDBMS based solutions. Having said that I found ArcSDE to be quite effective for orthophoto mosaics of aerial photography as I described earlier. The data that we used was: aerial photography: - approx 500 individual images from around fifteen runs of photography - approx 140 panelled ground control points and airbourne GPS - photography was scanned and aerotriangulated - imagery was then mosaiced, orthorectified and colour balanced - imagery then diced into around 70 RGB TIFF6 files, each around 1 GB, ~6 cm ground resolution. - imagery loaded into Oracle/ArcSDE - positional accuracy determined (~0.1m) using stats and spread of error viewed usinging krieging techniques. In short ESRI's approach with ArcSDE (as I understand it) is: - images broken down into small blobs (we used 128k x 128k tiles, LZ77 compressed TIFF) and loaded with one 128k blob per database record. - statistics calculated on imagery - 7 pyramid layers created This gave us the ability to: - store a relatively large amount imagery and utilise it as a single entity (e.g. a layer). - only retrieve the records (tiles) required for the geographic area being viewed. That is we did not need to load the entire mosaic into memory, just stream the records required. - only utilised an appropriate image sample for the viewing scale utilised via the pyramid layers (a common technique used by RS products). - if required, add additional data to the mosaic. - take advantage of corporate data management techniques as discussed previously. As Arnulf correctly identified, there is a black box behind the data storage. But this is equally true for the majority of spatial data that is under active management around the world. Ideally we would utilise an open format for storage and an open format for delivery. Also for Arnulf: - I think that the user requirement is there for storing raster data in a DB. We have had two uses identified by myself and by Ivan. - When you consider the complexities that Google must be facing with GE in trying to manage 256x256k tiles of imagery over the entire world, at multiple pyramid layers and with constant revision of imagery, you can soon see that a file based approach would lead to a major headache. - I personally think that the case for raster in a DB has been made. Now what I'd ideally like to find is a good solution for managing multi and hyper spectral data in an RDB with the ability to serve whatever band combination that a **user** requires via an appropriate standard (possibly via WMS 1.3+ or WCS). Does anyone know of any solutions, preferrably OS? I do recall a product that came out of a German Uni around 2003. There was some talk on GIS-L at the time, however it has slipped off my radar. Does anyone know what became of it. I do recall that they'd had discussions with Oracle. It was around this time that Oracle announced their Georaster format. Bruce Bannerman Notice: This email and any attachments may contain information that is personal, confidential, legally privileged and/or copyright.No part of it should be reproduced, adapted or communicated without the prior written consent of the copyright owner. It is the responsibility of the recipient to check for and remove viruses. If you have received this email in error, please notify the sender by return email, delete it from your system and destroy any copies. You are not authorised to use, communicate or rely on the information contained in this email. Please consider the environment before printing this email.
Re: [OSGeo-Discuss] Image Management in an RDBMS...(was OS Spatial environment 'sizing')
- When you consider the complexities that Google must be facing with GE in trying to manage 256x256k tiles of imagery over the entire world, at multiple pyramid layers and with constant revision of imagery, you can soon see that a file based approach would lead to a major headache. He he, I think I'd write that same sentence but substitute 'database approach' for 'file based approach'. I'd be pretty shocked if Google were using any kind of database for their tiles. They certainly aren't paying oracle or arcsde license fees. They could have a custom mysql solution, but I'd guess it's all on the Google File System: http://labs.google.com/papers/gfs.html Also, I think it's still in pretty beta development, but Geomatys has been working on PostGRID - http://seagis.sourceforge.net/postgrid/index.html and http://www.foss4g2007.org/presentations/view.php?abstract_id=225 have some information. I believe is pretty attached to java, but I think does some of what you want, managing the metadata in the database. Though I could be wrong about if it's close to what you're thinking of, my understanding of the raster side of the fence has never been that strong. best regards, Chris begin:vcard fn:Chris Holmes n:Holmes;Chris org:The Open Planning Project adr:;;349 W. 12th Street, #3;New York;NY;10014;USA email;internet:[EMAIL PROTECTED] title:Managing Director, Strategic Development x-mozilla-html:FALSE url:http://topp.openplans.org version:2.1 end:vcard ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] Image Management in an RDBMS...(was OS Spatial environment 'sizing')
On 20-Feb-08, at 4:29 PM, [EMAIL PROTECTED] wrote: - When you consider the complexities that Google must be facing with GE in trying to manage 256x256k tiles of imagery over the entire world, at multiple pyramid layers and with constant revision of imagery, you can soon see that a file based approach would lead to a major headache. Hi Bruce, Am I correct in believing that the two things people desire with images in an RDB, is having an abstract 1) storage framework (tables) and 2) a common access language (SQL) for managing the framework. You could have the most complex storage set up behind the scenes, but as long as the access interface plays well, the complexity could be minimised by good UI design. At least I think so, but haven't done it before. However, I did manage massive (at the time) amounts of vector files in the file system, and was dreaming about using a db. All the while I watched some others make the wholesale shift to vectors in a transactional db. I grimaced when asking them for data, only to wait while they batched up an SQL request to extract back into files -- what used to be a simple copy command to move a zip file into an FTP folder. I admired their commitment, but was frustrated by usability in the end (as were they). It was good food for thought nonetheless as I planned my own vector-in-db direction :) Some more recent raster in db discussion here: http://spatialgalaxy.net/2008/02/15/rasters-in-the-database-why-bother/ Tyler ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
