Hi Jon - alas for we ArcGIS users, which even here in 2012 is still a single-threaded application. Who needs more than one CPU core anyway? So your ArcGIS choice is to split your batch up into 12 groups and run the processes simultaneously (which will then take all your CPUs), or use something else.
Might I suggest a pretty simple solution which uses GDAL and MapServer instead? In the state of Maine we now serve all of our enterprise web mapping services (WMS) for imagery using GDAL and MapServer. Using gdalwarp and gdal_translate we convert whatever the source imagery is into JPEG2000 compressed tiles, and generate overview images using gdalwarp. This system takes our 3.5TB of source imagery down to about 700GB of JP2 files and is pretty darn fast. You could set something like this up on a simple PC (but don't use a USB drive for the data - put it on a fast internal drive) for a small group of users inside your organization, or on servers for a wide distribution to a large # of folks. Our system is designed to support about 2000 end-users and about 10,000 requests per hour, using a dual-CPU (quad-core) HP server. We use MS4W which is super-easy to install, and then scripts to generate the tiles. I'd be happy to share some of these with you as examples. ====================================== Michael Smith State GIS Manager, Maine Office of GIS State of Maine, Office of Information Technology michael.smith _at_ maine.gov 207-215-5530 Board Member, Maine GeoLibrary Board Member, Maine GIS Users Group State Rep, National States Geographic Information Council | MEGIS | GeoLibrary | MEGUG | NSGIC | State House Station 174 264 Civic Center Drive Augusta, ME 04333-0174 69 o 47' 49.5"W 44 o 20' 54.5"N Confidentiality Notice: This email message, including any attachments, is for the sole use of the intended recipients and may contain confidential and privileged information. If you are not the intended recipient, or an authorized agent of the intended recipient, please immediately contact the sender by reply email and destroy/delete all copies of the original message. Any unauthorized review, use, copying, forwarding, disclosure, or distribution by other than the intended recipient or authorized agent is prohibited. -----Original Message----- From: gdal-dev-boun...@lists.osgeo.org [mailto:gdal-dev-boun...@lists.osgeo.org] On Behalf Of jgamble Sent: Saturday, February 25, 2012 1:21 PM To: gdal-dev@lists.osgeo.org Subject: [gdal-dev] Switching from ArcGIS to GDAL Hi All, I've recently been tasked to cache several large imagery datasets using ArcGIS and have found that my bottleneck has been generating overviews for the datasets prior to caching. ArcGIS appears to be limiting itself to a single core while building overviews, which seems ridiculous on a 12 core machine ... which is why I am here for some help. I am using mosaic datasets in ArcGIS to manage my images. Footprints for the images have been created to clip out any bounding black pixels. Am I able to define overviews in ArcGIS and then use GDAL to build them with multiple cores ... Or at least utilize the footprints I've already defined in my mosaic dataset? Does GDAL have something similar to a mosaic dataset to manage images that I should be using? I'm extremely new to the thought of using GDAL, so any advice on how to go about utilizing it or at least pointing me in the right direction to some tutorials is much appreciated. Thanks Jon. -- View this message in context: http://osgeo-org.1560.n6.nabble.com/Switching-from-ArcGIS-to-GDAL-tp4507 244p4507244.html Sent from the GDAL - Dev mailing list archive at Nabble.com. _______________________________________________ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev _______________________________________________ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
