Re: [OSGeo-Discuss] 'lossless' JPEG2000
Hi Michael, We made some tests with tiles of 1000*1000 pixels, with 1 tiles, and the memory used is about 112 MB for the encoding and 114 MB for the decoding. If you don't want to use tiles, I don't think OpenJPEG can beat the commercial applications like Kakadu. What standard do you follow for metadata ? OGC GMLJP2, or do you include GeoTIFF information in a JP2 file like Luratech suggested to the JPEG committee ? Cheers, François Michael P. Gerlek a écrit : François: When you say Mega-Images (- geo-sized images), just how big are you talking about? If you are in the 10-100GB range, I/LizardTech would be very interested in talking with you about the project, and also about supporting some of the geo metadata conventions. (Especially if you can do GB-sized data sets in less than 1GB of RAM without requiring the image be tiled!) ((Do you have any benchmark data you can share?) -mpg -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of François-Olivier Devaux Sent: Tuesday, February 26, 2008 12:47 AM To: discuss@lists.osgeo.org Subject: [OSGeo-Discuss] 'lossless' JPEG2000 Hi, Norman Vine has pointed to me this discussion about JPEG 2000, and I thought it might be interesting to give you a small overview on JPEG 2000 and present the OpenJPEG library on which we are working. FIELDS WHERE JPEG 2000 IS USED JPEG 2000 is becoming the reference in image compression for professional applications, where precision and flexibility is really necessary. The most know field using JPEG 2000 is Digital Cinema, where JPEG 2000 has been favored against MPEG2 and H.264. Linked to that field, High Quality Broadcast applications are also turning to JPEG 2000 because of its quality and scalability (low resolution versions can be extracted directly from a high resolution sequence without any re-encoding, and JPEG 2000 sequences are encoded in intra which eases video editing). More close to your field is Archiving, where we are feeling a trend to select JPEG 2000 as compression algorithm http://www.egov.vic.gov.au/index.php?env=-inlink/detail:m1780- 1-1-8-s-0:l-9669-1-1-- Medical imaging applications, where lossless compression is a important requirement, are also taking full advantage of JPEG 2000 remote browsing possibilities (with the JPIP protocol) http://www.earthtimes.org/articles/show/aware-inc-to-demonstra te-groundbreaking-medical-imaging-streaming-solution-at- himss08,290686.shtml - JPEG 2000 FEATURES The JPEG 2000 features that are interesting for GeoSpatial Imagery is of course the ability to achieve lossless compression, the scalability (lower quality and resolutions as well as spatial areas can be extracted from a compressed file, without the need of decompression the entire file), the high precision (most codecs can at least handle 16 bits per component, and up to 256 components) and the fact that the core coding system can be obtained free of charge. JPEG 2000 also has an inherent robustness higher than most compression schemes (JPEG, ...) and a great protocol to interactively remotely browse images called JPIP. - OPENJPEG OpenJPEG, is an open-source JPEG 2000 library. It has been very recently remodeled by the CNES and the french company CS to meet the requirements of applications using Mega-Images (- geo-sized images). Independent access to tiles has been improved, in order to increase the library encoding and decoding performances. This new version should be made accessible to users at the beginning of March. We are very happy of the performances of this new version, and are open to new contributions. Regarding other JPEG 2000 open source solutions in your field, the GDAL library has a JPEG 2000 module that is based on Jasper, which is a great library, but has unfortunately not evolved for the last years. - Cheers, François ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
RE: [OSGeo-Discuss] 'lossless' JPEG2000
So that's 10GB of data, using tiles, at 100MB memory? That's good, and maybe requiring tiles for larger images is something I could get used to. What's the speed like? We use both the GMLJP2 standard and the GeoTIFF-tag approach. Gosh but I'd to get behind an open source geo-aware JP2 solution. -mpg From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of François-Olivier Devaux Sent: Wednesday, February 27, 2008 1:50 AM To: OSGeo Discussions Subject: Re: [OSGeo-Discuss] 'lossless' JPEG2000 Hi Michael, We made some tests with tiles of 1000*1000 pixels, with 1 tiles, and the memory used is about 112 MB for the encoding and 114 MB for the decoding. If you don't want to use tiles, I don't think OpenJPEG can beat the commercial applications like Kakadu. What standard do you follow for metadata ? OGC GMLJP2, or do you include GeoTIFF information in a JP2 file like Luratech suggested to the JPEG committee ? Cheers, François Michael P. Gerlek a écrit : François: When you say Mega-Images (- geo-sized images), just how big are you talking about? If you are in the 10-100GB range, I/LizardTech would be very interested in talking with you about the project, and also about supporting some of the geo metadata conventions. (Especially if you can do GB-sized data sets in less than 1GB of RAM without requiring the image be tiled!) ((Do you have any benchmark data you can share?) -mpg -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of François-Olivier Devaux Sent: Tuesday, February 26, 2008 12:47 AM To: discuss@lists.osgeo.org Subject: [OSGeo-Discuss] 'lossless' JPEG2000 Hi, Norman Vine has pointed to me this discussion about JPEG 2000, and I thought it might be interesting to give you a small overview on JPEG 2000 and present the OpenJPEG library on which we are working. FIELDS WHERE JPEG 2000 IS USED JPEG 2000 is becoming the reference in image compression for professional applications, where precision and flexibility is really necessary. The most know field using JPEG 2000 is Digital Cinema, where JPEG 2000 has been favored against MPEG2 and H.264. Linked to that field, High Quality Broadcast applications are also turning to JPEG 2000 because of its quality and scalability (low resolution versions can be extracted directly from a high resolution sequence without any re-encoding, and JPEG 2000 sequences are encoded in intra which eases video editing). More close to your field is Archiving, where we are feeling a trend to select JPEG 2000 as compression algorithm http://www.egov.vic.gov.au/index.php?env=-inlink/detail:m1780- 1-1-8-s-0:l-9669-1-1-- Medical imaging applications, where lossless compression is a important requirement, are also taking full advantage of JPEG 2000 remote browsing possibilities (with the JPIP protocol) http://www.earthtimes.org/articles/show/aware-inc-to-demonstra te-groundbreaking-medical-imaging-streaming-solution-at- himss08,290686.shtml - JPEG 2000 FEATURES The JPEG 2000 features that are interesting for GeoSpatial Imagery is of course the ability to achieve lossless compression, the scalability (lower quality and resolutions as well as spatial areas can be extracted from a compressed file, without
RE: [OSGeo-Discuss] 'lossless' JPEG2000
François: When you say Mega-Images (- geo-sized images), just how big are you talking about? If you are in the 10-100GB range, I/LizardTech would be very interested in talking with you about the project, and also about supporting some of the geo metadata conventions. (Especially if you can do GB-sized data sets in less than 1GB of RAM without requiring the image be tiled!) ((Do you have any benchmark data you can share?) -mpg -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of François-Olivier Devaux Sent: Tuesday, February 26, 2008 12:47 AM To: discuss@lists.osgeo.org Subject: [OSGeo-Discuss] 'lossless' JPEG2000 Hi, Norman Vine has pointed to me this discussion about JPEG 2000, and I thought it might be interesting to give you a small overview on JPEG 2000 and present the OpenJPEG library on which we are working. FIELDS WHERE JPEG 2000 IS USED JPEG 2000 is becoming the reference in image compression for professional applications, where precision and flexibility is really necessary. The most know field using JPEG 2000 is Digital Cinema, where JPEG 2000 has been favored against MPEG2 and H.264. Linked to that field, High Quality Broadcast applications are also turning to JPEG 2000 because of its quality and scalability (low resolution versions can be extracted directly from a high resolution sequence without any re-encoding, and JPEG 2000 sequences are encoded in intra which eases video editing). More close to your field is Archiving, where we are feeling a trend to select JPEG 2000 as compression algorithm http://www.egov.vic.gov.au/index.php?env=-inlink/detail:m1780- 1-1-8-s-0:l-9669-1-1-- Medical imaging applications, where lossless compression is a important requirement, are also taking full advantage of JPEG 2000 remote browsing possibilities (with the JPIP protocol) http://www.earthtimes.org/articles/show/aware-inc-to-demonstra te-groundbreaking-medical-imaging-streaming-solution-at- himss08,290686.shtml - JPEG 2000 FEATURES The JPEG 2000 features that are interesting for GeoSpatial Imagery is of course the ability to achieve lossless compression, the scalability (lower quality and resolutions as well as spatial areas can be extracted from a compressed file, without the need of decompression the entire file), the high precision (most codecs can at least handle 16 bits per component, and up to 256 components) and the fact that the core coding system can be obtained free of charge. JPEG 2000 also has an inherent robustness higher than most compression schemes (JPEG, ...) and a great protocol to interactively remotely browse images called JPIP. - OPENJPEG OpenJPEG, is an open-source JPEG 2000 library. It has been very recently remodeled by the CNES and the french company CS to meet the requirements of applications using Mega-Images (- geo-sized images). Independent access to tiles has been improved, in order to increase the library encoding and decoding performances. This new version should be made accessible to users at the beginning of March. We are very happy of the performances of this new version, and are open to new contributions. Regarding other JPEG 2000 open source solutions in your field, the GDAL library has a JPEG 2000 module that is based on Jasper, which is a great library, but has unfortunately not evolved for the last years. - Cheers, François ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
RE: [OSGeo-Discuss] 'lossless' JPEG2000
IMO: Michael, Again, I don't pretend to be an expert on JPEG2000. However, I'd like to know more about the format for future reference. Does the wiki article at the following URL represent a good overview of the format? http://en.wikipedia.org/wiki/JPEG_2000 If it is accurate, there is a section that leads me to conclude that the format is not suitable for a lot of remotely sensed spatial imagery: snip Color components transformation Initially, images have to be transformed from the RGB color space to another color space, leading to three components that are handled separately. There are two possible choices:... /snip If this *is* the case, then I wouldn't be able to use the format to store multi and hyper spectral imagery (ignoring other JP2 issues). As to what format are we using currently:The source format that the data came in with appropriate Geophysics, ERMapper and in some cases Erdas Imagine conversions. What are we using in the future: To be determined, probably a database oriented solution. As to data corruption: Many image processing algorithims and processes result in data loss. The aim for most people is to understand what is acceptable and to minimise the corruption of their data. In our situation, some of the imagery may result from many millions of dollars spent in capture and processing. Much of it is irreplacable. All of it must be protected for future use. Bruce Notice: This email and any attachments may contain information that is personal, confidential, legally privileged and/or copyright.No part of it should be reproduced, adapted or communicated without the prior written consent of the copyright owner. It is the responsibility of the recipient to check for and remove viruses. If you have received this email in error, please notify the sender by return email, delete it from your system and destroy any copies. You are not authorised to use, communicate or rely on the information contained in this email. Please consider the environment before printing this email. ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
RE: [OSGeo-Discuss] 'lossless' JPEG2000
I've not read the whole Wikipedia article, but the statement images have to be transformed from the RGB color space to another color space is indeed incorrect. Images that are 3-banded MAY be encoded with the YCC transform, but this is not required; images with some other number of bands do NOT undergo a color transform step. If you're using ERMapper (ECW) files now, you're already deep into the world of lossy transforms. Imagine files (.img) are lossless, I believe, so you're safe there. My offer to encode a few GB of sample data for you still holds :-) -mpg From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of [EMAIL PROTECTED] Sent: Tuesday, February 26, 2008 2:12 PM To: OSGeo Discussions Subject: RE: [OSGeo-Discuss] 'lossless' JPEG2000 IMO: Michael, Again, I don't pretend to be an expert on JPEG2000. However, I'd like to know more about the format for future reference. Does the wiki article at the following URL represent a good overview of the format? http://en.wikipedia.org/wiki/JPEG_2000 If it is accurate, there is a section that leads me to conclude that the format is not suitable for a lot of remotely sensed spatial imagery: snip Color components transformation Initially, images have to be transformed from the RGB color space http://en.wikipedia.org/wiki/Color_space to another color space, leading to three components that are handled separately. There are two possible choices:... /snip If this *is* the case, then I wouldn't be able to use the format to store multi and hyper spectral imagery (ignoring other JP2 issues). As to what format are we using currently:The source format that the data came in with appropriate Geophysics, ERMapper and in some cases Erdas Imagine conversions. What are we using in the future: To be determined, probably a database oriented solution. As to data corruption: Many image processing algorithims and processes result in data loss. The aim for most people is to understand what is acceptable and to minimise the corruption of their data. In our situation, some of the imagery may result from many millions of dollars spent in capture and processing. Much of it is irreplacable. All of it must be protected for future use. Bruce Notice: This email and any attachments may contain information that is personal, confidential, legally privileged and/or copyright. No part of it should be reproduced, adapted or communicated without the prior written consent of the copyright owner. It is the responsibility of the recipient to check for and remove viruses. If you have received this email in error, please notify the sender by return email, delete it from your system and destroy any copies. You are not authorised to use, communicate or rely on the information contained in this email. Please consider the environment before printing this email. ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
RE: [OSGeo-Discuss] 'lossless' JPEG2000
Yup: Kakadu is not Open Source, as per the OSI definition of the term. The only FOSS package I know of is OpenJpeg2000 (or something like that); unfortunately, however, it is not suitable for geo-sized imagery last time I looked. TotallyNotSpeakingForLizardTechNoWayNoHow Not a week goes by that I do not pause, look out my window, and sigh quietly over this issue. Economic realities are such that doing an open JP2 codec for geo folks would be tough. Kakadu is good enough and cheap enough for us commercial types, which disincentivizes a free version. Furthermore, every six months Kakadu comes out with a new release, and the gap widens further. Yes, an open version might serve to widen the overall JPEG 2000 marketspace -- but that's not a sure enough thing to merit commercial people investing money in. Someday I'd like to try and open a dialog with the OpenJP2 developers about this topic, but last I heard they were completely uninterested in supporting GB-sized datasets. /TotallyNotSpeakingForLizardTechNoWayNoHow (LZW tiffs are a reasonable option, as they are lossless and the LZW patent issues have faded into the sunset.) -mpg -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Christopher Schmidt Sent: Monday, February 25, 2008 10:18 AM To: discuss@lists.osgeo.org Subject: Re: [OSGeo-Discuss] 'lossless' JPEG2000 On Mon, Feb 25, 2008 at 09:27:22AM -0800, Michael P. Gerlek wrote: Bruce- Again, I'm not sure how to convince you of this... JP2 is inherently lossless just like GeoTIFF is; what arguments do you / would you find persuaive to use GeoTIFF? (alternatively, what do you use now that you trust?) I'm late in the game (having been in the Caymans all last week), but I find JP2 more difficult than GeoTIFF simply because the open source tools for working with JP2 through GDAL seem to be significantly flawed. The Kakadau (I think?) JP2 library seems to resolve a lot of this, but the end result is not an Open Source tool, so far as I'm aware: this matters more in terms of my personal (non-commercial) projects like OpenAerialMap, where I can't take a GeoTIFF, reencode it to JP2, and use that as my base imagery for 'free', either libre or gratis. I have had some success with JPG-compressed GeoTIFFs, but these are lossy, and that causes problems (obviously): uncompressed GeoTIFFs are also problematic due to their raw size. (This answer to the question may be irrelevant, as I'm jumping in in the middle of the thread. If so, I apologize.) Regards, -- Christopher Schmidt Web Developer ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
RE: [OSGeo-Discuss] 'lossless' JPEG2000
Christopher - You will very likely find that using different LZW compression options (particularly setting a small strip size) will slightly degrade compression performance while significantly improving read time. While I think your test data are valid, they only address one of many possible configurations and I wouldn't necessarily make broad generalizations about LZW from them. However, I have generally found that LZW compression for photographic data is indeed not a good choice; I'm surprised you got it to perform as well as you did (in compression). - Ed Ed McNierney Chief Mapmaker Demand Media / TopoZone.com 73 Princeton Street, Suite 305 North Chelmsford, MA 01863 Phone: 978-251-4242, Fax: 978-251-1396 [EMAIL PROTECTED] -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Christopher Schmidt Sent: Monday, February 25, 2008 8:57 PM To: discuss@lists.osgeo.org Subject: Re: [OSGeo-Discuss] 'lossless' JPEG2000 On Mon, Feb 25, 2008 at 04:31:34PM -0800, Michael P. Gerlek wrote: Yup: Kakadu is not Open Source, as per the OSI definition of the term. The only FOSS package I know of is OpenJpeg2000 (or something like that); unfortunately, however, it is not suitable for geo-sized imagery last time I looked. Yep. (LZW tiffs are a reasonable option, as they are lossless and the LZW patent issues have faded into the sunset.) The level of compression from LZW is poor by comparison, on aerial imagery: a test dataset compressed from 169MB to 83MB, and time to serve up a small portion was approximately the same as with JPG (6.1s jpg, 6.5s lzw, .5s uncompressed), but the JPG was compressed to 13MB, making it worth the cost; lzw doesn't pay for itself in the same way. (See numbers on http://wiki.openaerialmap.org/Serving_Data) Regards, -- Christopher Schmidt Web Developer ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] 'lossless' JPEG2000
On Mon, Feb 25, 2008 at 09:04:59PM -0500, Ed McNierney wrote: Christopher - You will very likely find that using different LZW compression options (particularly setting a small strip size) will slightly degrade compression performance while significantly improving read time. While I think your test data are valid, they only address one of many possible configurations and I wouldn't necessarily make broad generalizations about LZW from them. However, I have generally found that LZW compression for photographic data is indeed not a good choice; I'm surprised you got it to perform as well as you did (in compression). Yeah, I think we've stumbled back and forth across these numbers before. I'm aware that they're essentially 'back of the envelope': they weren't run entirely in isolation, they were only repeated a couple of times (half dozen rather than an order of magnitude more), they might have been cached in memory, etc. etc. etc. However, they do seem to serve as a good 'order of magnitude' measure of size and performance for compressing aerial imagery based on other similar experiments, and I have no evidence to seriously discount them, so I'm sticking to them. Regards, -- Christopher Schmidt Web Developer ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
RE: [OSGeo-Discuss] 'lossless' JPEG2000
Christopher - Let me add the evidence that I have found that reducing the strip size in LZW-compressed GeoTIFFs has, not surprisingly, a VERY large effect on read performance - about a factor of 10 in the particular cases I used. That indicates that the data you report might not be a good 'order of magnitude' measure of performance. I'm not talking about subtler effects of memory, caching, etc. but the substantial effect of changing the LZW strip size in a large image; the amount of work required to decompress a portion of that image is very, very different from the case in which the entire file is treated as one strip, due to the dictionary-building nature of the LZW algorithm. You're welcome to draw conclusions based on one data point; I am concerned that readers of your post may unreasonably extrapolate from them. You're welcome to stick to them, but I'm not sure I'd recommend that other users stick to them, at least in situations outside of your test environment. - Ed -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Christopher Schmidt Sent: Monday, February 25, 2008 9:18 PM To: discuss@lists.osgeo.org Subject: Re: [OSGeo-Discuss] 'lossless' JPEG2000 On Mon, Feb 25, 2008 at 09:04:59PM -0500, Ed McNierney wrote: Christopher - You will very likely find that using different LZW compression options (particularly setting a small strip size) will slightly degrade compression performance while significantly improving read time. While I think your test data are valid, they only address one of many possible configurations and I wouldn't necessarily make broad generalizations about LZW from them. However, I have generally found that LZW compression for photographic data is indeed not a good choice; I'm surprised you got it to perform as well as you did (in compression). Yeah, I think we've stumbled back and forth across these numbers before. I'm aware that they're essentially 'back of the envelope': they weren't run entirely in isolation, they were only repeated a couple of times (half dozen rather than an order of magnitude more), they might have been cached in memory, etc. etc. etc. However, they do seem to serve as a good 'order of magnitude' measure of size and performance for compressing aerial imagery based on other similar experiments, and I have no evidence to seriously discount them, so I'm sticking to them. Regards, -- Christopher Schmidt Web Developer ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] 'lossless' JPEG2000
On Mon, Feb 25, 2008 at 09:26:16PM -0500, Ed McNierney wrote: Christopher - Let me add the evidence that I have found that reducing the strip size in LZW-compressed GeoTIFFs has, not surprisingly, a VERY large effect on read performance - about a factor of 10 in the particular cases I used. That indicates that the data you report might not be a good 'order of magnitude' measure of performance. I'm not talking about subtler effects of memory, caching, etc. but the substantial effect of changing the LZW strip size in a large image; the amount of work required to decompress a portion of that image is very, very different from the case in which the entire file is treated as one strip, due to the dictionary-building nature of the LZW algorithm. Interesting, I didn't realize that the differences we were discussing were so large. Given that, perhaps I need to reevaluate: My knowledge of image compression is very poor (especially compared to yours!) and you're right in saying that it is poor practice to pass off my numbers without having fully understood them! Thanks for the feedback: now I'll just have to go read a bunch more on how to actually get better results, since clearly I haven't taken your feedback thus far as seriously as I should! Regards, -- Christopher Schmidt Web Developer ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
RE: [OSGeo-Discuss] 'lossless' JPEG2000
IMO: Michael, Thanks for the comments on this thread. I've had a couple of private emails expressing interest in the outcome, so I'll continue this conversation in public, rather than moving it offline. One of the problems that I have is that I understand that JPEG 2000 can be 'lossy' or 'non-lossy'. (Is there a way to tell if a JPEG2000 file is lossy or not?) I don't pretend to understand the maths behind wavelet compressions. I have also not seen 'proof' that would convince me I would be able to safely compress all of my imagery using JPEG2000, (potentially) throw away my source imagery and feel confident that I'd be able to run image processing routines on the radiometric 'numbers' of the imagery at some undefined point in the future with confidence in the integrity of the results. As a reminder, when talking about 'imagery', I'm using the term in its broadest sense to include data such as multi and hyperspectral data in the umbrella term 'imagery'. I'm not talking about only three bands displayed as Red, Green and Blue, but **all** the bands in the 'file'. The description of a test that I included in the early stages of this thread would give me a degree of confidence that JPEG 2000 was a suitable format for long term archival of image data. All that I'm seeing at the moment from many people and organisations is something to the effect of Trust me, your data is saved using a loss-less compression. Bruce [EMAIL PROTECTED] wrote on 26/02/2008 04:27:22 AM: Bruce- Again, I'm not sure how to convince you of this... JP2 is inherently lossless just like GeoTIFF is; what arguments do you / would you find persuaive to use GeoTIFF? (alternatively, what do you use now that you trust?) [feel free to take this to private email, this is probably a bit esoteric for the rest of the OSGeo crowd] -mpg Notice: This email and any attachments may contain information that is personal, confidential, legally privileged and/or copyright.No part of it should be reproduced, adapted or communicated without the prior written consent of the copyright owner. It is the responsibility of the recipient to check for and remove viruses. If you have received this email in error, please notify the sender by return email, delete it from your system and destroy any copies. You are not authorised to use, communicate or rely on the information contained in this email. Please consider the environment before printing this email. ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
RE: [OSGeo-Discuss] 'lossless' JPEG2000
IMO: Thanks for the reply Traian, I don't mean to be dismissive of this report, but I was hoping for something more definitive to prove that 'lossless' JPEG compressions did indeed protect the integrity of the data.. Perhaps its just my ignorance, but I was hoping for something along the lines of: - a study of a range of typical spatial 'imagery'. - evaluation of all spectral values for each pixel in each image before compression. - 'lossless' compression of the images - restoration of the compressed images - comparison of all spectral values for each pixel in each restored image against the original pre-compressed values. - definitive statement with reference to the study results. Bruce JPEG2K supports lossless via a reversible wavelet transform with integral coefficients (which make it reversible, and so lossless). Here is a reference: http://www.ece.uvic.ca/~mdadams/publications/pacrim2001.pdf Traian Notice: This email and any attachments may contain information that is personal, confidential, legally privileged and/or copyright.No part of it should be reproduced, adapted or communicated without the prior written consent of the copyright owner. It is the responsibility of the recipient to check for and remove viruses. If you have received this email in error, please notify the sender by return email, delete it from your system and destroy any copies. You are not authorised to use, communicate or rely on the information contained in this email. Please consider the environment before printing this email. ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
RE: [OSGeo-Discuss] 'lossless' JPEG2000
It is not possible to *prove* a mathematical relationship by example. One would have to study the set of all possible images in the world and make sure they have no loss in order to effectively prove the claim. On the other hand you can *disprove* it by providing a single counterexample. That is why I pointed to the math of the transform, which one has hope of showing is invertible for all cases. Anyway, I'm not that much of an expert on wavelet compression so I can't do much more than point to other people's documentation. Traian From: [EMAIL PROTECTED] [EMAIL PROTECTED] On Behalf Of [EMAIL PROTECTED] [EMAIL PROTECTED] Sent: Saturday, February 23, 2008 3:57 AM To: OSGeo Discussions Subject: RE: [OSGeo-Discuss] 'lossless' JPEG2000 IMO: Thanks for the reply Traian, I don't mean to be dismissive of this report, but I was hoping for something more definitive to prove that 'lossless' JPEG compressions did indeed protect the integrity of the data.. Perhaps its just my ignorance, but I was hoping for something along the lines of: - a study of a range of typical spatial 'imagery'. - evaluation of all spectral values for each pixel in each image before compression. - 'lossless' compression of the images - restoration of the compressed images - comparison of all spectral values for each pixel in each restored image against the original pre-compressed values. - definitive statement with reference to the study results. Bruce JPEG2K supports lossless via a reversible wavelet transform with integral coefficients (which make it reversible, and so lossless). Here is a reference: http://www.ece.uvic.ca/~mdadams/publications/pacrim2001.pdf Traian Notice: This email and any attachments may contain information that is personal, confidential, legally privileged and/or copyright. No part of it should be reproduced, adapted or communicated without the prior written consent of the copyright owner. It is the responsibility of the recipient to check for and remove viruses. If you have received this email in error, please notify the sender by return email, delete it from your system and destroy any copies. You are not authorised to use, communicate or rely on the information contained in this email. Please consider the environment before printing this email. ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
RE: [OSGeo-Discuss] 'lossless' JPEG2000
Bruce- It is not clear to me what sort of study you would need to convince you, as the ISO standard for encoding data into the JPEG-2000 file format is by construction mathematically and numerically lossless process. (Indeed, compression, i.e. throwing away bits so as to further reduce storage requirements, is actually not defined within the scope of the standard.) -mpg From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of [EMAIL PROTECTED] Sent: Saturday, February 23, 2008 1:58 AM To: OSGeo Discussions Subject: RE: [OSGeo-Discuss] 'lossless' JPEG2000 IMO: Thanks for the reply Traian, I don't mean to be dismissive of this report, but I was hoping for something more definitive to prove that 'lossless' JPEG compressions did indeed protect the integrity of the data.. Perhaps its just my ignorance, but I was hoping for something along the lines of: - a study of a range of typical spatial 'imagery'. - evaluation of all spectral values for each pixel in each image before compression. - 'lossless' compression of the images - restoration of the compressed images - comparison of all spectral values for each pixel in each restored image against the original pre-compressed values. - definitive statement with reference to the study results. Bruce JPEG2K supports lossless via a reversible wavelet transform with integral coefficients (which make it reversible, and so lossless). Here is a reference: http://www.ece.uvic.ca/~mdadams/publications/pacrim2001.pdf Traian Notice: This email and any attachments may contain information that is personal, confidential, legally privileged and/or copyright. No part of it should be reproduced, adapted or communicated without the prior written consent of the copyright owner. It is the responsibility of the recipient to check for and remove viruses. If you have received this email in error, please notify the sender by return email, delete it from your system and destroy any copies. You are not authorised to use, communicate or rely on the information contained in this email. Please consider the environment before printing this email. ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
RE: [OSGeo-Discuss] 'lossless' JPEG2000
IMO: Michael, My concern as a custodian of significant image resources is to ensure that the integrity of this data is protected and available for future analytical use by ourselves and by the public. As an example, to conduct multi-temporal analysis of 'imagery' to help understand big picture issues such as climate change. I understand that wavelet compressions such as MrSid and ecw are lossy compressions and JPEG 2000 can be 'lossless', or as often occurs, lossy. I'm currently seeing proposals to the effect: - wrt imagery, most people only want to look at pretty pictures - therefore we'll compress our imagery via wavelet compression and save a lot of disk space and ongoing SAN costs by backing up the source imagery to tape. Noone uses it anyway. I've been around long enough to expect problems from tape backups, and to not expect my data to be there when I request a restore. I can also see an increasing need for image analysis for big picture issues such as climate change and water shortage (in Australia). Therefore, naiave as it is, I want to be 'convinced' that our data is protected for future use before agreeing to such potentially irreversible proposals. Bruce [EMAIL PROTECTED] wrote on 24/02/2008 08:44:25 AM: Bruce- It is not clear to me what sort of study you would need to convince you, as the ISO standard for encoding data into the JPEG-2000 file format is by construction mathematically and numerically lossless process. (Indeed, compression, i.e. throwing away bits so as to further reduce storage requirements, is actually not defined within the scope of the standard.) -mpg Notice: This email and any attachments may contain information that is personal, confidential, legally privileged and/or copyright.No part of it should be reproduced, adapted or communicated without the prior written consent of the copyright owner. It is the responsibility of the recipient to check for and remove viruses. If you have received this email in error, please notify the sender by return email, delete it from your system and destroy any copies. You are not authorised to use, communicate or rely on the information contained in this email. Please consider the environment before printing this email. ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
