Re: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms
Excellent idea! It would need a little bit of content management to avoid getting spam in these fields and getting feedback preferably back to the source, but that might be overcome with some brainstorming :-) Cheers, Jeroen On Aug 26, 2009, at 4:51 AM, Ted Habermann wrote: Lots of discussions on standardization of data formats, lots of challenges there. One thing that has not been mentioned (I think) is the idea of standardizing responses from users about 1) uses of data and 2) limitations of data (files too big fits in here). Both of these are included in the MD_Usage object that is part of the ISO 19115 Standard included in GeoNetwork. Would be very cool to build into GeoNetwork the capability to accept user feedback about datasets and to associate that feedback with the appropriate datasets in the GeoNetwork catalog and to make it available to 1) data providers and 2) future users... Seems straightforward, Ted -- Ted Habermann === Enterprise Data Systems Group Leader NOAA, National Geophysical Data Center V: 303.497.6472 F: 303.497.6513 "If you want to go quickly, go alone. If you want to go far, go together" Old Proverb ted.haberm...@noaa.gov == ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms
Lots of discussions on standardization of data formats, lots of challenges there. One thing that has not been mentioned (I think) is the idea of standardizing responses from users about 1) uses of data and 2) limitations of data (files too big fits in here). Both of these are included in the MD_Usage object that is part of the ISO 19115 Standard included in GeoNetwork. Would be very cool to build into GeoNetwork the capability to accept user feedback about datasets and to associate that feedback with the appropriate datasets in the GeoNetwork catalog and to make it available to 1) data providers and 2) future users... Seems straightforward, Ted -- Ted Habermann === Enterprise Data Systems Group Leader NOAA, National Geophysical Data Center V: 303.497.6472 F: 303.497.6513 "If you want to go quickly, go alone. If you want to go far, go together" Old Proverb ted.haberm...@noaa.gov == ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
RE: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms
To be clear, it's not an effort I have the bandwidth for personally. But if there were qualified developers interested in taking it on, I might be in a position to offer project guidance and a small amount of funding. -mpg -Original Message- From: discuss-boun...@lists.osgeo.org [mailto:discuss-boun...@lists.osgeo.org] On Behalf Of Landon Blake Sent: Tuesday, August 25, 2009 11:12 AM To: OSGeo Discussions Subject: RE: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms OK. I'm not an expert on images, but I'd be interested in working with you. However, I avoid C++ like a lethal strain of the Swine Flu. :] I may give some more thought to some of Bob's ideas about making it easier to work with image tiles. Thanks. Landon Office Phone Number: (209) 946-0268 Cell Phone Number: (209) 992-0658 -Original Message- From: discuss-boun...@lists.osgeo.org [mailto:discuss-boun...@lists.osgeo.org] On Behalf Of Michael P. Gerlek Sent: Monday, August 24, 2009 3:19 PM To: OSGeo Discussions Subject: RE: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms I've not given it much thought recently, to be honest. I'd need to review the current state of things in OpenJp2 (or whatever it's called) to see where they are at, what changes would be realistic and viable, how amenable they'd be to taking patches versus a fork, etc. Done properly, the work would have no "geo" specific component at all -- it would just be a new version of some of the internal algorithms. The test case would simply be to encode and decode an 500 GB(?) raw file on a box with 2 GB (?) of RAM. I would certainly not want anyone to have to build a whole new jp2 library from scratch, if that's what you meant. I'd really only be interested in C++ (or possibly mono-safe C#). -mpg -Original Message- From: discuss-boun...@lists.osgeo.org [mailto:discuss-boun...@lists.osgeo.org] On Behalf Of Landon Blake Sent: Monday, August 24, 2009 4:11 PM To: OSGeo Discussions Subject: RE: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms MPG: When you say "effort" do you mean some sort of library to support JP2 geo side of things? What programming language would you be most interested in? C++? Landon Office Phone Number: (209) 946-0268 Cell Phone Number: (209) 992-0658 -Original Message- From: discuss-boun...@lists.osgeo.org [mailto:discuss-boun...@lists.osgeo.org] On Behalf Of Michael P. Gerlek Sent: Monday, August 24, 2009 1:59 PM To: OSGeo Discussions Subject: RE: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms It would not be a good SoC thing, due to the level of expertise and time required. I (LizardTech) would likely be willing to contribute to such an effort. -mpg -Original Message- From: discuss-boun...@lists.osgeo.org [mailto:discuss-boun...@lists.osgeo.org] On Behalf Of Fawcett, David Sent: Monday, August 24, 2009 8:11 AM To: OSGeo Discussions Subject: RE: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms I realize that there are likely not a large number of people who have the expertise and experience to write this kind of code. Is this a project that should be shopped around for funding? Google Summer of Code? A grant from our ~benevolent overlord Google? Some other foundation or org interested in open data formats? David. -Original Message- From: discuss-boun...@lists.osgeo.org [mailto:discuss-boun...@lists.osgeo.org] On Behalf Of Michael P. Gerlek Sent: Thursday, August 20, 2009 4:36 PM To: OSGeo Discussions Subject: RE: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms > Do you know why there hasn't been a broader adoption of JP2? Not through lack of trying on my part :-) I think the two biggest reasons are: (1) The algorithms for handling large images in memory really are rocket science, and no one in the FOSS community has gotten the "itch" sufficiently bad enough to go and do the work needed inside the existing open source packages. Hopefully someday someone will. ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss Warning: Information provided via electronic media is not guaranteed against defects including translation and transmission errors. If the reader is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this information in error, please notify the sender immediately. ___ Disc
RE: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms
OK. I'm not an expert on images, but I'd be interested in working with you. However, I avoid C++ like a lethal strain of the Swine Flu. :] I may give some more thought to some of Bob's ideas about making it easier to work with image tiles. Thanks. Landon Office Phone Number: (209) 946-0268 Cell Phone Number: (209) 992-0658 -Original Message- From: discuss-boun...@lists.osgeo.org [mailto:discuss-boun...@lists.osgeo.org] On Behalf Of Michael P. Gerlek Sent: Monday, August 24, 2009 3:19 PM To: OSGeo Discussions Subject: RE: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms I've not given it much thought recently, to be honest. I'd need to review the current state of things in OpenJp2 (or whatever it's called) to see where they are at, what changes would be realistic and viable, how amenable they'd be to taking patches versus a fork, etc. Done properly, the work would have no "geo" specific component at all -- it would just be a new version of some of the internal algorithms. The test case would simply be to encode and decode an 500 GB(?) raw file on a box with 2 GB (?) of RAM. I would certainly not want anyone to have to build a whole new jp2 library from scratch, if that's what you meant. I'd really only be interested in C++ (or possibly mono-safe C#). -mpg -Original Message- From: discuss-boun...@lists.osgeo.org [mailto:discuss-boun...@lists.osgeo.org] On Behalf Of Landon Blake Sent: Monday, August 24, 2009 4:11 PM To: OSGeo Discussions Subject: RE: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms MPG: When you say "effort" do you mean some sort of library to support JP2 geo side of things? What programming language would you be most interested in? C++? Landon Office Phone Number: (209) 946-0268 Cell Phone Number: (209) 992-0658 -Original Message- From: discuss-boun...@lists.osgeo.org [mailto:discuss-boun...@lists.osgeo.org] On Behalf Of Michael P. Gerlek Sent: Monday, August 24, 2009 1:59 PM To: OSGeo Discussions Subject: RE: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms It would not be a good SoC thing, due to the level of expertise and time required. I (LizardTech) would likely be willing to contribute to such an effort. -mpg -Original Message- From: discuss-boun...@lists.osgeo.org [mailto:discuss-boun...@lists.osgeo.org] On Behalf Of Fawcett, David Sent: Monday, August 24, 2009 8:11 AM To: OSGeo Discussions Subject: RE: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms I realize that there are likely not a large number of people who have the expertise and experience to write this kind of code. Is this a project that should be shopped around for funding? Google Summer of Code? A grant from our ~benevolent overlord Google? Some other foundation or org interested in open data formats? David. -Original Message- From: discuss-boun...@lists.osgeo.org [mailto:discuss-boun...@lists.osgeo.org] On Behalf Of Michael P. Gerlek Sent: Thursday, August 20, 2009 4:36 PM To: OSGeo Discussions Subject: RE: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms > Do you know why there hasn't been a broader adoption of JP2? Not through lack of trying on my part :-) I think the two biggest reasons are: (1) The algorithms for handling large images in memory really are rocket science, and no one in the FOSS community has gotten the "itch" sufficiently bad enough to go and do the work needed inside the existing open source packages. Hopefully someday someone will. ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss Warning: Information provided via electronic media is not guaranteed against defects including translation and transmission errors. If the reader is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this information in error, please notify the sender immediately. ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
RE: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms
I've not given it much thought recently, to be honest. I'd need to review the current state of things in OpenJp2 (or whatever it's called) to see where they are at, what changes would be realistic and viable, how amenable they'd be to taking patches versus a fork, etc. Done properly, the work would have no "geo" specific component at all -- it would just be a new version of some of the internal algorithms. The test case would simply be to encode and decode an 500 GB(?) raw file on a box with 2 GB (?) of RAM. I would certainly not want anyone to have to build a whole new jp2 library from scratch, if that's what you meant. I'd really only be interested in C++ (or possibly mono-safe C#). -mpg -Original Message- From: discuss-boun...@lists.osgeo.org [mailto:discuss-boun...@lists.osgeo.org] On Behalf Of Landon Blake Sent: Monday, August 24, 2009 4:11 PM To: OSGeo Discussions Subject: RE: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms MPG: When you say "effort" do you mean some sort of library to support JP2 geo side of things? What programming language would you be most interested in? C++? Landon Office Phone Number: (209) 946-0268 Cell Phone Number: (209) 992-0658 -Original Message- From: discuss-boun...@lists.osgeo.org [mailto:discuss-boun...@lists.osgeo.org] On Behalf Of Michael P. Gerlek Sent: Monday, August 24, 2009 1:59 PM To: OSGeo Discussions Subject: RE: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms It would not be a good SoC thing, due to the level of expertise and time required. I (LizardTech) would likely be willing to contribute to such an effort. -mpg -Original Message- From: discuss-boun...@lists.osgeo.org [mailto:discuss-boun...@lists.osgeo.org] On Behalf Of Fawcett, David Sent: Monday, August 24, 2009 8:11 AM To: OSGeo Discussions Subject: RE: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms I realize that there are likely not a large number of people who have the expertise and experience to write this kind of code. Is this a project that should be shopped around for funding? Google Summer of Code? A grant from our ~benevolent overlord Google? Some other foundation or org interested in open data formats? David. -Original Message- From: discuss-boun...@lists.osgeo.org [mailto:discuss-boun...@lists.osgeo.org] On Behalf Of Michael P. Gerlek Sent: Thursday, August 20, 2009 4:36 PM To: OSGeo Discussions Subject: RE: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms > Do you know why there hasn't been a broader adoption of JP2? Not through lack of trying on my part :-) I think the two biggest reasons are: (1) The algorithms for handling large images in memory really are rocket science, and no one in the FOSS community has gotten the "itch" sufficiently bad enough to go and do the work needed inside the existing open source packages. Hopefully someday someone will. ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss Warning: Information provided via electronic media is not guaranteed against defects including translation and transmission errors. If the reader is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this information in error, please notify the sender immediately. ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
RE: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms
MPG: When you say "effort" do you mean some sort of library to support JP2 geo side of things? What programming language would you be most interested in? C++? Landon Office Phone Number: (209) 946-0268 Cell Phone Number: (209) 992-0658 -Original Message- From: discuss-boun...@lists.osgeo.org [mailto:discuss-boun...@lists.osgeo.org] On Behalf Of Michael P. Gerlek Sent: Monday, August 24, 2009 1:59 PM To: OSGeo Discussions Subject: RE: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms It would not be a good SoC thing, due to the level of expertise and time required. I (LizardTech) would likely be willing to contribute to such an effort. -mpg -Original Message- From: discuss-boun...@lists.osgeo.org [mailto:discuss-boun...@lists.osgeo.org] On Behalf Of Fawcett, David Sent: Monday, August 24, 2009 8:11 AM To: OSGeo Discussions Subject: RE: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms I realize that there are likely not a large number of people who have the expertise and experience to write this kind of code. Is this a project that should be shopped around for funding? Google Summer of Code? A grant from our ~benevolent overlord Google? Some other foundation or org interested in open data formats? David. -Original Message- From: discuss-boun...@lists.osgeo.org [mailto:discuss-boun...@lists.osgeo.org] On Behalf Of Michael P. Gerlek Sent: Thursday, August 20, 2009 4:36 PM To: OSGeo Discussions Subject: RE: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms > Do you know why there hasn't been a broader adoption of JP2? Not through lack of trying on my part :-) I think the two biggest reasons are: (1) The algorithms for handling large images in memory really are rocket science, and no one in the FOSS community has gotten the "itch" sufficiently bad enough to go and do the work needed inside the existing open source packages. Hopefully someday someone will. ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss Warning: Information provided via electronic media is not guaranteed against defects including translation and transmission errors. If the reader is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this information in error, please notify the sender immediately. ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
RE: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms
It would not be a good SoC thing, due to the level of expertise and time required. I (LizardTech) would likely be willing to contribute to such an effort. -mpg -Original Message- From: discuss-boun...@lists.osgeo.org [mailto:discuss-boun...@lists.osgeo.org] On Behalf Of Fawcett, David Sent: Monday, August 24, 2009 8:11 AM To: OSGeo Discussions Subject: RE: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms I realize that there are likely not a large number of people who have the expertise and experience to write this kind of code. Is this a project that should be shopped around for funding? Google Summer of Code? A grant from our ~benevolent overlord Google? Some other foundation or org interested in open data formats? David. -Original Message- From: discuss-boun...@lists.osgeo.org [mailto:discuss-boun...@lists.osgeo.org] On Behalf Of Michael P. Gerlek Sent: Thursday, August 20, 2009 4:36 PM To: OSGeo Discussions Subject: RE: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms > Do you know why there hasn't been a broader adoption of JP2? Not through lack of trying on my part :-) I think the two biggest reasons are: (1) The algorithms for handling large images in memory really are rocket science, and no one in the FOSS community has gotten the "itch" sufficiently bad enough to go and do the work needed inside the existing open source packages. Hopefully someday someone will. ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms
Rene, > how could we standardize for those future uses? I was thinking more along the lines of a standard file size more than anything. Nto all deliverable are even able to accomplish capturing a whole contract in a single file, so if even a separation into more than one file is needed, why not come up with some form of sizing standard, I realize that there are uses other than those that I have, but some form of data tiling seems appropriate with this discussion. bobb >>> "René A. Enguehard" wrote: I agree, primarily because I just got a dataset from the city that was a 5Gb raster. I know hardrive space is cheap and so is processing power but still, it took literally hours to get anything meaningful out of it. Picking a more appropriate resolution, better compression and eventually switching file formats would have helped immensely but wasn't done since the prevailing attitude is that bigger is better. This attitude is really the same as in the programming world, where programs keep getting slower and slower (in terms of time complexity) but it's deemed "okay" since computers are also getting faster. I don't think this attitude is going to change any time soon though and making some form of standard would simply not work. How could we standardize what resolution and compression we should be using on specific datasets for specific applications? There are uses we haven't even thought up yet, how could we standardize for those future uses? Just my 0.02$ R Bob Basques wrote: > > All, > > > Ok, I'm probably going to get someone irritated, but here goes . . . > > > Why not approach this from the other end of the spectrum and work at > making the original files smaller. Work with the providers to make > the images smaller in the first place, or at least come up with a > maximum practical size to work with, I mean if this is the only (or > biggest reason) for implementing JP2, then getting folks to make the > smaller deliverables seems like a better long term approach. > > > Here's my reasoning, we're never (ever?) going to hit the top end on > how big files ever get, resolution just keeps going up and up, so > there is always going to be some upper limit that will need to be > breached somehow. Working out a proper method for segregating the > data up front (dare I say it), as some sort of standard (which can be > adjusted as time passes) will make everything work nicely, then all > will work with available tools when they are available, if tools to > handle larger datasets become available, and the community feels there > is a reason/need that these new larger files need to be handled, then > they get to change the standard. > > > bobb > > > > > > > > >>> "Fawcett, David" wrote: > > > I realize that there are likely not a large number of people who have > the expertise and experience to write this kind of code. > > Is this a project that should be shopped around for funding? Google > Summer of Code? A grant from our ~benevolent overlord Google? Some > other foundation or org interested in open data formats? > > David. > -----Original Message- > From: discuss-boun...@lists.osgeo.org > [mailto:discuss-boun...@lists.osgeo.org] On Behalf Of Michael P. Gerlek > Sent: Thursday, August 20, 2009 4:36 PM > To: OSGeo Discussions > Subject: RE: [OSGeo-Discuss] Open File Formats and Proprietary > Algorithms > > > > > Do you know why there hasn't been a broader adoption of JP2? > > Not through lack of trying on my part :-) > > I think the two biggest reasons are: > > (1) The algorithms for handling large images in memory really are rocket > science, and no one in the FOSS community has gotten the "itch" > sufficiently bad enough to go and do the work needed inside the existing > open source packages. Hopefully someday someone will. > > > ___ > Discuss mailing list > Discuss@lists.osgeo.org http://lists.> Discuss@lists.osgeo.org > http://lists.osgeo.org/mailman/listinfo/discuss > > > > ___ > Discuss mailing list > Discuss@lists.osgeo.org > http://lists.osgeo.org/mailman/listinfo/discuss > ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms
I agree, primarily because I just got a dataset from the city that was a 5Gb raster. I know hardrive space is cheap and so is processing power but still, it took literally hours to get anything meaningful out of it. Picking a more appropriate resolution, better compression and eventually switching file formats would have helped immensely but wasn't done since the prevailing attitude is that bigger is better. This attitude is really the same as in the programming world, where programs keep getting slower and slower (in terms of time complexity) but it's deemed "okay" since computers are also getting faster. I don't think this attitude is going to change any time soon though and making some form of standard would simply not work. How could we standardize what resolution and compression we should be using on specific datasets for specific applications? There are uses we haven't even thought up yet, how could we standardize for those future uses? Just my 0.02$ R Bob Basques wrote: All, Ok, I'm probably going to get someone irritated, but here goes . . . Why not approach this from the other end of the spectrum and work at making the original files smaller. Work with the providers to make the images smaller in the first place, or at least come up with a maximum practical size to work with, I mean if this is the only (or biggest reason) for implementing JP2, then getting folks to make the smaller deliverables seems like a better long term approach. Here's my reasoning, we're never (ever?) going to hit the top end on how big files ever get, resolution just keeps going up and up, so there is always going to be some upper limit that will need to be breached somehow. Working out a proper method for segregating the data up front (dare I say it), as some sort of standard (which can be adjusted as time passes) will make everything work nicely, then all will work with available tools when they are available, if tools to handle larger datasets become available, and the community feels there is a reason/need that these new larger files need to be handled, then they get to change the standard. bobb >>> "Fawcett, David" wrote: I realize that there are likely not a large number of people who have the expertise and experience to write this kind of code. Is this a project that should be shopped around for funding? Google Summer of Code? A grant from our ~benevolent overlord Google? Some other foundation or org interested in open data formats? David. -Original Message- From: discuss-boun...@lists.osgeo.org [mailto:discuss-boun...@lists.osgeo.org] On Behalf Of Michael P. Gerlek Sent: Thursday, August 20, 2009 4:36 PM To: OSGeo Discussions Subject: RE: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms > Do you know why there hasn't been a broader adoption of JP2? Not through lack of trying on my part :-) I think the two biggest reasons are: (1) The algorithms for handling large images in memory really are rocket science, and no one in the FOSS community has gotten the "itch" sufficiently bad enough to go and do the work needed inside the existing open source packages. Hopefully someday someone will. ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
RE: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms
All, Ok, I'm probably going to get someone irritated, but here goes . . . Why not approach this from the other end of the spectrum and work at making the original files smaller. Work with the providers to make the images smaller in the first place, or at least come up with a maximum practical size to work with, I mean if this is the only (or biggest reason) for implementing JP2, then getting folks to make the smaller deliverables seems like a better long term approach. Here's my reasoning, we're never (ever?) going to hit the top end on how big files ever get, resolution just keeps going up and up, so there is always going to be some upper limit that will need to be breached somehow. Working out a proper method for segregating the data up front (dare I say it), as some sort of standard (which can be adjusted as time passes) will make everything work nicely, then all will work with available tools when they are available, if tools to handle larger datasets become available, and the community feels there is a reason/need that these new larger files need to be handled, then they get to change the standard. bobb >>> "Fawcett, David" wrote: I realize that there are likely not a large number of people who have the expertise and experience to write this kind of code. Is this a project that should be shopped around for funding? Google Summer of Code? A grant from our ~benevolent overlord Google? Some other foundation or org interested in open data formats? David. -Original Message- From: discuss-boun...@lists.osgeo.org [mailto:discuss-boun...@lists.osgeo.org] On Behalf Of Michael P. Gerlek Sent: Thursday, August 20, 2009 4:36 PM To: OSGeo Discussions Subject: RE: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms > Do you know why there hasn't been a broader adoption of JP2? Not through lack of trying on my part :-) I think the two biggest reasons are: (1) The algorithms for handling large images in memory really are rocket science, and no one in the FOSS community has gotten the "itch" sufficiently bad enough to go and do the work needed inside the existing open source packages. Hopefully someday someone will. ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms
Fawcett, David wrote: I realize that there are likely not a large number of people who have the expertise and experience to write this kind of code. Is this a project that should be shopped around for funding? I'd say so. Google Summer of Code? IMHO, it definitely doesn't such timing/schedule. Best regards, -- Mateusz Loskot, http://mateusz.loskot.net ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
RE: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms
I realize that there are likely not a large number of people who have the expertise and experience to write this kind of code. Is this a project that should be shopped around for funding? Google Summer of Code? A grant from our ~benevolent overlord Google? Some other foundation or org interested in open data formats? David. -Original Message- From: discuss-boun...@lists.osgeo.org [mailto:discuss-boun...@lists.osgeo.org] On Behalf Of Michael P. Gerlek Sent: Thursday, August 20, 2009 4:36 PM To: OSGeo Discussions Subject: RE: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms > Do you know why there hasn't been a broader adoption of JP2? Not through lack of trying on my part :-) I think the two biggest reasons are: (1) The algorithms for handling large images in memory really are rocket science, and no one in the FOSS community has gotten the "itch" sufficiently bad enough to go and do the work needed inside the existing open source packages. Hopefully someday someone will. ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms
I regularly use the gdal tools and python to convert/reproject the NAIP county mosaics from UTM to NC State Plane in a Quarter Quad format. It really doesn't take that long on a 64-bit linux box with a couple of big hard drives. Once you get things out of MrSid format, things go pretty quickly. Doug Doug Newcomb USFWS Raleigh, NC 919-856-4520 ext. 14 doug_newc...@fws.gov - The opinions I express are my own and are not representative of the official policy of the U.S.Fish and Wildlife Service or Dept. of Interior. Life is too short for undocumented, proprietary data formats. Eric Wolf To Sent by: OSGeo Discussions discuss-boun...@l ists.osgeo.org cc Subject 08/20/2009 04:53 Re: [OSGeo-Discuss] Open File PMFormats and Proprietary Algorithms Please respond to OSGeo Discussions Interesting... I can understand why NAIP was in MRSID. It's a pretty large dataset - and I think .SID was more widely supported than JP2 until recently. The USDA site does provide links to PCI Geomatics FreeView, which can read .SID format but not save it. IrfanView, with a plugin, can read SID format and convert. So it's not a dead-end format. And it sure beats SDTS! I think data interchange and real interoperability has only recently been possible for large raster datasets. It's still a chore if you have to re-project large raster datasets. This may add some content to a research paper I'm working on. -Eric -=--=---===---=--=-=--=---==---=--=-=- Eric B. Wolf New! 720-334-7734 USGS Geographer Center of Excellence in GIScience PhD Student CU-Boulder - Geography On Thu, Aug 20, 2009 at 2:22 PM, Landon Blake wrote: Eric, The imagery I am talking about is from the USDA APFO: This FAQ contains a snippet about the format: http://www.fsa.usda.gov/FSA/apfoapp?area=home&subject=prog&topic=nai In an interesting turn of events I note that as of 2008, the USDA is releasing the county mosaics in JP2 format, not in MRSID. I am not sure what brought about this change, and I wasn’t aware that it had been made. The same web page indicates that there is a shapefile index for the individual image tiles. It appears that you can also download the county mosaics online. A lot of this has changed (improved) in the last couple of years. I’m glad I checked again. That being said, the principles from our discussion still apply. :] Landon Office Phone Number: (209) 946-0268 Cell Phone Number: (209) 992-0658 From: discuss-boun...@lists.osgeo.org [mailto: discuss-boun...@lists.osgeo.org] On Behalf Of Eric Wolf Sent: Thursday, August 20, 2009 1:15 PM To: OSGeo Discussions Subject: Re: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms The MRSID format is a very special case - and perhaps an opportunity for a new FOSS file format. MRSID is a lossless, fractal-based, multi-scale raster compression format. LizardTech has the algorithms to encode and decode MRSID locked up in copyrights, and I believe, patents. Even companies like ESRI shell out big bucks to LizardTech to be able to read and write the MRSID format. I guess I missed the context of the discussion. Is the government releasing certain data exclusively in this format? If so, I think the argument can be made against this practice. The different in compression between MRSID and gziped TIFFs isn't really that great in this day of cheap disks and fat pipes. -Eric -=--=---===---=--=-=--=---==---=--=-=- Eric B. Wolf New! 720-334-7734 USGS Geographer Center of Excellence in GIScience PhD Student CU-Boulder - Geography On Thu, Aug 20, 2009 at 12:27 PM, Landon Blake wrote: I realized that publishi
Re: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms [SEC=UNCLASSIFIED]
Bruce, For your information, I expect to be talking to Chris, from the National Archives Australia this coming week. Apparently they store their data sets in Open formats, but don't know how to store GIS datasets in open formats. I'm hoping that we can help. Bruce Bannerman wrote: IMO: Just another thought on this issue (though we do seem to be recycling arguments over the years...): Assuming that I have a very large archive of spatial data, be it imagery or any other spatial format and that I store my data in a variety of proprietary formats: In ten years from now, can I be sure that: - the company that created, understands, and holds the IP in the data format will still be around? - there will still be software that runs on the then current operating environment, that can read and 'fully exploit' the data in the proprietary standard? - that this future software will work seamlessly with my then current spatial environment? - if all of the above risks prove to eventuate, can I be sure that I'll be able to salvage my data into another format, retaining its complete semantic context? IMO, it is a high risk proposition to lock public (or private) archives away in proprietary data formats. It makes more sense to use open standards and formats that are publically available. Bruce Bannerman -Original Message- From: discuss-boun...@lists.osgeo.org [mailto:discuss-boun...@lists.osgeo.org] On Behalf Of Michael P. Gerlek Sent: Friday, 21 August 2009 6:55 AM To: OSGeo Discussions Subject: RE: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms Some clarifications: - MrSID has both lossy and lossless modes - MrSID is not fractal based; it uses wavelets (and arithmetic encoding) - you can't copyright algorithms; the MrSID source code certainly is, however - MrSID relies on a number of patents, not all of which are owned by LizardTech - reading MrSID does not require any fees; we have libraries you can download, although they are not open source That said, some editorial comments (although I'm now wishing I hadn't been so quick to rise to Landon's bait :-) - Some of you know the history of trying to open source MrSID; I won't go into that here, except to say that LizardTech doesn't own all of the required IP needed to make that happen. - If we are speaking of the NAIP data, then no, it is not exclusively available in MrSID format; it is also shipped as GeoTIFFs. - JPEG 2000 is a very robust open standard alternative to MrSID, and a number of players already support it (including LizardTech), but not enough to make it viable for certain domains like NAIP. - some of you also know the history on open JP2 support: there is today no open source implementation of JP2 that is suitable for geo work. Alas. -mpg From: discuss-boun...@lists.osgeo.org [mailto:discuss-boun...@lists.osgeo.org] On Behalf Of Eric Wolf Sent: Thursday, August 20, 2009 2:15 PM To: OSGeo Discussions Subject: Re: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms The MRSID format is a very special case - and perhaps an opportunity for a new FOSS file format. MRSID is a lossless, fractal-based, multi-scale raster compression format. LizardTech has the algorithms to encode and decode MRSID locked up in copyrights, and I believe, patents. Even companies like ESRI shell out big bucks to LizardTech to be able to read and write the MRSID format. I guess I missed the context of the discussion. Is the government releasing certain data exclusively in this format? If so, I think the argument can be made against this practice. The different in compression between MRSID and gziped TIFFs isn't really that great in this day of cheap disks and fat pipes. -Eric -=--=---===---=--=-=--=---==---=--=-=- Eric B. WolfNew! 720-334-7734 USGS Geographer Center of Excellence in GIScience PhD Student CU-Boulder - Geography ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss -- Cameron Shorter Geospatial Systems Architect Tel: +61 (0)2 8570 5050 Mob: +61 (0)419 142 254 Think Globally, Fix Locally Geospatial Solutions enhanced with Open Standards and Open Source http://www.lisasoft.com ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
RE: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms[SEC=UNCLASSIFIED]
All, Opticks is an open source remote sensing application and development framework. We recently started the process to add JPEG 2000 support to our framework. We picked OpenJpeg to add JPEG 2000 support to our application. They are also open source. We currently support importing JPEG 2000 files but we are currently limited to the 4GB memory size after decoding. Our plan is to continue development and to upgrade to OpenJpeg 2.0 once they have a stable release. That will allow Opticks to use a pager to display and support much larger files. Michael Considine -Original Message- From: discuss-boun...@lists.osgeo.org [mailto:discuss-boun...@lists.osgeo.org] On Behalf Of Bruce Bannerman Sent: Thursday, August 20, 2009 8:15 PM To: 'OSGeo Discussions' Subject: RE: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms[SEC=UNCLASSIFIED] IMO: Just another thought on this issue (though we do seem to be recycling arguments over the years...): Assuming that I have a very large archive of spatial data, be it imagery or any other spatial format and that I store my data in a variety of proprietary formats: In ten years from now, can I be sure that: - the company that created, understands, and holds the IP in the data format will still be around? - there will still be software that runs on the then current operating environment, that can read and 'fully exploit' the data in the proprietary standard? - that this future software will work seamlessly with my then current spatial environment? - if all of the above risks prove to eventuate, can I be sure that I'll be able to salvage my data into another format, retaining its complete semantic context? IMO, it is a high risk proposition to lock public (or private) archives away in proprietary data formats. It makes more sense to use open standards and formats that are publically available. Bruce Bannerman > -Original Message- > From: discuss-boun...@lists.osgeo.org > [mailto:discuss-boun...@lists.osgeo.org] On Behalf Of Michael > P. Gerlek > Sent: Friday, 21 August 2009 6:55 AM > To: OSGeo Discussions > Subject: RE: [OSGeo-Discuss] Open File Formats and > Proprietary Algorithms > > Some clarifications: > > > > - MrSID has both lossy and lossless modes > > - MrSID is not fractal based; it uses wavelets (and > arithmetic encoding) > > - you can't copyright algorithms; the MrSID source code > certainly is, however > > - MrSID relies on a number of patents, not all of which are > owned by LizardTech > > - reading MrSID does not require any fees; we have libraries > you can download, although they are not open source > > > > That said, some editorial comments (although I'm now wishing > I hadn't been so quick to rise to Landon's bait :-) > > > > - Some of you know the history of trying to open source > MrSID; I won't go into that here, except to say that > LizardTech doesn't own all of the required IP needed to make > that happen. > > - If we are speaking of the NAIP data, then no, it is not > exclusively available in MrSID format; it is also shipped as GeoTIFFs. > > - JPEG 2000 is a very robust open standard alternative to > MrSID, and a number of players already support it (including > LizardTech), but not enough to make it viable for certain > domains like NAIP. > > - some of you also know the history on open JP2 support: > there is today no open source implementation of JP2 that is > suitable for geo work. Alas. > > > > -mpg > > > > > > From: discuss-boun...@lists.osgeo.org > [mailto:discuss-boun...@lists.osgeo.org] On Behalf Of Eric Wolf > Sent: Thursday, August 20, 2009 2:15 PM > To: OSGeo Discussions > Subject: Re: [OSGeo-Discuss] Open File Formats and > Proprietary Algorithms > > > > The MRSID format is a very special case - and perhaps an > opportunity for a new FOSS file format. MRSID is a lossless, > fractal-based, multi-scale raster compression format. > LizardTech has the algorithms to encode and decode MRSID > locked up in copyrights, and I believe, patents. Even > companies like ESRI shell out big bucks to LizardTech to be > able to read and write the MRSID format. > > > > I guess I missed the context of the discussion. Is the > government releasing certain data exclusively in this format? > If so, I think the argument can be made against this > practice. The different in compression between MRSID and > gziped TIFFs isn't really that great in this day of cheap > disks and fat pipes. > > > > -Eric > > > -=--=---===---=--=-=--=---==---=--=-=- > Eric B. WolfNew! 72
RE: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms [SEC=UNCLASSIFIED]
IMO: Just another thought on this issue (though we do seem to be recycling arguments over the years...): Assuming that I have a very large archive of spatial data, be it imagery or any other spatial format and that I store my data in a variety of proprietary formats: In ten years from now, can I be sure that: - the company that created, understands, and holds the IP in the data format will still be around? - there will still be software that runs on the then current operating environment, that can read and 'fully exploit' the data in the proprietary standard? - that this future software will work seamlessly with my then current spatial environment? - if all of the above risks prove to eventuate, can I be sure that I'll be able to salvage my data into another format, retaining its complete semantic context? IMO, it is a high risk proposition to lock public (or private) archives away in proprietary data formats. It makes more sense to use open standards and formats that are publically available. Bruce Bannerman > -Original Message- > From: discuss-boun...@lists.osgeo.org > [mailto:discuss-boun...@lists.osgeo.org] On Behalf Of Michael > P. Gerlek > Sent: Friday, 21 August 2009 6:55 AM > To: OSGeo Discussions > Subject: RE: [OSGeo-Discuss] Open File Formats and > Proprietary Algorithms > > Some clarifications: > > > > - MrSID has both lossy and lossless modes > > - MrSID is not fractal based; it uses wavelets (and > arithmetic encoding) > > - you can't copyright algorithms; the MrSID source code > certainly is, however > > - MrSID relies on a number of patents, not all of which are > owned by LizardTech > > - reading MrSID does not require any fees; we have libraries > you can download, although they are not open source > > > > That said, some editorial comments (although I'm now wishing > I hadn't been so quick to rise to Landon's bait :-) > > > > - Some of you know the history of trying to open source > MrSID; I won't go into that here, except to say that > LizardTech doesn't own all of the required IP needed to make > that happen. > > - If we are speaking of the NAIP data, then no, it is not > exclusively available in MrSID format; it is also shipped as GeoTIFFs. > > - JPEG 2000 is a very robust open standard alternative to > MrSID, and a number of players already support it (including > LizardTech), but not enough to make it viable for certain > domains like NAIP. > > - some of you also know the history on open JP2 support: > there is today no open source implementation of JP2 that is > suitable for geo work. Alas. > > > > -mpg > > > > > > From: discuss-boun...@lists.osgeo.org > [mailto:discuss-boun...@lists.osgeo.org] On Behalf Of Eric Wolf > Sent: Thursday, August 20, 2009 2:15 PM > To: OSGeo Discussions > Subject: Re: [OSGeo-Discuss] Open File Formats and > Proprietary Algorithms > > > > The MRSID format is a very special case - and perhaps an > opportunity for a new FOSS file format. MRSID is a lossless, > fractal-based, multi-scale raster compression format. > LizardTech has the algorithms to encode and decode MRSID > locked up in copyrights, and I believe, patents. Even > companies like ESRI shell out big bucks to LizardTech to be > able to read and write the MRSID format. > > > > I guess I missed the context of the discussion. Is the > government releasing certain data exclusively in this format? > If so, I think the argument can be made against this > practice. The different in compression between MRSID and > gziped TIFFs isn't really that great in this day of cheap > disks and fat pipes. > > > > -Eric > > > -=--=---===---=--=-=--=---==---=--=-=- > Eric B. WolfNew! 720-334-7734 > USGS Geographer > Center of Excellence in GIScience > PhD Student > CU-Boulder - Geography > > > > > > > ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
RE: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms
Good post Christopher. I will think about what you have said. In the meantime, I won't be using any big images. :] Landon Office Phone Number: (209) 946-0268 Cell Phone Number: (209) 992-0658 -Original Message- From: discuss-boun...@lists.osgeo.org [mailto:discuss-boun...@lists.osgeo.org] On Behalf Of Christopher Schmidt Sent: Thursday, August 20, 2009 2:50 PM To: OSGeo Discussions Subject: Re: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms On Thu, Aug 20, 2009 at 01:57:16PM -0700, Landon Blake wrote: > MPG: > > Thanks for the clarification. > > When you said "there is today no open source implementation of JP2 that > is suitable for geo work" do you mean that there is no open source > library that can read and write JP2? If so, who is using the format? There are: 1. Several non-open source implementations (most of which cost money) which work at geo-sized JP2 images. 2. Many use cases of JPEG2000 which involve imagery at sizes that are less than geo. (This is the much more common case, in my research.) > Do you know why there hasn't been a broader adoption of JP2? I'm not sure what your definition is of "broader adoption"; many of the datasources I worked with for OAM were provided in either JP2 or MrSID formats. I would almost always go with MrSID, because I could: * Work with it easily, and for free * It was typically significantly smaller. Perhaps you're asking why there hasn't been more open source software written to handle large, highly compressed JP2 images better -- to which I would point out that there isn't *any* format that has good open source support for large, highly compressed images. (gzipped TIFFs work to some extent, but don't compare to the benefits gained by JP2 or MrSID in many cases.) It's a hard problem, and -- given that the major players see the costs to 'pay to play' as being trivial (and they typically are, in the big scheme of things), not in a situation where it's likely that the people with ots of money ar ein a position to spend it on open source, rather than simply paying a smaller amount for existing non-opensource solutions. Despite the claims that 'disks are cheap and bandwidth is free', many providers *are* limited by bandwidth: MassGIS, for example, had to put in cash for a costly upgrade to their badnwidth solely due to the demand put on their servers by people downloading aerial imagery. Those funds could have gone to funding more open geodata, but instead were used to maek the data that already existed more readily available. These things *do* matter, and MrSID offers, by far, the best 'bang for the buck' for amount of data per byte of download. This applies even more at the consumer end; when you talk about consuming data, MrSID is even *more* user-friendly, because the users (who have limited bandwidth) are able to open it more easily. Additionally, many viewers which include MrSID support are able to display larger images -- due to the MrSID library -- than they would be by opening the entire image in RAM or something similar. Many of my friends have used MrSID for looking at thigns like Shakespear's Folios, because tools like IfranView include it by default, and the tool "Just Works" better than anything else. I believe that the important things in terms of delivering public content to users are: * License -- Are they allowed to do what they want with it? * Ease of use -- Is it *possible* For them to do what they want with it, including downloading it in the first place? * Openness -- Can they do what htey want with it with free/open tools? If the formwer two are true, then the latter -- openness -- can be handled by third parties. Imagine that you have two options: * Data provided online, for users to download, in MrSID * Data provided on CDs, for users to have shipped to them, in GeoTIFF (The latter will almost always have a non-trivial fee, because it involves person time, but ignore that for the time being.) If these are your options -- and this *is* the case for a non-zero number of imagery providers -- which one would you prefer to use? Best Regards, -- Christopher Schmidt Web Developer ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss Warning: Information provided via electronic media is not guaranteed against defects including translation and transmission errors. If the reader is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this information in error, please notify the sender immediately. ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms
On Thu, Aug 20, 2009 at 01:57:16PM -0700, Landon Blake wrote: > MPG: > > Thanks for the clarification. > > When you said "there is today no open source implementation of JP2 that > is suitable for geo work" do you mean that there is no open source > library that can read and write JP2? If so, who is using the format? There are: 1. Several non-open source implementations (most of which cost money) which work at geo-sized JP2 images. 2. Many use cases of JPEG2000 which involve imagery at sizes that are less than geo. (This is the much more common case, in my research.) > Do you know why there hasn't been a broader adoption of JP2? I'm not sure what your definition is of "broader adoption"; many of the datasources I worked with for OAM were provided in either JP2 or MrSID formats. I would almost always go with MrSID, because I could: * Work with it easily, and for free * It was typically significantly smaller. Perhaps you're asking why there hasn't been more open source software written to handle large, highly compressed JP2 images better -- to which I would point out that there isn't *any* format that has good open source support for large, highly compressed images. (gzipped TIFFs work to some extent, but don't compare to the benefits gained by JP2 or MrSID in many cases.) It's a hard problem, and -- given that the major players see the costs to 'pay to play' as being trivial (and they typically are, in the big scheme of things), not in a situation where it's likely that the people with ots of money ar ein a position to spend it on open source, rather than simply paying a smaller amount for existing non-opensource solutions. Despite the claims that 'disks are cheap and bandwidth is free', many providers *are* limited by bandwidth: MassGIS, for example, had to put in cash for a costly upgrade to their badnwidth solely due to the demand put on their servers by people downloading aerial imagery. Those funds could have gone to funding more open geodata, but instead were used to maek the data that already existed more readily available. These things *do* matter, and MrSID offers, by far, the best 'bang for the buck' for amount of data per byte of download. This applies even more at the consumer end; when you talk about consuming data, MrSID is even *more* user-friendly, because the users (who have limited bandwidth) are able to open it more easily. Additionally, many viewers which include MrSID support are able to display larger images -- due to the MrSID library -- than they would be by opening the entire image in RAM or something similar. Many of my friends have used MrSID for looking at thigns like Shakespear's Folios, because tools like IfranView include it by default, and the tool "Just Works" better than anything else. I believe that the important things in terms of delivering public content to users are: * License -- Are they allowed to do what they want with it? * Ease of use -- Is it *possible* For them to do what they want with it, including downloading it in the first place? * Openness -- Can they do what htey want with it with free/open tools? If the formwer two are true, then the latter -- openness -- can be handled by third parties. Imagine that you have two options: * Data provided online, for users to download, in MrSID * Data provided on CDs, for users to have shipped to them, in GeoTIFF (The latter will almost always have a non-trivial fee, because it involves person time, but ignore that for the time being.) If these are your options -- and this *is* the case for a non-zero number of imagery providers -- which one would you prefer to use? Best Regards, -- Christopher Schmidt Web Developer ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
RE: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms
I'll mention too the question of patents and JP2, since this thread is bound to
get into THAT issue too before long :-)
Some of the algorithms within the JP2 standard (from ISO) are patented.
However, the companies in question have agreed to not exercise their rights on
those patents for any implementation of the standard. That is, if you write a
ISO-compliant JP2 encoder, Company X won't come after you. This is a good
thing, and is not uncommon practice for some standards groups. It's better for
us than the RAND ("reasonable and non-discriminatory") clauses that get used by
some groups.
However, there is an interesting philosophical consideration for the open
source community here.
Let's say I write a nice, compliant MpgJp2 library on Monday and open source
it. Landon looks at my code and, smart cookie that he is, realizes that he
could improve the overall compression ratio by tweaking one of the core
algorithms. He forks my code, makes the change, and posts the SunburnedJp2
library to the web on Tuesday night. Cool. We like that. Open source in
action.
But wait -- Wednesday morning, he finds an email from Company X's lawyers in
his inbox: he is now in violation of X's patent, because he is not using the
patent within the bounds of a "compliant JP2 encoder". He broke the file
format. ["You break it, you buy it"?] It's not a "JPEG 2000" library anymore.
Some open source partisans may therefore consider the JP2 standard to not be
truly open enough.
I'm sure there are other standards with this same problem, although I don't
know of any offhand.
-mpg
From: discuss-boun...@lists.osgeo.org [mailto:discuss-boun...@lists.osgeo.org]
On Behalf Of Landon Blake
Sent: Thursday, August 20, 2009 2:57 PM
To: OSGeo Discussions
Subject: RE: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms
MPG:
Thanks for the clarification.
When you said "there is today no open source implementation of JP2 that is
suitable for geo work" do you mean that there is no open source library that
can read and write JP2? If so, who is using the format?
Do you know why there hasn't been a broader adoption of JP2?
(I should also add the MPG helped me publish a short article in support for
open file formats, so I know he is on our side.) :]
Landon
Office Phone Number: (209) 946-0268
Cell Phone Number: (209) 992-0658
From: discuss-boun...@lists.osgeo.org [mailto:discuss-boun...@lists.osgeo.org]
On Behalf Of Michael P. Gerlek
Sent: Thursday, August 20, 2009 1:55 PM
To: OSGeo Discussions
Subject: RE: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms
Some clarifications:
- MrSID has both lossy and lossless modes
- MrSID is not fractal based; it uses wavelets (and arithmetic encoding)
- you can't copyright algorithms; the MrSID source code certainly is, however
- MrSID relies on a number of patents, not all of which are owned by LizardTech
- reading MrSID does not require any fees; we have libraries you can download,
although they are not open source
That said, some editorial comments (although I'm now wishing I hadn't been so
quick to rise to Landon's bait :-)
- Some of you know the history of trying to open source MrSID; I won't go into
that here, except to say that LizardTech doesn't own all of the required IP
needed to make that happen.
- If we are speaking of the NAIP data, then no, it is not exclusively available
in MrSID format; it is also shipped as GeoTIFFs.
- JPEG 2000 is a very robust open standard alternative to MrSID, and a number
of players already support it (including LizardTech), but not enough to make it
viable for certain domains like NAIP.
- some of you also know the history on open JP2 support: there is today no open
source implementation of JP2 that is suitable for geo work. Alas.
-mpg
From: discuss-boun...@lists.osgeo.org [mailto:discuss-boun...@lists.osgeo.org]
On Behalf Of Eric Wolf
Sent: Thursday, August 20, 2009 2:15 PM
To: OSGeo Discussions
Subject: Re: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms
The MRSID format is a very special case - and perhaps an opportunity for a new
FOSS file format. MRSID is a lossless, fractal-based, multi-scale raster
compression format. LizardTech has the algorithms to encode and decode MRSID
locked up in copyrights, and I believe, patents. Even companies like ESRI shell
out big bucks to LizardTech to be able to read and write the MRSID format.
I guess I missed the context of the discussion. Is the government releasing
certain data exclusively in this format? If so, I think the argument can be
made against this practice. The different in compression between MRSID and
gziped TIFFs isn't really that great in this day of cheap disks and fat pipes.
-Eric
-=--=---===---=--=-=--=---=---
RE: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms
Landon asked: > When you said "there is today no open source implementation of JP2 that is > suitable for geo > work" do you mean that there is no open source library that can read and > write JP2? If so, > who is using the format? There are a few implementations of JP2 around. The Kakadu library, which is extremely compliant and featureful and robust (and correspondingly extremely big and complicated and scary) is the best-known package: it is available only via licensing fees. LizardTech uses Kakadu, in fact, and a number of geo vendors use either Kakadu directly or LizardTech's packaging of it. The ER Mapper folks had a JP2 solution at one time, but I never understood their licensing terms to be OSI compliant -- and since they got bought out by Leica I've sort of stopped tracking that issue. If anyone has any current info, I'd like to hear it. There are a couple truly open source libraries, but none have been written in such a way as to be able to support geo-sized imagery (>500MB, say). Doing the wavelet algorithms efficiently for large data sets requires rocket science. > Do you know why there hasn't been a broader adoption of JP2? Not through lack of trying on my part :-) I think the two biggest reasons are: (1) The algorithms for handling large images in memory really are rocket science, and no one in the FOSS community has gotten the "itch" sufficiently bad enough to go and do the work needed inside the existing open source packages. Hopefully someday someone will. (2) MrSID (and, perhaps, ECW) are widely used and supported. Philosophical motivations aside, MrSID and ECW have historically gotten the job done and so the need for JP2 just isn't as high as it otherwise might be. That said, NGA is a good counter-example. They support JP2 in a number of areas already and have mandates to broaden that support. -mpg ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
RE: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms
MPG: Thanks for the clarification. When you said "there is today no open source implementation of JP2 that is suitable for geo work" do you mean that there is no open source library that can read and write JP2? If so, who is using the format? Do you know why there hasn't been a broader adoption of JP2? (I should also add the MPG helped me publish a short article in support for open file formats, so I know he is on our side.) :] Landon Office Phone Number: (209) 946-0268 Cell Phone Number: (209) 992-0658 From: discuss-boun...@lists.osgeo.org [mailto:discuss-boun...@lists.osgeo.org] On Behalf Of Michael P. Gerlek Sent: Thursday, August 20, 2009 1:55 PM To: OSGeo Discussions Subject: RE: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms Some clarifications: - MrSID has both lossy and lossless modes - MrSID is not fractal based; it uses wavelets (and arithmetic encoding) - you can't copyright algorithms; the MrSID source code certainly is, however - MrSID relies on a number of patents, not all of which are owned by LizardTech - reading MrSID does not require any fees; we have libraries you can download, although they are not open source That said, some editorial comments (although I'm now wishing I hadn't been so quick to rise to Landon's bait :-) - Some of you know the history of trying to open source MrSID; I won't go into that here, except to say that LizardTech doesn't own all of the required IP needed to make that happen. - If we are speaking of the NAIP data, then no, it is not exclusively available in MrSID format; it is also shipped as GeoTIFFs. - JPEG 2000 is a very robust open standard alternative to MrSID, and a number of players already support it (including LizardTech), but not enough to make it viable for certain domains like NAIP. - some of you also know the history on open JP2 support: there is today no open source implementation of JP2 that is suitable for geo work. Alas. -mpg From: discuss-boun...@lists.osgeo.org [mailto:discuss-boun...@lists.osgeo.org] On Behalf Of Eric Wolf Sent: Thursday, August 20, 2009 2:15 PM To: OSGeo Discussions Subject: Re: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms The MRSID format is a very special case - and perhaps an opportunity for a new FOSS file format. MRSID is a lossless, fractal-based, multi-scale raster compression format. LizardTech has the algorithms to encode and decode MRSID locked up in copyrights, and I believe, patents. Even companies like ESRI shell out big bucks to LizardTech to be able to read and write the MRSID format. I guess I missed the context of the discussion. Is the government releasing certain data exclusively in this format? If so, I think the argument can be made against this practice. The different in compression between MRSID and gziped TIFFs isn't really that great in this day of cheap disks and fat pipes. -Eric -=--=---===---=--=-=--=---==---=--=-=- Eric B. WolfNew! 720-334-7734 USGS Geographer Center of Excellence in GIScience PhD Student CU-Boulder - Geography Warning: Information provided via electronic media is not guaranteed against defects including translation and transmission errors. If the reader is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this information in error, please notify the sender immediately.___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms
Thanks for the clarification, Michael! And your comments about IP may also add to the paper I am developing (or another). I have a theme I plan to develop at some point - mostly dealing with the inherent limitations of copyrighted software in an era of cloud computing... -Eric -=--=---===---=--=-=--=---==---=--=-=- Eric B. WolfNew! 720-334-7734 USGS Geographer Center of Excellence in GIScience PhD Student CU-Boulder - Geography On Thu, Aug 20, 2009 at 2:54 PM, Michael P. Gerlek wrote: > Some clarifications: > > > > - MrSID has both lossy and lossless modes > > - MrSID is not fractal based; it uses wavelets (and arithmetic encoding) > > - you can’t copyright algorithms; the MrSID source code certainly is, > however > > - MrSID relies on a number of patents, not all of which are owned by > LizardTech > > - reading MrSID does not require any fees; we have libraries you can > download, although they are not open source > > > > That said, some editorial comments (although I’m now wishing I hadn’t been > so quick to rise to Landon’s bait :-) > > > > - Some of you know the history of trying to open source MrSID; I won’t go > into that here, except to say that LizardTech doesn’t own all of the > required IP needed to make that happen. > > - If we are speaking of the NAIP data, then no, it is not exclusively > available in MrSID format; it is also shipped as GeoTIFFs. > > - JPEG 2000 is a very robust open standard alternative to MrSID, and a > number of players already support it (including LizardTech), but not enough > to make it viable for certain domains like NAIP. > > - some of you also know the history on open JP2 support: there is today no > open source implementation of JP2 that is suitable for geo work. Alas. > > > > -mpg > > > > > > *From:* discuss-boun...@lists.osgeo.org [mailto: > discuss-boun...@lists.osgeo.org] *On Behalf Of *Eric Wolf > *Sent:* Thursday, August 20, 2009 2:15 PM > *To:* OSGeo Discussions > *Subject:* Re: [OSGeo-Discuss] Open File Formats and Proprietary > Algorithms > > > > The MRSID format is a very special case - and perhaps an opportunity for a > new FOSS file format. MRSID is a lossless, fractal-based, multi-scale raster > compression format. LizardTech has the algorithms to encode and decode MRSID > locked up in copyrights, and I believe, patents. Even companies like ESRI > shell out big bucks to LizardTech to be able to read and write the MRSID > format. > > > > I guess I missed the context of the discussion. Is the government releasing > certain data exclusively in this format? If so, I think the argument can be > made against this practice. The different in compression between MRSID and > gziped TIFFs isn't really that great in this day of cheap disks and fat > pipes. > > > > -Eric > > > -=--=---===---=--=-=--=---==---=--=-=- > Eric B. WolfNew! 720-334-7734 > USGS Geographer > Center of Excellence in GIScience > PhD Student > CU-Boulder - Geography > > > > > ___ > Discuss mailing list > Discuss@lists.osgeo.org > http://lists.osgeo.org/mailman/listinfo/discuss > > ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
RE: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms
Some clarifications: - MrSID has both lossy and lossless modes - MrSID is not fractal based; it uses wavelets (and arithmetic encoding) - you can't copyright algorithms; the MrSID source code certainly is, however - MrSID relies on a number of patents, not all of which are owned by LizardTech - reading MrSID does not require any fees; we have libraries you can download, although they are not open source That said, some editorial comments (although I'm now wishing I hadn't been so quick to rise to Landon's bait :-) - Some of you know the history of trying to open source MrSID; I won't go into that here, except to say that LizardTech doesn't own all of the required IP needed to make that happen. - If we are speaking of the NAIP data, then no, it is not exclusively available in MrSID format; it is also shipped as GeoTIFFs. - JPEG 2000 is a very robust open standard alternative to MrSID, and a number of players already support it (including LizardTech), but not enough to make it viable for certain domains like NAIP. - some of you also know the history on open JP2 support: there is today no open source implementation of JP2 that is suitable for geo work. Alas. -mpg From: discuss-boun...@lists.osgeo.org [mailto:discuss-boun...@lists.osgeo.org] On Behalf Of Eric Wolf Sent: Thursday, August 20, 2009 2:15 PM To: OSGeo Discussions Subject: Re: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms The MRSID format is a very special case - and perhaps an opportunity for a new FOSS file format. MRSID is a lossless, fractal-based, multi-scale raster compression format. LizardTech has the algorithms to encode and decode MRSID locked up in copyrights, and I believe, patents. Even companies like ESRI shell out big bucks to LizardTech to be able to read and write the MRSID format. I guess I missed the context of the discussion. Is the government releasing certain data exclusively in this format? If so, I think the argument can be made against this practice. The different in compression between MRSID and gziped TIFFs isn't really that great in this day of cheap disks and fat pipes. -Eric -=--=---===---=--=-=--=---==---=--=-=- Eric B. WolfNew! 720-334-7734 USGS Geographer Center of Excellence in GIScience PhD Student CU-Boulder - Geography ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms
Interesting... I can understand why NAIP was in MRSID. It's a pretty large dataset - and I think .SID was more widely supported than JP2 until recently. The USDA site does provide links to PCI Geomatics FreeView, which can read .SID format but not save it. IrfanView, with a plugin, can read SID format and convert. So it's not a dead-end format. And it sure beats SDTS! I think data interchange and real interoperability has only recently been possible for large raster datasets. It's still a chore if you have to re-project large raster datasets. This may add some content to a research paper I'm working on. -Eric -=--=---===---=--=-=--=---==---=--=-=- Eric B. WolfNew! 720-334-7734 USGS Geographer Center of Excellence in GIScience PhD Student CU-Boulder - Geography On Thu, Aug 20, 2009 at 2:22 PM, Landon Blake wrote: > Eric, > > > > The imagery I am talking about is from the USDA APFO: > > > > > > This FAQ contains a snippet about the format: > > http://www.fsa.usda.gov/FSA/apfoapp?area=home&subject=prog&topic=nai > > > > In an interesting turn of events I note that as of 2008, the USDA is > releasing the county mosaics in JP2 format, not in MRSID. I am not sure what > brought about this change, and I wasn’t aware that it had been made. The > same web page indicates that there is a shapefile index for the individual > image tiles. > > > > It appears that you can also download the county mosaics online. > > > > A lot of this has changed (improved) in the last couple of years. I’m glad > I checked again. That being said, the principles from our discussion still > apply. :] > > > > *Landon* > > Office Phone Number: (209) 946-0268 > > Cell Phone Number: (209) 992-0658 > > > > > -- > > *From:* discuss-boun...@lists.osgeo.org [mailto: > discuss-boun...@lists.osgeo.org] *On Behalf Of *Eric Wolf > *Sent:* Thursday, August 20, 2009 1:15 PM > *To:* OSGeo Discussions > *Subject:* Re: [OSGeo-Discuss] Open File Formats and Proprietary > Algorithms > > > > The MRSID format is a very special case - and perhaps an opportunity for a > new FOSS file format. MRSID is a lossless, fractal-based, multi-scale raster > compression format. LizardTech has the algorithms to encode and decode MRSID > locked up in copyrights, and I believe, patents. Even companies like ESRI > shell out big bucks to LizardTech to be able to read and write the MRSID > format. > > > > I guess I missed the context of the discussion. Is the government releasing > certain data exclusively in this format? If so, I think the argument can be > made against this practice. The different in compression between MRSID and > gziped TIFFs isn't really that great in this day of cheap disks and fat > pipes. > > > > -Eric > > > -=--=---===---=--=-=--=---==---=--=-=- > Eric B. WolfNew! 720-334-7734 > USGS Geographer > Center of Excellence in GIScience > PhD Student > CU-Boulder - Geography > > > On Thu, Aug 20, 2009 at 12:27 PM, Landon Blake wrote: > > I realized that publishing a spec for a file format like MRSID isn’t as > clear cut as I had at first thought. If the MRSID software uses a fancy > top-secret compression/decompression algorithm to move data to and from the > file format knowing only the structure of the format would do no good. You’d > have to release the details of the algorithm as well. > > > > I still don’t think proprietary file formats are a good idea for government > data released to the public, but I admit that having a company like > LizardTech publish a spec for something like MRSID is not necessarily a > simple task. No doubt a lot of time and money goes into developing those > algorithms. > > > > This makes me wonder about algorithms used to purposefully encrypt binary > file formats. That is another can of worms. It looks like the easiest thing > to do is to start with a file format that was designed to be open from the > very beginning. > > > > Landon > > > > > > *Warning:** > *Information provided via electronic media is not guaranteed against > defects including translation and transmission errors. If the reader is not > the intended recipient, you are hereby notified that any dissemination, > distribution or copying of this communication is strictly prohibited. If you > have received this information in error, please notify the sender > immediately. > > > ___ > Discuss mailing list > Discuss@lists.osgeo.org > http://lists.osgeo.org/mailman/listinfo/discuss > > > > ___ > Discuss mailing list > Discuss@lists.osgeo.org > http://lists.osgeo.org/mailman/listinfo/discuss > > ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
RE: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms
Eric, The imagery I am talking about is from the USDA APFO: This FAQ contains a snippet about the format: http://www.fsa.usda.gov/FSA/apfoapp?area=home&subject=prog&topic=nai In an interesting turn of events I note that as of 2008, the USDA is releasing the county mosaics in JP2 format, not in MRSID. I am not sure what brought about this change, and I wasn't aware that it had been made. The same web page indicates that there is a shapefile index for the individual image tiles. It appears that you can also download the county mosaics online. A lot of this has changed (improved) in the last couple of years. I'm glad I checked again. That being said, the principles from our discussion still apply. :] Landon Office Phone Number: (209) 946-0268 Cell Phone Number: (209) 992-0658 From: discuss-boun...@lists.osgeo.org [mailto:discuss-boun...@lists.osgeo.org] On Behalf Of Eric Wolf Sent: Thursday, August 20, 2009 1:15 PM To: OSGeo Discussions Subject: Re: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms The MRSID format is a very special case - and perhaps an opportunity for a new FOSS file format. MRSID is a lossless, fractal-based, multi-scale raster compression format. LizardTech has the algorithms to encode and decode MRSID locked up in copyrights, and I believe, patents. Even companies like ESRI shell out big bucks to LizardTech to be able to read and write the MRSID format. I guess I missed the context of the discussion. Is the government releasing certain data exclusively in this format? If so, I think the argument can be made against this practice. The different in compression between MRSID and gziped TIFFs isn't really that great in this day of cheap disks and fat pipes. -Eric -=--=---===---=--=-=--=---==---=--=-=- Eric B. WolfNew! 720-334-7734 USGS Geographer Center of Excellence in GIScience PhD Student CU-Boulder - Geography On Thu, Aug 20, 2009 at 12:27 PM, Landon Blake wrote: I realized that publishing a spec for a file format like MRSID isn't as clear cut as I had at first thought. If the MRSID software uses a fancy top-secret compression/decompression algorithm to move data to and from the file format knowing only the structure of the format would do no good. You'd have to release the details of the algorithm as well. I still don't think proprietary file formats are a good idea for government data released to the public, but I admit that having a company like LizardTech publish a spec for something like MRSID is not necessarily a simple task. No doubt a lot of time and money goes into developing those algorithms. This makes me wonder about algorithms used to purposefully encrypt binary file formats. That is another can of worms. It looks like the easiest thing to do is to start with a file format that was designed to be open from the very beginning. Landon Warning: Information provided via electronic media is not guaranteed against defects including translation and transmission errors. If the reader is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this information in error, please notify the sender immediately. ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] Open File Formats and Proprietary Algorithms
The MRSID format is a very special case - and perhaps an opportunity for a new FOSS file format. MRSID is a lossless, fractal-based, multi-scale raster compression format. LizardTech has the algorithms to encode and decode MRSID locked up in copyrights, and I believe, patents. Even companies like ESRI shell out big bucks to LizardTech to be able to read and write the MRSID format. I guess I missed the context of the discussion. Is the government releasing certain data exclusively in this format? If so, I think the argument can be made against this practice. The different in compression between MRSID and gziped TIFFs isn't really that great in this day of cheap disks and fat pipes. -Eric -=--=---===---=--=-=--=---==---=--=-=- Eric B. WolfNew! 720-334-7734 USGS Geographer Center of Excellence in GIScience PhD Student CU-Boulder - Geography On Thu, Aug 20, 2009 at 12:27 PM, Landon Blake wrote: > I realized that publishing a spec for a file format like MRSID isn’t as > clear cut as I had at first thought. If the MRSID software uses a fancy > top-secret compression/decompression algorithm to move data to and from the > file format knowing only the structure of the format would do no good. You’d > have to release the details of the algorithm as well. > > > > I still don’t think proprietary file formats are a good idea for government > data released to the public, but I admit that having a company like > LizardTech publish a spec for something like MRSID is not necessarily a > simple task. No doubt a lot of time and money goes into developing those > algorithms. > > > > This makes me wonder about algorithms used to purposefully encrypt binary > file formats. That is another can of worms. It looks like the easiest thing > to do is to start with a file format that was designed to be open from the > very beginning. > > > > Landon > > > > > *Warning: > *Information provided via electronic media is not guaranteed against > defects including translation and transmission errors. If the reader is > not the intended recipient, you are hereby notified that any dissemination, > distribution or copying of this communication is strictly prohibited. If you > have received this information in error, please notify the sender > immediately. > > ___ > Discuss mailing list > Discuss@lists.osgeo.org > http://lists.osgeo.org/mailman/listinfo/discuss > > ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
[OSGeo-Discuss] Open File Formats and Proprietary Algorithms
I realized that publishing a spec for a file format like MRSID isn't as clear cut as I had at first thought. If the MRSID software uses a fancy top-secret compression/decompression algorithm to move data to and from the file format knowing only the structure of the format would do no good. You'd have to release the details of the algorithm as well. I still don't think proprietary file formats are a good idea for government data released to the public, but I admit that having a company like LizardTech publish a spec for something like MRSID is not necessarily a simple task. No doubt a lot of time and money goes into developing those algorithms. This makes me wonder about algorithms used to purposefully encrypt binary file formats. That is another can of worms. It looks like the easiest thing to do is to start with a file format that was designed to be open from the very beginning. Landon Warning: Information provided via electronic media is not guaranteed against defects including translation and transmission errors. If the reader is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this information in error, please notify the sender immediately.___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
