Re: [gdal-dev] Misc. subjects : OSGeo Vienna code sprint, release plans, GDAL 2.0
On Tue, Apr 1, 2014 at 5:57 PM, Kyle Shannon k...@pobox.com wrote: On Tue, Apr 1, 2014 at 12:36 PM, Etienne Tourigny etourigny@gmail.com wrote: The 2 objections I have with json are : - it is so verbose that editing by hand is not as easy as .csv - the xml tags make file size much larger than .csv files, unless they would be stored in a compressed file (gzip) On the other hand, who messes with theses files on a regular basis anyway? It seems like a nice idea. In what ways would it be better than Even's suggestion to use sqlite? On Mon, Mar 31, 2014 at 4:37 PM, Dmitriy Baryshnikov bishop@gmail.com wrote: Hi, I think the JSON format is good for metadata storing and representation. JSON support to store string, digits, bool, dates, etc. Write such data and read such data from files. But need good library or set of classes to work with it in memory representation. For example I like wxJSON - it support to get value by name or iterate values by index. int majorVer = root[wxWidgets][Version][Major].AsInt(); root[wxWidgets][Authors][0] = J. Smart; root[wxWidgets][Authors][1] = V. Zeitling; root[wxWidgets][Authors][2] = R. Roebling; It consist only 3 files (json reader, json writer and json value). It seems to me that json-c library is more complicated. Unfortunately wxJSON cannot be used in GDAL as it have bindings to wxWidgets library, but the approach is interesting. By the way wxWidgets have more democratic license instead of qt. Best regards, Dmitry 31.03.2014 23:03, Even Rouault пишет: Hi Etienne, Thanks for your ideas. Hi all, I have a few suggestions for gdal 2.0, based on my personal experience in learning to use, enhance and maintain gdal/ogr code. - replace cpl/csl/string/xml code with a mainstream, modern cross-platform toolkit such as QT, boost, etc. QT is certainly a dependency we wouldn't want to draw. Too big for some embededded usage, and it would make GDAL to be practially bound by the LGPL. I guess standard C++ libraries classes, or perhaps boost, should do the job for what you mention below. While cpl/csl classes are robust and do the job, they are not well documented and not very intuitive for a new gdal coder. This is from my personal experience, some may not agree. They are also not used outside gdal, as such do not benefit from enhancements as other toolkits. Well, at least, MapServer uses a few CPL functions : CPL minixml, CPLMalloc/CPLFree, CPLReadDir, CPLFormFilename, CPLGetFilename, CSLInsertString, etc.. The drawback is that some data/methods would not be easily available in c and other bindings. Also it might not be available for all platforms. Existing code would still be able to use cpl/csl code but be deprecated until a given release. - upgrade/migrate gdal support file (files in .csv format such as gcs.csv) reading to use containers (e.g. hash maps) instead of reading the relevant .csv files every time Current reading of gdal support files is sub-optimal as any query for support data requires reading the relevant support file(s). It would be more efficient to read these once and cache into a container (such as hash map). This could be done using a cross-platform toolkit mentioned earlier. Another option would be to use a .sqlite database with proper indices. SQLite would become a runtime requirement, but I don't think that would be a big problem. - modify metadata treatment to be able to store/retrieve data in formats other than strings (e.g. floats, doubles, boolean), and query the actual type of a given metadata item. Currently there is no way to know if a given metadata item represents an integer, float, double (or string). In the netcdf driver, I overcame this limitation in 2 ways: 1) parsing the value to test if it is an int, float or double (potentially error-prone) 2) add an extra metadata item to specify the type of the data represented (cumbersome and non-standard) My suggestion would be to be able to store metadata value along with its data type (default string as previously). This would require some kind of container for internal representation (e.g. in qt: QMapQString, QVariant) and a way to represent this on file (as .aux.xml or otherwise in files that support metadata) such as gtiff. Values could be stored as strings to avoid big/little endian conversion problems. Data type could be stored in a (hidden) metadata domain, one item for each real metadata key. Interesting idea. Well, we need coders now ;-) Even ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org
Re: [gdal-dev] Misc. subjects : OSGeo Vienna code sprint, release plans, GDAL 2.0
On Mon, Mar 31, 2014 at 4:03 PM, Even Rouault even.roua...@mines-paris.orgwrote: Hi Etienne, Thanks for your ideas. Hi all, I have a few suggestions for gdal 2.0, based on my personal experience in learning to use, enhance and maintain gdal/ogr code. - replace cpl/csl/string/xml code with a mainstream, modern cross-platform toolkit such as QT, boost, etc. QT is certainly a dependency we wouldn't want to draw. Too big for some embededded usage, and it would make GDAL to be practially bound by the LGPL. I guess standard C++ libraries classes, or perhaps boost, should do the job for what you mention below. In light of this QT is not a candidate, but on the other hand standard c++ library classes only deal with containers right? Are there any cross-platfform, convenience classes? I'm thinking along the lines of things like file paths, xml and csv file reading/writing support. While cpl/csl classes are robust and do the job, they are not well documented and not very intuitive for a new gdal coder. This is from my personal experience, some may not agree. They are also not used outside gdal, as such do not benefit from enhancements as other toolkits. Well, at least, MapServer uses a few CPL functions : CPL minixml, CPLMalloc/CPLFree, CPLReadDir, CPLFormFilename, CPLGetFilename, CSLInsertString, etc.. I don't know much about MapServer, but it seems that GDAL is an optional dependency. Could the CPL code be decoupled from GDAL as an external? Would it make sense or just too much a hassle? The drawback is that some data/methods would not be easily available in c and other bindings. Also it might not be available for all platforms. Existing code would still be able to use cpl/csl code but be deprecated until a given release. - upgrade/migrate gdal support file (files in .csv format such as gcs.csv) reading to use containers (e.g. hash maps) instead of reading the relevant .csv files every time Current reading of gdal support files is sub-optimal as any query for support data requires reading the relevant support file(s). It would be more efficient to read these once and cache into a container (such as hash map). This could be done using a cross-platform toolkit mentioned earlier. Another option would be to use a .sqlite database with proper indices. SQLite would become a runtime requirement, but I don't think that would be a big problem. .sqlite wouls be more compact, but not as easy to edit. - modify metadata treatment to be able to store/retrieve data in formats other than strings (e.g. floats, doubles, boolean), and query the actual type of a given metadata item. Currently there is no way to know if a given metadata item represents an integer, float, double (or string). In the netcdf driver, I overcame this limitation in 2 ways: 1) parsing the value to test if it is an int, float or double (potentially error-prone) 2) add an extra metadata item to specify the type of the data represented (cumbersome and non-standard) My suggestion would be to be able to store metadata value along with its data type (default string as previously). This would require some kind of container for internal representation (e.g. in qt: QMapQString, QVariant) and a way to represent this on file (as .aux.xml or otherwise in files that support metadata) such as gtiff. Values could be stored as strings to avoid big/little endian conversion problems. Data type could be stored in a (hidden) metadata domain, one item for each real metadata key. Interesting idea. Well, we need coders now ;-) Even -- Geospatial professional services http://even.rouault.free.fr/services.html ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Misc. subjects : OSGeo Vienna code sprint, release plans, GDAL 2.0
The 2 objections I have with json are : - it is so verbose that editing by hand is not as easy as .csv - the xml tags make file size much larger than .csv files, unless they would be stored in a compressed file (gzip) On the other hand, who messes with theses files on a regular basis anyway? It seems like a nice idea. In what ways would it be better than Even's suggestion to use sqlite? On Mon, Mar 31, 2014 at 4:37 PM, Dmitriy Baryshnikov bishop@gmail.comwrote: Hi, I think the JSON format is good for metadata storing and representation. JSON support to store string, digits, bool, dates, etc. Write such data and read such data from files. But need good library or set of classes to work with it in memory representation. For example I like wxJSON - it support to get value by name or iterate values by index. int majorVer = root[wxWidgets][Version][Major].AsInt http://wxcode.sourceforge.net/docs/wxjson/classwxJSONValue.html#9223209e7eea4dd4d668869b69560acb(); root[wxWidgets][Authors][0] = J. Smart; root[wxWidgets][Authors][1] = V. Zeitling; root[wxWidgets][Authors][2] = R. Roebling; It consist only 3 files (json reader, json writer and json value). It seems to me that json-c library is more complicated. Unfortunately wxJSON cannot be used in GDAL as it have bindings to wxWidgets library, but the approach is interesting. By the way wxWidgets have more democratic license instead of qt. Best regards, Dmitry 31.03.2014 23:03, Even Rouault пишет: Hi Etienne, Thanks for your ideas. Hi all, I have a few suggestions for gdal 2.0, based on my personal experience in learning to use, enhance and maintain gdal/ogr code. - replace cpl/csl/string/xml code with a mainstream, modern cross-platform toolkit such as QT, boost, etc. QT is certainly a dependency we wouldn't want to draw. Too big for some embededded usage, and it would make GDAL to be practially bound by the LGPL. I guess standard C++ libraries classes, or perhaps boost, should do the job for what you mention below. While cpl/csl classes are robust and do the job, they are not well documented and not very intuitive for a new gdal coder. This is from my personal experience, some may not agree. They are also not used outside gdal, as such do not benefit from enhancements as other toolkits. Well, at least, MapServer uses a few CPL functions : CPL minixml, CPLMalloc/CPLFree, CPLReadDir, CPLFormFilename, CPLGetFilename, CSLInsertString, etc.. The drawback is that some data/methods would not be easily available in c and other bindings. Also it might not be available for all platforms. Existing code would still be able to use cpl/csl code but be deprecated until a given release. - upgrade/migrate gdal support file (files in .csv format such as gcs.csv) reading to use containers (e.g. hash maps) instead of reading the relevant .csv files every time Current reading of gdal support files is sub-optimal as any query for support data requires reading the relevant support file(s). It would be more efficient to read these once and cache into a container (such as hash map). This could be done using a cross-platform toolkit mentioned earlier. Another option would be to use a .sqlite database with proper indices. SQLite would become a runtime requirement, but I don't think that would be a big problem. - modify metadata treatment to be able to store/retrieve data in formats other than strings (e.g. floats, doubles, boolean), and query the actual type of a given metadata item. Currently there is no way to know if a given metadata item represents an integer, float, double (or string). In the netcdf driver, I overcame this limitation in 2 ways: 1) parsing the value to test if it is an int, float or double (potentially error-prone) 2) add an extra metadata item to specify the type of the data represented (cumbersome and non-standard) My suggestion would be to be able to store metadata value along with its data type (default string as previously). This would require some kind of container for internal representation (e.g. in qt: QMapQString, QVariant) and a way to represent this on file (as .aux.xml or otherwise in files that support metadata) such as gtiff. Values could be stored as strings to avoid big/little endian conversion problems. Data type could be stored in a (hidden) metadata domain, one item for each real metadata key. Interesting idea. Well, we need coders now ;-) Even ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Misc. subjects : OSGeo Vienna code sprint, release plans, GDAL 2.0
On Tue, Apr 1, 2014 at 12:36 PM, Etienne Tourigny etourigny@gmail.com wrote: The 2 objections I have with json are : - it is so verbose that editing by hand is not as easy as .csv - the xml tags make file size much larger than .csv files, unless they would be stored in a compressed file (gzip) On the other hand, who messes with theses files on a regular basis anyway? It seems like a nice idea. In what ways would it be better than Even's suggestion to use sqlite? On Mon, Mar 31, 2014 at 4:37 PM, Dmitriy Baryshnikov bishop@gmail.com wrote: Hi, I think the JSON format is good for metadata storing and representation. JSON support to store string, digits, bool, dates, etc. Write such data and read such data from files. But need good library or set of classes to work with it in memory representation. For example I like wxJSON - it support to get value by name or iterate values by index. int majorVer = root[wxWidgets][Version][Major].AsInt(); root[wxWidgets][Authors][0] = J. Smart; root[wxWidgets][Authors][1] = V. Zeitling; root[wxWidgets][Authors][2] = R. Roebling; It consist only 3 files (json reader, json writer and json value). It seems to me that json-c library is more complicated. Unfortunately wxJSON cannot be used in GDAL as it have bindings to wxWidgets library, but the approach is interesting. By the way wxWidgets have more democratic license instead of qt. Best regards, Dmitry 31.03.2014 23:03, Even Rouault пишет: Hi Etienne, Thanks for your ideas. Hi all, I have a few suggestions for gdal 2.0, based on my personal experience in learning to use, enhance and maintain gdal/ogr code. - replace cpl/csl/string/xml code with a mainstream, modern cross-platform toolkit such as QT, boost, etc. QT is certainly a dependency we wouldn't want to draw. Too big for some embededded usage, and it would make GDAL to be practially bound by the LGPL. I guess standard C++ libraries classes, or perhaps boost, should do the job for what you mention below. While cpl/csl classes are robust and do the job, they are not well documented and not very intuitive for a new gdal coder. This is from my personal experience, some may not agree. They are also not used outside gdal, as such do not benefit from enhancements as other toolkits. Well, at least, MapServer uses a few CPL functions : CPL minixml, CPLMalloc/CPLFree, CPLReadDir, CPLFormFilename, CPLGetFilename, CSLInsertString, etc.. The drawback is that some data/methods would not be easily available in c and other bindings. Also it might not be available for all platforms. Existing code would still be able to use cpl/csl code but be deprecated until a given release. - upgrade/migrate gdal support file (files in .csv format such as gcs.csv) reading to use containers (e.g. hash maps) instead of reading the relevant .csv files every time Current reading of gdal support files is sub-optimal as any query for support data requires reading the relevant support file(s). It would be more efficient to read these once and cache into a container (such as hash map). This could be done using a cross-platform toolkit mentioned earlier. Another option would be to use a .sqlite database with proper indices. SQLite would become a runtime requirement, but I don't think that would be a big problem. - modify metadata treatment to be able to store/retrieve data in formats other than strings (e.g. floats, doubles, boolean), and query the actual type of a given metadata item. Currently there is no way to know if a given metadata item represents an integer, float, double (or string). In the netcdf driver, I overcame this limitation in 2 ways: 1) parsing the value to test if it is an int, float or double (potentially error-prone) 2) add an extra metadata item to specify the type of the data represented (cumbersome and non-standard) My suggestion would be to be able to store metadata value along with its data type (default string as previously). This would require some kind of container for internal representation (e.g. in qt: QMapQString, QVariant) and a way to represent this on file (as .aux.xml or otherwise in files that support metadata) such as gtiff. Values could be stored as strings to avoid big/little endian conversion problems. Data type could be stored in a (hidden) metadata domain, one item for each real metadata key. Interesting idea. Well, we need coders now ;-) Even ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev What problem are we trying to solve exactly? Is CPL* thought to be buggy because it isn't exposed to the wild? As far as the csv and support files, is it a performance issue we are trying to
Re: [gdal-dev] Misc. subjects : OSGeo Vienna code sprint, release plans, GDAL 2.0
Hi all, I have a few suggestions for gdal 2.0, based on my personal experience in learning to use, enhance and maintain gdal/ogr code. - replace cpl/csl/string/xml code with a mainstream, modern cross-platform toolkit such as QT, boost, etc. While cpl/csl classes are robust and do the job, they are not well documented and not very intuitive for a new gdal coder. This is from my personal experience, some may not agree. They are also not used outside gdal, as such do not benefit from enhancements as other toolkits. The drawback is that some data/methods would not be easily available in c and other bindings. Also it might not be available for all platforms. Existing code would still be able to use cpl/csl code but be deprecated until a given release. - upgrade/migrate gdal support file (files in .csv format such as gcs.csv) reading to use containers (e.g. hash maps) instead of reading the relevant .csv files every time Current reading of gdal support files is sub-optimal as any query for support data requires reading the relevant support file(s). It would be more efficient to read these once and cache into a container (such as hash map). This could be done using a cross-platform toolkit mentioned earlier. - modify metadata treatment to be able to store/retrieve data in formats other than strings (e.g. floats, doubles, boolean), and query the actual type of a given metadata item. Currently there is no way to know if a given metadata item represents an integer, float, double (or string). In the netcdf driver, I overcame this limitation in 2 ways: 1) parsing the value to test if it is an int, float or double (potentially error-prone) 2) add an extra metadata item to specify the type of the data represented (cumbersome and non-standard) My suggestion would be to be able to store metadata value along with its data type (default string as previously). This would require some kind of container for internal representation (e.g. in qt: QMapQString, QVariant) and a way to represent this on file (as .aux.xml or otherwise in files that support metadata) such as gtiff. Values could be stored as strings to avoid big/little endian conversion problems. Data type could be stored in a (hidden) metadata domain, one item for each real metadata key. cheers, Etienne On Thu, Feb 13, 2014 at 7:14 PM, Even Rouault even.roua...@mines-paris.orgwrote: Hi, I've confirmed my presence to the OSGeo Vienna code sprint ( http://wiki.osgeo.org/wiki/Vienna_Code_Sprint_2014 ). Are there folks that will be there and indent doing some work on GDAL ? Any particular topics of interest ? It could be the opportunity to take a crack at some changes that have been mentionned for some time and are listed at http://trac.osgeo.org/gdal/wiki/GDAL20Changes We should decide how to manage the 2.0 transition. I think that the 2.0 number should be reserved as the opportunity of introducing breaking changes in the API / ABI. But this might take a long time to implement all that is listed above, so there's a risk we end up with a trunk that is never ready for release for years. Not a good thing. On the other hand, we could just be more modest and take breaking changes as they are introduced and raise the major version number (so the yearly version after 2.0 could be 3.0). This will require projects using GDAL to adapt multiple times. An alternative would be to prepare the API for new features even if they are not implemented, but that's a difficult exercise and there's a risk that at implementation time, the API doesn't fit. Any thoughts ? Currently we have no such breakage in trunk so it could qualify as GDAL 1.11. Perhaps we should just release it as such for now before the bigger changes ? Somes topics I can see for GDAL 2.0 that impact API/ABI : - well, the mythological unification of OGR driver model with GDAL driver model. - XYZM support - Curve geometries - 64 bit integer support Other possible structural changes : - Change of master version control system: switch to git / GitHub ? - New build system : cmake ? Of course all of this will more likely happen if contributors or funders show up ! Even -- Geospatial professional services http://even.rouault.free.fr/services.html ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Misc. subjects : OSGeo Vienna code sprint, release plans, GDAL 2.0
Hi Etienne, Thanks for your ideas. Hi all, I have a few suggestions for gdal 2.0, based on my personal experience in learning to use, enhance and maintain gdal/ogr code. - replace cpl/csl/string/xml code with a mainstream, modern cross-platform toolkit such as QT, boost, etc. QT is certainly a dependency we wouldn't want to draw. Too big for some embededded usage, and it would make GDAL to be practially bound by the LGPL. I guess standard C++ libraries classes, or perhaps boost, should do the job for what you mention below. While cpl/csl classes are robust and do the job, they are not well documented and not very intuitive for a new gdal coder. This is from my personal experience, some may not agree. They are also not used outside gdal, as such do not benefit from enhancements as other toolkits. Well, at least, MapServer uses a few CPL functions : CPL minixml, CPLMalloc/CPLFree, CPLReadDir, CPLFormFilename, CPLGetFilename, CSLInsertString, etc.. The drawback is that some data/methods would not be easily available in c and other bindings. Also it might not be available for all platforms. Existing code would still be able to use cpl/csl code but be deprecated until a given release. - upgrade/migrate gdal support file (files in .csv format such as gcs.csv) reading to use containers (e.g. hash maps) instead of reading the relevant .csv files every time Current reading of gdal support files is sub-optimal as any query for support data requires reading the relevant support file(s). It would be more efficient to read these once and cache into a container (such as hash map). This could be done using a cross-platform toolkit mentioned earlier. Another option would be to use a .sqlite database with proper indices. SQLite would become a runtime requirement, but I don't think that would be a big problem. - modify metadata treatment to be able to store/retrieve data in formats other than strings (e.g. floats, doubles, boolean), and query the actual type of a given metadata item. Currently there is no way to know if a given metadata item represents an integer, float, double (or string). In the netcdf driver, I overcame this limitation in 2 ways: 1) parsing the value to test if it is an int, float or double (potentially error-prone) 2) add an extra metadata item to specify the type of the data represented (cumbersome and non-standard) My suggestion would be to be able to store metadata value along with its data type (default string as previously). This would require some kind of container for internal representation (e.g. in qt: QMapQString, QVariant) and a way to represent this on file (as .aux.xml or otherwise in files that support metadata) such as gtiff. Values could be stored as strings to avoid big/little endian conversion problems. Data type could be stored in a (hidden) metadata domain, one item for each real metadata key. Interesting idea. Well, we need coders now ;-) Even -- Geospatial professional services http://even.rouault.free.fr/services.html ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Misc. subjects : OSGeo Vienna code sprint, release plans, GDAL 2.0
On 31 March 2014 20:17, Etienne Tourigny etourigny@gmail.com wrote: I have a few suggestions for gdal 2.0, based on my personal experience in learning to use, enhance and maintain gdal/ogr code. - replace cpl/csl/string/xml code with a mainstream, modern cross-platform toolkit such as QT, boost, etc. IMHO, there is no need to look that far, at Qt or Boost. There is plenty of features in C++, some features of C++11 could be conditionally-enabled as well. Replacing some of the homegrown utilities with C++ std would make things slightly better in terms of API discoverability, self-documented, easier to use, ... https://trac.osgeo.org/gdal/ticket/1629 Best regards, -- Mateusz Łoskot, http://mateusz.loskot.net ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Misc. subjects : OSGeo Vienna code sprint, release plans, GDAL 2.0
Hi, I think the JSON format is good for metadata storing and representation. JSON support to store string, digits, bool, dates, etc. Write such data and read such data from files. But need good library or set of classes to work with it in memory representation. For example I like wxJSON - it support to get value by name or iterate values by index. int majorVer = root[wxWidgets][Version][Major].AsInt http://wxcode.sourceforge.net/docs/wxjson/classwxJSONValue.html#9223209e7eea4dd4d668869b69560acb(); root[wxWidgets][Authors][0] =J. Smart; root[wxWidgets][Authors][1] =V. Zeitling; root[wxWidgets][Authors][2] =R. Roebling; It consist only 3 files (json reader, json writer and json value). It seems to me that json-c library is more complicated. Unfortunately wxJSON cannot be used in GDAL as it have bindings to wxWidgets library, but the approach is interesting. By the way wxWidgets have more democratic license instead of qt. Best regards, Dmitry 31.03.2014 23:03, Even Rouault ?: Hi Etienne, Thanks for your ideas. Hi all, I have a few suggestions for gdal 2.0, based on my personal experience in learning to use, enhance and maintain gdal/ogr code. - replace cpl/csl/string/xml code with a mainstream, modern cross-platform toolkit such as QT, boost, etc. QT is certainly a dependency we wouldn't want to draw. Too big for some embededded usage, and it would make GDAL to be practially bound by the LGPL. I guess standard C++ libraries classes, or perhaps boost, should do the job for what you mention below. While cpl/csl classes are robust and do the job, they are not well documented and not very intuitive for a new gdal coder. This is from my personal experience, some may not agree. They are also not used outside gdal, as such do not benefit from enhancements as other toolkits. Well, at least, MapServer uses a few CPL functions : CPL minixml, CPLMalloc/CPLFree, CPLReadDir, CPLFormFilename, CPLGetFilename, CSLInsertString, etc.. The drawback is that some data/methods would not be easily available in c and other bindings. Also it might not be available for all platforms. Existing code would still be able to use cpl/csl code but be deprecated until a given release. - upgrade/migrate gdal support file (files in .csv format such as gcs.csv) reading to use containers (e.g. hash maps) instead of reading the relevant .csv files every time Current reading of gdal support files is sub-optimal as any query for support data requires reading the relevant support file(s). It would be more efficient to read these once and cache into a container (such as hash map). This could be done using a cross-platform toolkit mentioned earlier. Another option would be to use a .sqlite database with proper indices. SQLite would become a runtime requirement, but I don't think that would be a big problem. - modify metadata treatment to be able to store/retrieve data in formats other than strings (e.g. floats, doubles, boolean), and query the actual type of a given metadata item. Currently there is no way to know if a given metadata item represents an integer, float, double (or string). In the netcdf driver, I overcame this limitation in 2 ways: 1) parsing the value to test if it is an int, float or double (potentially error-prone) 2) add an extra metadata item to specify the type of the data represented (cumbersome and non-standard) My suggestion would be to be able to store metadata value along with its data type (default string as previously). This would require some kind of container for internal representation (e.g. in qt: QMapQString, QVariant) and a way to represent this on file (as .aux.xml or otherwise in files that support metadata) such as gtiff. Values could be stored as strings to avoid big/little endian conversion problems. Data type could be stored in a (hidden) metadata domain, one item for each real metadata key. Interesting idea. Well, we need coders now ;-) Even ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Misc. subjects : OSGeo Vienna code sprint, release plans, GDAL 2.0
On 31 March 2014 21:37, Dmitriy Baryshnikov bishop@gmail.com wrote: Hi, I think the JSON format is good for metadata storing and representation. JSON support to store string, digits, bool, dates, etc. IMO it's great idea. It consist only 3 files (json reader, json writer and json value). It seems to me that json-c library is more complicated. Unfortunately wxJSON cannot be used in GDAL as it have bindings to wxWidgets library, but the approach is interesting. By the way wxWidgets have more democratic license instead of qt. A header-only MIT-licensed rapidjson is worth to check https://code.google.com/p/rapidjson/ Best regards, -- Mateusz Łoskot, http://mateusz.loskot.net ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Misc. subjects : OSGeo Vienna code sprint, release plans, GDAL 2.0
Hi Mateus, That's really what I'm talking about. Good example, but need some testing. Best regards, Dmitry 31.03.2014 23:44, Mateusz Łoskot пишет: On 31 March 2014 21:37, Dmitriy Baryshnikov bishop@gmail.com wrote: Hi, I think the JSON format is good for metadata storing and representation. JSON support to store string, digits, bool, dates, etc. IMO it's great idea. It consist only 3 files (json reader, json writer and json value). It seems to me that json-c library is more complicated. Unfortunately wxJSON cannot be used in GDAL as it have bindings to wxWidgets library, but the approach is interesting. By the way wxWidgets have more democratic license instead of qt. A header-only MIT-licensed rapidjson is worth to check https://code.google.com/p/rapidjson/ Best regards, ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Misc. subjects : OSGeo Vienna code sprint, release plans, GDAL 2.0
On 3/31/2014 1:03 PM, Even Rouault wrote: Hi Etienne, Thanks for your ideas. Hi all, I have a few suggestions for gdal 2.0, based on my personal experience in learning to use, enhance and maintain gdal/ogr code. - replace cpl/csl/string/xml code with a mainstream, modern cross-platform toolkit such as QT, boost, etc. QT is certainly a dependency we wouldn't want to draw. Too big for some embededded usage, and it would make GDAL to be practially bound by the LGPL. I guess standard C++ libraries classes, or perhaps boost, should do the job for what you mention below. +1 for the idea of moving to either well-supported toolkits or more extensive use of the std library or features of C++11 While cpl/csl classes are robust and do the job, they are not well documented and not very intuitive for a new gdal coder. This is from my personal experience, some may not agree. They are also not used outside gdal, as such do not benefit from enhancements as other toolkits. Well, at least, MapServer uses a few CPL functions : CPL minixml, CPLMalloc/CPLFree, CPLReadDir, CPLFormFilename, CPLGetFilename, CSLInsertString, etc.. To the extent that these remain necessary, making them a thin wrapper of whatever standard lib/toolkit is adopted would both improve the effective documentation, as well as make them compatible with the library/toolkit (esp memory allocation) ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Misc. subjects : OSGeo Vienna code sprint, release plans, GDAL 2.0
Le mardi 01 avril 2014 00:18:11, David Strip a écrit : On 3/31/2014 1:03 PM, Even Rouault wrote: Hi Etienne, Thanks for your ideas. Hi all, I have a few suggestions for gdal 2.0, based on my personal experience in learning to use, enhance and maintain gdal/ogr code. - replace cpl/csl/string/xml code with a mainstream, modern cross-platform toolkit such as QT, boost, etc. QT is certainly a dependency we wouldn't want to draw. Too big for some embededded usage, and it would make GDAL to be practially bound by the LGPL. I guess standard C++ libraries classes, or perhaps boost, should do the job for what you mention below. +1 for the idea of moving to either well-supported toolkits or more extensive use of the std library or features of C++11 While cpl/csl classes are robust and do the job, they are not well documented and not very intuitive for a new gdal coder. This is from my personal experience, some may not agree. They are also not used outside gdal, as such do not benefit from enhancements as other toolkits. Well, at least, MapServer uses a few CPL functions : CPL minixml, CPLMalloc/CPLFree, CPLReadDir, CPLFormFilename, CPLGetFilename, CSLInsertString, etc.. To the extent that these remain necessary, making them a thin wrapper of whatever standard lib/toolkit is adopted would both improve the effective documentation, as well as make them compatible with the library/toolkit (esp memory allocation) Honestly, on that point, the migration cost vs the benefit does not seem obvious to me. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev -- Geospatial professional services http://even.rouault.free.fr/services.html ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Misc. subjects : OSGeo Vienna code sprint, release plans, GDAL 2.0
Hi Even, I plan to participate in code sprint too and work on GDAL. In addition to the mentioned tasks I would like to discuss some politics and direction to add new functionality to GDAL. Best regards, Dmitry 14.02.2014 1:14, Even Rouault пишет: Hi, I've confirmed my presence to the OSGeo Vienna code sprint ( http://wiki.osgeo.org/wiki/Vienna_Code_Sprint_2014 ). Are there folks that will be there and indent doing some work on GDAL ? Any particular topics of interest ? It could be the opportunity to take a crack at some changes that have been mentionned for some time and are listed at http://trac.osgeo.org/gdal/wiki/GDAL20Changes We should decide how to manage the 2.0 transition. I think that the 2.0 number should be reserved as the opportunity of introducing breaking changes in the API / ABI. But this might take a long time to implement all that is listed above, so there's a risk we end up with a trunk that is never ready for release for years. Not a good thing. On the other hand, we could just be more modest and take breaking changes as they are introduced and raise the major version number (so the yearly version after 2.0 could be 3.0). This will require projects using GDAL to adapt multiple times. An alternative would be to prepare the API for new features even if they are not implemented, but that's a difficult exercise and there's a risk that at implementation time, the API doesn't fit. Any thoughts ? Currently we have no such breakage in trunk so it could qualify as GDAL 1.11. Perhaps we should just release it as such for now before the bigger changes ? Somes topics I can see for GDAL 2.0 that impact API/ABI : - well, the mythological unification of OGR driver model with GDAL driver model. - XYZM support - Curve geometries - 64 bit integer support Other possible structural changes : - Change of master version control system: switch to git / GitHub ? - New build system : cmake ? Of course all of this will more likely happen if contributors or funders show up ! Even ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Misc. subjects : OSGeo Vienna code sprint, release plans, GDAL 2.0
Selon Dmitriy Baryshnikov bishop@gmail.com: Hi Even, I plan to participate in code sprint too and work on GDAL. In addition to the mentioned tasks I would like to discuss some politics and direction to add new functionality to GDAL. Great, perhaps you can share some of your thoughts ahead of time ? Best regards, Dmitry 14.02.2014 1:14, Even Rouault пОÑеÑ: Hi, I've confirmed my presence to the OSGeo Vienna code sprint ( http://wiki.osgeo.org/wiki/Vienna_Code_Sprint_2014 ). Are there folks that will be there and indent doing some work on GDAL ? Any particular topics of interest ? It could be the opportunity to take a crack at some changes that have been mentionned for some time and are listed at http://trac.osgeo.org/gdal/wiki/GDAL20Changes We should decide how to manage the 2.0 transition. I think that the 2.0 number should be reserved as the opportunity of introducing breaking changes in the API / ABI. But this might take a long time to implement all that is listed above, so there's a risk we end up with a trunk that is never ready for release for years. Not a good thing. On the other hand, we could just be more modest and take breaking changes as they are introduced and raise the major version number (so the yearly version after 2.0 could be 3.0). This will require projects using GDAL to adapt multiple times. An alternative would be to prepare the API for new features even if they are not implemented, but that's a difficult exercise and there's a risk that at implementation time, the API doesn't fit. Any thoughts ? Currently we have no such breakage in trunk so it could qualify as GDAL 1.11. Perhaps we should just release it as such for now before the bigger changes ? Somes topics I can see for GDAL 2.0 that impact API/ABI : - well, the mythological unification of OGR driver model with GDAL driver model. - XYZM support - Curve geometries - 64 bit integer support Other possible structural changes : - Change of master version control system: switch to git / GitHub ? - New build system : cmake ? Of course all of this will more likely happen if contributors or funders show up ! Even ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Misc. subjects : OSGeo Vienna code sprint, release plans, GDAL 2.0
Hi Even, On code sprint I can work on: - unification of OGR driver model with GDAL driver - XYZM - cmake for GDAL That about thoughts - e.g. now I work on linear referencing using OGR datasources. As the absence of M value I use other algorithms. So I have ogrlineref application which can create needed data for linear referencing and using it for get: 1) point on path from linear position 2) linear position from point 3) sublinestring from two linear positions Also, I added few new methods to OGRLineString: virtual double Project(const OGRPoint *) const; virtual OGRLineString* getSubLine(double, double, int) const; I can commit all this, but I don't sure - is this a GDAL scope? And there is a border for functionality which is not applicable to GDAL. Also I have some doubts about breaking ABI as a result of adding new methods to the OGRLineString. Best regards, Dmitry 21.02.2014 14:07, Even Rouault пишет: Selon Dmitriy Baryshnikov bishop@gmail.com: Hi Even, I plan to participate in code sprint too and work on GDAL. In addition to the mentioned tasks I would like to discuss some politics and direction to add new functionality to GDAL. Great, perhaps you can share some of your thoughts ahead of time ? Best regards, Dmitry 14.02.2014 1:14, Even Rouault пОÑеÑ: Hi, I've confirmed my presence to the OSGeo Vienna code sprint ( http://wiki.osgeo.org/wiki/Vienna_Code_Sprint_2014 ). Are there folks that will be there and indent doing some work on GDAL ? Any particular topics of interest ? It could be the opportunity to take a crack at some changes that have been mentionned for some time and are listed at http://trac.osgeo.org/gdal/wiki/GDAL20Changes We should decide how to manage the 2.0 transition. I think that the 2.0 number should be reserved as the opportunity of introducing breaking changes in the API / ABI. But this might take a long time to implement all that is listed above, so there's a risk we end up with a trunk that is never ready for release for years. Not a good thing. On the other hand, we could just be more modest and take breaking changes as they are introduced and raise the major version number (so the yearly version after 2.0 could be 3.0). This will require projects using GDAL to adapt multiple times. An alternative would be to prepare the API for new features even if they are not implemented, but that's a difficult exercise and there's a risk that at implementation time, the API doesn't fit. Any thoughts ? Currently we have no such breakage in trunk so it could qualify as GDAL 1.11. Perhaps we should just release it as such for now before the bigger changes ? Somes topics I can see for GDAL 2.0 that impact API/ABI : - well, the mythological unification of OGR driver model with GDAL driver model. - XYZM support - Curve geometries - 64 bit integer support Other possible structural changes : - Change of master version control system: switch to git / GitHub ? - New build system : cmake ? Of course all of this will more likely happen if contributors or funders show up ! Even ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Misc. subjects : OSGeo Vienna code sprint, release plans, GDAL 2.0
Selon Dmitriy Baryshnikov bishop@gmail.com: Hi Even, On code sprint I can work on: - unification of OGR driver model with GDAL driver - XYZM - cmake for GDAL That about thoughts - e.g. now I work on linear referencing using OGR datasources. As the absence of M value I use other algorithms. So I have ogrlineref application which can create needed data for linear referencing and using it for get: 1) point on path from linear position 2) linear position from point 3) sublinestring from two linear positions Also, I added few new methods to OGRLineString: virtual double Project(const OGRPoint *) const; virtual OGRLineString* getSubLine(double, double, int) const; I just have had a look at those 2 in https://github.com/nextgis/gdal-svn/compare/cmake4gdal. That looks fine generally. I have just seen that GEOSProject_r isn't available in GEOS 3.1.0 which is our new minimal required GEOS version, but it has been added in 3.2.0. You should just conditionalize the implementation of Project method with the GEOS version. And perhaps also add Doxygen comments of the functions. I can commit all this, but I don't sure - is this a GDAL scope? Apart from the above remarks, I'd say: go ahead and commit it. And there is a border for functionality which is not applicable to GDAL. Also I have some doubts about breaking ABI as a result of adding new methods to the OGRLineString. Such breakage of the C++ ABI is within our usual rules. Even ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Misc. subjects : OSGeo Vienna code sprint, release plans, GDAL 2.0
On Thu, Feb 13, 2014 at 2:14 PM, Even Rouault even.roua...@mines-paris.org wrote: Hi, I've confirmed my presence to the OSGeo Vienna code sprint ( http://wiki.osgeo.org/wiki/Vienna_Code_Sprint_2014 ). Are there folks that will be there and indent doing some work on GDAL ? Any particular topics of interest ? It could be the opportunity to take a crack at some changes that have been mentionned for some time and are listed at http://trac.osgeo.org/gdal/wiki/GDAL20Changes We should decide how to manage the 2.0 transition. I think that the 2.0 number should be reserved as the opportunity of introducing breaking changes in the API / ABI. But this might take a long time to implement all that is listed above, so there's a risk we end up with a trunk that is never ready for release for years. Not a good thing. On the other hand, we could just be more modest and take breaking changes as they are introduced and raise the major version number (so the yearly version after 2.0 could be 3.0). This will require projects using GDAL to adapt multiple times. An alternative would be to prepare the API for new features even if they are not implemented, but that's a difficult exercise and there's a risk that at implementation time, the API doesn't fit. Any thoughts ? Currently we have no such breakage in trunk so it could qualify as GDAL 1.11. Perhaps we should just release it as such for now before the bigger changes ? Somes topics I can see for GDAL 2.0 that impact API/ABI : - well, the mythological unification of OGR driver model with GDAL driver model. - XYZM support - Curve geometries - 64 bit integer support Other possible structural changes : - Change of master version control system: switch to git / GitHub ? - New build system : cmake ? Of course all of this will more likely happen if contributors or funders show up ! Even -- Geospatial professional services http://even.rouault.free.fr/services.html ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev Hi, I meant to bring this up in my earlier email, but it slipped my mind. Or maybe I didn't want to open a can of worms. What exactly would 'unification' entail? I assume all ogr drivers would have to inherit GDALMajorObject (and probably OGRDataSource and OGRLayer maybe?) and then ogr drivers would have to populate metadata. I also assume that this would replace GetCapabilities(). Can someone elaborate on the scope of such a change? Thanks. -- Kyle ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Misc. subjects : OSGeo Vienna code sprint, release plans, GDAL 2.0
Hi, I meant to bring this up in my earlier email, but it slipped my mind. Or maybe I didn't want to open a can of worms. What exactly would 'unification' entail? I assume all ogr drivers would have to inherit GDALMajorObject (and probably OGRDataSource and OGRLayer maybe?) and then ogr drivers would have to populate metadata. I also assume that this would replace GetCapabilities(). Can someone elaborate on the scope of such a change? Thanks. Yes, probably things in that direction. Well that could be a whole subject to discuss in face to face for people present in Vienna. A further step would be to have drivers that would do both the raster and vector side at the same time (think of Geopackage for example). A dataset/datasource would have raster band and vector layers. -- Geospatial professional services http://even.rouault.free.fr/services.html ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Misc. subjects : OSGeo Vienna code sprint, release plans, GDAL 2.0
Even, thanks for the notes. On Sat, Feb 15, 2014 at 11:57 AM, Even Rouault even.roua...@mines-paris.org wrote: Thanks for your thoughs Kyle. I expect more developers and PSC members to express theirs too. How long would the stable branches be maintained? Would we handle as we do now, with one stable branch and one development branch, or would we backport bug fixes to n branches (3.1, 2.4, etc)? Would this require maintaining 3 branches? stable, trunk, and api_break? Any thoughts? What would be the api_break branch, as opposed to trunk I mean ? Maintaining 2 branches in addition to the development branch seems to be a bit too much work. Well, backporting is not that difficult generally, but releasing a version is an effort that takes some energy and time, so we would have likely difficulty in making the necessary releases. But anyone wanting to maintain a branch can do it, so there's no need to set that policy in stone. Sorry, I chose poor wording, but you answered my question in regard to two branches. I agree with your assessment. An alternative would be to prepare the API for new features even if they are not implemented, but that's a difficult exercise and there's a risk that at implementation time, the API doesn't fit. Any thoughts ? Currently we have no such breakage in trunk so it could qualify as GDAL 1.11. Perhaps we should just release it as such for now before the bigger changes ? Maybe release 1.11 soon, and take a crack at 2.0 changes before the next release? This would probably require a concerted effort from developers or funders, as Even mentioned in regard to the sprint. Somes topics I can see for GDAL 2.0 that impact API/ABI : - well, the mythological unification of OGR driver model with GDAL driver model. - XYZM support - Curve geometries - 64 bit integer support The 64-bit integer changes seem fairly straight forward. I see XYZM support up for GSoC again, maybe it'll get picked up. I have no idea what curve support would entail. Well, new geometry types and enhancements in drivers that would support them (PostGIS, ...). And also likely adapt all existing drivers that have write support so they can deal with the new geometry types : ignore them, or take them into account. Other possible structural changes : - Change of master version control system: switch to git / GitHub ? - New build system : cmake ? What are the benefits here? Is travis integration easier? Well, I put that on the table since it is sometimes mentionned by developers, but the effort vs benefit ratio is not completely obvious for existing GDAL svn commiters. git transition would be mainly to keep up with what developers are of will soon be familiar with. Someone pointed me recently that GitHub also exposed git repositories as subversion repositories (which I experimented a bit), so that could satisfy most developers. git has the benefit of easing porting patches between branches, and making contributions from casual developers easier. Since the git mirror already exists, the transition to github would essentially require porting the Trac database to Github tickets (we could benefit from MapServer experience that has done that move before) I have no preference as to which scm is used, although if it allows more people to use or contribute, I'm all for it. This is probably not realistic, but it also may be a chance to purge some older/invalid tickets from the trac database. I imagine there is no way of really doing this without a human checking for validity though. I believe someone has a cmake port floating around on github, any comments there? The effort seems to have stalled. The benefit here would be the unification of the Unix and Windows makefiles, but the complexity of GDAL dependencies makes the porting effort rather repelling... I can imagine... -- Geospatial professional services http://even.rouault.free.fr/services.html Thanks. -- Kyle ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Misc. subjects : OSGeo Vienna code sprint, release plans, GDAL 2.0
Dmitriy, What version of cmake is required. On Sat, Feb 15, 2014 at 1:31 PM, Dmitriy Baryshnikov bishop@gmail.com wrote: Hi, As cmake4gdal developer I think there is no problem with cmake. By now we main code is cmaked, and deal only with some drivers (GDAL or OGR), which needed cmake scripts. I make needed cmake scripts for drivers what I use in may work. Are any of your Find* scripts being pushed back into the cmake code base, or do they live exclusively in the GDAL source tree? It's strange to me that cmake doesn't come bundled with find scripts for libs like sqlite and mysql, they are fairly common. So, with some help we can do all cmake scripts rather fast. The current implementation is here: https://github.com/nextgis/gdal-svn/tree/cmake4gdal and https://github.com/aashish24/gdal-svn/tree/cmake4gdal The branches include not the latest GDAL repository commits, as they can include makefile chages, so there is some delay as I synchronize the branches. Now there are scripts to: 1) all libraries - CPL, GDAL, OGR 2) apps - gdaladdo, gdalbuildvrt, gdalinfo, gdallocationinfo, gdalwarp, gdal_contour, gdal_grid, gdal_translate, ogr2ogr, ogrinfo, test_ogrsf 3) GDAL drivers - bmp, dimap, gif, tiff, hfa, iso8211, jpeg, map, mem, ozi, png, postgisraster, raw, saga, til, vrt, wms 4) OGR drivers - csv, dxf, generic, geojson, gml, gpx, kml, libkml, mem, mitab, mysql, pg, s57, shape, sqlite, sxf, vrt, wfs Best regards, Dmitry 15.02.2014 22:57, Even Rouault пишет: Thanks for your thoughs Kyle. I expect more developers and PSC members to express theirs too. How long would the stable branches be maintained? Would we handle as we do now, with one stable branch and one development branch, or would we backport bug fixes to n branches (3.1, 2.4, etc)? Would this require maintaining 3 branches? stable, trunk, and api_break? Any thoughts? What would be the api_break branch, as opposed to trunk I mean ? Maintaining 2 branches in addition to the development branch seems to be a bit too much work. Well, backporting is not that difficult generally, but releasing a version is an effort that takes some energy and time, so we would have likely difficulty in making the necessary releases. But anyone wanting to maintain a branch can do it, so there's no need to set that policy in stone. An alternative would be to prepare the API for new features even if they are not implemented, but that's a difficult exercise and there's a risk that at implementation time, the API doesn't fit. Any thoughts ? Currently we have no such breakage in trunk so it could qualify as GDAL 1.11. Perhaps we should just release it as such for now before the bigger changes ? Maybe release 1.11 soon, and take a crack at 2.0 changes before the next release? This would probably require a concerted effort from developers or funders, as Even mentioned in regard to the sprint. Somes topics I can see for GDAL 2.0 that impact API/ABI : - well, the mythological unification of OGR driver model with GDAL driver model. - XYZM support - Curve geometries - 64 bit integer support The 64-bit integer changes seem fairly straight forward. I see XYZM support up for GSoC again, maybe it'll get picked up. I have no idea what curve support would entail. Well, new geometry types and enhancements in drivers that would support them (PostGIS, ...). And also likely adapt all existing drivers that have write support so they can deal with the new geometry types : ignore them, or take them into account. Other possible structural changes : - Change of master version control system: switch to git / GitHub ? - New build system : cmake ? What are the benefits here? Is travis integration easier? Well, I put that on the table since it is sometimes mentionned by developers, but the effort vs benefit ratio is not completely obvious for existing GDAL svn commiters. git transition would be mainly to keep up with what developers are of will soon be familiar with. Someone pointed me recently that GitHub also exposed git repositories as subversion repositories (which I experimented a bit), so that could satisfy most developers. git has the benefit of easing porting patches between branches, and making contributions from casual developers easier. Since the git mirror already exists, the transition to github would essentially require porting the Trac database to Github tickets (we could benefit from MapServer experience that has done that move before) I believe someone has a cmake port floating around on github, any comments there? The effort seems to have stalled. The benefit here would be the unification of the Unix and Windows makefiles, but the complexity of GDAL dependencies makes the porting effort rather repelling... ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Misc. subjects : OSGeo Vienna code sprint, release plans, GDAL 2.0
Hi Kyle, The minimum version cmake is 2.8.8 because it have some required functionality: add_library(name OBJECT src...) and TARGET_OBJECTS:objlib some discussion can be found here: http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/3379 But in scripts I set VERSION 2.8.10 - this is historical. What about pushing Find scripts back: as the work is not finished and GDAL and other required libraries changing, I think it's rather early to do. Though can be done. Best regards, Dmitry 16.02.2014 21:49, Kyle Shannon пишет: Dmitriy, What version of cmake is required. On Sat, Feb 15, 2014 at 1:31 PM, Dmitriy Baryshnikov bishop@gmail.com wrote: Hi, As cmake4gdal developer I think there is no problem with cmake. By now we main code is cmaked, and deal only with some drivers (GDAL or OGR), which needed cmake scripts. I make needed cmake scripts for drivers what I use in may work. Are any of your Find* scripts being pushed back into the cmake code base, or do they live exclusively in the GDAL source tree? It's strange to me that cmake doesn't come bundled with find scripts for libs like sqlite and mysql, they are fairly common. So, with some help we can do all cmake scripts rather fast. The current implementation is here: https://github.com/nextgis/gdal-svn/tree/cmake4gdal and https://github.com/aashish24/gdal-svn/tree/cmake4gdal The branches include not the latest GDAL repository commits, as they can include makefile chages, so there is some delay as I synchronize the branches. Now there are scripts to: 1) all libraries - CPL, GDAL, OGR 2) apps - gdaladdo, gdalbuildvrt, gdalinfo, gdallocationinfo, gdalwarp, gdal_contour, gdal_grid, gdal_translate, ogr2ogr, ogrinfo, test_ogrsf 3) GDAL drivers - bmp, dimap, gif, tiff, hfa, iso8211, jpeg, map, mem, ozi, png, postgisraster, raw, saga, til, vrt, wms 4) OGR drivers - csv, dxf, generic, geojson, gml, gpx, kml, libkml, mem, mitab, mysql, pg, s57, shape, sqlite, sxf, vrt, wfs Best regards, Dmitry 15.02.2014 22:57, Even Rouault пишет: Thanks for your thoughs Kyle. I expect more developers and PSC members to express theirs too. How long would the stable branches be maintained? Would we handle as we do now, with one stable branch and one development branch, or would we backport bug fixes to n branches (3.1, 2.4, etc)? Would this require maintaining 3 branches? stable, trunk, and api_break? Any thoughts? What would be the api_break branch, as opposed to trunk I mean ? Maintaining 2 branches in addition to the development branch seems to be a bit too much work. Well, backporting is not that difficult generally, but releasing a version is an effort that takes some energy and time, so we would have likely difficulty in making the necessary releases. But anyone wanting to maintain a branch can do it, so there's no need to set that policy in stone. An alternative would be to prepare the API for new features even if they are not implemented, but that's a difficult exercise and there's a risk that at implementation time, the API doesn't fit. Any thoughts ? Currently we have no such breakage in trunk so it could qualify as GDAL 1.11. Perhaps we should just release it as such for now before the bigger changes ? Maybe release 1.11 soon, and take a crack at 2.0 changes before the next release? This would probably require a concerted effort from developers or funders, as Even mentioned in regard to the sprint. Somes topics I can see for GDAL 2.0 that impact API/ABI : - well, the mythological unification of OGR driver model with GDAL driver model. - XYZM support - Curve geometries - 64 bit integer support The 64-bit integer changes seem fairly straight forward. I see XYZM support up for GSoC again, maybe it'll get picked up. I have no idea what curve support would entail. Well, new geometry types and enhancements in drivers that would support them (PostGIS, ...). And also likely adapt all existing drivers that have write support so they can deal with the new geometry types : ignore them, or take them into account. Other possible structural changes : - Change of master version control system: switch to git / GitHub ? - New build system : cmake ? What are the benefits here? Is travis integration easier? Well, I put that on the table since it is sometimes mentionned by developers, but the effort vs benefit ratio is not completely obvious for existing GDAL svn commiters. git transition would be mainly to keep up with what developers are of will soon be familiar with. Someone pointed me recently that GitHub also exposed git repositories as subversion repositories (which I experimented a bit), so that could satisfy most developers. git has the benefit of easing porting patches between branches, and making contributions from casual developers easier. Since the git mirror already exists, the transition to github would essentially require porting the Trac database to Github tickets (we could benefit from MapServer
Re: [gdal-dev] Misc. subjects : OSGeo Vienna code sprint, release plans, GDAL 2.0
Hi, Just a few thoughts/questions... On Thu, Feb 13, 2014 at 2:14 PM, Even Rouault even.roua...@mines-paris.org wrote: Hi, I've confirmed my presence to the OSGeo Vienna code sprint ( http://wiki.osgeo.org/wiki/Vienna_Code_Sprint_2014 ). Are there folks that will be there and indent doing some work on GDAL ? Any particular topics of interest ? It could be the opportunity to take a crack at some changes that have been mentionned for some time and are listed at http://trac.osgeo.org/gdal/wiki/GDAL20Changes We should decide how to manage the 2.0 transition. I think that the 2.0 number should be reserved as the opportunity of introducing breaking changes in the API / ABI. But this might take a long time to implement all that is listed above, so there's a risk we end up with a trunk that is never ready for release for years. Not a good thing. On the other hand, we could just be more modest and take breaking changes as they are introduced and raise the major version number (so the yearly version after 2.0 could be 3.0). This will require projects using GDAL to adapt multiple times. How long would the stable branches be maintained? Would we handle as we do now, with one stable branch and one development branch, or would we backport bug fixes to n branches (3.1, 2.4, etc)? Would this require maintaining 3 branches? stable, trunk, and api_break? Any thoughts? An alternative would be to prepare the API for new features even if they are not implemented, but that's a difficult exercise and there's a risk that at implementation time, the API doesn't fit. Any thoughts ? Currently we have no such breakage in trunk so it could qualify as GDAL 1.11. Perhaps we should just release it as such for now before the bigger changes ? Maybe release 1.11 soon, and take a crack at 2.0 changes before the next release? This would probably require a concerted effort from developers or funders, as Even mentioned in regard to the sprint. Somes topics I can see for GDAL 2.0 that impact API/ABI : - well, the mythological unification of OGR driver model with GDAL driver model. - XYZM support - Curve geometries - 64 bit integer support The 64-bit integer changes seem fairly straight forward. I see XYZM support up for GSoC again, maybe it'll get picked up. I have no idea what curve support would entail. Other possible structural changes : - Change of master version control system: switch to git / GitHub ? - New build system : cmake ? What are the benefits here? Is travis integration easier? I believe someone has a cmake port floating around on github, any comments there? Of course all of this will more likely happen if contributors or funders show up ! Even -- Geospatial professional services http://even.rouault.free.fr/services.html ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev Regards, -- Kyle ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Misc. subjects : OSGeo Vienna code sprint, release plans, GDAL 2.0
Thanks for your thoughs Kyle. I expect more developers and PSC members to express theirs too. How long would the stable branches be maintained? Would we handle as we do now, with one stable branch and one development branch, or would we backport bug fixes to n branches (3.1, 2.4, etc)? Would this require maintaining 3 branches? stable, trunk, and api_break? Any thoughts? What would be the api_break branch, as opposed to trunk I mean ? Maintaining 2 branches in addition to the development branch seems to be a bit too much work. Well, backporting is not that difficult generally, but releasing a version is an effort that takes some energy and time, so we would have likely difficulty in making the necessary releases. But anyone wanting to maintain a branch can do it, so there's no need to set that policy in stone. An alternative would be to prepare the API for new features even if they are not implemented, but that's a difficult exercise and there's a risk that at implementation time, the API doesn't fit. Any thoughts ? Currently we have no such breakage in trunk so it could qualify as GDAL 1.11. Perhaps we should just release it as such for now before the bigger changes ? Maybe release 1.11 soon, and take a crack at 2.0 changes before the next release? This would probably require a concerted effort from developers or funders, as Even mentioned in regard to the sprint. Somes topics I can see for GDAL 2.0 that impact API/ABI : - well, the mythological unification of OGR driver model with GDAL driver model. - XYZM support - Curve geometries - 64 bit integer support The 64-bit integer changes seem fairly straight forward. I see XYZM support up for GSoC again, maybe it'll get picked up. I have no idea what curve support would entail. Well, new geometry types and enhancements in drivers that would support them (PostGIS, ...). And also likely adapt all existing drivers that have write support so they can deal with the new geometry types : ignore them, or take them into account. Other possible structural changes : - Change of master version control system: switch to git / GitHub ? - New build system : cmake ? What are the benefits here? Is travis integration easier? Well, I put that on the table since it is sometimes mentionned by developers, but the effort vs benefit ratio is not completely obvious for existing GDAL svn commiters. git transition would be mainly to keep up with what developers are of will soon be familiar with. Someone pointed me recently that GitHub also exposed git repositories as subversion repositories (which I experimented a bit), so that could satisfy most developers. git has the benefit of easing porting patches between branches, and making contributions from casual developers easier. Since the git mirror already exists, the transition to github would essentially require porting the Trac database to Github tickets (we could benefit from MapServer experience that has done that move before) I believe someone has a cmake port floating around on github, any comments there? The effort seems to have stalled. The benefit here would be the unification of the Unix and Windows makefiles, but the complexity of GDAL dependencies makes the porting effort rather repelling... -- Geospatial professional services http://even.rouault.free.fr/services.html ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Misc. subjects : OSGeo Vienna code sprint, release plans, GDAL 2.0
On Thu, Feb 13, 2014 at 1:14 PM, Even Rouault even.roua...@mines-paris.orgwrote: Currently we have no such breakage in trunk so it could qualify as GDAL 1.11. Perhaps we should just release it as such for now before the bigger changes ? Even, I think that 2.0 could be kept for breaking changes, and that if we aren't going to accomplish those big projects we should just aim for a 1.11 release. Somes topics I can see for GDAL 2.0 that impact API/ABI : - well, the mythological unification of OGR driver model with GDAL driver model. - XYZM support - Curve geometries - 64 bit integer support I think the above could potentially justify a 2.0 release. I think a change to github or cmake do not because they don't affect applications using GDAL. Best regards, Frank Other possible structural changes : - Change of master version control system: switch to git / GitHub ? - New build system : cmake ? Of course all of this will more likely happen if contributors or funders show up ! Even -- Geospatial professional services http://even.rouault.free.fr/services.html ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev -- ---+-- I set the clouds in motion - turn up | Frank Warmerdam, warmer...@pobox.com light and sound - activate the windows | http://pobox.com/~warmerdam and watch the world go round - Rush| Geospatial Software Developer ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Misc. subjects : OSGeo Vienna code sprint, release plans, GDAL 2.0
Hi, As cmake4gdal developer I think there is no problem with cmake. By now we main code is cmaked, and deal only with some drivers (GDAL or OGR), which needed cmake scripts. I make needed cmake scripts for drivers what I use in may work. So, with some help we can do all cmake scripts rather fast. The current implementation is here: https://github.com/nextgis/gdal-svn/tree/cmake4gdal and https://github.com/aashish24/gdal-svn/tree/cmake4gdal The branches include not the latest GDAL repository commits, as they can include makefile chages, so there is some delay as I synchronize the branches. Now there are scripts to: 1) all libraries - CPL, GDAL, OGR 2) apps - gdaladdo, gdalbuildvrt, gdalinfo, gdallocationinfo, gdalwarp, gdal_contour, gdal_grid, gdal_translate, ogr2ogr, ogrinfo, test_ogrsf 3) GDAL drivers - bmp, dimap, gif, tiff, hfa, iso8211, jpeg, map, mem, ozi, png, postgisraster, raw, saga, til, vrt, wms 4) OGR drivers - csv, dxf, generic, geojson, gml, gpx, kml, libkml, mem, mitab, mysql, pg, s57, shape, sqlite, sxf, vrt, wfs Best regards, Dmitry 15.02.2014 22:57, Even Rouault пишет: Thanks for your thoughs Kyle. I expect more developers and PSC members to express theirs too. How long would the stable branches be maintained? Would we handle as we do now, with one stable branch and one development branch, or would we backport bug fixes to n branches (3.1, 2.4, etc)? Would this require maintaining 3 branches? stable, trunk, and api_break? Any thoughts? What would be the api_break branch, as opposed to trunk I mean ? Maintaining 2 branches in addition to the development branch seems to be a bit too much work. Well, backporting is not that difficult generally, but releasing a version is an effort that takes some energy and time, so we would have likely difficulty in making the necessary releases. But anyone wanting to maintain a branch can do it, so there's no need to set that policy in stone. An alternative would be to prepare the API for new features even if they are not implemented, but that's a difficult exercise and there's a risk that at implementation time, the API doesn't fit. Any thoughts ? Currently we have no such breakage in trunk so it could qualify as GDAL 1.11. Perhaps we should just release it as such for now before the bigger changes ? Maybe release 1.11 soon, and take a crack at 2.0 changes before the next release? This would probably require a concerted effort from developers or funders, as Even mentioned in regard to the sprint. Somes topics I can see for GDAL 2.0 that impact API/ABI : - well, the mythological unification of OGR driver model with GDAL driver model. - XYZM support - Curve geometries - 64 bit integer support The 64-bit integer changes seem fairly straight forward. I see XYZM support up for GSoC again, maybe it'll get picked up. I have no idea what curve support would entail. Well, new geometry types and enhancements in drivers that would support them (PostGIS, ...). And also likely adapt all existing drivers that have write support so they can deal with the new geometry types : ignore them, or take them into account. Other possible structural changes : - Change of master version control system: switch to git / GitHub ? - New build system : cmake ? What are the benefits here? Is travis integration easier? Well, I put that on the table since it is sometimes mentionned by developers, but the effort vs benefit ratio is not completely obvious for existing GDAL svn commiters. git transition would be mainly to keep up with what developers are of will soon be familiar with. Someone pointed me recently that GitHub also exposed git repositories as subversion repositories (which I experimented a bit), so that could satisfy most developers. git has the benefit of easing porting patches between branches, and making contributions from casual developers easier. Since the git mirror already exists, the transition to github would essentially require porting the Trac database to Github tickets (we could benefit from MapServer experience that has done that move before) I believe someone has a cmake port floating around on github, any comments there? The effort seems to have stalled. The benefit here would be the unification of the Unix and Windows makefiles, but the complexity of GDAL dependencies makes the porting effort rather repelling... ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
[gdal-dev] Misc. subjects : OSGeo Vienna code sprint, release plans, GDAL 2.0
Hi, I've confirmed my presence to the OSGeo Vienna code sprint ( http://wiki.osgeo.org/wiki/Vienna_Code_Sprint_2014 ). Are there folks that will be there and indent doing some work on GDAL ? Any particular topics of interest ? It could be the opportunity to take a crack at some changes that have been mentionned for some time and are listed at http://trac.osgeo.org/gdal/wiki/GDAL20Changes We should decide how to manage the 2.0 transition. I think that the 2.0 number should be reserved as the opportunity of introducing breaking changes in the API / ABI. But this might take a long time to implement all that is listed above, so there's a risk we end up with a trunk that is never ready for release for years. Not a good thing. On the other hand, we could just be more modest and take breaking changes as they are introduced and raise the major version number (so the yearly version after 2.0 could be 3.0). This will require projects using GDAL to adapt multiple times. An alternative would be to prepare the API for new features even if they are not implemented, but that's a difficult exercise and there's a risk that at implementation time, the API doesn't fit. Any thoughts ? Currently we have no such breakage in trunk so it could qualify as GDAL 1.11. Perhaps we should just release it as such for now before the bigger changes ? Somes topics I can see for GDAL 2.0 that impact API/ABI : - well, the mythological unification of OGR driver model with GDAL driver model. - XYZM support - Curve geometries - 64 bit integer support Other possible structural changes : - Change of master version control system: switch to git / GitHub ? - New build system : cmake ? Of course all of this will more likely happen if contributors or funders show up ! Even -- Geospatial professional services http://even.rouault.free.fr/services.html ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
