On Tue, Apr 1, 2014 at 5:57 PM, Kyle Shannon <k...@pobox.com> wrote: > On Tue, Apr 1, 2014 at 12:36 PM, Etienne Tourigny > <etourigny....@gmail.com> wrote: > > The 2 objections I have with json are : > > > > - it is so verbose that editing by hand is not as easy as .csv > > - the xml tags make file size much larger than .csv files, unless they > would > > be stored in a compressed file (gzip) > > > > On the other hand, who messes with theses files on a regular basis > anyway? > > > > It seems like a nice idea. In what ways would it be better than Even's > > suggestion to use sqlite? > > > > > > On Mon, Mar 31, 2014 at 4:37 PM, Dmitriy Baryshnikov < > bishop....@gmail.com> > > wrote: > >> > >> Hi, > >> > >> I think the JSON format is good for metadata storing and representation. > >> JSON support to store string, digits, bool, dates, etc. Write such data > and > >> read such data from files. > >> But need good library or set of classes to work with it in memory > >> representation. > >> For example I like wxJSON - it support to get value by name or iterate > >> values by index. > >> > >> int majorVer = root["wxWidgets"]["Version"]["Major"].AsInt(); > >> > >> > >> root["wxWidgets"]["Authors"][0] = "J. Smart"; > >> root["wxWidgets"]["Authors"][1] = "V. Zeitling"; > >> root["wxWidgets"]["Authors"][2] = "R. Roebling"; > >> > >> > >> It consist only 3 files (json reader, json writer and json value). > >> > >> It seems to me that json-c library is more complicated. Unfortunately > >> wxJSON cannot be used in GDAL as it have bindings to wxWidgets library, > but > >> the approach is interesting. By the way wxWidgets have more democratic > >> license instead of qt. > >> > >> Best regards, > >> Dmitry > >> > >> 31.03.2014 23:03, Even Rouault пишет: > >> > >> Hi Etienne, > >> > >> Thanks for your ideas. > >> > >> Hi all, > >> > >> I have a few suggestions for gdal 2.0, based on my personal experience > in > >> learning to use, enhance and maintain gdal/ogr code. > >> > >> - replace cpl/csl/string/xml code with a mainstream, modern > cross-platform > >> toolkit such as QT, boost, etc. > >> > >> QT is certainly a dependency we wouldn't want to draw. Too big for some > >> embededded usage, and it would make GDAL to be practially bound by the > >> LGPL. > >> I guess standard C++ libraries classes, or perhaps boost, should do the > >> job > >> for what you mention below. > >> > >> While cpl/csl classes are robust and "do the job", they are not well > >> documented and not very intuitive for a new gdal coder. This is from my > >> personal experience, some may not agree. > >> They are also not used outside gdal, as such do not benefit from > >> enhancements as other toolkits. > >> > >> Well, at least, MapServer uses a few CPL functions : CPL minixml, > >> CPLMalloc/CPLFree, CPLReadDir, CPLFormFilename, CPLGetFilename, > >> CSLInsertString, etc.. > >> > >> The drawback is that some data/methods would not be easily available in > c > >> and other bindings. Also it might not be available for all platforms. > >> Existing code would still be able to use cpl/csl code but be deprecated > >> until a given release. > >> > >> > >> - upgrade/migrate gdal support file (files in .csv format such as > gcs.csv) > >> reading to use containers (e.g. hash maps) instead of reading the > relevant > >> .csv files every time > >> > >> Current reading of gdal support files is sub-optimal as any query for > >> support data requires reading the relevant support file(s). It would be > >> more efficient to read these once and cache into a container (such as > hash > >> map). This could be done using a cross-platform toolkit mentioned > earlier. > >> > >> Another option would be to use a .sqlite database with proper indices. > >> SQLite > >> would become a runtime requirement, but I don't think that would be a > big > >> problem. > >> > >> - modify metadata treatment to be able to store/retrieve data in formats > >> other than strings (e.g. floats, doubles, boolean), and query the actual > >> type of a given metadata item. > >> > >> Currently there is no way to know if a given metadata item represents an > >> integer, float, double (or string). > >> > >> In the netcdf driver, I overcame this limitation in 2 ways: > >> 1) parsing the value to test if it is an int, float or double > (potentially > >> error-prone) > >> 2) add an extra metadata item to specify the type of the data > represented > >> (cumbersome and non-standard) > >> > >> My suggestion would be to be able to store metadata value along with its > >> data type (default string as previously). This would require some kind > of > >> container for internal representation (e.g. in qt: QMap<QString, > >> QVariant>) > >> and a way to represent this on file (as .aux.xml or otherwise in files > >> that > >> support metadata) such as gtiff. Values could be stored as strings to > >> avoid > >> big/little endian conversion problems. Data type could be stored in a > >> (hidden) metadata domain, one item for each "real" metadata key. > >> > >> Interesting idea. > >> > >> Well, we need coders now ;-) > >> > >> Even > >> > >> > >> > >> _______________________________________________ > >> gdal-dev mailing list > >> gdal-dev@lists.osgeo.org > >> http://lists.osgeo.org/mailman/listinfo/gdal-dev > > > > > > > > _______________________________________________ > > gdal-dev mailing list > > gdal-dev@lists.osgeo.org > > http://lists.osgeo.org/mailman/listinfo/gdal-dev > > What problem are we trying to solve exactly? Is CPL* thought to be > buggy because it isn't exposed to the wild?
I thought that gdal 2.0 could be a good occasion to upgrade/modernize some of the legacy code in gdal, which is a bit hard to learn for newcomers and not that well documented. > As far as the csv and > support files, is it a performance issue we are trying to solve?. Although I don't have any benchmarks to support this, I perceive that the way the .csv reading is handled is not very efficient. I recall reading some comments in the code that this should be fixed, but that never happened. > Is > the metadata system insufficient for some purposes? It may be in > Etienne's case, due to typing, but are there other examples? I'm just > curious. Thanks. > I find that having metadata values as strings can be limiting and can induce errors an inaccuracies due to rounding. But again - these are merely suggestions - I think things work well as they are but could need some improvements and modernizing/standardization. cheers Etienne > > -- > Kyle >
_______________________________________________ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev