just spent some time with GMLReader, which is actually our JMLReader and according to my finding it get's some magnitudes faster when the date parsing is commented out (in GMLInputTemplate.getColumnValue()).
had a short look at the FlexibleDateParser and have the impression that regex patterns are not precompiled but recompiled on every usage. btw. that would be a classic in terms of parsing slowdowns. ..ede On 7/29/2017 13:03, Michaël Michaud wrote: > Hi, > > I also remembered that jml reader was quite slow compared with shp, but my > last tests show me only slight differences. > > (Maybe shapefile reader has slowed down with my last commits - attempts to > make it more robust with shp coming from esri or qgis) > > Now, reading a big file (888000 features) with complex polygons and 3 > attributes is just a bit longer with jml (38 vs 32s) > For a file containing 1M of simple features (squares), it is even faster with > jml (14 vs 23). Reading more attributes seems longer with jml. > > I would say : > > reading complex geometries is much longer with jml > reading attributes is longer with jml > reading simple geometries is longer with shx/shp > > Jukka, have you examples where jml is much longer that shp ? > > Michaël > > > Le 28/07/2017 à 17:25, edgar.sol...@web.de a écrit : >> On 28.07.2017 17:06, Rahkonen Jukka (MML) wrote: >>> Hi, >>> >>> I noticed with a dataset having 1.2 million features with lots of >>> attributes that OpenJUMP is rather slow in parsing its own native JML >>> format. It takes about 30 minutes to open the file. Shapefiles are much >>> faster but they do not suit me because long strings are truncated. Can the >>> hardcore developers guess what is the bottle neck with JML/XML parsing and >>> if there could be some place for improvements? >>> >> hi Jukka, >> >> could you provide said dataset privately? >> >> does it make a difference when you cut down the number of attributes? >> >> ..ede >> >> ------------------------------------------------------------------------------ >> Check out the vibrant tech community on one of the world's most >> engaging tech sites, Slashdot.org! http://sdm.link/slashdot >> _______________________________________________ >> Jump-pilot-devel mailing list >> Jump-pilot-devel@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel >> > > > ------------------------------------------------------------------------------ > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > _______________________________________________ > Jump-pilot-devel mailing list > Jump-pilot-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________ Jump-pilot-devel mailing list Jump-pilot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel
