Jukka, sorry, r5482 is the latest. commit msg stays the same. ..ede
On 7/30/2017 18:44, edgar.sol...@web.de wrote: > Jukka, > > can you try r4682? > >> Log Message: >> ----------- >> speeding up JML/GML reader when reading time attributes by parsing them lazy >> (during access later) >> - adding FlexibleFeature, FlexibleFeatureSchema >> - porting GMLReader to use the flexible classes > > ..ede > > On 7/29/2017 22:50, Rahkonen Jukka (MML) wrote: >> Hi, >> >> I took some timings with todays nightly build by opening my dataset with >> 1101817 polygons and quite many attributes. >> >> Opening time >> JML: 25 minutes 14 sec >> Shapefile: 1 minute 28 sec >> GeoJSON: 45 seconds >> >> Seems that there must be something suboptimal in the JML driver and that Ede >> did really nice work with the GeoJSON driver. >> >> -Jukka- >> >> >> ________________________________________ >> Lähettäjä: edgar.sol...@web.de <edgar.sol...@web.de> >> Lähetetty: 29. heinäkuuta 2017 22:25 >> Vastaanottaja: OpenJump develop and use >> Aihe: Re: [JPP-Devel] Slow parsing of JML files >> >> just spent some time with GMLReader, which is actually our JMLReader and >> according to my finding it get's some magnitudes faster when the date >> parsing is commented out (in GMLInputTemplate.getColumnValue()). >> >> had a short look at the FlexibleDateParser and have the impression that >> regex patterns are not precompiled but recompiled on every usage. btw. that >> would be a classic in terms of parsing slowdowns. >> >> ..ede >> >> On 7/29/2017 13:03, Michaël Michaud wrote: >>> Hi, >>> >>> I also remembered that jml reader was quite slow compared with shp, but my >>> last tests show me only slight differences. >>> >>> (Maybe shapefile reader has slowed down with my last commits - attempts to >>> make it more robust with shp coming from esri or qgis) >>> >>> Now, reading a big file (888000 features) with complex polygons and 3 >>> attributes is just a bit longer with jml (38 vs 32s) >>> For a file containing 1M of simple features (squares), it is even faster >>> with jml (14 vs 23). Reading more attributes seems longer with jml. >>> >>> I would say : >>> >>> reading complex geometries is much longer with jml >>> reading attributes is longer with jml >>> reading simple geometries is longer with shx/shp >>> >>> Jukka, have you examples where jml is much longer that shp ? >>> >>> Michaël >>> >>> >>> Le 28/07/2017 à 17:25, edgar.sol...@web.de a écrit : >>>> On 28.07.2017 17:06, Rahkonen Jukka (MML) wrote: >>>>> Hi, >>>>> >>>>> I noticed with a dataset having 1.2 million features with lots of >>>>> attributes that OpenJUMP is rather slow in parsing its own native JML >>>>> format. It takes about 30 minutes to open the file. Shapefiles are much >>>>> faster but they do not suit me because long strings are truncated. Can >>>>> the hardcore developers guess what is the bottle neck with JML/XML >>>>> parsing and if there could be some place for improvements? >>>>> >>>> hi Jukka, >>>> >>>> could you provide said dataset privately? >>>> >>>> does it make a difference when you cut down the number of attributes? >>>> >>>> ..ede >>>> >>>> ------------------------------------------------------------------------------ >>>> Check out the vibrant tech community on one of the world's most >>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot >>>> _______________________________________________ >>>> Jump-pilot-devel mailing list >>>> Jump-pilot-devel@lists.sourceforge.net >>>> https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel >>>> >>> >>> >>> ------------------------------------------------------------------------------ >>> Check out the vibrant tech community on one of the world's most >>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot >>> _______________________________________________ >>> Jump-pilot-devel mailing list >>> Jump-pilot-devel@lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel >> >> >> ------------------------------------------------------------------------------ >> Check out the vibrant tech community on one of the world's most >> engaging tech sites, Slashdot.org! http://sdm.link/slashdot >> _______________________________________________ >> Jump-pilot-devel mailing list >> Jump-pilot-devel@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel >> >> ------------------------------------------------------------------------------ >> Check out the vibrant tech community on one of the world's most >> engaging tech sites, Slashdot.org! http://sdm.link/slashdot >> _______________________________________________ >> Jump-pilot-devel mailing list >> Jump-pilot-devel@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel >> > > > ------------------------------------------------------------------------------ > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > _______________________________________________ > Jump-pilot-devel mailing list > Jump-pilot-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel > ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________ Jump-pilot-devel mailing list Jump-pilot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel
