[Geotools-devel] Could I get more GitHub permissions?
Hi, I've been working in the GT codebase and I'd like to have a bit more permission. I cannot re-run failed builds and I cannot ask for PR reviews. Thanks a lot, Dave ___ GeoTools-Devel mailing list GeoTools-Devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
Re: [Geotools-devel] GeoPKG - issue with determining Geometry type (polygon vs multipolygon)
I fixed the Natural Earth GeoPKG just by changing the "POLYGON" to "MULTIPOLYGON" in the GEOPKG_GEOMETRY_COLUMNS table. QGIS is fine with this, and so is GeoServer (it will convert all the Polygons to Multipolygon - which is lossless). Not a great solution, but works fine... Dave On Wed, May 13, 2020 at 9:57 AM Jody Garnett wrote: > Andrea I am personally in favour of throwing an exception if the data is > inconsistent, but I respect that some data is not going to be fixed... > > Would a connection parameter work? Or were you thinking a system property > flag ... > > Something like "check_geometry_type": do a pass through the geometry to > detect this problem and override to MultiPolygon when creating the > FeatureType. > > I considered a flag to force "multi" but that would end up applying to all > tables even when it is not needed. > -- > Jody Garnett > > > On Wed, 13 May 2020 at 00:38, Andrea Aime > wrote: > >> Goin back to the original mail >> >> On Wed, May 13, 2020 at 2:12 AM David Blasby >> wrote: >> >>> >>> >>> Looking at the gpkg_geometry_column tables; >>> [image: image.png] >>> >>> We can see it's marked as "Polygon" (not multipolygon). >>> >>> During reading, we get to these lines; >>> >>> >>> https://github.com/geotools/geotools/blob/master/modules/library/jdbc/src/main/java/org/geotools/jdbc/JDBCFeatureReader.java#L409 >>> [image: image.png] >>> >>> Here, value is a Multipolygon, but the binding is a Polygon. The >>> multipolygon is converted to a polygon, leading to the issues above. >>> >> >> This is the however the correct behavior, as consistently expected across >> all GeoTools. >> >> >>> We could just throw an exception? Dont do the conversion? We could also >>> (similar to shapefiles) have polygon (and line/point) geopkgs always >>> represented as the corresponding multi*. However, that's a bit >>> aggressive... >>> >> >> See my other mail, I'd add a flag to enable this. >> I've also asked for confirmation on the GeoPackage list... hasn't shown >> up in the archives yet, so cannot link it... I'll follow up here with >> an answer as soon as I get one >> >> Cheers >> Andrea >> >> == GeoServer Professional Services from the experts! Visit >> http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf >> Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa >> (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 >> http://www.geo-solutions.it http://twitter.com/geosolutions_it >> --- *Con riferimento >> alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - >> Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni >> circostanza inerente alla presente email (il suo contenuto, gli eventuali >> allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i >> destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per >> errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le >> sarei comunque grato se potesse darmene notizia. This email is intended >> only for the person or entity to which it is addressed and may contain >> information that is privileged, confidential or otherwise protected from >> disclosure. We remind that - as provided by European Regulation 2016/679 >> “GDPR” - copying, dissemination or use of this e-mail or the information >> herein by anyone other than the intended recipient is prohibited. If you >> have received this email by mistake, please notify us immediately by >> telephone or e-mail.* >> ___ >> GeoTools-Devel mailing list >> GeoTools-Devel@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/geotools-devel >> > ___ GeoTools-Devel mailing list GeoTools-Devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
Re: [Geotools-devel] GeoPKG - issue with determining Geometry type (polygon vs multipolygon)
Oh - I reported the issue to the Natural Earth people - https://github.com/nvkelso/natural-earth-vector/issues/343 On Tue, May 12, 2020 at 5:11 PM David Blasby wrote: > Hi, > > I was trying to use the Natural Earth GeoPackage (countries) - link below. > > However, I found the rendering was very "wrong"; > [image: image.png] > This triangle is actually 3 small islands - it's represented in the > GeoPackage as a Multipolygon (one polygon for each island). > > This works fine in QGIS. > > Looking at the gpkg_geometry_column tables; > [image: image.png] > > We can see it's marked as "Polygon" (not multipolygon). > > During reading, we get to these lines; > > > https://github.com/geotools/geotools/blob/master/modules/library/jdbc/src/main/java/org/geotools/jdbc/JDBCFeatureReader.java#L409 > [image: image.png] > > Here, value is a Multipolygon, but the binding is a Polygon. The > multipolygon is converted to a polygon, leading to the issues above. > > I'm not sure what should happen here? > > We could just throw an exception? Dont do the conversion? We could also > (similar to shapefiles) have polygon (and line/point) geopkgs always > represented as the corresponding multi*. However, that's a bit > aggressive... > > Doesn't seem to be a good solution here... > > Ideas? > > > (note - this is 250mb) > http://naciscdn.org/naturalearth/packages/natural_earth_vector.gpkg.zip > > Thanks, > David > ___ GeoTools-Devel mailing list GeoTools-Devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
[Geotools-devel] GeoPKG - issue with determining Geometry type (polygon vs multipolygon)
Hi, I was trying to use the Natural Earth GeoPackage (countries) - link below. However, I found the rendering was very "wrong"; [image: image.png] This triangle is actually 3 small islands - it's represented in the GeoPackage as a Multipolygon (one polygon for each island). This works fine in QGIS. Looking at the gpkg_geometry_column tables; [image: image.png] We can see it's marked as "Polygon" (not multipolygon). During reading, we get to these lines; https://github.com/geotools/geotools/blob/master/modules/library/jdbc/src/main/java/org/geotools/jdbc/JDBCFeatureReader.java#L409 [image: image.png] Here, value is a Multipolygon, but the binding is a Polygon. The multipolygon is converted to a polygon, leading to the issues above. I'm not sure what should happen here? We could just throw an exception? Dont do the conversion? We could also (similar to shapefiles) have polygon (and line/point) geopkgs always represented as the corresponding multi*. However, that's a bit aggressive... Doesn't seem to be a good solution here... Ideas? (note - this is 250mb) http://naciscdn.org/naturalearth/packages/natural_earth_vector.gpkg.zip Thanks, David ___ GeoTools-Devel mailing list GeoTools-Devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
Re: [Geotools-devel] geopkg issue
I put together a pull request - very minor change. I also put in a test case that messes with the timezone -- hopefully this will make sure this doesn't happen again. https://github.com/geotools/geotools/pull/2837 Thanks for the help/ideas/code Andrea! Couldn't have done it without you! dave On Tue, Mar 10, 2020 at 11:19 AM Andrea Aime wrote: > I've found the flags related to local date handling: > > https://github.com/geotools/geotools/blob/029dc5c8ccc503de3cfd7f4d1c939d8056c260c4/modules/library/metadata/src/main/java/org/geotools/util/factory/GeoTools.java#L239 > > https://github.com/geotools/geotools/blob/029dc5c8ccc503de3cfd7f4d1c939d8056c260c4/modules/library/metadata/src/main/java/org/geotools/util/factory/Hints.java#L992 > > However they seem to be affecting only GML encoding, not JDBC code > directly. > I'm guessing that in JDBC land java.sql.Date fields are treated as without > a time zone? Not sure. > See also this page of documentation about PostgreSQL: > https://www.postgresql.org/docs/9.1/datatype-datetime.html > Quoting: "The SQL standard requires that writing just timestamp be > equivalent to timestamp without time zone" > > Cheers > Andrea > > > On Tue, Mar 10, 2020 at 7:01 PM David Blasby > wrote: > >> I'll look into the problem this afternoon. >> >> The timezone handling in sql lite is very limited and confusing. >> >> Cheers, >> David >> ___ >> GeoTools-Devel mailing list >> GeoTools-Devel@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/geotools-devel >> > > > -- > > Regards, Andrea Aime == GeoServer Professional Services from the experts! > Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime > @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 > Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 > 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it > --- *Con riferimento > alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - > Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni > circostanza inerente alla presente email (il suo contenuto, gli eventuali > allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i > destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per > errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le > sarei comunque grato se potesse darmene notizia. This email is intended > only for the person or entity to which it is addressed and may contain > information that is privileged, confidential or otherwise protected from > disclosure. We remind that - as provided by European Regulation 2016/679 > “GDPR” - copying, dissemination or use of this e-mail or the information > herein by anyone other than the intended recipient is prohibited. If you > have received this email by mistake, please notify us immediately by > telephone or e-mail.* > ___ GeoTools-Devel mailing list GeoTools-Devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
[Geotools-devel] geopkg issue
I'll look into the problem this afternoon. The timezone handling in sql lite is very limited and confusing. Cheers, David ___ GeoTools-Devel mailing list GeoTools-Devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
[Geotools-devel] [JIRA] (GEOT-5443) NetCDF-4: Test Case's sample data file doesn't load in geoserver/geotools (likely not COARDS)
Title: Message Title
David Blasby created an issue
GeoTools / GEOT-5443
NetCDF-4: Test Case's sample data file doesn't load in geoserver/geotools (likely not COARDS)
Issue Type:
Bug
Assignee:
Unassigned
Created:
17/Jun/16 11:54 PM
Priority:
Medium
Reporter:
David Blasby
I'm using the geotools test cases NetCDF-4 test file temperatureisobaricNC4.nc. Data file is here; https://github.com/geotools/geotools/blob/42c4cc0c3d850b1dea50c07108f171aace35a798/modules/plugin/coverage-multidim/netcdf/src/test/resources/org/geotools/coverage/io/netcdf/test-data/temperatureisobaricNC4.nc It gives exceptions when trying to add it as a datastore. The actual exception is; Caused by: java.lang.ClassCastException: java.lang.Double cannot be cast to java.util.Date at org.geotools.imageio.netcdf.VariableAdapter.getTimeValueByIndex(VariableAdapter.java:1292) at org.geotools.imageio.netcdf.VariableAdapter.createFeature(VariableAdapter.java:1172) at org.geotools.imageio.netcdf.VariableAdapter.getFeatures(VariableAdapter.java:1138) at org.geotools.imageio.netcdf.NetCDFImageReader.initIndex(NetCDFImageReader.java:333) This is because it's getting confused as to which dimension is the Time dimension. Here's the info from ncdump; netcdf temperatureisobaricNC4 { dimensions: isobaric = 2 ; time = 1 ; lat = 65 ; lon = 47 ; variables: double isobaric(isobaric) ; isobaric:long_name = "isobaric" ; isobaric:description = "isobaric" ; isobaric:units = "Pa" ; double time(time) ; time:long_name = "time" ; time:description = "time" ; time:units = "seconds since 1970-01-01 00:00:00 UTC" ; float lat(lat) ; lat:long_name = "latitude" ; lat:units = "degrees_north" ; float lon(lon) ; lon:long_name = "longitude" ; lon:units = "degrees_east" ; float Temperature_isobaric(isobaric, time, lat, lon) ; ... Looks like the COARDS conventions say; Order of dimensions: If any or all of the dimensions of a variable have the interpretations of "date or time" (a.k.a. "T"), "height or depth" (a.k.a. "Z"), "latitude" (a.k.a. "Y"), or "longitude" (a.k.a. "X&
[Geotools-devel] [JIRA] (GEOT-5322) PostGISDialect is using ST_Simplify instead of ST_SimplifyPreserveTopology
Title: Message Title David Blasby created an issue GeoTools / GEOT-5322 PostGISDialect is using ST_Simplify instead of ST_SimplifyPreserveTopology Issue Type: Bug Assignee: Unassigned Components: jdbc-postgis plugin Created: 11/Dec/15 11:09 PM Priority: Medium Reporter: David Blasby org.geotools.data.postgis.PostGISDialect This uses ST_Simplify() instead of ST_SimplifyPreserveTopology() - see lines 333 and 344. ST_Simplify will often create extremely small polygons for polygons that have a portion that is quite thin (when the result should be a large polygon). ST_SimplifyPreserveTopology() is much better at avoiding this (however, its a bit slower). Add Comment
[Geotools-devel] possible IP issue in renderer
Jessie,
As we talked about in the IRC.
The intersection() function at the very bottom of the
LabelCacheDefault.java should be:
//djb: replaced because old one was from sun's Rectangle class
private Envelope intersection(Envelope e1,Envelope e2)
{
Envelope r = e1.intersection(e2);
if (r.getWidth() 0)
return null;
if (r.getHeight() 0)
return null;
return r;
}
Could you make sure this is tracked and doesnt creep back in?
dave
---
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642
___
Geotools-devel mailing list
Geotools-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geotools-devel
[Geotools-devel] IP issue - adding LGPL and Public Domain code to geotools archive
When I modified the PostGIS driver to (a) use Base64 and (b) optimize WKB parsing, I added two file: http://svn.geotools.org/geotools/trunk/gt/plugin/postgis/src/org/geotools/data/postgis/attributeio/Base64.java http://svn.geotools.org/geotools/trunk/gt/plugin/postgis/src/org/geotools/data/postgis/attributeio/WKBReader.java If you look at the headers, the first (Base64.java) is tagged as Public Domain, and the 2nd (WKBReader.java) is based on the JTS LGPL code for its WKBReader code. Both of them had to be modified. I talked to jody about whats supposed to happen when you do this, but he didnt know - so I'm asking the list. 1. leave it as is (ie. in the SVN repository) 2. create two jars with one class file in them, and stick them in the .jar repository and include them using maven. Which are we supposed to do? dave --- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Geotools-devel mailing list Geotools-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
Re: [Geotools-devel] IP issue - adding LGPL and Public Domain code to geotools archive
I also noticed that the intersection() function at the very bottom of LabelCacheDefault.java is very similiar (but not exact) to Sun's Rectangle.intersection() function. I've replaced this function with a call to JTS's Envelope.intersection() function. I believe this happened because JTS 1.6 didnt have the Envelope.intersection() function, while JTS 1.7 does. I've fixed this is 2.1.x, 2.2.x, trunk, and FM branches. dave --- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Geotools-devel mailing list Geotools-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
Re: [Geotools-devel] FM planning
My rough idea of a project plan for the FM branch would be: - write test cases for everything up to SimpleFeature - port shapefile and postgis over to use SimpleFeature - slam the change onto trunk, and explore the rest of the API as aspects of the orphaned complex-feature branch trickle in out of their long cold night I would recommend having support for postgis, shapefile, oracle, ArcSDE, and DB2 datastores as these seem to be the ones that people actually care about. Are there any non-trivial datastores that actually produce complex features? I'm just concerned that if FM merges-in as a half-a-product, you're not going to find too many people who will actually go about using/testing/fixing it as there's a no reason to actually upgrade from 2.2.x to FM. If you step back and imagine yourself as someone looking at geotools from the outside, I'm sure they would consider the FM *fully* merged in when: 0) finish the FM branch a) its actually merged in on trunk b) geotools declares it stable c) there's at least 1 datastore that readswrites complex features (+ all the old ones doing simple features) d) both udig and geoserver have transitioned to it (+ whomever else) e) both udig and geoserver split out releases based on it f) bugs reported get fixed g) continue (e) and (f) until the rate of report is about what it is now The minium for letting people continue with the tasks you listed that require FM is probably (b), but (a) shouldnt really occur until the FM branch is pretty much good and stable. I would expect (b) to come quickly after (a) - but our experience with 2.2.x may contradict that. I think to do any planning, we need to know what needs to be done on the FM branch to get done (ie. 0 above), hopefully with some time estimates. Then we need to know who's able to do what, and what kind of time availability they have. The last update on FM was almost a month ago: I have the core modules (api,main,refrencing,...) compiling against the new feature model. Some of the tests in main still fail... Anyone have the current status? Warm fuzzies would be good. So: a) need to define scope for what 'state' the FM needs to be in before it gets merged onto trunk. + I argued, above, for a fairly complete version of it, but arguments can certainly be made for 'less'. b) we need some type of todo list for getting the current FM from what it is now to whats required in (a) + dont forget a testing and bug-fixing plan c) we need to define our resource and get them up to speed. You mentioned that the FM-required projects will be starting mid-june to early july. I must admit that I'm a bit skeptical of geotools actually being able to move that quickly - someone's going to have to throw some $$ or resources at it. Anyways, could we start by getting a list of FM 'stakeholders'? Here's my guess (hopefully I havent missed anyone): * Justin (TOPP) * Chris (TOPP) * Jody (RR) * RobA (SCO) * Gabriel (AXIOS) * Bryce (?) * GridCoverage folks (Alessio volunteered) It would be great if they could reply and say what there interest is in FM (even if just I think it will be good in the future), what type of timeline constraints they have (if any), and what kind of resources they have that they could throw at it (if any). dave ps. At least two people mentioned that GC 'requires' or affects FM, which (if true) would mean that there needs to be some planning between FM and GC. This could really throw a wrench in the works if FM need modification to properly accommodate GCs (or visa-versa). --- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Geotools-devel mailing list Geotools-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
Re: [Geotools-devel] Tasks needing to be completed for 2.2.0 release
Cory Horner wrote: As mentioned in the meeting: http://docs.codehaus.org/display/GEOTOOLS/Tasks+for+2.2.0+release ...are the tasks needed to be done! Please review and edit to your liking. I think you should add PostGIS datastore to Milestone 3 because Geotools isnt really useful without it. dave --- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Geotools-devel mailing list Geotools-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
Re: [Geotools-devel] FM planning
(resending because my email was having problems) My rough idea of a project plan for the FM branch would be: - write test cases for everything up to SimpleFeature - port shapefile and postgis over to use SimpleFeature - slam the change onto trunk, and explore the rest of the API as aspects of the orphaned complex-feature branch trickle in out of their long cold night I would recommend having support for postgis, shapefile, oracle, ArcSDE, and DB2 datastores as these seem to be the ones that people actually care about. Are there any non-trivial datastores that actually produce complex features? I'm just concerned that if FM merges-in as a half-a-product, you're not going to find too many people who will actually go about using/testing/fixing it as there's a no reason to actually upgrade from 2.2.x to FM. If you step back and imagine yourself as someone looking at geotools from the outside, I'm sure they would consider the FM *fully* merged in when: 0) finish the FM branch a) its actually merged in on trunk b) geotools declares it stable c) there's at least 1 datastore that readswrites complex features (+ all the old ones doing simple features) d) both udig and geoserver have transitioned to it (+ whomever else) e) both udig and geoserver split out releases based on it f) bugs reported get fixed g) continue (e) and (f) until the rate of report is about what it is now The minium for letting people continue with the tasks you listed that require FM is probably (b), but (a) shouldnt really occur until the FM branch is pretty much good and stable. I would expect (b) to come quickly after (a) - but our experience with 2.2.x may contradict that. I think to do any planning, we need to know what needs to be done on the FM branch to get done (ie. 0 above), hopefully with some time estimates. Then we need to know who's able to do what, and what kind of time availability they have. The last update on FM was almost a month ago: I have the core modules (api,main,refrencing,...) compiling against the new feature model. Some of the tests in main still fail... Anyone have the current status? Warm fuzzies would be good. So: a) need to define scope for what 'state' the FM needs to be in before it gets merged onto trunk. + I argued, above, for a fairly complete version of it, but arguments can certainly be made for 'less'. b) we need some type of todo list for getting the current FM from what it is now to whats required in (a) + dont forget a testing and bug-fixing plan c) we need to define our resource and get them up to speed. You mentioned that the FM-required projects will be starting mid-june to early july. I must admit that I'm a bit skeptical of geotools actually being able to move that quickly - someone's going to have to throw some $$ or resources at it. Anyways, could we start by getting a list of FM 'stakeholders'? Here's my guess (hopefully I havent missed anyone): * Justin (TOPP) * Chris (TOPP) * Jody (RR) * RobA (SCO) * Gabriel (AXIOS) * Bryce (?) * GridCoverage folks (Alessio volunteered) It would be great if they could reply and say what there interest is in FM (even if just I think it will be good in the future), what type of timeline constraints they have (if any), and what kind of resources they have that they could throw at it (if any). dave ps. At least two people mentioned that GC 'requires' or affects FM, which (if true) would mean that there needs to be some planning between FM and GC. This could really throw a wrench in the works if FM need modification to properly accommodate GCs (or visa-versa). --- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Geotools-devel mailing list Geotools-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
Re: [Geotools-devel] Re: Renderer improvements meeting part 2 (Grid Coverge Merge to Trunk)
(resending because my email was having problems) Martin Desruisseaux wrote: - The GO-1 module has basic classes for a new renderer with more rigourous axis support. Martin, I've heard about the GO-1 renderer, but I dont know anything about it. Could you give a few sentances on what it is? dave --- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Geotools-devel mailing list Geotools-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
Re: [Geotools-devel] FM IRC Tour day 1
Jody Garnett wrote: We spent the day fighting with Maven (heck it is just like normal geotools development). The goal is to get the FM branch ready for group development, rather then just one person hacking in eclipse. Thanks to those who showed up, the positive feedback on the TypeBuilder was appreciated (on behalf of Justin of course). Until tomorrow, same bat channel same bat place. Did you save the log - there's probably info in there worth saving. dave --- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Geotools-devel mailing list Geotools-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
Re: [Geotools-devel] FM Planning
Here's a quick question for you, jody. When is udig planning on transitioning to the FM? I'm not sure if you have a enough info to answer that, but what does your gut say? dave --- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Geotools-devel mailing list Geotools-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
Re: [Geotools-devel] PostGIS not releasing connections on OutOfMemory
FYI - this have been patched in the latest Postgresql drivers. I worked around this problem so we dont need to upgrade right away. Maybe upgrade the PostgreSQL JDBC driver next time there's an offical postgresql JDBC release? [EMAIL PROTECTED] wrote: I've applied a patch for this to 8.0, 8.1, and HEAD cvs branches and new official releases should hopefully be out soon. dave David Blasby wrote: Okay, I've found a pretty nasty bug in the PostGIS driver. My dataset is a table with about 250 rows in it. Each row has the VMAP0 dataset polygon of a country in. Each row is large. If you look closely at the JDBC datastore, you'll notice that it will fetch 200 rows at a time - which means its pretty much loading in the entire dataset for each query - thats about 250mb. This doesnt fit into the memory allocated to geoserver (especially if you're doing multiple requests at the same time), so an OutOfMemory error gets thrown and the reader is closed. So far, so good. Closing the reader will release the result set and the statement - this frees up about 60mb of memory. It then tries to close the DB connection. This fails - most likely because it was in the middle of reading a tuple when it was rudely interupted by running out of memory. It actually runs out of memory in PGStream.ReceiveTupleV3(), called from QueryExecutorImpl.processResults(). Later, the connection is closed and the PooledConnectionImpl$ConnectionHandler.invoke() is called, which (because we're not AutoCommit) attempt to rollback the transaction with a con.rollback(). This is where the problem occurs. Inside QueryExecutorImpl.processResults(), it gets a \ from the server (highly likely from the bytea returned in the original query). It, of course, doesnt understand this and throws an An I/O error occured while sending to the backend. error. The end result is that the connection doesnt appear to be closed and released back to the connectionpool. This means that connections to the database are being leaked, and probably a fair amount of memory. This, of course, causes the OutOfMemory error to happen more often. I'm probably going to have to talk to the postgresql JDBC people, but I thought I would ask here first incase people know something about. Are we doing anything funny with the connection pooling - I noticed there was a funny proxy object there? dave --- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=lnkkid=110944bid=241720dat=121642 ___ Geotools-devel mailing list Geotools-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel --- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Geotools-devel mailing list Geotools-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
[Geotools-devel] Renderer improvements meeting part 2 (Grid Coverge Merge to Trunk)
IF YOU ARE INTERESTED IN GRID COVERAGES OR THINK GRID COVERAGE WORK WILL AFFECT YOU - PLEASE SPEAK UP NOW. We didnt have enough time to discuss all the details, so we're having another meeting next week. Also, we've been discussing merging in the Grid Coverage renderer to trunk, which could affect the FM folks. http://timeanddate.com/worldclock/fixedtime.html?day=8month=5year=2006hour=22min=0sec=0p1=37 Please see the meeting logs ( http://docs.codehaus.org/display/GEOTOOLS/2006/05/01/Renderer+IRC ). Folks are interested in merging in the Grid Coverage Branch. The timing may be in conflict with the FM merge (see other message) Simone and Alessio can give some detail about what going on in the branch, and what needs to be done to bring it on to trunk. I, unfortunately, dont know anything about the branch. I believe they they will be ready with 2d grids next month and 5D grids soon (and is dealt within the branch's current structure). Their branch is currently nicely aligned with whats in geotools so there shouldnt be a problem with merging. The outstanding issues are concerned with the Streaming Renderer. Here's a meeting outline (feel free to modify): a) Simone and Alessio will talk about what needs to be done on the GridCoverage branch and when it will be ready to merge in to geotools trunk. There's a lot of functionality there, and they will also talk about how this will affect current users and developers so there is a smooth transition. b) Streaming Renderer Issues: 1. CRS. It appears that if we use a ReferencedEnvelope a bit differently than we are now, the Streaming Renderer will be functioning according to spec and work with the GridCoverage stuff. Unfortunately, many people do not make requests to the renderer according to spec so we may need some hints that says renderer according to spec or renderer according to what everyone actually does. Hopefully Martin (and Frank W) will (hopefully) have something to say about this. SA already have it working, so its mostly a case of seeing how it affect folks and making sure its properly done. 2. Interaction with the Grid Coverage Renderer. Simone and Alessio said this is easy and already have something simple. c) Java GDAL bindings There are several java folks interested in getting GDAL working well in JAVA. FrankW said that GDAL is moving more towards having thread-safe read-only access to rasters. Simone and Alessio also mentioned that making GDAL working will with JAI/geotools would also be a big win. Not sure what the timeframe is for this, but I think 3 months was mentioned. Its not a needed component for Grid Coverage support -- it just open up the possibility of interaction with GDAL. If you're interested in this, or know someone who is, please let them know so everyone can work together. I have a *little* time to donate to this. I dont know anything about the Grid Coverage branch, but I do know about the Streaming Renderer and willing to help out there. Who else has some time to help with this? Even if its just kicking the tires... I'm looking forward to seeing the Grid Coverage stuff come onto branch, because its the first step in getting the WCS/WMS-with-raster support in Geoserver's trunk. I also know there are several other people who want Coverage support in geotools. Please send your thoughs, and join us next week at the meeting. dave --- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Geotools-devel mailing list Geotools-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
Re: [Geotools-devel] Maven 2 build status
Martin (and others), Thanks a lot for this update, its given me a lot of info. dave --- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Geotools-devel mailing list Geotools-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
[Geotools-devel] FM merge - when and what (wrt Grid Coverage Merge)
During the Grid Coverage/Renderer improvements IRC today, we talked about merging the grid coverage branch onto trunk. See: http://docs.codehaus.org/display/GEOTOOLS/2006/05/01/Renderer+IRC and the Renderer improvements meeting part 2 (Grid Coverge Merge to Trunk) message thread. I know almost nothing about the FM branch, and the last I heard was that it was going to be another 6 months before its done. Apparently, thats incorrect, and its going to be finished up next month (june) because there's a lot of OWS-4 work thats depending on it. Perhaps we can get a check-in from the folks who are working on FM, or who are current stalled (or will soon be stalled) waiting for it - at the very least get a list of the FM 'stakeholders'. The main problem is that a Grid Coverage merge could quite possibily step on the toes of the FM folks (see the IRC discussion linked above). Trying to do the Grid Coverage merge and FM merge at the same time is almost certainly going to cause problems, so it seems like we should either: a) merge current trunk (I believe the main changes are changes to expression - correct me if I'm wrong) with Grid Coverage Branch + Renderering Improvments - this would be a 2.3 release. 2.4 would then be 2.3 (ie. coverages) + FM. OR b) wait for the FM merge to occur (current trunk + FM = 2.3) then merge the Grid Coverage stuff in later (2.4 = trunk + FM + Grid). There's other options, but I thought I would just throw this out so that the FM folks could start talking. Whats the plan? (In terms of those working on it, and those waiting on it) Is doing a Grid merge going to cause problems? If so, what do we do? I dont think there's a pressing need right now, today to merge the grid stuff in (I could be wrong, though), but I know a lot of people have been waiting for it for a while. dave ps. Jody's kindly offered (free-as-in-beer-and-as-in-speech) FM walk-throughs all this week on IRC. pps. There is starting to be a fairly large divergence between 2.2.x and trunk. This is making it difficult to move 2.2.x patches to trunk - not all bug fixes are finding there way to trunk. This seems to indicate that we should kick out a 2.3 fairly soon. ppps. Justin has said that the FM branch is basically a re-write of the geotools core. It certainly makes sense to call the end result geotools 3 to recognize this. --- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Geotools-devel mailing list Geotools-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
Re: [Geotools-devel] Transaction woes
Either way I feel like using Transaction hints to do rendering stuff is stretching the design a slight bit too far. Perhaps put hints in Query itself? Thats what I first suggested, but jody said the way to go was to put it in Transaction. But, then it affect all future Queries on that Transaction which is an un-wanted. I like the idea of tagging the Query object since its really easy to understand and find. dave --- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Geotools-devel mailing list Geotools-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
[Geotools-devel] Query w/o FIDs
Okay, I finished investigating making Queries w/o FIDs.
According to the profiler, I was expecting an approximately 15-25% speed
increase.
The actual results were not as expected (for reading and rendering
100,000 lines - average 83 points/line in the original data, very high
decimation during rendering):
My laptop (single processor, windows) about 5% faster.
Sigma (2 processor, linux) about 11% faster.
The dataset appears to fit in the disk cache - the first run takes about
3 minutes on my laptop (with the disk going crazy), and about 30 seconds
each request after that. On sigma the first run takes about 1 minute
(it has a raid array) and the rest about 15 seconds.
My bet is that there is IO (ie. socket xfer, round trips to the DB) that
is starting to be the limiting factor.
Changes
-
There's no changes if you dont set the transaction property
doNotGetFIDS, but if you do set it:
jdbc.QueryData constructor:
//DJB: allow for transaction-level FID mapping
if (transactionSaysDontGetFIDS(transaction))
{
this.mapper = featureTypeInfo.getFIDMapper(); //old way
}
else
{
this.mapper = new NullFIDMapper2(); //dont bother getting
anything for the FIDS
}
jdbc1datastore
constructQuery() has two changes (1) added a Transaction parameter (2)
it has code like the above to construct the appropriate FID Mapper.
Renderer
a) added doesntHaveFIDFilter(Query/Filter)
+ looks in the filter for a FIDFilter element
b) query now looks like this (FeatureSource doesnt allow you to set a
Transaction):
...
if ( (featureSource instanceof FeatureStore)
(doesntHaveFIDFilter(query) ) )
{
try{
FeatureStore fs = (FeatureStore) featureSource;
if (fs.getTransaction() == Transaction.AUTO_COMMIT)
{
// play it safe, only update the transaction info if
its an auto_commit
//it logically possible that someone could be
using the Transaction to do future (or past) processing.
//We dont want to affect a future Query
//thats not possible with an AUTO_COMMIT so its
safe.
Transaction t= new DefaultTransaction();
t.putProperty(doNotGetFIDS,Boolean.TRUE);
fs.setTransaction(t);
}
}
I'm not sure how this will affect the other JDBC datastore, but
theoretically there shouldnt be any problems - as long as their either
use both the above function (jdbc.QueryData/jdbc1datastore) or neither.
I'll commit this next week incase anyone wants to comment in the next
few days.
dave
---
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642
___
Geotools-devel mailing list
Geotools-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geotools-devel
Re: [Geotools-devel] StreamingRenderer errors and Decimator
Cory Horner wrote:
Hi folks,
We've been getting strange errors in uDig when rendering from PostGIS
(and possibly others). Everything appeared to draw correctly, but
some exceptions occurred including this:
Caused by: java.lang.ArrayIndexOutOfBoundsException: 1
at
org.geotools.renderer.lite.LiteShape2.getBounds2D(LiteShape2.java:538)
at sun.java2d.pipe.AlphaPaintPipe.startSequence(Unknown Source)
at
sun.java2d.pipe.SpanShapeRenderer$Composite.startSequence(Unknown Source)
at sun.java2d.pipe.SpanShapeRenderer.renderSpans(
This has happened for a long time, but now the decimation is more
aggressive so it occurs more often.
I havent been able to get the Geoserver WMS to call that function during
a render- how are you getting it called? Whats the full stack trace?
Here's a better way to do it (use getEnvelopeInternal()) :
public Rectangle2D getBounds2D() {
Envelope env = geometry.getEnvelopeInternal();
return new Rectangle2D.Double(env.getXMin(), env.getYMin(),
env.getWidth(), env.getHeight() );
}
---
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642
___
Geotools-devel mailing list
Geotools-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geotools-devel
Re: [Geotools-devel] StreamingRenderer errors and Decimator
Strange you must be using linux or something - on my machine (windows),
java always uses the DuctusRenderer, not the SpanShapeRenderer.
That is a much better solution. In the cases that would return an
envelope of zero width and height -- would we want to actually return
that or null?
Return the zero-width box - it still has data in it. It could be doing
some type of clipping...
dave
Cory Horner wrote:
David Blasby wrote:
This has happened for a long time, but now the decimation is more
aggressive so it occurs more often.
I havent been able to get the Geoserver WMS to call that function -
how are you getting it called?
It is purely a PostGIS datastore in uDig.
Whats the full stack trace?
Unfortunately i've already cleared my log file, but here's most of the
stack trace...
!ENTRY net.refractions.udig.project 2 0 2006-04-20 11:47:49.328
!MESSAGE Problem rendering: 1
!STACK 0
net.refractions.udig.project.render.RenderException: Problem rendering: 1
at
net.refractions.udig.render.internal.feature.basic.BasicFeatureRenderer.render(BasicFeatureRenderer.java:250)
at
net.refractions.udig.render.internal.feature.basic.BasicFeatureRenderer.render(BasicFeatureRenderer.java:193)
at
net.refractions.udig.project.internal.render.impl.TilingRenderer.render(TilingRenderer.java:68)
at
net.refractions.udig.project.internal.render.impl.RenderExecutorImpl$RenderJob.startRendering(RenderExecutorImpl.java:275)
at
net.refractions.udig.project.internal.render.impl.RenderExecutorImpl$RenderJob.run(RenderExecutorImpl.java:336)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:76)
Caused by: java.lang.ArrayIndexOutOfBoundsException: 1
at
org.geotools.renderer.lite.LiteShape2.getBounds2D(LiteShape2.java:538)
at sun.java2d.pipe.AlphaPaintPipe.startSequence(Unknown Source)
at
sun.java2d.pipe.SpanShapeRenderer$Composite.startSequence(Unknown Source)
at sun.java2d.pipe.SpanShapeRenderer.renderSpans(Unknown Source)
at sun.java2d.pipe.SpanShapeRenderer.renderPath(Unknown Source)
at sun.java2d.pipe.SpanShapeRenderer.fill(Unknown Source)
at sun.java2d.SunGraphics2D.fill(Unknown Source)
at
org.geotools.renderer.lite.StyledShapePainter.paint(StyledShapePainter.java:191)
at
org.geotools.renderer.lite.StreamingRenderer.processSymbolizers(StreamingRenderer.java:1299)
at
org.geotools.renderer.lite.StreamingRenderer.process(StreamingRenderer.java:1213)
at
org.geotools.renderer.lite.StreamingRenderer.processStylers(StreamingRenderer.java:1163)
at
org.geotools.renderer.lite.StreamingRenderer.paint(StreamingRenderer.java:503)
at
org.geotools.renderer.lite.StreamingRenderer.paint(StreamingRenderer.java:409)
at
net.refractions.udig.render.internal.feature.basic.BasicFeatureRenderer.render(BasicFeatureRenderer.java:239)
... 5 more
Here's a better way to do it (use getEnvelopeInternal()) :
public Rectangle2D getBounds2D() {
Envelope env = geometry.getEnvelopeInternal();
return new Rectangle2D.Double(env.getXMin(), env.getYMin(),
env.getWidth(), env.getHeight() );
}
That is a much better solution. In the cases that would return an
envelope of zero width and height -- would we want to actually return
that or null?
Thanks,
Cory.
---
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job
easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache
Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642
___
Geotools-devel mailing list
Geotools-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geotools-devel
---
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642
___
Geotools-devel mailing list
Geotools-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geotools-devel
Re: [Geotools-devel] 'Like' filters and the default SQLEncoder
Saul,
I noticed this too. I added the SQL92 like-encoder a little while ago
and tested in in postgis. I dont have oracle, mysql, or SDE here to
test, so I didnt add it to them. I did, however, send a message out
asking folks to test and add it to the filter capabilities if it worked.
I think there might be a probably with how mysql deals with like
queries, but I'm not sure.
dave
ps. Add the like-filter-to-SQL encoder just to do feature searching!
You can see it in action here: http://sigma.openplans.org
Saul Farber wrote:
Hello all,
So I'm creating a community-mapbuilder application, and I think to
myself, hey, I'll have a 'search' feature here'.
I create the page, have it submit a filter like this to my SDE-backed
geoserver featureCollection:
Filter
PropertyIsLike wildcard,singlechar,escape
PropertyNameSTREET_NAME/PropertyName
Literal%MAIN%/Literal
/PropertyIsLike
/Filter
However, it gives me back EVERY feature in the collection.
Hmm. I try a few different things, but while it's definitely getting
the LIKE filter into its internal Query representation: (here's a log
snippit)
[FINE] org.vfny.geoserver.wfs.responses.FeatureResponse - Query is
Query []
feature type: massgis:GISDATA.NAVTEQRDS_ARC
filter: [ ST_NAME is like %MAIN% ]
[properties: ST_NAME]
To gt2: Query: []
feature type: GISDATA.NAVTEQRDS_ARC
filter: [ ST_NAME is like %MAIN% ]
[properties: ST_NAME]
It's just NOT working.
So I delve a bit deeper, and have the ArcSDEQuery tell me what filter
it's using. In getSeSQLConstruct() there's a test like this (around
line 983):
if (!Filter.NONE.equals(sqlFilter)) {
...get the where clause for this query...
}
So I added an else to this if statement like so:
else {
LOGGER.warning(Filter was 'FILTER.NONE');
}
Lo-and-behold, the line Filter was 'FILTER.NONE' appears in my log.
Weird! This is totally a supported filter as far as SDE is
concerned...why wouldn't it be showing up?
So I trace back and up through the geotools stack, and discover than
in the base-class org.geotools.filter.SQLEncoder on line 175 (also in
GT-2.2.x, by the way) the method createFilterCapabilities() creates
the default filter capabilities for a generic SQL encoder...and
*doesn't include* LIKE as a supported filter.
I went and added the line:
capabilities.addType(AbstractFilter.LIKE);
to that method, and everything worked just fine for me.
After all that, my question is as follows:
* Is this intentional? Is 'LIKE' excluded from the default filter
capabilities because some query engines just don't support 'LIKE' at
all? Or is this an oversight?
If it's an oversight, I can provide a patch to 2.2.x and 2.1.x
quickly...it's a one-liner.
If it's NOT an oversight, I can provide a patch to 2.2.x and 2.1.x to
get SDE support for LIKE filters, because (after all) it *does*
in-fact support them.
Anyone else bumped into this?
--saul
---
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job
easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache
Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642
___
Geotools-devel mailing list
Geotools-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geotools-devel
---
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642
___
Geotools-devel mailing list
Geotools-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geotools-devel
[Geotools-devel] renderer changes commited
Changes - NOTE: test cases are mostly turned off for maven 2 and non-functional in maven 1. I ran the tests in eclipse and they pass. NOTE: I had awful trouble moving my changes from 2.2.x to trunk. As I mentioned last week, I made a bunch of changes to improve performance and fix bugs. I'm now commiting the changes, so I'm telling you again. Everything should work as before, but be faster. I made a few changes to the labeling algorithm so if you dont like the new way let me know whats different and why it looks bad. The new labeling algorithm is faster. 1. main * added LiteCoordinateSequence ...Factory to org.geotools.util so it can be shared. * added StyleAttributeExtractorTruncated which is a subclass of StyleAttributeExtractor but doesnt pickup 'things' that are directly grabbed from the feature during rendering -- ie. Geometry, TextSymbolizer's label, TextSymbolizer's priority. See the javadoc. This fixed a bug in the renderer and gave a free performance boost. * StyleAttributeExtractor - added javadocs * CompareFilterImpl - fixed several bugs increased performance (unable to apply to trunk) * JDBC1DataStore -- changed default fetch size from 200 to 1000 2. PostGIS * added (optimized) Base64.java class. This is a public domain library. * changed WKB parser to optimized WKB parser (slightly modified JTS version) * WKB parser uses ObjectInputStream.readDoubles() native method to parse doubles. Falls back to a 100% pure java method if there's problems. * uses the LiteCoordinateSequenceFactory to make Geometries instead of the default JTS version. 3. Renderer * moved LiteCoordinateSequence ...Factory to org.geotools.util so it can be shared. * changed the line labeler so it networks using an approx O(n) algorithm instead of the old O(N*N) * line labeler now chooses the rotation of the line based on the instantaneous slope of the line at the label point instead of the slope between the 1st and last point. Some cases this is better, some cases its slight worse. Its easy to change back if there's feedback. * new line iterator LiteIterator2 * decimation changed so its more efficient. * use LiteCoordinateSequences --- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=lnkkid=110944bid=241720dat=121642 ___ Geotools-devel mailing list Geotools-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
Re: [Geotools-devel] renderer changes commited
Andrea Aime wrote:
Hum I'm wondering what changes did you made. It's easy to make
decimation more efficient
by making it more aggressive, but a properly working decimation
generates results that cannot
be distinguished from not having applied decimation at all.
When I coded it, I made sure the decimation distance was not over 1
pixel on screen, but I
don't really know what happened in the meantime.
It, basically, wasnt actually decimating. It would, however, generate
lines with 0 or 1 point in them, and polygon rings with 0, 1, 2 points
and 1st point != last point. These geometry issues were causing
problems, which this version solves.
Note - MathTransform in this code is always an affine transform (see my
earlier message about CRS being done in the wrong place).
Also note that this is always done on a copy of the geometry since its
destructive.
A better way to do this would be to generate a spanx/spany for each
geometry instead of one thats good for all geometries, then
(hopefully) you'll be able to do this in 1 step instead of 2.
dave
/**
* 1. remove any points that are within the spanx,spany. We
ALWAYS keep 1st and last point
* 2. transform to screen coordinates
* 3. remove any points that are close (span 1)
*
* @param seq
* @param tranform
*/
private final void
decimateTransformGeneralize(LiteCoordinateSequence seq,MathTransform
transform) throws TransformException
{
//decimates before XFORM
int ncoords = seq.size();
double originalOrds[] = seq.getXYArray(); // 2*#of points
if (ncoords 2)
{
if (ncoords ==1) // 1 coordinate -- just xform it
{
double[] newCoordsXformed2= new double[2];
transform.transform(originalOrds, 0,
newCoordsXformed2, 0, 1);
seq.setArray(newCoordsXformed2);
return;
}
else
return; // ncoords =0
}
// unfortunately, we have to keep things in double precion
until after the transform or we could move things.
double[] allCoords = new double[ncoords*2]; // preallocate --
might not be full (throw away Z)
allCoords[0] = originalOrds[0]; //allways have 1st one
allCoords[1] = originalOrds[1];
int actualCoords = 1;
double lastX = allCoords[0];
double lastY = allCoords[1];
for (int t=1;t(ncoords-1);t++)
{
//see if this one should be added
double x = originalOrds[t*2];
double y = originalOrds[t*2+1];
if ( (Math.abs(x-lastX ) spanx) || ( Math.abs(y-lastY
)) spany)
{
allCoords[actualCoords*2] = x;
allCoords[actualCoords*2+1] = y;
lastX = x;
lastY = y;
actualCoords++;
}
}
allCoords[actualCoords*2] = originalOrds[(ncoords-1)*2];
//always have last one
allCoords[actualCoords*2+1] = originalOrds[(ncoords-1)*2+1];
actualCoords++;
double[] newCoordsXformed;
//DO THE XFORM
if ((transform == null) || (transform.isIdentity()) ) // no
actual xform
{
newCoordsXformed = allCoords;
}
else
{
newCoordsXformed= new double[actualCoords*2];
transform.transform(allCoords, 0, newCoordsXformed, 0,
actualCoords);
}
//GENERLIZE -- we should be in screen space so spanx=spany=1.0
// unfortunately, we have to keep things in double
precion until after the transform or we could move things.
double[] finalCoords = new double[ncoords*2]; //
preallocate -- might not be full (throw away Z)
finalCoords[0] = newCoordsXformed[0]; //allways have 1st one
finalCoords[1] = newCoordsXformed[1];
int actualCoordsGen = 1;
lastX = newCoordsXformed[0];
lastY = newCoordsXformed[1];
for (int t=1;t(actualCoords-1);t++)
{
//see if this one should be added
double x = newCoordsXformed[t*2];
double y = newCoordsXformed[t*2+1];
if ( (Math.abs(x-lastX ) 1) || ( Math.abs(y-lastY
)) 1)
{
finalCoords[actualCoordsGen*2] = x;
finalCoords[actualCoordsGen*2+1] = y;
lastX = x;
lastY = y;
actualCoordsGen++;
}
}
finalCoords[actualCoordsGen*2] =
[Geotools-devel] caching hashcode calculation in Symbolizer
Jody, My plan to not generate FIDs for Rendering only had a 5% speed increase. I'm not sure why the profiler was off by so much - perhaps IO is a limiting factor. Well, the next biggest time sink seems to be calculating hashcodes for the Symbolizers (and their sub-objects). I was thinking of caching the hashcode and updating it with any changes to it (or any children). There seems to be listeners on the Symbolizers and their sub-objects (ie. LineSymbolizer.getStroke().getColor()). How are you supposed to do this? dave --- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=lnkkid=110944bid=241720dat=121642 ___ Geotools-devel mailing list Geotools-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
[Geotools-devel] StreamingRenderer -- changes to do....
(sending this out quickly for the IRC meeting) Okay, I've been fixing up the StreamingRenderer the last little while, and here's what I think needs to be done. Most of these are just little clean-up to get rid of the evidence of a long period of people slapping stuff into it. A) The path through the renderer is a bit confused. For example, where is CRS transformation done? Its actually done by wrapping the FeatureReader in a CRS-transforming feature reader. But, there's also code (now in LiteShape2 - it used to be partially in LiteShape2 and partially in the Graphics2D segment Iterators) that takes a geometry, decimates it, transforms it,then does a pixel-level generalization. The CRS transformation code should be done here since you can save a bunch of complex math by generalizing first. Its also just really confusing to read since there's 3 sets of generalization code! There's other cases of confusion. This is mostly a clean-up than writing new code - just make the path through the renderer obvious and itemize the steps (AND WRITE DOX). Currently its confusing, mostly because the original renderer was dead-simple and the new one was made by bolting tonnes of extras on. B) All the Graphics2D Iterators need to be re-written. I just rewrote the LiteIterator (which can get re-used by all the other Iterators). The Iterators are extreamly difficult to read. Once (A) is done, this should be much easier since they seem to be doing too much right now. C) The renderer should be segmented into two classes. The first one should be concerned with setting things up. These are things like dealing with the MapContext, optimizing the styles, getting optimized FeatureReaders. The second will basically take the FeatureTypeStyle[] (which has the Graphics2D and actual Styles) and FeatureReader and do the actual drawing. This break is currently in the Renderer approximately at the function processStylers()/processSymbolizers(). This will allow for more code-reuse, and simplify the current (giant) class. Its not too much work since the renderer is conceptually already broken up like this. It will make it very easy to write a parallel renderer that would render more than one layer at a time (good for multiprocessor machines and/or datasets that block while reading from the disk/network/socket). The end result will be a renderer that is much easier to understand and maintain. I'd bet there will be minor speed improvements too. Anyone got any time to help in this? dave --- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=lnkkid=110944bid=241720dat=121642 ___ Geotools-devel mailing list Geotools-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
[Geotools-devel] Sending Filters down to Datastore (WMS/Style)
I've upgraded the StreamingRenderer so that it smarter about sending Filter in the datastore. Jessie did the original work, I just generalized it a bit. You can set a renderer hint maxFiltersToSendToDatastore (Integer) to limit the number of Filters that get sent to the datastore. The default is currently 5. For access to postgis datastore (or any DB datastore) this tends to do quite well in speeding things up if you're not going to be rendering all your rows. dave --- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=lnkkid=110944bid=241720dat=121642 ___ Geotools-devel mailing list Geotools-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
[Geotools-devel] PostgisPermissionOnlineTest -- not running local and hangs remotely
The PostgisPermissionOnlineTest doesnt work on a local database and it just hangs when it runs remotely. It appears to be accessing a restricted table in the database, so this looks like another missing data problem. There's no real comments in the test case, so I have no idea what its supposed to be doing. dave --- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=lnkkid=110944bid=241720dat=121642 ___ Geotools-devel mailing list Geotools-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
Re: [Geotools-devel] unable to build shapefile (again) 2.2.x
http://jira.codehaus.org/browse/GEOT-832 Jody Garnett wrote: David Blasby wrote: I'm unable to build shapefile using maven 1 or maven 2. Same problem as before. We applied a patch to close the readers ... so it must be a different problem then we fixed? Can you give Jesse a Jira number - he is module maintainer on this one. Jody --- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=lnkkid=110944bid=241720dat=121642 ___ Geotools-devel mailing list Geotools-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
[Geotools-devel] Getting Writers to use transactions
Jody, I modified my program so it looks like this (using a
DefaultTransaction ):
private void process() throws Exception
{
DefaultTransaction trans = new DefaultTransaction();
FeatureWriter aWriter = ds.getFeatureWriter(table_name,trans);
while (aWriter.hasNext())
{
Feature f = aWriter.next(); // get a feature
Geometry g = (Geometry) f.getAttribute(source_column);
Geometry g_gen = generalize(g,tolerance);
f.setAttribute(dest_colum,g_gen); // modify it
aWriter.write(); // write it to DB
}
trans.commit();
aWriter.close();
}
But its still not doing it in a transaction.
dave
---
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnkkid=110944bid=241720dat=121642
___
Geotools-devel mailing list
Geotools-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geotools-devel
[Geotools-devel] Problems with feature writing Geometry.equals(Geometry) vs Geometry.equalsExact(Geometry)
Jody,
I looked more into the featurewriting stuff that we talked about last
night -- here's what I found.
I think there's a problem in how Feature implements equals. Here's
where it came up.
I wrote a tiny program that updates a table in my database:
/**
* similiar to:
* update table name set destination geometry =
generalize(source geometry name,tolerance);
*
* but the generalize() function in here is much smarter than the one
in postgis
...
*/
Here's my main loop (taken from the documentation page
http://docs.codehaus.org/display/GEOTOOLS/Data+Writing;):
private void process() throws Exception
{
FeatureWriter aWriter = ds.getFeatureWriter(table_name,
((FeatureStore) ds.getFeatureSource(table_name))
.getTransaction());
while (aWriter.hasNext())
{
Feature f = aWriter.next(); // get a feature
Geometry g = (Geometry) f.getAttribute(source_column);
Geometry g_gen = generalize(g,tolerance); // less point!!
f.setAttribute(dest_colum,g_gen); // modify it
aWriter.write(); // write it to DB
}
aWriter.close();
}
Here's my problems.
1. This isnt happening in a transaction - it put my database into an
invalid state :(
2. DOCUMENTATION: its really weird seeing your writer acting as a
reader. Might want to emphasis that in the documentation. I put a note
at the top so others wouldnt be confused. You should also include a
brief summary of FeatureStore/FeatureSource/FeatureXYZ since that is
also confusing (ie. you ask for a FeatureSource, but you really get a
FeatureStore!).
3. Unfortunately, my program doesnt work! As I mentioned, this has to
do with Feature.equals().
When the aWriter.write(); occurs it gets passed off to the
JDBCFeatureWriter class which quickly does a:
live.equals(current) on line 193 (2.2.x)
in DefaultFeature (line 480) we see this:
NOTE: THIS CODE IS DUPLICATED IN
DataUtilities#attributesEqual()
if (!((Geometry) attributes[i]).equals(
(Geometry) otherAtt)) {
return false;
}
I dont think this is the correct behavior.
a) this takes a *long time* to compute
b) if your geometries are invalid, it will likely throw an
exception and you're screwed (thats what happened to me)
c) Geometry.equals(Geometry) probably doesnt do what you
think/want it to do.
For example, these 3 geometries are all equal:
LINESTRING(0 0, 10 10)
LINESTRING(0 0, 5 5, 10 10) // extra point that coincident
with the segment
LINESTRING(10 10,0 0) // backwards
NOTE: this means you cannot flip the direction of lines in
Geotools -- it'll never get written out.
The alternative is to use Geometry.equalsExact(Geometry). This is a
relatively quick method that checks to see if 2 geometries are the
same type, have the same number of points, and all the points are equal
and in the same order. Basically, what most people mean when they say
two geometries are equal.
So, how about changing all occurances of Geometry.equals(Geometry) with
Geometry.equalsExact(Geometry)? Is this going to cause problems?
I found these two places where it happens, but I'm sure there are others.
dave
---
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnkkid=110944bid=241720dat=121642
___
Geotools-devel mailing list
Geotools-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geotools-devel
Re: [Geotools-devel] unable to build shapefile (again) 2.2.x
This problem has been going for almost 2 months. I've only been able to build 2.2.x ONCE. I say we just delete the test case - this is been going on WAY too long and cost me way too much time. dave --- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=lnkkid=110944bid=241720dat=121642 ___ Geotools-devel mailing list Geotools-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
Re: [Geotools-devel] Moving to TOPP confluence (Geoserver and Geotools)
Paul Ramsey wrote: On 13-Mar-06, at 11:08 AM, [EMAIL PROTECTED] wrote: ps. Jody -- you put udig on its own confluence so I'd like to hear your experience. It was as advertised. Take the backup XML file and upload, content appears. Excellent. Did you try to move over JIRA issues? dave --- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=lnkkid=110944bid=241720dat=121642 ___ Geotools-devel mailing list Geotools-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
[Geotools-devel] LiteralLiteral -- null in 2.2.x, in 2.1.x
I noticed this change. I think this is the incorrect interpretation since there is a PropertyIsNull test in Filter. dave --- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=lnkkid=110944bid=241720dat=121642 ___ Geotools-devel mailing list Geotools-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
[Geotools-devel] Re: LiteralLiteral -- null in 2.2.x, in 2.1.x (for database datastores)
David Blasby wrote: I noticed this change. I think this is the incorrect interpretation since there is a PropertyIsNull test in Filter. oops, I noticed that PropertyIsNull is interpreted as [ interstate = null ] which causes exceptions because the right hand side of the = is 'null' (not something that means null). I'll update the postgis driver to handle this case, but its likely that PropertyIsNull.../PropertyIsNull AND PropertyIsEqualTo ... LiteralLiteral /PropertyIsEqualTo Will cause exceptions in *MOST* datastores. I'm only changing this in the postgis driver (SQLEncoderPostgis) because I believe its done differently in different databases. SELECT * FROM table WHERE column isnull; -- postgresql SELECT * FROM table WHERE isnull(column); -- oracle??? Other databases will have problems too. dave --- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=lnkkid=110944bid=241720dat=121642 ___ Geotools-devel mailing list Geotools-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
Re: [Geotools-devel] PostGIS driver -- upgrade to newer Postgresql JDBC driver?
Martin Desruisseaux wrote: I already posted an email suggesting the same (for different reason) two weeks ago. Since I got no objection, I already upgrated the Maven 2 build on trunk to the following version: 8.1-404.jdbc3 This is 404 instead of 405 only because the 404 version was the only one available on http://www.ibiblio.org/maven2/ (the Maven 2 repository) last time I looked. Okay - I was using the maven 1 build because of my problems with building using it. I'm not fussy about the 404 or 405 build. I'm on the postgresql JDBC mailing list for at least the next little while, I'll update folks here if they do anything exciting. dave --- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=lnkkid=110944bid=241720dat=121642 ___ Geotools-devel mailing list Geotools-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
[Geotools-devel] PostGIS driver -- upgrade to newer Postgresql JDBC driver?
I've noticed a few problems with an older version of the Postgresql JDBC driver. This had to do with detecting encoding of data in the database. I've been using: http://jdbc.postgresql.org/download/postgresql-8.1-405.jdbc3.jar and have found that it works better. I'd like to suggest that we: 1. update geotools so it uses this jar instead of an earlier one (requires maven 1 and maven 2 magic) 2. update geoserver so it also uses this jar (just replace the jdbcPostresql.jar jar with the above one) I have no idea how to do (1), but (2) is trivial. dave ps. this is the latest stable release of the postgresql JDBC driver. The one that appears to be currently used is postgresql-74.213.jar. --- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=lnkkid=110944bid=241720dat=121642 ___ Geotools-devel mailing list Geotools-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
[Geotools-devel] 2.2.x -- now working with maven 1
I've been able to get 2.2.x working with maven 1. Still no go with maven 2. For those of you who are interested, there's a geoserver branch (branches/1_3_0a_geotools_2_2_x) that is the current trunk of geoserver (to be 1.3.1 soonish) moved to 2.2.x. dave --- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=lnkkid=110944bid=241720dat=121642 ___ Geotools-devel mailing list Geotools-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
[Geotools-devel] 2.2.x -- not building with maven 1 or maven 2 (different problems)
(I picked up yesterday's changes, but 'no go') With Maven 2, I'm getting the same missing-JAI library-like error as before in plugin/image. It builds/tests fine with Maven 1, so I'm betting this is some type of Maven 2 configuration problem. == java.lang.NoClassDefFoundError: com/sun/media/imageio/stream/FileChannelImageInputStream at org.geotools.gce.image.WorldImageReaderTest.read(WorldImageReaderTest.java:100) == I'm still getting the same shapefile errors with maven 1 and maven2: - --- Testcase: testConcurrentReadWrite(org.geotools.data.shapefile.ShapefileReadWriteTest): Caused an ERROR C:\DOCUME~1\DAVIDB~1\LOCALS~1\Temp\test-shp27495.prj (The requested operation cannot be performed on a file with a user-mapped section open) java.io.FileNotFoundException: C:\DOCUME~1\DAVIDB~1\LOCALS~1\Temp\test-shp27495.prj (The requested operation cannot be performed on a file with a user-mapped section open) at java.io.FileOutputStream.open(Native Method) at java.io.FileOutputStream.init(FileOutputStream.java:179) at java.io.FileOutputStream.init(FileOutputStream.java:131) at org.geotools.data.shapefile.ShapefileDataStore.copyAndDelete(ShapefileDataStore.java:1087) at org.geotools.data.shapefile.ShapefileDataStore.createSchema(ShapefileDataStore.java:780) at org.geotools.data.shapefile.ShapefileReadWriteTest.test(ShapefileReadWriteTest.java:164) at org.geotools.data.shapefile.ShapefileReadWriteTest.test(ShapefileReadWriteTest.java:153) at org.geotools.data.shapefile.ShapefileReadWriteTest.testConcurrentReadWrite(ShapefileReadWriteTest.java:133) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) --- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=lnkkid=110944bid=241720dat=121642 ___ Geotools-devel mailing list Geotools-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
Re: [Geotools-devel] 2.2.x -- not building with maven 1 or maven 2 (different problems)
I spent a lot of time looking at whats was going on, but that code is *extreamly* difficult to read. As far as I can tell, it looks like it has one thread reading the shapefile and another writing to it. That should *always* throw an exception as the write operation seems to replace the original file which seems pretty bad is someone's reading the file that's being replaced. But, its failing on the .prj file and the reader is reading the .shp file. In fact, it appears the file that the thread is reading has **nothing at all** to do with the shapefile! The thread seems to be reading some random file it creates itself while the shapefile datastore system is using its own files. Weird. I think the test case is a bit screwy, but there's no comments to say what its testing so I have no idea if its doing what its supposed to do! The shapefile datastore doesnt have a nice summary of what it does, so its hard to figure out too (like what, exactly, does the useMemoryMap do). Perhaps someone who knows the datastore (and the test cases) can take a closer look at. But, it seemeds to have fixed itself. I dont know why, but I: 1. killed all the files in my tmp directory 2. re-checked out the project. 3. put the project in eclipse (maven eclipseAll) so I could debug the testcase So, others who have troubles building the shapefile plugin can try the above and see if it works. --- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=lnkkid=110944bid=241720dat=121642 ___ Geotools-devel mailing list Geotools-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
[Geotools-devel] 2.2.x -- oracle spatial not compiling (Failed to download ojdbc-14.jar)
+ | Executing clean,jar:install geotools-oraclespatial | Memory: 34M/59M + Attempting to download ojdbc-14.jar. WARNING: Failed to download ojdbc-14.jar. BUILD FAILED I looked at the project.xml and I see: !-- To use for real - download the ojdbc14.jar from oracle: copy ojdbc14.jar to ~/.maven/repository/oracle/jars/ojdbc-14.jar The test cases will try and connect to an oracle server on localhost; or on the server indicated by a .properties file. -- dependency artifactIdojdbc/artifactId groupIdoracle/groupId version14/version /dependency !-- To use for fake: Uncomment the following section, it downloads a jar full of Mock Objects so you can compile without downloading the real thing from oracle. -- !--dependency artifactIddummy_spatial/artifactId groupIdoracle/groupId version8.1.8/version /dependency-- !-- end of optional dependencies -- So, if I comment the first part and uncomment the 2nd, I can at least get the module to build. Is this something we can change in SVN because I'm sure everyone will be having the same problem. dave --- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=lnkkid=110944bid=241720dat=121642 ___ Geotools-devel mailing list Geotools-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
[Geotools-devel] TransformerBase --- indentation causing problems
Cory,
You added to TransformerBase.java (commit message GEOT-806: XML
Indentation fixed, line 71):
if (indentation -1) {
tFactory.setAttribute(indent-number, new
Integer(indentation));
}
This is causing:
Caused by: java.lang.IllegalArgumentException: Not supported: indent-number
at
org.apache.xalan.processor.TransformerFactoryImpl.setAttribute(TransformerFactoryImpl.java:582)
at
org.geotools.xml.transform.TransformerBase.createTransformer(TransformerBase.java:72)
at
org.vfny.geoserver.wms.responses.helpers.WMSCapsTransformer.createTransformer(WMSCapsTransformer.java:101)
at
org.geotools.xml.transform.TransformerBase$Task.init(TransformerBase.java:250)
at
org.geotools.xml.transform.TransformerBase.createTransformTask(TransformerBase.java:139)
at
org.geotools.xml.transform.TransformerBase.transform(TransformerBase.java:120)
at
org.geotools.xml.transform.TransformerBase.transform(TransformerBase.java:100)
...
Is this a java 1.5 option?
dave
---
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnkkid=110944bid=241720dat=121642
___
Geotools-devel mailing list
Geotools-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geotools-devel
[Geotools-devel] 2.2.x and maven 2 and WorldImageWriterTest (plugin/image)
Folks told me to go to maven 2 for 2.2.x. I did, but I'm getting different errors. I have JAI installed. I verified that com/sun/media/imageio/stream/FileChannelImageInputStream is in C:\j2sdk1.4.2_07\jre\lib\ext\jai_imageio.jar. Any idea whats wrong? dave --- Battery: org.geotools.gce.image.WorldImageWriterTest --- Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0.032 sec testWrite(org.geotools.gce.image.WorldImageWriterTest) Time elapsed: 0 sec ERROR! ... java.lang.NoClassDefFoundError: com/sun/media/imageio/stream/FileChannelImageInputStream at org.geotools.gce.image.WorldImageWriterTest.write(WorldImageWriterTest.java:130) at org.geotools.gce.image.WorldImageWriterTest.testWrite(WorldImageWriterTest.java:100) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at junit.framework.TestCase.runTest(TestCase.java:154) at junit.framework.TestCase.runBare(TestCase.java:127) at junit.framework.TestResult$1.protect(TestResult.java:106) at junit.framework.TestResult.runProtected(TestResult.java:124) at junit.framework.TestResult.run(TestResult.java:109) at junit.framework.TestCase.run(TestCase.java:118) at junit.framework.TestSuite.runTest(TestSuite.java:208) at junit.framework.TestSuite.run(TestSuite.java:203) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.apache.maven.surefire.battery.JUnitBattery.executeJUnit(JUnitBattery.java:242) at org.apache.maven.surefire.battery.JUnitBattery.execute(JUnitBattery.java:216) at org.apache.maven.surefire.Surefire.executeBattery(Surefire.java:215) at org.apache.maven.surefire.Surefire.run(Surefire.java:163) at org.apache.maven.surefire.Surefire.run(Surefire.java:87) at org.apache.maven.surefire.Surefire.run(Surefire.java:63) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.apache.maven.surefire.SurefireBooter.main(SurefireBooter.java:728) --- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=lnkkid=110944bid=241720dat=121642 ___ Geotools-devel mailing list Geotools-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
Re: [Geotools-devel] our stable branch isnt very stable (ShapefileReadWriteTest FAILED) -- 2.2.x
Thanks a lot, oliver. Changes should be commited to the 2.2.x AND trunk branches. Putting it in 2.2.x means that everyone using the stable branch gets your changes and putting it in trunk means that your changes will make it to the next stable version (and for the folks currently using trunk). 2.1.x doesnt have the problem, so you dont have to fix it there! dave --- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=lnkkid=110944bid=241720dat=121642 ___ Geotools-devel mailing list Geotools-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
Re: [Geoserver-devel] Re: [Geotools-devel] PostGIS not releasing connections on OutOfMemory
Most of the connection pooling is actually done by the Postgresql JDBC driver, but there's a proxy object on top of it thats hard to look inside. I talked to the Postgresql JDBC mailing list and they said oops - it should reset the stream so its in a consistent state. Dont know when this will actually be fixed, but hopefully soon. dave --- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=lnkkid=110944bid=241720dat=121642 ___ Geotools-devel mailing list Geotools-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
[Geotools-devel] our stable branch isnt very stable (ShapefileReadWriteTest FAILED) -- 2.2.x
I have a few patches I just did for 2.1.x - the bugs are still in 2.2.x and trunk. I'd like to move the patches forward, but I'm unable to build 2.2.x. I'm also stalled moving geoserver from 2.1.x to 2.2.x and have been for a while. I'm actually using geoserver quite a bit; this is prime testing (and fixing) time that I'd like to be putting to 2.2.x, but its going to 2.1.x. Is anyone working on this? I took a quick look, but was unable to figure out what was going wrong. dave Testcase: testConcurrentReadWrite(org.geotools.data.shapefile.ShapefileReadWriteTest): Caused an ERROR C:\DOCUME~1\DAVIDB~1\LOCALS~1\Temp\test-shp63644.prj (The requested operation cannot be performed on a file with a user-mapped section open) java.io.FileNotFoundException: C:\DOCUME~1\DAVIDB~1\LOCALS~1\Temp\test-shp63644.prj (The requested operation cannot be performed on a file with a user-mapped section open) at java.io.FileOutputStream.open(Native Method) at java.io.FileOutputStream.init(FileOutputStream.java:179) at java.io.FileOutputStream.init(FileOutputStream.java:131) at org.geotools.data.shapefile.ShapefileDataStore.copyAndDelete(ShapefileDataStore.java:1083) at org.geotools.data.shapefile.ShapefileDataStore.createSchema(ShapefileDataStore.java:778) at org.geotools.data.shapefile.ShapefileReadWriteTest.test(ShapefileReadWriteTest.java:164) at org.geotools.data.shapefile.ShapefileReadWriteTest.test(ShapefileReadWriteTest.java:153) at org.geotools.data.shapefile.ShapefileReadWriteTest.testConcurrentReadWrite(ShapefileReadWriteTest.java:133) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) There's also a bunch of these warnings: Mar 23, 2006 11:27:35 AM org.geotools.data.shapefile.dbf.DbaseFileHeader warn WARNING: Field Length for SAMP_POP set to 19 Preserving length, but should be set to Max of 18 for dbase III specification. --- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=lnkkid=110944bid=241720dat=121642 ___ Geotools-devel mailing list Geotools-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
Re: [Geotools-devel] our stable branch isnt very stable (ShapefileReadWriteTest FAILED) -- 2.2.x
Jody Garnett wrote: Jesse Eichar wrote: Jody said he had some people working on the problem so I haven't been looking into it. I'll see if he is still on this. Yes oliver is working on this, I hope to hear back tomorrow myself. Fraid we were stuck in a meeting all day today... On a related note I am stuck on the state of jira, and needing to go through and remove the code deprecated from the 2.0.x release. I hope to put in some good time tomorrow on this stuff, and then move onto oracle datastore QA. thanks! dave --- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=lnkkid=110944bid=241720dat=121642 ___ Geotools-devel mailing list Geotools-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
[Geotools-devel] PostGIS not releasing connections on OutOfMemory
Okay, I've found a pretty nasty bug in the PostGIS driver. My dataset is a table with about 250 rows in it. Each row has the VMAP0 dataset polygon of a country in. Each row is large. If you look closely at the JDBC datastore, you'll notice that it will fetch 200 rows at a time - which means its pretty much loading in the entire dataset for each query - thats about 250mb. This doesnt fit into the memory allocated to geoserver (especially if you're doing multiple requests at the same time), so an OutOfMemory error gets thrown and the reader is closed. So far, so good. Closing the reader will release the result set and the statement - this frees up about 60mb of memory. It then tries to close the DB connection. This fails - most likely because it was in the middle of reading a tuple when it was rudely interupted by running out of memory. It actually runs out of memory in PGStream.ReceiveTupleV3(), called from QueryExecutorImpl.processResults(). Later, the connection is closed and the PooledConnectionImpl$ConnectionHandler.invoke() is called, which (because we're not AutoCommit) attempt to rollback the transaction with a con.rollback(). This is where the problem occurs. Inside QueryExecutorImpl.processResults(), it gets a \ from the server (highly likely from the bytea returned in the original query). It, of course, doesnt understand this and throws an An I/O error occured while sending to the backend. error. The end result is that the connection doesnt appear to be closed and released back to the connectionpool. This means that connections to the database are being leaked, and probably a fair amount of memory. This, of course, causes the OutOfMemory error to happen more often. I'm probably going to have to talk to the postgresql JDBC people, but I thought I would ask here first incase people know something about. Are we doing anything funny with the connection pooling - I noticed there was a funny proxy object there? dave --- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=lnkkid=110944bid=241720dat=121642 ___ Geotools-devel mailing list Geotools-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
