Re: [JPP-Devel] Performance issue with large polygons with holes in ShapefileReader
Hi Martin,
Thanks for the hints,
Now the question is : do we prefer speed or robustness in case of weird
polygons
Or should we add options to our shapefile reader :
In this case, weird polygons are polygons where some shells are
described as if they were holes...
There's a few fixes required:
1. if only one shell is present, do not run the PIP code, but simply
assign the holes to the shell. (This fix has been made in the current
GeoTools PolygonHandler code, and could be copied from there)
Currently, we check if holes are inside a shell with a PIP test and in
case a hole don't
find any shell, it is considered itself as a shell.
Not sure how often it happens and how arcgis would read such a polygon.
Also I don't remember what happens if we let holes which happen to be
out of the shell in a Polygon.
Would need to create such a shapefile to test, as I don't think I have one.
2. this can be extended to checking for only one *candidate* shell (a
candidate shell is one whose envelope contains the hole envelope). This
code is not yet developed.
Same case as above. If a single candidate shell is found, do we still
make a PIP test
to be sure the hole is included ? Currently, we do.
3. the code uses ArrayList.indexOf. This is inefficient, since it uses
equals(), which does a full equality comparison. Instead, an iterator
and == should be used instead (code below)
Thanks, will test that as soon as I could create a monstruous polygon.
The performance improvement from just #1 was dramatic - reading a 1M
feature shapefile went from over 1000 s to 32 s.
Wow, your polygon should be really complex !
Will do some test and I'll report my results on this list.
Michaël
PolygonHandler change:
((ArrayList) holesForShells.get(findIndex(shells,
minShell))).add(testHole);
/**
* Finds a object in a list. Should be much faster than indexof
*
* @param list
* @param o
* @return
*/
private static int findIndex(ArrayList list, Object o)
{
int n = list.size();
for (int i = 0; i n; i++) {
if (list.get(i) == o)
return i;
}
return -1;
}
--
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
___
Jump-pilot-devel mailing list
Jump-pilot-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel
--
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
___
Jump-pilot-devel mailing list
Jump-pilot-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel
Re: [JPP-Devel] Performance issue with large polygons with holes in ShapefileReader
Hi Martin,
The performance improvement from just #1 was dramatic - reading a 1M
feature shapefile went from over 1000 s to 32 s.
Wonder how complex your shapefile is.
It took me hours to create the following 65Mo shapefile, but it has been
loaded in 11 s only
Min Max Moyenne Total
Pts 4 398604 3492.4928774928776 3677595
Trous 0 480437.41310541310541 39396
Composants 1 71 1.8034188034188035 1899
Surface 0.041523783176671714 1.1578715743783949E7 93654.73981537568
9.86184410255906E7
Longueur 1.553765588506568 456680.2181261205 4068.589468188681
4284224.710002681
Michaël
Wow, your polygon should be really complex !
Will do some test and I'll report my results on this list.
Michaël
PolygonHandler change:
((ArrayList) holesForShells.get(findIndex(shells,
minShell))).add(testHole);
/**
* Finds a object in a list. Should be much faster than indexof
*
* @param list
* @param o
* @return
*/
private static int findIndex(ArrayList list, Object o)
{
int n = list.size();
for (int i = 0; i n; i++) {
if (list.get(i) == o)
return i;
}
return -1;
}
--
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
___
Jump-pilot-devel mailing list
Jump-pilot-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel
--
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
___
Jump-pilot-devel mailing list
Jump-pilot-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel
--
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2___
Jump-pilot-devel mailing list
Jump-pilot-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel
Re: [JPP-Devel] Performance issue with large polygons with holes in ShapefileReader
Hi,
Here are some test cases done with a 200 Mb shapefile (thanks Jukka)
containing : 221770 polygons
worst case : 287273 pts and 5484 holes
Dataset : Current OJ code / 1st Martin's optimization / 3rd Martin's
optimization / 1st+3rd
whole shapefile : 41 s / 24 s / 24 s / 11 s
worst polygon : 4.6 s / 0.16 s / 3.3 s / 0.13 s
==
1st optimization has a big impact on worst case and a medium impact on a
whole file with mixed cases
3rd optimization has a small impact on worst case and a medium impact on
a whole file with mixed cases
I'll commit the 3rd optimization anyway has it has no side effect.
For the 1st one, i'll try to check if it may have negative impact on
some shapefile (if I can find or produce such shapefiles)
regards,
Michaël
Hi Martin,
The performance improvement from just #1 was dramatic - reading a 1M
feature shapefile went from over 1000 s to 32 s.
Wonder how complex your shapefile is.
It took me hours to create the following 65Mo shapefile, but it has
been loaded in 11 s only
Min Max Moyenne Total
Pts 4 398604 3492.4928774928776 3677595
Trous 0 480437.41310541310541 39396
Composants 1 71 1.8034188034188035 1899
Surface 0.041523783176671714 1.1578715743783949E7
93654.73981537568 9.86184410255906E7
Longueur 1.553765588506568 456680.2181261205 4068.589468188681
4284224.710002681
Michaël
Wow, your polygon should be really complex !
Will do some test and I'll report my results on this list.
Michaël
PolygonHandler change:
((ArrayList) holesForShells.get(findIndex(shells,
minShell))).add(testHole);
/**
* Finds a object in a list. Should be much faster than indexof
*
* @param list
* @param o
* @return
*/
private static int findIndex(ArrayList list, Object o)
{
int n = list.size();
for (int i = 0; i n; i++) {
if (list.get(i) == o)
return i;
}
return -1;
}
--
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
___
Jump-pilot-devel mailing list
Jump-pilot-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel
--
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
___
Jump-pilot-devel mailing list
Jump-pilot-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel
--
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
___
Jump-pilot-devel mailing list
Jump-pilot-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel
--
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2___
Jump-pilot-devel mailing list
Jump-pilot-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel
Re: [JPP-Devel] Performance issue with large polygons with holes in ShapefileReader
Hi Michaël,
Do you think this optimization will have an effect on the Shapefile with
overlapping shells issue that we worked in 2008?
see:
http://www.mail-archive.com/jump-pilot-devel@lists.sourceforge.net/msg04403.html
regards,
Larry
2012/4/20 Michaël Michaud michael.mich...@free.fr
Hi,
Here are some test cases done with a 200 Mb shapefile (thanks Jukka)
containing : 221770 polygons
worst case : 287273 pts and 5484 holes
Dataset : Current OJ code / 1st Martin's optimization / 3rd Martin's
optimization / 1st+3rd
whole shapefile : 41 s / 24 s / 24 s / 11 s
worst polygon : 4.6 s / 0.16 s / 3.3 s / 0.13 s
==
1st optimization has a big impact on worst case and a medium impact on a
whole file with mixed cases
3rd optimization has a small impact on worst case and a medium impact on a
whole file with mixed cases
I'll commit the 3rd optimization anyway has it has no side effect.
For the 1st one, i'll try to check if it may have negative impact on some
shapefile (if I can find or produce such shapefiles)
regards,
Michaël
Hi Martin,
The performance improvement from just #1 was dramatic - reading a 1M
feature shapefile went from over 1000 s to 32 s.
Wonder how complex your shapefile is.
It took me hours to create the following 65Mo shapefile, but it has been
loaded in 11 s only
Min Max Moyenne Total Pts 4 398604 3492.4928774928776 3677595
Trous 0 4804 37.41310541310541 39396 Composants 1 71 1.8034188034188035 1899
Surface 0.041523783176671714 1.1578715743783949E7 93654.73981537568
9.86184410255906E7 Longueur 1.553765588506568 456680.2181261205
4068.589468188681 4284224.710002681
Michaël
Wow, your polygon should be really complex !
Will do some test and I'll report my results on this list.
Michaël
PolygonHandler change:
((ArrayList) holesForShells.get(findIndex(shells,
minShell))).add(testHole);
/**
* Finds a object in a list. Should be much faster than indexof
*
* @param list
* @param o
* @return
*/
private static int findIndex(ArrayList list, Object o)
{
int n = list.size();
for (int i = 0; i n; i++) {
if (list.get(i) == o)
return i;
}
return -1;
}
--
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it
FREE!http://p.sf.net/sfu/Boundary-d2dvs2
___
Jump-pilot-devel mailing
listJump-pilot-devel@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/jump-pilot-devel
--
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it
FREE!http://p.sf.net/sfu/Boundary-d2dvs2
___
Jump-pilot-devel mailing
listJump-pilot-devel@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/jump-pilot-devel
--
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it
FREE!http://p.sf.net/sfu/Boundary-d2dvs2
___
Jump-pilot-devel mailing
listJump-pilot-devel@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/jump-pilot-devel
--
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
___
Jump-pilot-devel mailing list
Jump-pilot-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel
--
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2___
Jump-pilot-devel mailing list
Jump-pilot-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel
Re: [JPP-Devel] Performance issue with large polygons with holes in ShapefileReader
Hi Larry,
Thanks for the link,
I was just looking for it.
I found also this thread, but this is not strictly the same problem.
Anyway, it should be good to test again those cases :
http://groups.google.com/group/openjump-users/browse_thread/thread/55e12e96507f3b80/fb152c3d3f87d88d%3Fq%3D%2522Becker%2BBecker%2522%23fb152c3d3f87d88dei=iGwTS6eaOpW8Qpmqic0Osa=tct=rescd=4source=groupsusg=AFQjCNEJRegXqGvSzL2nLwg1KQYKUoCaqA
Michaël
Hi Michaël,
Do you think this optimization will have an effect on the Shapefile
with overlapping shells issue that we worked in 2008?
see:
http://www.mail-archive.com/jump-pilot-devel@lists.sourceforge.net/msg04403.html
regards,
Larry
2012/4/20 Michaël Michaud michael.mich...@free.fr
mailto:michael.mich...@free.fr
Hi,
Here are some test cases done with a 200 Mb shapefile (thanks Jukka)
containing : 221770 polygons
worst case : 287273 pts and 5484 holes
Dataset : Current OJ code / 1st Martin's optimization / 3rd
Martin's optimization / 1st+3rd
whole shapefile : 41 s / 24 s / 24 s / 11 s
worst polygon : 4.6 s / 0.16 s / 3.3 s / 0.13 s
==
1st optimization has a big impact on worst case and a medium
impact on a whole file with mixed cases
3rd optimization has a small impact on worst case and a medium
impact on a whole file with mixed cases
I'll commit the 3rd optimization anyway has it has no side effect.
For the 1st one, i'll try to check if it may have negative impact
on some shapefile (if I can find or produce such shapefiles)
regards,
Michaël
Hi Martin,
The performance improvement from just #1 was dramatic - reading a 1M
feature shapefile went from over 1000 s to 32 s.
Wonder how complex your shapefile is.
It took me hours to create the following 65Mo shapefile, but it
has been loaded in 11 s only
Min Max Moyenne Total
Pts 4 398604 3492.4928774928776 3677595
Trous 0 480437.41310541310541 39396
Composants 1 71 1.8034188034188035 1899
Surface 0.0415237831766717141.1578715743783949E7
93654.73981537568 9.86184410255906E7
Longueur1.553765588506568 456680.2181261205
4068.589468188681 4284224.710002681
Michaël
Wow, your polygon should be really complex !
Will do some test and I'll report my results on this list.
Michaël
PolygonHandler change:
((ArrayList) holesForShells.get(findIndex(shells,
minShell))).add(testHole);
/**
* Finds a object in a list. Should be much faster than indexof
*
* @param list
* @param o
* @return
*/
private static int findIndex(ArrayList list, Object o)
{
int n = list.size();
for (int i = 0; i n; i++) {
if (list.get(i) == o)
return i;
}
return -1;
}
--
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
___
Jump-pilot-devel mailing list
Jump-pilot-devel@lists.sourceforge.net
mailto:Jump-pilot-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel
--
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
___
Jump-pilot-devel mailing list
Jump-pilot-devel@lists.sourceforge.net
mailto:Jump-pilot-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel
--
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
___
Jump-pilot-devel mailing list
Jump-pilot-devel@lists.sourceforge.net
mailto:Jump-pilot-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel
--
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
Re: [JPP-Devel] Performance issue with large polygons with holes in ShapefileReader
Hi Larry,
Do you think this optimization will have an effect on the Shapefile
with overlapping shells issue that we worked in 2008?
It should not.
It takes place after the change made in 2008
- 2008 : if no hole, check if some shells are overlapping and chenge
them to hole
- 2012 : if there is one shell only, consider every hole is a hole of
this shell without PIP checking
If you want to double check or to add it in skyjump, you can see diff here
http://jump-pilot.svn.sourceforge.net/viewvc/jump-pilot/core/trunk/src/org/geotools/shapefile/PolygonHandler.java?r1=2110r2=2830sortby=date
I checked with Martin's file that no single square millimeter has been lost,
but I have no shapefile with overlapping shells to test that 2008 patch
is not affected.
Michaël
see:
http://www.mail-archive.com/jump-pilot-devel@lists.sourceforge.net/msg04403.html
regards,
Larry
2012/4/20 Michaël Michaud michael.mich...@free.fr
mailto:michael.mich...@free.fr
Hi,
Here are some test cases done with a 200 Mb shapefile (thanks Jukka)
containing : 221770 polygons
worst case : 287273 pts and 5484 holes
Dataset : Current OJ code / 1st Martin's optimization / 3rd
Martin's optimization / 1st+3rd
whole shapefile : 41 s / 24 s / 24 s / 11 s
worst polygon : 4.6 s / 0.16 s / 3.3 s / 0.13 s
==
1st optimization has a big impact on worst case and a medium
impact on a whole file with mixed cases
3rd optimization has a small impact on worst case and a medium
impact on a whole file with mixed cases
I'll commit the 3rd optimization anyway has it has no side effect.
For the 1st one, i'll try to check if it may have negative impact
on some shapefile (if I can find or produce such shapefiles)
regards,
Michaël
Hi Martin,
The performance improvement from just #1 was dramatic - reading a 1M
feature shapefile went from over 1000 s to 32 s.
Wonder how complex your shapefile is.
It took me hours to create the following 65Mo shapefile, but it
has been loaded in 11 s only
Min Max Moyenne Total
Pts 4 398604 3492.4928774928776 3677595
Trous 0 480437.41310541310541 39396
Composants 1 71 1.8034188034188035 1899
Surface 0.0415237831766717141.1578715743783949E7
93654.73981537568 9.86184410255906E7
Longueur1.553765588506568 456680.2181261205
4068.589468188681 4284224.710002681
Michaël
Wow, your polygon should be really complex !
Will do some test and I'll report my results on this list.
Michaël
PolygonHandler change:
((ArrayList) holesForShells.get(findIndex(shells,
minShell))).add(testHole);
/**
* Finds a object in a list. Should be much faster than indexof
*
* @param list
* @param o
* @return
*/
private static int findIndex(ArrayList list, Object o)
{
int n = list.size();
for (int i = 0; i n; i++) {
if (list.get(i) == o)
return i;
}
return -1;
}
--
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
___
Jump-pilot-devel mailing list
Jump-pilot-devel@lists.sourceforge.net
mailto:Jump-pilot-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel
--
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
___
Jump-pilot-devel mailing list
Jump-pilot-devel@lists.sourceforge.net
mailto:Jump-pilot-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel
--
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
___
Jump-pilot-devel mailing list
Jump-pilot-devel@lists.sourceforge.net
mailto:Jump-pilot-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel
Re: [JPP-Devel] Performance issue with large polygons with holes in ShapefileReader
Martin, Thanks for the file, very impressive (did not guess where it was - but only searched in Canada). You convinced me. We had two previous discussions on the list about preserving weird shapefile as much as possible, but none of them was about holes lying out of their shell. So let's go where geotools already is. I had a quick look on your suggestion #2, but could not see a way to improve. For more than one shell case, we don't know which hole is associated with which shell before the test (except if we assume that the description of a shell is immediately followed by the description of its holes) Don't know if we should assume this. For more than one shell case, I just changed the order of tests to do a bit more envelope tests and a bit less PiP tests but I could not measure the difference on your file. Michaël Very thorough investigation, Michael. The shapefile I was using is here: http://maps.cmparks.net/geoserver/metroparks/ows?service=WFSversion=1.0.0request=GetFeaturetypeName=metroparks:tpi_1outputFormat=SHAPE-ZIP ( http://tinyurl.com/c9k26gp ) It is pretty gnarly - over 1M polygons, and has the following stats for the largest (vertices/holes): 18527 1112 58597 5526 31087 1598 24152 1318 17989 1386 20280 1098 20815 1336 44871 1204 61496 3024 28370 1120 46058 2156 21970 1546 91379 3034 121028 8012 81172 4544 28544 1272 20524 1074 6 1992 2282758 169278 1166731 94382 241350 14250 142028 5922 139035 5690 33610 1344 83112 3134 38464 1048 GeoTools has had optimization #1 for quite a while, I think, if that's any reassurance. On 4/20/2012 6:29 AM, Michaël Michaud wrote: Hi, Here are some test cases done with a 200 Mb shapefile (thanks Jukka) containing : 221770 polygons worst case : 287273 pts and 5484 holes Dataset : Current OJ code / 1st Martin's optimization / 3rd Martin's optimization / 1st+3rd whole shapefile : 41 s / 24 s / 24 s / 11 s worst polygon : 4.6 s / 0.16 s / 3.3 s / 0.13 s == 1st optimization has a big impact on worst case and a medium impact on a whole file with mixed cases 3rd optimization has a small impact on worst case and a medium impact on a whole file with mixed cases I'll commit the 3rd optimization anyway has it has no side effect. For the 1st one, i'll try to check if it may have negative impact on some shapefile (if I can find or produce such shapefiles) -- For Developers, A Lot Can Happen In A Second. Boundary is the first to Know...and Tell You. Monitor Your Applications in Ultra-Fine Resolution. Try it FREE! http://p.sf.net/sfu/Boundary-d2dvs2 ___ Jump-pilot-devel mailing list Jump-pilot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel -- For Developers, A Lot Can Happen In A Second. Boundary is the first to Know...and Tell You. Monitor Your Applications in Ultra-Fine Resolution. Try it FREE! http://p.sf.net/sfu/Boundary-d2dvs2___ Jump-pilot-devel mailing list Jump-pilot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel
[JPP-Devel] [ jump-pilot-Bugs-3519929 ] PostGIS : do not respect date datatype
Bugs item #3519929, was opened at 2012-04-20 11:59 Message generated for change (Tracker Item Submitted) made by michaudm You can respond by visiting: https://sourceforge.net/tracker/?func=detailatid=679906aid=3519929group_id=118054 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Plug-In Group: None Status: Open Resolution: None Priority: 6 Private: No Submitted By: michael michaud (michaudm) Assigned to: michael michaud (michaudm) Summary: PostGIS : do not respect date datatype Initial Comment: Date/TimeStamp table fields create String OpenJUMP attributes. It throws exceptions if one want to update (insert) a feature from OpenJUMP to PostGIS -- You can respond by visiting: https://sourceforge.net/tracker/?func=detailatid=679906aid=3519929group_id=118054 -- For Developers, A Lot Can Happen In A Second. Boundary is the first to Know...and Tell You. Monitor Your Applications in Ultra-Fine Resolution. Try it FREE! http://p.sf.net/sfu/Boundary-d2dvs2 ___ Jump-pilot-devel mailing list Jump-pilot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel
[JPP-Devel] [ jump-pilot-Bugs-3519931 ] PostGIS : do not manage geometry dimension
Bugs item #3519931, was opened at 2012-04-20 12:02 Message generated for change (Tracker Item Submitted) made by michaudm You can respond by visiting: https://sourceforge.net/tracker/?func=detailatid=679906aid=3519931group_id=118054 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Plug-In Group: None Status: Open Resolution: None Priority: 4 Private: No Submitted By: michael michaud (michaudm) Assigned to: michael michaud (michaudm) Summary: PostGIS : do not manage geometry dimension Initial Comment: If I have a geometry table in PostGIS with a constraint on geometry dimension (3d), I cannot update it as geometry returned is 2d only. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detailatid=679906aid=3519931group_id=118054 -- For Developers, A Lot Can Happen In A Second. Boundary is the first to Know...and Tell You. Monitor Your Applications in Ultra-Fine Resolution. Try it FREE! http://p.sf.net/sfu/Boundary-d2dvs2 ___ Jump-pilot-devel mailing list Jump-pilot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel
[JPP-Devel] [ jump-pilot-Bugs-3519933 ] PostGIS 'SAVE DATASET AS' write error
Bugs item #3519933, was opened at 2012-04-20 12:42 Message generated for change (Tracker Item Submitted) made by You can respond by visiting: https://sourceforge.net/tracker/?func=detailatid=679906aid=3519933group_id=118054 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: OpenJUMP - Menu - Layer Group: MacOSX Status: Open Resolution: None Priority: 5 Private: No Submitted By: https://www.google.com/accounts () Assigned to: Nobody/Anonymous (nobody) Summary: PostGIS 'SAVE DATASET AS' write error Initial Comment: When attempting to save an edited PostGIS layer, it fails, throwing the error as seen in the screenshot attachment. I am connecting to a PostgreSQL 9.1/PostGIS 2.x database table. I took a local shapefile layer that has the example same schema as does have the remote PostGIS layer, and I copied a feature from this local shapefile layer, and pasted it into the remote PostIGS layer, and then attempted to save it, at which pont I receive this error. Thanks, Eric Jarvies -- You can respond by visiting: https://sourceforge.net/tracker/?func=detailatid=679906aid=3519933group_id=118054 -- For Developers, A Lot Can Happen In A Second. Boundary is the first to Know...and Tell You. Monitor Your Applications in Ultra-Fine Resolution. Try it FREE! http://p.sf.net/sfu/Boundary-d2dvs2 ___ Jump-pilot-devel mailing list Jump-pilot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel
[JPP-Devel] [ jump-pilot-Bugs-3519933 ] PostGIS 'SAVE DATASET AS' write error
Bugs item #3519933, was opened at 2012-04-20 12:42 Message generated for change (Comment added) made by michaudm You can respond by visiting: https://sourceforge.net/tracker/?func=detailatid=679906aid=3519933group_id=118054 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: OpenJUMP - Menu - Layer Group: MacOSX Status: Open Resolution: None Priority: 5 Private: No Submitted By: Eric Jarvies () Assigned to: Nobody/Anonymous (nobody) Summary: PostGIS 'SAVE DATASET AS' write error Initial Comment: When attempting to save an edited PostGIS layer, it fails, throwing the error as seen in the screenshot attachment. I am connecting to a PostgreSQL 9.1/PostGIS 2.x database table. I took a local shapefile layer that has the example same schema as does have the remote PostGIS layer, and I copied a feature from this local shapefile layer, and pasted it into the remote PostIGS layer, and then attempted to save it, at which pont I receive this error. Thanks, Eric Jarvies -- Comment By: michael michaud (michaudm) Date: 2012-04-20 14:31 Message: Eric, Can you confirm : - that you are using PostGIS 1.5.0 - that you are trying to save a layer loaded from postgis not the one loaded from the shapefile Can you please tel me exactly the options used in the database connexion panel (as the message you get should only happen if you try to create a new table) Can you also tel me if your postgis table has a reference in the geometry_columns table, what is its srid there, and what is the srid in your OpenJUMP remote layer (Layer Change the SRID) Thanks for your help -- You can respond by visiting: https://sourceforge.net/tracker/?func=detailatid=679906aid=3519933group_id=118054 -- For Developers, A Lot Can Happen In A Second. Boundary is the first to Know...and Tell You. Monitor Your Applications in Ultra-Fine Resolution. Try it FREE! http://p.sf.net/sfu/Boundary-d2dvs2 ___ Jump-pilot-devel mailing list Jump-pilot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel
[JPP-Devel] [ jump-pilot-Bugs-3519929 ] PostGIS : do not respect date datatype
Bugs item #3519929, was opened at 2012-04-20 11:59 Message generated for change (Comment added) made by michaudm You can respond by visiting: https://sourceforge.net/tracker/?func=detailatid=679906aid=3519929group_id=118054 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Plug-In Group: None Status: Closed Resolution: Fixed Priority: 6 Private: No Submitted By: michael michaud (michaudm) Assigned to: michael michaud (michaudm) Summary: PostGIS : do not respect date datatype Initial Comment: Date/TimeStamp table fields create String OpenJUMP attributes. It throws exceptions if one want to update (insert) a feature from OpenJUMP to PostGIS -- Comment By: michael michaud (michaudm) Date: 2012-04-20 15:12 Message: Fixed - will be available in next release (152) -- You can respond by visiting: https://sourceforge.net/tracker/?func=detailatid=679906aid=3519929group_id=118054 -- For Developers, A Lot Can Happen In A Second. Boundary is the first to Know...and Tell You. Monitor Your Applications in Ultra-Fine Resolution. Try it FREE! http://p.sf.net/sfu/Boundary-d2dvs2 ___ Jump-pilot-devel mailing list Jump-pilot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel
[JPP-Devel] [ jump-pilot-Bugs-3519933 ] PostGIS 'SAVE DATASET AS' write error
Bugs item #3519933, was opened at 2012-04-20 12:42 Message generated for change (Comment added) made by mentaer You can respond by visiting: https://sourceforge.net/tracker/?func=detailatid=679906aid=3519933group_id=118054 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: OpenJUMP - Menu - Layer Group: MacOSX Status: Open Resolution: None Priority: 5 Private: No Submitted By: Eric Jarvies () Assigned to: Nobody/Anonymous (nobody) Summary: PostGIS 'SAVE DATASET AS' write error Initial Comment: When attempting to save an edited PostGIS layer, it fails, throwing the error as seen in the screenshot attachment. I am connecting to a PostgreSQL 9.1/PostGIS 2.x database table. I took a local shapefile layer that has the example same schema as does have the remote PostGIS layer, and I copied a feature from this local shapefile layer, and pasted it into the remote PostIGS layer, and then attempted to save it, at which pont I receive this error. Thanks, Eric Jarvies -- Comment By: mentaer (mentaer) Date: 2012-04-20 15:43 Message: I wrote in my notes for OJ Horae that saving to a file under MacOSX does work with OJ 1.5.x - but not to PostGIS. So it might be that this is a MacOSX specific issue??? -- Comment By: michael michaud (michaudm) Date: 2012-04-20 14:31 Message: Eric, Can you confirm : - that you are using PostGIS 1.5.0 - that you are trying to save a layer loaded from postgis not the one loaded from the shapefile Can you please tel me exactly the options used in the database connexion panel (as the message you get should only happen if you try to create a new table) Can you also tel me if your postgis table has a reference in the geometry_columns table, what is its srid there, and what is the srid in your OpenJUMP remote layer (Layer Change the SRID) Thanks for your help -- You can respond by visiting: https://sourceforge.net/tracker/?func=detailatid=679906aid=3519933group_id=118054 -- For Developers, A Lot Can Happen In A Second. Boundary is the first to Know...and Tell You. Monitor Your Applications in Ultra-Fine Resolution. Try it FREE! http://p.sf.net/sfu/Boundary-d2dvs2 ___ Jump-pilot-devel mailing list Jump-pilot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel
Re: [JPP-Devel] Performance issue with large polygons with holes in ShapefileReader
The file I sent is in Ohio, I think. It was sent to me by Stephen Mather, who I met in Washington DC at FOSS4GNA last week. It's pretty excessive - not the normal kind of shapefile, I think. But still, it's nice to optimize the corner cases, especially if they take 10x longer to run. I did realize that the use case for GeoTools (in GeoServer at least) is a bit different to OJ. GeoServer doesn't cache data in memory, so it's going to be reading data many times, so every second is precious. And there's no expectation it will fix bad data - it just has to provide something that mirrors what was read. OJ on the other hand might be expected to try to fix some of the bad data, and users might be willing to spend more time on load to get that functionality. Anyway, fix #3 is good in any case, and it seems like a good thing that you added fix #1 too. Fix #2 was a bit more speculative, and it only applies in very specific circumstances, so it's probably not as critical. (BTW, the idea was not to assume to which shell holes were assigned to, but to check if only one shell could actually contain a given hole, by virtue of the envelope intersection test). On 4/20/2012 11:02 AM, Michaël Michaud wrote: Martin, Thanks for the file, very impressive (did not guess where it was - but only searched in Canada). You convinced me. We had two previous discussions on the list about preserving weird shapefile as much as possible, but none of them was about holes lying out of their shell. So let's go where geotools already is. I had a quick look on your suggestion #2, but could not see a way to improve. For more than one shell case, we don't know which hole is associated with which shell before the test (except if we assume that the description of a shell is immediately followed by the description of its holes) Don't know if we should assume this. For more than one shell case, I just changed the order of tests to do a bit more envelope tests and a bit less PiP tests but I could not measure the difference on your file. Michaël Very thorough investigation, Michael. The shapefile I was using is here: http://maps.cmparks.net/geoserver/metroparks/ows?service=WFSversion=1.0.0request=GetFeaturetypeName=metroparks:tpi_1outputFormat=SHAPE-ZIP ( http://tinyurl.com/c9k26gp ) It is pretty gnarly - over 1M polygons, and has the following stats for the largest (vertices/holes): 18527 1112 58597 5526 31087 1598 24152 1318 17989 1386 20280 1098 20815 1336 44871 1204 61496 3024 28370 1120 46058 2156 21970 1546 91379 3034 121028 8012 81172 4544 28544 1272 20524 1074 6 1992 2282758 169278 1166731 94382 241350 14250 142028 5922 139035 5690 33610 1344 83112 3134 38464 1048 GeoTools has had optimization #1 for quite a while, I think, if that's any reassurance. On 4/20/2012 6:29 AM, Michaël Michaud wrote: Hi, Here are some test cases done with a 200 Mb shapefile (thanks Jukka) containing : 221770 polygons worst case : 287273 pts and 5484 holes Dataset : Current OJ code / 1st Martin's optimization / 3rd Martin's optimization / 1st+3rd whole shapefile : 41 s / 24 s / 24 s / 11 s worst polygon : 4.6 s / 0.16 s / 3.3 s / 0.13 s == 1st optimization has a big impact on worst case and a medium impact on a whole file with mixed cases 3rd optimization has a small impact on worst case and a medium impact on a whole file with mixed cases I'll commit the 3rd optimization anyway has it has no side effect. For the 1st one, i'll try to check if it may have negative impact on some shapefile (if I can find or produce such shapefiles) -- For Developers, A Lot Can Happen In A Second. Boundary is the first to Know...and Tell You. Monitor Your Applications in Ultra-Fine Resolution. Try it FREE! http://p.sf.net/sfu/Boundary-d2dvs2 ___ Jump-pilot-devel mailing list Jump-pilot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel -- For Developers, A Lot Can Happen In A Second. Boundary is the first to Know...and Tell You. Monitor Your Applications in Ultra-Fine Resolution. Try it FREE! http://p.sf.net/sfu/Boundary-d2dvs2 ___ Jump-pilot-devel mailing list Jump-pilot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel No virus found in this message. Checked by AVG - www.avg.com http://www.avg.com Version: 2012.0.1913 / Virus Database: 2411/4948 - Release Date: 04/20/12 -- For Developers, A Lot Can Happen In A Second. Boundary is the first to Know...and Tell You. Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
