Tessellate exception in Elasticsearch
Hi, This is an error which we get in Elasticsearch when trying to index geo_shape fields but it seems this can be narrowed down to a problem in Lucene. We can reproduce the problem withe ES 6.8.x and ES 7.7.x. This is the error we are getting: Caused by: java.lang.IllegalArgumentException: Unable to Tessellate shape [[9.3213479767, -3.20048586995] [-17.71666808212, -42.558438073] [-1.052667893531, -53.756582018] [14.88742787267, -64.4683260198] [32.67469200505, -76.4213498962] [57.8286121207, -93.3246459366] [87.639876011, -113.3577339016] [87.883524044, -113.5214459883] [47.6075479995, -48.6889979781] [-3.927812089984, 26.37826614264] [79.4449960563, 98.8751621119] [73.8492839626, 90.7297539996] [41.9057321381, 44.2310018589] [9.3213479767, -3.20048586995] ]. Possible malformed shape detected. at org.apache.lucene.geo.Tessellator.tessellate(Tessellator.java:114) ~[lucene-sandbox-7.7.3.jar:7.7.3 1a0d2a901dfec93676b0fe8be425101ceb754b85 - noble - 2020-04-21 10:31:55] at org.apache.lucene.document.LatLonShape.createIndexableFields(LatLonShape.java:73) ~[lucene-sandbox-7.7.3.jar:7.7.3 1a0d2a901dfec93676b0fe8be425101ceb754b85 - noble - 2020-04-21 10:31:55] at org.elasticsearch.index.mapper.GeoShapeFieldMapper.indexShape(GeoShapeFieldMapper.java:146) ~[elasticsearch-6.8.9.jar:6.8.9] This is a very basic geometry. Could someone please explain why this shape is invalid? [cid:ea7c35c0-0708-49be-a450-c5440a1b2b26] Thanks in advance, Wouter Claeys
Re: Tessellate exception in Elasticsearch
Hi, I think your polygon has intersecting edges but it is difficult to reproduce with that output. Could you provide the original polygon you are trying to index? Thanks! On Thu, Jun 4, 2020 at 11:30 AM Claeys Wouter wrote: > Hi, > > This is an error which we get in Elasticsearch when trying to index > geo_shape fields but it seems this can be narrowed down to a problem in > Lucene. We can reproduce the problem withe ES 6.8.x and ES 7.7.x. This is > the error we are getting: > > Caused by: java.lang.IllegalArgumentException: Unable to Tessellate shape > [[9.3213479767, -3.20048586995] [-17.71666808212, > -42.558438073] [-1.052667893531, -53.756582018] > [14.88742787267, -64.4683260198] [32.67469200505, > -76.4213498962] [57.8286121207, -93.3246459366] > [87.639876011, -113.3577339016] [87.883524044, > -113.5214459883] [47.6075479995, -48.6889979781] > [-3.927812089984, 26.37826614264] [79.4449960563, > 98.8751621119] [73.8492839626, 90.7297539996] > [41.9057321381, 44.2310018589] [9.3213479767, > -3.20048586995] ]. Possible malformed shape detected. > at > org.apache.lucene.geo.Tessellator.tessellate(Tessellator.java:114) > ~[lucene-sandbox-7.7.3.jar:7.7.3 1a0d2a901dfec93676b0fe8be425101ceb754b85 - > noble - 2020-04-21 10:31:55] > at > org.apache.lucene.document.LatLonShape.createIndexableFields(LatLonShape.java:73) > ~[lucene-sandbox-7.7.3.jar:7.7.3 1a0d2a901dfec93676b0fe8be425101ceb754b85 - > noble - 2020-04-21 10:31:55] > at > org.elasticsearch.index.mapper.GeoShapeFieldMapper.indexShape(GeoShapeFieldMapper.java:146) > ~[elasticsearch-6.8.9.jar:6.8.9] > > This is a very basic geometry. Could someone please explain why this shape > is invalid? > > > > > Thanks in advance, > > Wouter Claeys >
Re: Tessellate exception in Elasticsearch
Hi, This is the original polygon: { "crs":{ "type":"name", "properties":{ "name":"urn:ogc:def:crs:EPSG::31370" } }, "type":"MultiPolygon", "coordinates":[ [ [ [ 171044.231002, 175818.094268 ], [ 170996.799514, 175850.678652 ], [ 170957.441562, 175877.716668 ], [ 170946.243418, 175861.052668 ], [ 170935.531674, 175845.112572 ], [ 170923.57865, 175827.325308 ], [ 170906.675354, 175802.171388 ], [ 170886.642266, 175772.360124 ], [ 170886.478554, 175772.116476 ], [ 170951.311002, 175727.607548 ], [ 171026.378266, 175676.072188 ], [ 171098.875162, 175780.555004 ], [ 171090.729754, 175786.150716 ], [ 171044.231002, 175818.094268 ] ] ] ] } Thanks! Van: Ignacio Vera Sequeiros Verzonden: donderdag 4 juni 2020 12:24 Aan: java-user@lucene.apache.org Onderwerp: Re: Tessellate exception in Elasticsearch Hi, I think your polygon has intersecting edges but it is difficult to reproduce with that output. Could you provide the original polygon you are trying to index? Thanks! On Thu, Jun 4, 2020 at 11:30 AM Claeys Wouter wrote: > Hi, > > This is an error which we get in Elasticsearch when trying to index > geo_shape fields but it seems this can be narrowed down to a problem in > Lucene. We can reproduce the problem withe ES 6.8.x and ES 7.7.x. This is > the error we are getting: > > Caused by: java.lang.IllegalArgumentException: Unable to Tessellate shape > [[9.3213479767, -3.20048586995] [-17.71666808212, > -42.558438073] [-1.052667893531, -53.756582018] > [14.88742787267, -64.4683260198] [32.67469200505, > -76.4213498962] [57.8286121207, -93.3246459366] > [87.639876011, -113.3577339016] [87.883524044, > -113.5214459883] [47.6075479995, -48.6889979781] > [-3.927812089984, 26.37826614264] [79.4449960563, > 98.8751621119] [73.8492839626, 90.7297539996] > [41.9057321381, 44.2310018589] [9.3213479767, > -3.20048586995] ]. Possible malformed shape detected. > at > org.apache.lucene.geo.Tessellator.tessellate(Tessellator.java:114) > ~[lucene-sandbox-7.7.3.jar:7.7.3 1a0d2a901dfec93676b0fe8be425101ceb754b85 - > noble - 2020-04-21 10:31:55] > at > org.apache.lucene.document.LatLonShape.createIndexableFields(LatLonShape.java:73) > ~[lucene-sandbox-7.7.3.jar:7.7.3 1a0d2a901dfec93676b0fe8be425101ceb754b85 - > noble - 2020-04-21 10:31:55] > at > org.elasticsearch.index.mapper.GeoShapeFieldMapper.indexShape(GeoShapeFieldMapper.java:146) > ~[elasticsearch-6.8.9.jar:6.8.9] > > This is a very basic geometry. Could someone please explain why this shape > is invalid? > > > > > Thanks in advance, > > Wouter Claeys >
Re: Tessellate exception in Elasticsearch
I think this is not a lucene issue. Elasticsearch geo_shape only supports (and it assumes) polygons on the WGS-84 reference system. On Thu, Jun 4, 2020 at 1:38 PM Claeys Wouter wrote: > Hi, > > This is the original polygon: > > { >"crs":{ > "type":"name", > "properties":{ > "name":"urn:ogc:def:crs:EPSG::31370" > } >}, >"type":"MultiPolygon", >"coordinates":[ > [ > [ > [ >171044.231002, >175818.094268 > ], > [ >170996.799514, >175850.678652 > ], > [ >170957.441562, >175877.716668 > ], > [ >170946.243418, >175861.052668 > ], > [ >170935.531674, >175845.112572 > ], > [ >170923.57865, >175827.325308 > ], > [ >170906.675354, >175802.171388 > ], > [ >170886.642266, >175772.360124 > ], > [ >170886.478554, >175772.116476 > ], > [ >170951.311002, >175727.607548 > ], > [ >171026.378266, >175676.072188 > ], > [ >171098.875162, >175780.555004 > ], > [ >171090.729754, >175786.150716 > ], > [ >171044.231002, >175818.094268 > ] > ] > ] >] > } > > Thanks! > > > Van: Ignacio Vera Sequeiros > Verzonden: donderdag 4 juni 2020 12:24 > Aan: java-user@lucene.apache.org > Onderwerp: Re: Tessellate exception in Elasticsearch > > Hi, > > I think your polygon has intersecting edges but it is difficult to > reproduce with that output. Could you provide the original polygon you are > trying to index? > > Thanks! > > On Thu, Jun 4, 2020 at 11:30 AM Claeys Wouter > > wrote: > > > Hi, > > > > This is an error which we get in Elasticsearch when trying to index > > geo_shape fields but it seems this can be narrowed down to a problem in > > Lucene. We can reproduce the problem withe ES 6.8.x and ES 7.7.x. This is > > the error we are getting: > > > > Caused by: java.lang.IllegalArgumentException: Unable to Tessellate shape > > [[9.3213479767, -3.20048586995] [-17.71666808212, > > -42.558438073] [-1.052667893531, -53.756582018] > > [14.88742787267, -64.4683260198] [32.67469200505, > > -76.4213498962] [57.8286121207, -93.3246459366] > > [87.639876011, -113.3577339016] [87.883524044, > > -113.5214459883] [47.6075479995, -48.6889979781] > > [-3.927812089984, 26.37826614264] [79.4449960563, > > 98.8751621119] [73.8492839626, 90.7297539996] > > [41.9057321381, 44.2310018589] [9.3213479767, > > -3.20048586995] ]. Possible malformed shape detected. > > at > > org.apache.lucene.geo.Tessellator.tessellate(Tessellator.java:114) > > ~[lucene-sandbox-7.7.3.jar:7.7.3 > 1a0d2a901dfec93676b0fe8be425101ceb754b85 - > > noble - 2020-04-21 10:31:55] > > at > > > org.apache.lucene.document.LatLonShape.createIndexableFields(LatLonShape.java:73) > > ~[lucene-sandbox-7.7.3.jar:7.7.3 > 1a0d2a901dfec93676b0fe8be425101ceb754b85 - > > noble - 2020-04-21 10:31:55] > > at > > > org.elasticsearch.index.mapper.GeoShapeFieldMapper.indexShape(GeoShapeFieldMapper.java:146) > > ~[elasticsearch-6.8.9.jar:6.8.9] > > > > This is a very basic geometry. Could someone please explain why this > shape > > is invalid? > > > > > > > > > > Thanks in advance, > > > > Wouter Claeys > > >
Re: Tessellate exception in Elasticsearch
Thanks for the help! This isn't very clear in the Elasticsearch docs. Upon converting to WGS-84 everything seems to index fine. Van: Ignacio Vera Verzonden: donderdag 4 juni 2020 14:01 Aan: java-user@lucene.apache.org Onderwerp: Re: Tessellate exception in Elasticsearch I think this is not a lucene issue. Elasticsearch geo_shape only supports (and it assumes) polygons on the WGS-84 reference system. On Thu, Jun 4, 2020 at 1:38 PM Claeys Wouter wrote: > Hi, > > This is the original polygon: > > { >"crs":{ > "type":"name", > "properties":{ > "name":"urn:ogc:def:crs:EPSG::31370" > } >}, >"type":"MultiPolygon", >"coordinates":[ > [ > [ > [ >171044.231002, >175818.094268 > ], > [ >170996.799514, >175850.678652 > ], > [ >170957.441562, >175877.716668 > ], > [ >170946.243418, >175861.052668 > ], > [ >170935.531674, >175845.112572 > ], > [ >170923.57865, >175827.325308 > ], > [ >170906.675354, >175802.171388 > ], > [ >170886.642266, >175772.360124 > ], > [ >170886.478554, >175772.116476 > ], > [ >170951.311002, >175727.607548 > ], > [ >171026.378266, >175676.072188 > ], > [ >171098.875162, >175780.555004 > ], > [ >171090.729754, >175786.150716 > ], > [ >171044.231002, >175818.094268 > ] > ] > ] >] > } > > Thanks! > > > Van: Ignacio Vera Sequeiros > Verzonden: donderdag 4 juni 2020 12:24 > Aan: java-user@lucene.apache.org > Onderwerp: Re: Tessellate exception in Elasticsearch > > Hi, > > I think your polygon has intersecting edges but it is difficult to > reproduce with that output. Could you provide the original polygon you are > trying to index? > > Thanks! > > On Thu, Jun 4, 2020 at 11:30 AM Claeys Wouter > > wrote: > > > Hi, > > > > This is an error which we get in Elasticsearch when trying to index > > geo_shape fields but it seems this can be narrowed down to a problem in > > Lucene. We can reproduce the problem withe ES 6.8.x and ES 7.7.x. This is > > the error we are getting: > > > > Caused by: java.lang.IllegalArgumentException: Unable to Tessellate shape > > [[9.3213479767, -3.20048586995] [-17.71666808212, > > -42.558438073] [-1.052667893531, -53.756582018] > > [14.88742787267, -64.4683260198] [32.67469200505, > > -76.4213498962] [57.8286121207, -93.3246459366] > > [87.639876011, -113.3577339016] [87.883524044, > > -113.5214459883] [47.6075479995, -48.6889979781] > > [-3.927812089984, 26.37826614264] [79.4449960563, > > 98.8751621119] [73.8492839626, 90.7297539996] > > [41.9057321381, 44.2310018589] [9.3213479767, > > -3.20048586995] ]. Possible malformed shape detected. > > at > > org.apache.lucene.geo.Tessellator.tessellate(Tessellator.java:114) > > ~[lucene-sandbox-7.7.3.jar:7.7.3 > 1a0d2a901dfec93676b0fe8be425101ceb754b85 - > > noble - 2020-04-21 10:31:55] > > at > > > org.apache.lucene.document.LatLonShape.createIndexableFields(LatLonShape.java:73) > > ~[lucene-sandbox-7.7.3.jar:7.7.3 > 1a0d2a901dfec93676b0fe8be425101ceb754b85 - > > noble - 2020-04-21 10:31:55] > > at > > > org.elasticsearch.index.mapper.GeoShapeFieldMapper.indexShape(GeoShapeFieldMapper.java:146) > > ~[elasticsearch-6.8.9.jar:6.8.9] > > > > This is a very basic geometry. Could someone please explain why this > shape > > is invalid? > > > > > > > > > > Thanks in advance, > > > > Wouter Claeys > > >
Re: Tessellate exception in Elasticsearch
Hi, Yes. With different projections there is one issue: Elasticsearch only converts the polygon points to wgs84. But depending on the projection, the lines between the points may have a different shape in reality (no longer lines, but maybe curves), but as only the line endpoints are converted they get straight, which may cause a shape of the polygon, where in wgs84 projection some lines may overlap other lines. The projection converters are nice in Elasticsearch, but far from useable for real use cases. Because the earth is not rectangular and flat. 🤣 Uwe Am June 4, 2020 1:24:28 PM UTC schrieb Claeys Wouter : >Thanks for the help! This isn't very clear in the Elasticsearch docs. >Upon converting to WGS-84 everything seems to index fine. > > >Van: Ignacio Vera >Verzonden: donderdag 4 juni 2020 14:01 >Aan: java-user@lucene.apache.org >Onderwerp: Re: Tessellate exception in Elasticsearch > >I think this is not a lucene issue. Elasticsearch geo_shape only >supports >(and it assumes) polygons on the WGS-84 reference system. > >On Thu, Jun 4, 2020 at 1:38 PM Claeys Wouter > >wrote: > >> Hi, >> >> This is the original polygon: >> >> { >>"crs":{ >> "type":"name", >> "properties":{ >> "name":"urn:ogc:def:crs:EPSG::31370" >> } >>}, >>"type":"MultiPolygon", >>"coordinates":[ >> [ >> [ >> [ >>171044.231002, >>175818.094268 >> ], >> [ >>170996.799514, >>175850.678652 >> ], >> [ >>170957.441562, >>175877.716668 >> ], >> [ >>170946.243418, >>175861.052668 >> ], >> [ >>170935.531674, >>175845.112572 >> ], >> [ >>170923.57865, >>175827.325308 >> ], >> [ >>170906.675354, >>175802.171388 >> ], >> [ >>170886.642266, >>175772.360124 >> ], >> [ >>170886.478554, >>175772.116476 >> ], >> [ >>170951.311002, >>175727.607548 >> ], >> [ >>171026.378266, >>175676.072188 >> ], >> [ >>171098.875162, >>175780.555004 >> ], >> [ >>171090.729754, >>175786.150716 >> ], >> [ >>171044.231002, >>175818.094268 >> ] >> ] >> ] >>] >> } >> >> Thanks! >> >> >> Van: Ignacio Vera Sequeiros >> Verzonden: donderdag 4 juni 2020 12:24 >> Aan: java-user@lucene.apache.org >> Onderwerp: Re: Tessellate exception in Elasticsearch >> >> Hi, >> >> I think your polygon has intersecting edges but it is difficult to >> reproduce with that output. Could you provide the original polygon >you are >> trying to index? >> >> Thanks! >> >> On Thu, Jun 4, 2020 at 11:30 AM Claeys Wouter >> > >> wrote: >> >> > Hi, >> > >> > This is an error which we get in Elasticsearch when trying to index >> > geo_shape fields but it seems this can be narrowed down to a >problem in >> > Lucene. We can reproduce the problem withe ES 6.8.x and ES 7.7.x. >This is >> > the error we are getting: >> > >> > Caused by: java.lang.IllegalArgumentException: Unable to Tessellate >shape >> > [[9.3213479767, -3.20048586995] [-17.71666808212, >> > -42.558438073] [-1.052667893531, -53.756582018] >> > [14.88742787267, -64.4683260198] [32.67469200505, >> > -76.4213498962] [57.8286121207, -93.3246459366] >> > [87.639876011, -113.3577339016] [87.883524044, >> > -113.5214459883] [47.6075479995, -48.6889979781] >> > [-3.927812089984, 26.37826614264] [79.4449960563, >> > 98.8751621119] [73.8492839626, 90.7297539996] >> > [41.9057321381, 44.2310018589] [9.3213479767, >> > -3.20048586995] ]. Possible malformed shape detected. >> > at >> > org.apache.lucene.geo.Tessellator.tessellate(Tessellator.java:114) >> > ~[lucene-sandbox-7.7.3.jar:7.7.3 >> 1a0d2a901dfec93676b0fe8be425101ceb754b85 - >> > noble - 2020-04-21 10:31:55] >> > at >> > >> >org.apache.lucene.document.LatLonShape.createIndexableFields(LatLonShape.java:73) >> > ~[lucene-sandbox-7.7.3.jar:7.7.3 >> 1a0d2a901dfec93676b0fe8be425101ceb754b85 - >> > noble - 2020-04-21 10:31:55] >> > at >> > >> >org.elasticsearch.index.mapper.GeoShapeFieldMapper.indexShape(GeoShapeFieldMapper.java:146) >> > ~[elasticsearch-6.8.9.jar:6.8.9] >> > >> > This is a very basic geometry. Could someone please explain why >this >
Re: Tessellate exception in Elasticsearch
Ah sorry, I misunderstood your schema and input document. The CRS in your json is not used by ES, it just assumes WGS84. So the polygon is plain invalid, as if you calculate it modulo 360 degrees, the polygon is just wrong and has many overlapping lines. I was thinking you want to test the new CRS features in recent ES development. My fault. Uwe Am June 4, 2020 1:40:51 PM UTC schrieb Uwe Schindler : >Hi, > >Yes. With different projections there is one issue: Elasticsearch only >converts the polygon points to wgs84. But depending on the projection, >the lines between the points may have a different shape in reality (no >longer lines, but maybe curves), but as only the line endpoints are >converted they get straight, which may cause a shape of the polygon, >where in wgs84 projection some lines may overlap other lines. > >The projection converters are nice in Elasticsearch, but far from >useable for real use cases. Because the earth is not rectangular and >flat. 🤣 > >Uwe > >Am June 4, 2020 1:24:28 PM UTC schrieb Claeys Wouter >: >>Thanks for the help! This isn't very clear in the Elasticsearch docs. >>Upon converting to WGS-84 everything seems to index fine. >> >> >>Van: Ignacio Vera >>Verzonden: donderdag 4 juni 2020 14:01 >>Aan: java-user@lucene.apache.org >>Onderwerp: Re: Tessellate exception in Elasticsearch >> >>I think this is not a lucene issue. Elasticsearch geo_shape only >>supports >>(and it assumes) polygons on the WGS-84 reference system. >> >>On Thu, Jun 4, 2020 at 1:38 PM Claeys Wouter >> >>wrote: >> >>> Hi, >>> >>> This is the original polygon: >>> >>> { >>>"crs":{ >>> "type":"name", >>> "properties":{ >>> "name":"urn:ogc:def:crs:EPSG::31370" >>> } >>>}, >>>"type":"MultiPolygon", >>>"coordinates":[ >>> [ >>> [ >>> [ >>>171044.231002, >>>175818.094268 >>> ], >>> [ >>>170996.799514, >>>175850.678652 >>> ], >>> [ >>>170957.441562, >>>175877.716668 >>> ], >>> [ >>>170946.243418, >>>175861.052668 >>> ], >>> [ >>>170935.531674, >>>175845.112572 >>> ], >>> [ >>>170923.57865, >>>175827.325308 >>> ], >>> [ >>>170906.675354, >>>175802.171388 >>> ], >>> [ >>>170886.642266, >>>175772.360124 >>> ], >>> [ >>>170886.478554, >>>175772.116476 >>> ], >>> [ >>>170951.311002, >>>175727.607548 >>> ], >>> [ >>>171026.378266, >>>175676.072188 >>> ], >>> [ >>>171098.875162, >>>175780.555004 >>> ], >>> [ >>>171090.729754, >>>175786.150716 >>> ], >>> [ >>>171044.231002, >>>175818.094268 >>> ] >>> ] >>> ] >>>] >>> } >>> >>> Thanks! >>> >>> >>> Van: Ignacio Vera Sequeiros >>> Verzonden: donderdag 4 juni 2020 12:24 >>> Aan: java-user@lucene.apache.org >>> Onderwerp: Re: Tessellate exception in Elasticsearch >>> >>> Hi, >>> >>> I think your polygon has intersecting edges but it is difficult to >>> reproduce with that output. Could you provide the original polygon >>you are >>> trying to index? >>> >>> Thanks! >>> >>> On Thu, Jun 4, 2020 at 11:30 AM Claeys Wouter > >>> wrote: >>> >>> > Hi, >>> > >>> > This is an error which we get in Elasticsearch when trying to >index >>> > geo_shape fields but it seems this can be narrowed down to a >>problem in >>> > Lucene. We can reproduce the problem withe ES 6.8.x and ES 7.7.x. >>This is >>> > the error we are getting: >>> > >>> > Caused by: java.lang.IllegalArgumentException: Unable to >Tessellate >>shape >>> > [[9.3213479767, -3.20048586995] [-17.71666808212, >>> > -42.558438073] [-1.052667893531, -53.756582018] >>> > [14.88742787267, -64.4683260198] [32.67469200505, >>> > -76.4213498962] [57.8286121207, -93.3246459366] >>> > [87.639876011, -113.3577339016] [87.883524044, >>> > -113.5214459883] [47.6075479995, -48.6889979781] >>> > [-3.927812089984, 26.37826614264] [79.4449960563, >>> > 98.8751621119] [73.8492839626, 90.7297539996] >>> > [41.9057321381, 44.2310018589] [9.3213479767, >>> > -3.20048586995] ]. Possible malformed shape detected. >>> > at >>> > org.apache.lucene.geo.Tessellator.tessellate(Tessellator.java:114) >>> > ~[lucene-sand