RE: [WKT] Spatial Searching
Thanks for the links Chris. I think my approach in SOLR-2155 complies with those rules. The only part that concerns me wether this is true is the rule regarding the default action of a build script: YOU MUST NOT distribute build scripts or documentation within an Apache product with the purpose of causing the default/standard build of an Apache product to include any part of a prohibited work. It's not saying specifically we can't compile against LGPL, it's ambiguously saying include. I take that to mean the result of the build -- e.g. class and jar files, which may not include LGPL. RE SIS... I wonder how the expertise on that project compares to that of JTS's Martin Davis -- an expert, and the library has been in use for 10 years. ~ David From: Mattmann, Chris A (388J) [chris.a.mattm...@jpl.nasa.gov] Sent: Tuesday, March 29, 2011 1:00 AM To: solr-user@lucene.apache.org Cc: Adam Estrada Subject: Re: [WKT] Spatial Searching LGPL licenses and Apache aren't exactly compatible, see: http://www.apache.org/legal/3party.html#transition-examples-lgpl http://www.apache.org/legal/resolved.html#category-x In practice, this was the reason we started the SIS project. Cheers, Chris On Mar 28, 2011, at 11:16 AM, Smiley, David W. wrote: (This is one of those messages that I would have responded to at the time if I only noticed it.) There is not yet indexing of arbitrary shapes (i.e. your data can only be points), but with SOLR-2155 you can query via WKT thanks to JTS. If you want to index shapes then you'll have to wait a month or two for work that is underway right now. It's coming; be patient. I don't see the LGPL licensing as a problem; it's *L*GPL, not GPL, after all. In SOLR-2155 the patch I take measures to download this library dynamically at build time and compile against it. JTS need not ship with Solr; the user can get it themselves if they want this capability. Non-JTS query shapes should work without the presence of JTS. ~ David Smiley Author: http://www.packtpub.com/solr-1-4-enterprise-search-server/ On Feb 8, 2011, at 11:18 PM, Adam Estrada wrote: I just came across a ~nudge post over in the SIS list on what the status is for that project. This got me looking more in to spatial mods with Solr4.0. I found this enhancement in Jira. https://issues.apache.org/jira/browse/SOLR-2155. In this issue, David mentions that he's already integrated JTS in to Solr4.0 for querying on polygons stored as WKT. It's relatively easy to get WKT strings in to Solr but does the Field type exist yet? Is there a patch or something that I can test out? Here's how I would do it using GDAL/OGR and the already existing csv update handler. http://www.gdal.org/ogr/drv_csv.html ogr2ogr -f CSV output.csv input.shp -lco GEOMETRY=AS_WKT This converts a shapefile to a csv with the geometries in tact in the form of WKT. You can then get the data in to Solr by running the following command. curl http://localhost:8983/solr/update/csv?commit=trueseparator=%2Cfieldnames=id,attr1,attr2,attr3,geomstream.file=C:\tmp\output.csvoverwrite=truestream.contentType=text/plain;charset=utf-8; There are lots of flavors of geometries so I suspect that this will be a daunting task but because JTS recognizes each geometry type it should be possible to work with them. Does anyone know of a patch or even when this functionality might be included in to Solr4.0? I need to query for polygons ;-) Thanks, Adam ++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++
Re: [WKT] Spatial Searching
Does anyone know of a patch or even when this functionality might be included in to Solr4.0? I need to query for polygons ;-) check: http://code.google.com/p/lucene-spatial-playground/ This is my sketch / soon-to-be-proposal for what I think lucene spatial should look like. It includes a WKTField that can do complex geometry queries: https://lucene-spatial-playground.googlecode.com/svn/trunk/spatial-lucene/src/main/java/org/apache/lucene/spatial/search/jts/ ryan
Re: [WKT] Spatial Searching
On Mar 29, 2011, at 8:12 AM, Mattmann, Chris A (388J) wrote: RE SIS... I wonder how the expertise on that project compares to that of JTS's Martin Davis -- an expert, and the library has been in use for 10 years. Time will tell. I'd favor the Apache model where instead of name dropping we rely on the collective expertise p a group of like minded individuals Lucene was originally the work of one expert. That's not a bad way to start. wunder -- Walter Underwood
Re: [WKT] Spatial Searching
(This is one of those messages that I would have responded to at the time if I only noticed it.) There is not yet indexing of arbitrary shapes (i.e. your data can only be points), but with SOLR-2155 you can query via WKT thanks to JTS. If you want to index shapes then you'll have to wait a month or two for work that is underway right now. It's coming; be patient. I don't see the LGPL licensing as a problem; it's *L*GPL, not GPL, after all. In SOLR-2155 the patch I take measures to download this library dynamically at build time and compile against it. JTS need not ship with Solr; the user can get it themselves if they want this capability. Non-JTS query shapes should work without the presence of JTS. ~ David Smiley Author: http://www.packtpub.com/solr-1-4-enterprise-search-server/ On Feb 8, 2011, at 11:18 PM, Adam Estrada wrote: I just came across a ~nudge post over in the SIS list on what the status is for that project. This got me looking more in to spatial mods with Solr4.0. I found this enhancement in Jira. https://issues.apache.org/jira/browse/SOLR-2155. In this issue, David mentions that he's already integrated JTS in to Solr4.0 for querying on polygons stored as WKT. It's relatively easy to get WKT strings in to Solr but does the Field type exist yet? Is there a patch or something that I can test out? Here's how I would do it using GDAL/OGR and the already existing csv update handler. http://www.gdal.org/ogr/drv_csv.html ogr2ogr -f CSV output.csv input.shp -lco GEOMETRY=AS_WKT This converts a shapefile to a csv with the geometries in tact in the form of WKT. You can then get the data in to Solr by running the following command. curl http://localhost:8983/solr/update/csv?commit=trueseparator=%2Cfieldnames=id,attr1,attr2,attr3,geomstream.file=C:\tmp\output.csvoverwrite=truestream.contentType=text/plain;charset=utf-8; There are lots of flavors of geometries so I suspect that this will be a daunting task but because JTS recognizes each geometry type it should be possible to work with them. Does anyone know of a patch or even when this functionality might be included in to Solr4.0? I need to query for polygons ;-) Thanks, Adam
Re: [WKT] Spatial Searching
Outstanding! Thanks David...I can't wait to take a look at it. Adam Sent from my iPhone On Mar 28, 2011, at 2:16 PM, Smiley, David W. dsmi...@mitre.org wrote: (This is one of those messages that I would have responded to at the time if I only noticed it.) There is not yet indexing of arbitrary shapes (i.e. your data can only be points), but with SOLR-2155 you can query via WKT thanks to JTS. If you want to index shapes then you'll have to wait a month or two for work that is underway right now. It's coming; be patient. I don't see the LGPL licensing as a problem; it's *L*GPL, not GPL, after all. In SOLR-2155 the patch I take measures to download this library dynamically at build time and compile against it. JTS need not ship with Solr; the user can get it themselves if they want this capability. Non-JTS query shapes should work without the presence of JTS. ~ David Smiley Author: http://www.packtpub.com/solr-1-4-enterprise-search-server/ On Feb 8, 2011, at 11:18 PM, Adam Estrada wrote: I just came across a ~nudge post over in the SIS list on what the status is for that project. This got me looking more in to spatial mods with Solr4.0. I found this enhancement in Jira. https://issues.apache.org/jira/browse/SOLR-2155. In this issue, David mentions that he's already integrated JTS in to Solr4.0 for querying on polygons stored as WKT. It's relatively easy to get WKT strings in to Solr but does the Field type exist yet? Is there a patch or something that I can test out? Here's how I would do it using GDAL/OGR and the already existing csv update handler. http://www.gdal.org/ogr/drv_csv.html ogr2ogr -f CSV output.csv input.shp -lco GEOMETRY=AS_WKT This converts a shapefile to a csv with the geometries in tact in the form of WKT. You can then get the data in to Solr by running the following command. curl http://localhost:8983/solr/update/csv?commit=trueseparator=%2Cfieldnames=id,attr1,attr2,attr3,geomstream.file=C:\tmp\output.csvoverwrite=truestream.contentType=text/plain;charset=utf-8; There are lots of flavors of geometries so I suspect that this will be a daunting task but because JTS recognizes each geometry type it should be possible to work with them. Does anyone know of a patch or even when this functionality might be included in to Solr4.0? I need to query for polygons ;-) Thanks, Adam
Re: [WKT] Spatial Searching
LGPL licenses and Apache aren't exactly compatible, see: http://www.apache.org/legal/3party.html#transition-examples-lgpl http://www.apache.org/legal/resolved.html#category-x In practice, this was the reason we started the SIS project. Cheers, Chris On Mar 28, 2011, at 11:16 AM, Smiley, David W. wrote: (This is one of those messages that I would have responded to at the time if I only noticed it.) There is not yet indexing of arbitrary shapes (i.e. your data can only be points), but with SOLR-2155 you can query via WKT thanks to JTS. If you want to index shapes then you'll have to wait a month or two for work that is underway right now. It's coming; be patient. I don't see the LGPL licensing as a problem; it's *L*GPL, not GPL, after all. In SOLR-2155 the patch I take measures to download this library dynamically at build time and compile against it. JTS need not ship with Solr; the user can get it themselves if they want this capability. Non-JTS query shapes should work without the presence of JTS. ~ David Smiley Author: http://www.packtpub.com/solr-1-4-enterprise-search-server/ On Feb 8, 2011, at 11:18 PM, Adam Estrada wrote: I just came across a ~nudge post over in the SIS list on what the status is for that project. This got me looking more in to spatial mods with Solr4.0. I found this enhancement in Jira. https://issues.apache.org/jira/browse/SOLR-2155. In this issue, David mentions that he's already integrated JTS in to Solr4.0 for querying on polygons stored as WKT. It's relatively easy to get WKT strings in to Solr but does the Field type exist yet? Is there a patch or something that I can test out? Here's how I would do it using GDAL/OGR and the already existing csv update handler. http://www.gdal.org/ogr/drv_csv.html ogr2ogr -f CSV output.csv input.shp -lco GEOMETRY=AS_WKT This converts a shapefile to a csv with the geometries in tact in the form of WKT. You can then get the data in to Solr by running the following command. curl http://localhost:8983/solr/update/csv?commit=trueseparator=%2Cfieldnames=id,attr1,attr2,attr3,geomstream.file=C:\tmp\output.csvoverwrite=truestream.contentType=text/plain;charset=utf-8; There are lots of flavors of geometries so I suspect that this will be a daunting task but because JTS recognizes each geometry type it should be possible to work with them. Does anyone know of a patch or even when this functionality might be included in to Solr4.0? I need to query for polygons ;-) Thanks, Adam ++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++
Re: [WKT] Spatial Searching
The show stopper for JTS is it's license, unfortunately. Otherwise, I think it would be done already! We could, since it's LGPL, make it an optional dependency, assuming someone can stub it out. On Feb 8, 2011, at 11:18 PM, Adam Estrada wrote: I just came across a ~nudge post over in the SIS list on what the status is for that project. This got me looking more in to spatial mods with Solr4.0. I found this enhancement in Jira. https://issues.apache.org/jira/browse/SOLR-2155. In this issue, David mentions that he's already integrated JTS in to Solr4.0 for querying on polygons stored as WKT. It's relatively easy to get WKT strings in to Solr but does the Field type exist yet? Is there a patch or something that I can test out? Here's how I would do it using GDAL/OGR and the already existing csv update handler. http://www.gdal.org/ogr/drv_csv.html ogr2ogr -f CSV output.csv input.shp -lco GEOMETRY=AS_WKT This converts a shapefile to a csv with the geometries in tact in the form of WKT. You can then get the data in to Solr by running the following command. curl http://localhost:8983/solr/update/csv?commit=trueseparator=%2Cfieldnames=id,attr1,attr2,attr3,geomstream.file=C:\tmp\output.csvoverwrite=truestream.contentType=text/plain;charset=utf-8; There are lots of flavors of geometries so I suspect that this will be a daunting task but because JTS recognizes each geometry type it should be possible to work with them. Does anyone know of a patch or even when this functionality might be included in to Solr4.0? I need to query for polygons ;-) Thanks, Adam -- Grant Ingersoll http://www.lucidimagination.com/
Re: [WKT] Spatial Searching
How could i stub this out not being a java guy? What is needed in order to do this? Licensing is always going to be an issue with JTS which is why I am interested in the project SIS sitting in incubation right now. I willing to put forth the effort if I had a little direction from the peanut gallery ;-) Adam On Feb 9, 2011, at 7:03 AM, Grant Ingersoll gsing...@apache.org wrote: The show stopper for JTS is it's license, unfortunately. Otherwise, I think it would be done already! We could, since it's LGPL, make it an optional dependency, assuming someone can stub it out. On Feb 8, 2011, at 11:18 PM, Adam Estrada wrote: I just came across a ~nudge post over in the SIS list on what the status is for that project. This got me looking more in to spatial mods with Solr4.0. I found this enhancement in Jira. https://issues.apache.org/jira/browse/SOLR-2155. In this issue, David mentions that he's already integrated JTS in to Solr4.0 for querying on polygons stored as WKT. It's relatively easy to get WKT strings in to Solr but does the Field type exist yet? Is there a patch or something that I can test out? Here's how I would do it using GDAL/OGR and the already existing csv update handler. http://www.gdal.org/ogr/drv_csv.html ogr2ogr -f CSV output.csv input.shp -lco GEOMETRY=AS_WKT This converts a shapefile to a csv with the geometries in tact in the form of WKT. You can then get the data in to Solr by running the following command. curl http://localhost:8983/solr/update/csv?commit=trueseparator=%2Cfieldnames=id,attr1,attr2,attr3,geomstream.file=C:\tmp\output.csvoverwrite=truestream.contentType=text/plain;charset=utf-8; There are lots of flavors of geometries so I suspect that this will be a daunting task but because JTS recognizes each geometry type it should be possible to work with them. Does anyone know of a patch or even when this functionality might be included in to Solr4.0? I need to query for polygons ;-) Thanks, Adam -- Grant Ingersoll http://www.lucidimagination.com/
Re: [WKT] Spatial Searching
Thought I would share this on web mapping...it's a great write up and something to consider when talking about working with spatial data. http://www.tokumine.com/2010/09/20/gis-data-payload-sizes/ Adam On Feb 9, 2011, at 7:03 AM, Grant Ingersoll gsing...@apache.org wrote: The show stopper for JTS is it's license, unfortunately. Otherwise, I think it would be done already! We could, since it's LGPL, make it an optional dependency, assuming someone can stub it out. On Feb 8, 2011, at 11:18 PM, Adam Estrada wrote: I just came across a ~nudge post over in the SIS list on what the status is for that project. This got me looking more in to spatial mods with Solr4.0. I found this enhancement in Jira. https://issues.apache.org/jira/browse/SOLR-2155. In this issue, David mentions that he's already integrated JTS in to Solr4.0 for querying on polygons stored as WKT. It's relatively easy to get WKT strings in to Solr but does the Field type exist yet? Is there a patch or something that I can test out? Here's how I would do it using GDAL/OGR and the already existing csv update handler. http://www.gdal.org/ogr/drv_csv.html ogr2ogr -f CSV output.csv input.shp -lco GEOMETRY=AS_WKT This converts a shapefile to a csv with the geometries in tact in the form of WKT. You can then get the data in to Solr by running the following command. curl http://localhost:8983/solr/update/csv?commit=trueseparator=%2Cfieldnames=id,attr1,attr2,attr3,geomstream.file=C:\tmp\output.csvoverwrite=truestream.contentType=text/plain;charset=utf-8; There are lots of flavors of geometries so I suspect that this will be a daunting task but because JTS recognizes each geometry type it should be possible to work with them. Does anyone know of a patch or even when this functionality might be included in to Solr4.0? I need to query for polygons ;-) Thanks, Adam -- Grant Ingersoll http://www.lucidimagination.com/
Re: [WKT] Spatial Searching
Grant, How could i stub this out not being a java guy? What is needed in order to do this? Licensing is always going to be an issue with JTS which is why I am interested in the project SIS sitting in incubation right now. I'm willing to put forth the effort if I had a little direction on how to implement it from the peanut gallery ;-) Adam On Feb 9, 2011, at 7:03 AM, Grant Ingersoll wrote: The show stopper for JTS is it's license, unfortunately. Otherwise, I think it would be done already! We could, since it's LGPL, make it an optional dependency, assuming someone can stub it out. On Feb 8, 2011, at 11:18 PM, Adam Estrada wrote: I just came across a ~nudge post over in the SIS list on what the status is for that project. This got me looking more in to spatial mods with Solr4.0. I found this enhancement in Jira. https://issues.apache.org/jira/browse/SOLR-2155. In this issue, David mentions that he's already integrated JTS in to Solr4.0 for querying on polygons stored as WKT. It's relatively easy to get WKT strings in to Solr but does the Field type exist yet? Is there a patch or something that I can test out? Here's how I would do it using GDAL/OGR and the already existing csv update handler. http://www.gdal.org/ogr/drv_csv.html ogr2ogr -f CSV output.csv input.shp -lco GEOMETRY=AS_WKT This converts a shapefile to a csv with the geometries in tact in the form of WKT. You can then get the data in to Solr by running the following command. curl http://localhost:8983/solr/update/csv?commit=trueseparator=%2Cfieldnames=id,attr1,attr2,attr3,geomstream.file=C:\tmp\output.csvoverwrite=truestream.contentType=text/plain;charset=utf-8; There are lots of flavors of geometries so I suspect that this will be a daunting task but because JTS recognizes each geometry type it should be possible to work with them. Does anyone know of a patch or even when this functionality might be included in to Solr4.0? I need to query for polygons ;-) Thanks, Adam -- Grant Ingersoll http://www.lucidimagination.com/
Re: [WKT] Spatial Searching
Thought I would share this on web mapping...it's a great write up and something to consider when talking about working with spatial data. http://www.tokumine.com/2010/09/20/gis-data-payload-sizes/ Adam On Feb 9, 2011, at 7:03 AM, Grant Ingersoll wrote: The show stopper for JTS is it's license, unfortunately. Otherwise, I think it would be done already! We could, since it's LGPL, make it an optional dependency, assuming someone can stub it out. On Feb 8, 2011, at 11:18 PM, Adam Estrada wrote: I just came across a ~nudge post over in the SIS list on what the status is for that project. This got me looking more in to spatial mods with Solr4.0. I found this enhancement in Jira. https://issues.apache.org/jira/browse/SOLR-2155. In this issue, David mentions that he's already integrated JTS in to Solr4.0 for querying on polygons stored as WKT. It's relatively easy to get WKT strings in to Solr but does the Field type exist yet? Is there a patch or something that I can test out? Here's how I would do it using GDAL/OGR and the already existing csv update handler. http://www.gdal.org/ogr/drv_csv.html ogr2ogr -f CSV output.csv input.shp -lco GEOMETRY=AS_WKT This converts a shapefile to a csv with the geometries in tact in the form of WKT. You can then get the data in to Solr by running the following command. curl http://localhost:8983/solr/update/csv?commit=trueseparator=%2Cfieldnames=id,attr1,attr2,attr3,geomstream.file=C:\tmp\output.csvoverwrite=truestream.contentType=text/plain;charset=utf-8; There are lots of flavors of geometries so I suspect that this will be a daunting task but because JTS recognizes each geometry type it should be possible to work with them. Does anyone know of a patch or even when this functionality might be included in to Solr4.0? I need to query for polygons ;-) Thanks, Adam -- Grant Ingersoll http://www.lucidimagination.com/
[WKT] Spatial Searching
I just came across a ~nudge post over in the SIS list on what the status is for that project. This got me looking more in to spatial mods with Solr4.0. I found this enhancement in Jira. https://issues.apache.org/jira/browse/SOLR-2155. In this issue, David mentions that he's already integrated JTS in to Solr4.0 for querying on polygons stored as WKT. It's relatively easy to get WKT strings in to Solr but does the Field type exist yet? Is there a patch or something that I can test out? Here's how I would do it using GDAL/OGR and the already existing csv update handler. http://www.gdal.org/ogr/drv_csv.html ogr2ogr -f CSV output.csv input.shp -lco GEOMETRY=AS_WKT This converts a shapefile to a csv with the geometries in tact in the form of WKT. You can then get the data in to Solr by running the following command. curl http://localhost:8983/solr/update/csv?commit=trueseparator=%2Cfieldnames=id,attr1,attr2,attr3,geomstream.file=C:\tmp\output.csvoverwrite=truestream.contentType=text/plain;charset=utf-8; There are lots of flavors of geometries so I suspect that this will be a daunting task but because JTS recognizes each geometry type it should be possible to work with them. Does anyone know of a patch or even when this functionality might be included in to Solr4.0? I need to query for polygons ;-) Thanks, Adam
Re: [WKT] Spatial Searching
+1 to David's patch from SOLR-2155. It would be great to implement. Great job using GDAL on converting the WKT Adam! Cheers, Chris On Feb 8, 2011, at 8:18 PM, Adam Estrada wrote: I just came across a ~nudge post over in the SIS list on what the status is for that project. This got me looking more in to spatial mods with Solr4.0. I found this enhancement in Jira. https://issues.apache.org/jira/browse/SOLR-2155. In this issue, David mentions that he's already integrated JTS in to Solr4.0 for querying on polygons stored as WKT. It's relatively easy to get WKT strings in to Solr but does the Field type exist yet? Is there a patch or something that I can test out? Here's how I would do it using GDAL/OGR and the already existing csv update handler. http://www.gdal.org/ogr/drv_csv.html ogr2ogr -f CSV output.csv input.shp -lco GEOMETRY=AS_WKT This converts a shapefile to a csv with the geometries in tact in the form of WKT. You can then get the data in to Solr by running the following command. curl http://localhost:8983/solr/update/csv?commit=trueseparator=%2Cfieldnames=id,attr1,attr2,attr3,geomstream.file=C:\tmp\output.csvoverwrite=truestream.contentType=text/plain;charset=utf-8; There are lots of flavors of geometries so I suspect that this will be a daunting task but because JTS recognizes each geometry type it should be possible to work with them. Does anyone know of a patch or even when this functionality might be included in to Solr4.0? I need to query for polygons ;-) Thanks, Adam ++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++
