subject:"\[jira\] \[Commented\] $SOLR\-2155$ Geospatial search using geohash prefixes"

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254357#comment-13254357
]

Bill Bell commented on SOLR-2155:
-

Could it be if they only have 1 entry in the store_geohash field?

Geospatial search using geohash prefixes

Key: SOLR-2155
URL: https://issues.apache.org/jira/browse/SOLR-2155
Project: Solr
Issue Type: Improvement
Reporter: David Smiley
Attachments: GeoHashPrefixFilter.patch, GeoHashPrefixFilter.patch,
GeoHashPrefixFilter.patch,
SOLR-2155_GeoHashPrefixFilter_with_sorting_no_poly.patch, SOLR.2155.p3.patch,
SOLR.2155.p3tests.patch, Solr2155-1.0.2-project.zip,
Solr2155-1.0.3-project.zip, Solr2155-1.0.4-project.zip,
Solr2155-for-1.0.2-3.x-port.patch

{panel:title=NOTICE} There is a .zip attached to the issue with a .jar you
can drop in to Solr 3.x. Lucene 4's new spatial module is largely based on
this code. The Solr 4 glue for it should come very soon but as of this
writing it's hosted temporarily at Spatial4j.com. For more information on
using SOLR-2155 with Solr 3, see
http://wiki.apache.org/solr/SpatialSearch#SOLR-2155 This JIRA issue is
closed because it won't be committed in its current form.
{panel}
There currently isn't a solution in Solr for doing geospatial filtering on
documents that have a variable number of points. This scenario occurs when
there is location extraction (i.e. via a gazateer) occurring on free text.
None, one, or many geospatial locations might be extracted from any given
document and users want to limit their search results to those occurring in a
user-specified area.
I've implemented this by furthering the GeoHash based work in Lucene/Solr
with a geohash prefix based filter. A geohash refers to a lat-lon box on the
earth. Each successive character added further subdivides the box into a 4x8
(or 8x4 depending on the even/odd length of the geohash) grid. The first
step in this scheme is figuring out which geohash grid squares cover the
user's search query. I've added various extra methods to GeoHashUtils (and
added tests) to assist in this purpose. The next step is an actual Lucene
Filter, GeoHashPrefixFilter, that uses these geohash prefixes in
TermsEnum.seek() to skip to relevant grid squares in the index. Once a
matching geohash grid is found, the points therein are compared against the
user's query to see if it matches. I created an abstraction GeoShape
extended by subclasses named PointDistance... and CartesianBox to support
different queried shapes so that the filter need not care about these details.
This work was presented at LuceneRevolution in Boston on October 8th.

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254356#comment-13254356
]

Bill Bell commented on SOLR-2155:
-

I tried SOLR 2155 1.0.3 and the same results.

I am able to simplify the issue with a simple query:

{code}
http://localhost:8983/solr/citystateprovider/select?fl=score,display_name,store_lat_lon,store_geohash,city_stateq.alt=*:*d=100pt=39.740112,-104.984856defType=dismaxrows=100echoParams=allsfield=store_geohashsort=geodisth%28%29%20asc;
{code}

AND the one that works:

{code}

http://localhost:8983/solr/citystateprovider/select?fl=score,display_name,store_lat_lon,store_geohash,city_stateq.alt=*:*pt=39.740112,-104.984856defType=dismaxrows=100echoParams=allsfield=store_lat_lonsort=geodist%28%29%20asc;

{code}

Geospatial search using geohash prefixes

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254359#comment-13254359
]

Bill Bell commented on SOLR-2155:
-

Ohh. This is interesting, if I switch to sort using the new geodisth() for the
sfield=store_lat_lon fields I get the same results as what I use
sfield=store_geohash (the wrong ones).

Geospatial search using geohash prefixes

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254392#comment-13254392
]

Bill Bell commented on SOLR-2155:
-

The distance is calculated properly, but the sort is not working. Probably due
to the wrong function being called.

You need to add a test case to check the sort order coming back from Solr.

Geospatial search using geohash prefixes

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254398#comment-13254398
]

Bill Bell commented on SOLR-2155:
-

sort is definitely not working in all cases using geohash field.

I am going to compare with other sorts... Maybe we need to extend another class?

Geospatial search using geohash prefixes

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

2012-04-14 Thread David Smiley (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254269#comment-13254269
]

David Smiley commented on SOLR-2155:

Weird. Is this reproducible using an index of just these 4 documents (all geo
single-valued, apparently) with your query here?

Geospatial search using geohash prefixes

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

2012-03-29 Thread Harley Parks (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242025#comment-13242025
]

Harley Parks commented on SOLR-2155:

Thanks David. so, i will be sure to add the custom cache.

running into some issues when using a range query on a multiValue GeoHash.

sometimes the values are outside of the range provided.

is that expected? and if so, why?
Thanks

Geospatial search using geohash prefixes

There currently isn't a solution in Solr for doing geospatial filtering on
documents that have a variable number of points. This scenario occurs when
there is location extraction (i.e. via a gazateer) occurring on free text.
None, one, or many geospatial locations might be extracted from any given
document and users want to limit their search results to those occurring in a
user-specified area.
I've implemented this by furthering the GeoHash based work in Lucene/Solr
with a geohash prefix based filter. A geohash refers to a lat-lon box on the
earth. Each successive character added further subdivides the box into a 4x8
(or 8x4 depending on the even/odd length of the geohash) grid. The first
step in this scheme is figuring out which geohash grid squares cover the
user's search query. I've added various extra methods to GeoHashUtils (and
added tests) to assist in this purpose. The next step is an actual Lucene
Filter, GeoHashPrefixFilter, that uses these geohash prefixes in
TermsEnum.seek() to skip to relevant grid squares in the index. Once a
matching geohash grid is found, the points therein are compared against the
user's query to see if it matches. I created an abstraction GeoShape
extended by subclasses named PointDistance... and CartesianBox to support
different queried shapes so that the filter need not care about these details.
This work was presented at LuceneRevolution in Boston on October 8th.

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

2012-03-29 Thread Harley Parks (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242031#comment-13242031
]

Harley Parks commented on SOLR-2155:

my assumption is that coordinates are lat, lng
and, if north is up on the page, the range is from the Lower-Left TO
Upper-Right.

examples of range query:

q=GeoTagGeoHash:[-15,149 TO -4,165]
q=GeoTagGeoHash:[19,-160 TO 23,-154]
q=GeoTagGeoHash:[10,20 TO 30,40]

Geospatial search using geohash prefixes

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

2012-03-29 Thread David Smiley (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242042#comment-13242042
]

David Smiley commented on SOLR-2155:

I don't recall ever declaring that the range query syntax works. I reviewed the
code and I didn't override getRangeQuery() which is required for it to work
correctly -- instead the parent class's implementation is in effect which
unsurprisingly doesn't work. Instead, use my query parser to supply the query
box.

Geospatial search using geohash prefixes

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

2012-03-27 Thread David Smiley (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13239218#comment-13239218
]

David Smiley commented on SOLR-2155:

Bill,
I don't think 1.0.4 introduced any problem as it was fairly trivial. The INFO
log message tells me you are using multi-value sorting which has to put all the
values into memory after each commit. Did a commit happen prior to this log
message? FYI you should put a warming query into newSearcher for any sorting
you do in Solr so that a user never sees the time hit for loading related
caches. Mikhail Khludnev suggested the fieldValueCache be explicitly
configured but I don't see it being relevant. AFAIK each sorted or faceted
multi-valued field gets one entry in that cache.

Geospatial search using geohash prefixes

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

2012-03-27 Thread Bill Bell (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13239230#comment-13239230
]

Bill Bell commented on SOLR-2155:
-

It seems that a max size of 10 would be too small. If we have an average of 3
geohash fields per doc, and we have 2M rows how do we set these caches?

We will get back to you on when it occurs. We don't commit during the day.
Maybe during a gc?

Geospatial search using geohash prefixes

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

2012-03-27 Thread Harley Parks (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13239710#comment-13239710
]

Harley Parks commented on SOLR-2155:

interesting... conversation regarding both tuning and data requirements.
I would guess that the size needs to be in power of 2's

since the example given is related to a custom type, it should be noted that a
fieldValueCache is by default created, for each document id.

the example given by default in solr 3.4 if needed:
fieldValueCache class=solr.FastLRUCache
size=512
autowarmCount=128
showItems=32 /

Additionally, the custom cache uses the same name, which might not matter, or
does it? the custom cache may override the fieldValueCache created by default.

Geospatial search using geohash prefixes

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

2012-03-27 Thread Harley Parks (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13239711#comment-13239711
]

Harley Parks commented on SOLR-2155:

interesting... conversation regarding both tuning and data requirements.
I would guess that the size needs to be in power of 2's

since the example given is related to a custom type, it should be noted that a
fieldValueCache is by default created, for each document id.

the example given by default in solr 3.4 if needed:
fieldValueCache class=solr.FastLRUCache
size=512
autowarmCount=128
showItems=32 /

Additionally, the custom cache uses the same name, which might not matter, or
does it? the custom cache may override the fieldValueCache created by default.

Geospatial search using geohash prefixes

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

2012-03-27 Thread William Bell

Harley,

I don't really think this is the same thing. Although I think it is
poor form to name it the same, these are 2 different caches. I am
still not sure what increasing the custom cache does. I looked at the
code and it is not clear. I did confirm that this seems t be happening
NOT on commit, but throughout the day. Maybe on Garbage collection?

On Tue, Mar 27, 2012 at 11:42 AM, Harley Parks (Commented) (JIRA)
j...@apache.org wrote:

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13239711#comment-13239711
]

Harley Parks commented on SOLR-2155:

interesting... conversation regarding both tuning and data requirements.
I would guess that the size needs to be in power of 2's

since the example given is related to a custom type, it should be noted that
a fieldValueCache is by default created, for each document id.

the example given by default in solr 3.4 if needed:
fieldValueCache class=solr.FastLRUCache
size=512
autowarmCount=128
showItems=32 /

Additionally, the custom cache uses the same name, which might not matter, or
does it? the custom cache may override the fieldValueCache created by default.

Geospatial search using geohash prefixes

Key: SOLR-2155
URL: https://issues.apache.org/jira/browse/SOLR-2155
Project: Solr
Issue Type: Improvement
Reporter: David Smiley
Attachments: GeoHashPrefixFilter.patch, GeoHashPrefixFilter.patch,
GeoHashPrefixFilter.patch,
SOLR-2155_GeoHashPrefixFilter_with_sorting_no_poly.patch,
SOLR.2155.p3.patch, SOLR.2155.p3tests.patch, Solr2155-1.0.2-project.zip,
Solr2155-1.0.3-project.zip, Solr2155-1.0.4-project.zip,
Solr2155-for-1.0.2-3.x-port.patch

There currently isn't a solution in Solr for doing geospatial filtering on
documents that have a variable number of points. This scenario occurs when
there is location extraction (i.e. via a gazateer) occurring on free text.
None, one, or many geospatial locations might be extracted from any given
document and users want to limit their search results to those occurring in
a user-specified area.
I've implemented this by furthering the GeoHash based work in Lucene/Solr
with a geohash prefix based filter. A geohash refers to a lat-lon box on
the earth. Each successive character added further subdivides the box into
a 4x8 (or 8x4 depending on the even/odd length of the geohash) grid. The
first step in this scheme is figuring out which geohash grid squares cover
the user's search query. I've added various extra methods to GeoHashUtils
(and added tests) to assist in this purpose. The next step is an actual
Lucene Filter, GeoHashPrefixFilter, that uses these geohash prefixes in
TermsEnum.seek() to skip to relevant grid squares in the index. Once a
matching geohash grid is found, the points therein are compared against the
user's query to see if it matches. I created an abstraction GeoShape
extended by subclasses named PointDistance... and CartesianBox to
support different queried shapes so that the filter need not care about
these details.
This work was presented at LuceneRevolution in Boston on October 8th.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA
administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

--
Bill Bell
billnb...@gmail.com
cell 720-256-8076

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

2012-03-27 Thread David Smiley (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240190#comment-13240190
]

David Smiley commented on SOLR-2155:

Ok so I got to the bottom of this. When I originally coded the cache entry
thing, my intention was to use the same cache as UnInvertedField (multi-value
faceting) -- THE fieldValueCache since it seemed related. But I did that
wrong and instead it looked up a custom user cache by that same name. The
difference is that the official one is configured with fieldValueCache and
the other is cache name=fieldValueCache (documented in the read me). In
hind site, I should have chosen some special name like solr2155. Now, if you
don't configure the cache then *each* time you try to sort, it has to rebuild
the cache. So basically it's required to add this cache configuration assuming
your sorting.

Geospatial search using geohash prefixes

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

2012-03-26 Thread Bill Bell (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13238921#comment-13238921
]

Bill Bell commented on SOLR-2155:
-

David,

We are seeing weird slow performance on your new 1.0.4 release.

INFO: [providersearch] webapp=/solr path=/select
params={d=160.9344facet=falsewt=jsonrows=6start=0pt=42.6450,-73.facet.field=5star_45f.5star_45.facet.mincount=1qt=providersearchspecdistfq=specialties_ids:(45+)qq=city_state_lower:albany,+nyf.5star_45.facet.limit=-1}
hits=960 status=0 QTime=8222

Hitting that with a slightly different lat long comes back almost instantly.
I'm not sure why sometimes they take seconds instead of milliseconds. There is
also this log entry a few lines before the long query:

Mar 22, 2012 11:26:29 AM solr2155.solr.search.function.GeoHashValueSource init
INFO: field 'store_geohash' in RAM: loaded min/avg/max per doc #:
(1,1.1089503,11) #2270017

Are we missing something? Shall we go back to 1.0.3 ?

Shall I increase the following? What does this actually do?

cache name=fieldValueCache
class=solr.FastLRUCache size=10 initialSize=1
autowarmCount=1/

Geospatial search using geohash prefixes

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

2012-03-21 Thread Harley Parks (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13235330#comment-13235330
]

Harley Parks commented on SOLR-2155:

Just doing some testing on the new jar file.
are there rules on how to structure bounding box. lower left is south west and
upper right is north east?

using the gh_geohash, was tricky as it's coordinates are flipped: long,lat to
get west, south, east, north box.
but works!
{!gh_geofilt%20sfield=GeoTagGeoHash%20box=129,-16,-180,1}

q=*:*fq={!geofilt sfield=GeoTagGeoHash pt=60,100 d=1}

ranged query works too:
q=GeoTagGeoHash:[-15,149 TO -4,165]
q=GeoTagGeoHash:[19,-160 TO 23,-154]

good stuff.

Geospatial search using geohash prefixes

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

2012-03-21 Thread Harley Parks (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13235332#comment-13235332
]

Harley Parks commented on SOLR-2155:

Just doing some testing on the new jar file.
are there rules on how to structure bounding box. lower left is south west and
upper right is north east?

using the gh_geohash, was tricky as it's coordinates are flipped: long,lat to
get west, south, east, north box.
but works!
{!gh_geofilt%20sfield=GeoTagGeoHash%20box=129,-16,-180,1}

q=*:*fq={!geofilt sfield=GeoTagGeoHash pt=60,100 d=1}

ranged query works too:
q=GeoTagGeoHash:[-15,149 TO -4,165]
q=GeoTagGeoHash:[19,-160 TO 23,-154]

good stuff.

Geospatial search using geohash prefixes

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

2012-03-19 Thread Harley Parks (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13233208#comment-13233208
]

Harley Parks commented on SOLR-2155:

Nice! Thanks David, that really helps us out.

Geospatial search using geohash prefixes

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

2012-03-19 Thread Bill Bell (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13233230#comment-13233230
]

Bill Bell commented on SOLR-2155:
-

I already had updated http://wiki.apache.org/solr/SpatialSearchDev with the
SOLR-2155 info.

Geospatial search using geohash prefixes

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

2012-03-17 Thread Bill Bell (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13232133#comment-13232133
]

Bill Bell commented on SOLR-2155:
-

I did figure out the {!gh_geofilt}

The parameters are point and radius, not pt and d.

Radius also is in meters not km.

The performance of this gh geofilt is also most the same as geofilt. So not
sure why you would need it.

Geospatial search using geohash prefixes

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

2012-03-17 Thread Harley Parks (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13232139#comment-13232139
]

Harley Parks commented on SOLR-2155:

Bill:

the main advantage is gh_geofilt (or the name in queryParser settup in
solrconfig.xml) can be used as part of the query versus the filter... at least
that is what I thought... here are my test strings in solr/admin/form

it might be helpful to be able to pass parameters in like
geodist(sfield,lat,lng).
gh_geofilt(sfield,lat,lng,d)

this worked for me in the solr/admin/form:

!gh_geofilt sfield=GeoTagGeoHash pt=-9,160 d=5

this also worked for me in the solr/admin/form:

!gh_geofilt sfield=GeoTagGeoHash point=-9,160 radius=5

but this also worked:
!geofilt sfield=GeoTagGeoHash point=-9,160 radius=5
!geofilt sfield=GeoTagGeoHash pt=-9,160 d=5

Geospatial search using geohash prefixes

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

2012-03-17 Thread Bill Bell (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13232156#comment-13232156
]

Bill Bell commented on SOLR-2155:
-

D is in km and radius is in meters. So Ou would need radius=5000

Geofilt does to use point and radius as far as I can tell.

Also hg_geofilt does not use pt and d.

I just don't want people confused.

Geospatial search using geohash prefixes

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

2012-03-17 Thread Bill Bell

I updated the Spatial wiki

Sent from my Mobile device
720-256-8076

On Mar 13, 2012, at 8:31 AM, David Smiley (Commented) (JIRA)
j...@apache.org wrote:

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13228408#comment-13228408
]

David Smiley commented on SOLR-2155:

Harley,
Apparently I haven't been clear because this question does come up often,
and I sympathize with you all because the comments on this issue are
ridiculously long. What I should have done and still can do is add info to
the Solr wiki. Ever since ~September 2011, you no longer need to patch Solr
and you can use any 3x release. The specific comment with further info
announcing this is:
https://issues.apache.org/jira/browse/SOLR-2155?focusedCommentId=13117350page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13117350
If you look at the attachments to the issue, you'll notice the latest version
is 1.0.3.

When Solr 3.6 comes out (soon!), my co-author and I will write an online
addendum to discuss the changes in Solr 3.5 Solr 3.6 that affect the
content of the book, or are interesting things that we would have written
about if we were still writing it. I'll add a clarification to the existing
info box on 144 mentioning SOLR-2155 that this feature is available in plugin
form to 3x without patching Solr.

Geospatial search using geohash prefixes

Key: SOLR-2155
URL: https://issues.apache.org/jira/browse/SOLR-2155
Project: Solr
Issue Type: Improvement
Reporter: David Smiley
Attachments: GeoHashPrefixFilter.patch, GeoHashPrefixFilter.patch,
GeoHashPrefixFilter.patch,
SOLR-2155_GeoHashPrefixFilter_with_sorting_no_poly.patch,
SOLR.2155.p3.patch, SOLR.2155.p3tests.patch, Solr2155-1.0.2-project.zip,
Solr2155-1.0.3-project.zip, Solr2155-for-1.0.2-3.x-port.patch

There currently isn't a solution in Solr for doing geospatial filtering on
documents that have a variable number of points. This scenario occurs when
there is location extraction (i.e. via a gazateer) occurring on free text.
None, one, or many geospatial locations might be extracted from any given
document and users want to limit their search results to those occurring in
a user-specified area.
I've implemented this by furthering the GeoHash based work in Lucene/Solr
with a geohash prefix based filter. A geohash refers to a lat-lon box on
the earth. Each successive character added further subdivides the box into
a 4x8 (or 8x4 depending on the even/odd length of the geohash) grid. The
first step in this scheme is figuring out which geohash grid squares cover
the user's search query. I've added various extra methods to GeoHashUtils
(and added tests) to assist in this purpose. The next step is an actual
Lucene Filter, GeoHashPrefixFilter, that uses these geohash prefixes in
TermsEnum.seek() to skip to relevant grid squares in the index. Once a
matching geohash grid is found, the points therein are compared against the
user's query to see if it matches. I created an abstraction GeoShape
extended by subclasses named PointDistance... and CartesianBox to
support different queried shapes so that the filter need not care about
these details.
This work was presented at LuceneRevolution in Boston on October 8th.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA
administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13229928#comment-13229928
]

Harley Parks commented on SOLR-2155:

Sorry, about the editing.

But Thanks For the Feedback.

so perhaps there is something wrong with my configuration, since the search
results do not return lat, long but the geohash string.

The maven build went great.

I did finally figure out how to use the geofilt function using the pt and d.

I am reindexing each time, but yes, I delete the data folder, and reindex.

I am placing the jar file into the tomcats solr/lib folder, after a restart,
and after changing solrconfig and schema, geohash string is displayed, not the
lat,long.
Schema
Field Type:
fieldType name=geohash class=solr2155.solr.schema.GeoHashField
length=12/
this is the field:
field name=GeoTagGeoHash type=geohash indexed=true stored=true
multiValued=true /

this is the info from solr/admin/ field types - GEOHASH
Field Type: geohash
Fields: GEOTAGGEOHASH
Tokenized: true
Class Name: solr2155.solr.schema.GeoHashField
Index Analyzer: org.apache.solr.analysis.TokenizerChain
Tokenizer Class: solr2155.solr.schema.GeoHashField$1
Query Analyzer: org.apache.solr.schema.FieldType$DefaultAnalyzer

Still, the query returns values like:

arr name=GeoTagGeoHash
strrw3sh9g8c6mx/str
strrw3f3xc9dnh3/str
strrw3ckbue74y7/str
/arr

so, if this is not the right, is there anything I can do to troubleshoot?

Geospatial search using geohash prefixes

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

2012-03-15 Thread Bill Bell (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13229945#comment-13229945
]

Bill Bell commented on SOLR-2155:
-

David,

What is an example URL call for multiValued field? Does geofilt work?

/select?q=*:*fq={!geofilt}sort=geodist() ascsfield=store_hashd=10

Or do we need to use gh_geofilt? like this?

/select?q=*:*fq={!gh_geofilt}sort=geodist() ascsfield=store_hashd=10

Geospatial search using geohash prefixes

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230289#comment-13230289
]

Harley Parks commented on SOLR-2155:

Bill:
the doc's on queryParser state the name of the function can then be used as the
main query, gh_geofilt, perhaps something like: /select?q={!gh_geofilt}... but,
good question.
geofilt is working for me on multivalued fields.

my issue is the query result returns the geohash string, not the geohash lat,
long.
In building the v 1.0.3 jar file for solr2155, I used jdk 6. as I didn't see
any errors, so hopefully, that's fine.
so, I'm going to see if solr 3.5 will perhaps resolve my issue.

Geospatial search using geohash prefixes

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230346#comment-13230346
]

Harley Parks commented on SOLR-2155:

Oh.. I may have messed up my build, since i did not include the solr 3.4 jar
files in the class path...
is there an enviorment variable that maven will use? such as CLASSPATH or a lib
folder in the project being built?

Geospatial search using geohash prefixes

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230398#comment-13230398
]

Harley Parks commented on SOLR-2155:

All of the Class Paths in the solr1.0.3 project point to apache solr 3.4
libraries on the apache website... so no action needed, to answer my own
question. I'm stumped.

Geospatial search using geohash prefixes

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230781#comment-13230781
]

Harley Parks commented on SOLR-2155:

I used a fresh install of solr 3.4, Just in case our implementation had
something to do with it, still not able to return the lat,long and only get the
geohash.

what's interesting is I can change the field's type between solr-2155's geohash
solr's geohash, the query returned format changes from geohash to lat,long
without reindexing.

Where in the code does the geohash value get returned in a query?

Help.

Geospatial search using geohash prefixes

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230843#comment-13230843
]

Harley Parks commented on SOLR-2155:

Sorry, for the endless droning.
But, for now, the plan is to store the latlong and the geohash in two fields.
search on the geohash, and map on the latlong.

Geospatial search using geohash prefixes

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

2012-03-14 Thread Harley Parks (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13229423#comment-13229423
]

Harley Parks commented on SOLR-2155:

did i need to rebuild the index after making changes to the schema?
What does a valid url query for geohash look like?

Geospatial search using geohash prefixes

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

2012-03-14 Thread Harley Parks (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13229777#comment-13229777
]

Harley Parks commented on SOLR-2155:

Wait... there is GeoHashUtils.decode(geohashString)... I wonder if I can
create a field in solr that returns lat, lng pair?

Geospatial search using geohash prefixes

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

2012-03-14 Thread David Smiley (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13229869#comment-13229869
]

David Smiley commented on SOLR-2155:

Harley,
You shouldn't need to know a thing about geohashes to use SOLR-2155. You use
it identically to LatLonType insofar as you add the data in lat,lon format and
get it out the same way in search results, and you can use the built-in geofilt
query parser (my gh_geofilt is has other options and is optional). Perhaps you
are inadvertently using Solr's geohash field? And/or maybe you forgot to
reindex?

p.s. please use the JIRA comment edit feature sparingly; it sends interested
parties notifications each time.

Geospatial search using geohash prefixes

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

2012-03-13 Thread Erick Erickson

Be brave, just go for building it G...
It's actually surprisingly easy.

Start here:

http://wiki.apache.org/solr/HowToContribute

you should be able to get this all running (assuming
you have svn, ant and a jdk) in about 15 minutes,
exclusive of the checkout time. I was amazed first
time I tried it.

One note: Solr 3.x all go against the 1.5 JDK. It should
work against the 1.6, but note that the release was compiled
with 1.5.

Applying the patches is patching source code and recompiling

Best
Erick

On Tue, Mar 13, 2012 at 12:17 AM, Harley Parks (Commented) (JIRA)
j...@apache.org wrote:

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13228218#comment-13228218
]

Harley Parks commented on SOLR-2155:

So, basic question.. perhaps needs to be posted else where.
I'm working with Solr 3.4, using the GeoHash to store multiple locations for
a document.
if geofilt or geodist doesn't work with the GeoHash, is the only way to add
this patch into solr 3.4?
I'm using a tomcat and solr, and jumping to 4.0 might be a while, even if
it's released soon.

I'm not real clear how to apply the patch, as I would need basically create
the solr.war file from the source... and compile all of the other sources...
painful, but once setup, perhaps rewarding.

Ideally, I would have a jar file from the patch, that I drop into the
solr/lib and make the needed changes to the config files.

So, I'm real interested in getting something stable, so i'm watching the
above mentioned links.

David Smiley, would you be able to amend your book - Apache Solr 3 ESS, which
mentions solr-2155, to include how to implement this patch? or do I need to
get brave, and build the source?

Geospatial search using geohash prefixes

Key: SOLR-2155
URL: https://issues.apache.org/jira/browse/SOLR-2155
Project: Solr
Issue Type: Improvement
Reporter: David Smiley
Attachments: GeoHashPrefixFilter.patch, GeoHashPrefixFilter.patch,
GeoHashPrefixFilter.patch,
SOLR-2155_GeoHashPrefixFilter_with_sorting_no_poly.patch,
SOLR.2155.p3.patch, SOLR.2155.p3tests.patch, Solr2155-1.0.2-project.zip,
Solr2155-1.0.3-project.zip, Solr2155-for-1.0.2-3.x-port.patch

There currently isn't a solution in Solr for doing geospatial filtering on
documents that have a variable number of points. This scenario occurs when
there is location extraction (i.e. via a gazateer) occurring on free text.
None, one, or many geospatial locations might be extracted from any given
document and users want to limit their search results to those occurring in
a user-specified area.
I've implemented this by furthering the GeoHash based work in Lucene/Solr
with a geohash prefix based filter. A geohash refers to a lat-lon box on
the earth. Each successive character added further subdivides the box into
a 4x8 (or 8x4 depending on the even/odd length of the geohash) grid. The
first step in this scheme is figuring out which geohash grid squares cover
the user's search query. I've added various extra methods to GeoHashUtils
(and added tests) to assist in this purpose. The next step is an actual
Lucene Filter, GeoHashPrefixFilter, that uses these geohash prefixes in
TermsEnum.seek() to skip to relevant grid squares in the index. Once a
matching geohash grid is found, the points therein are compared against the
user's query to see if it matches. I created an abstraction GeoShape
extended by subclasses named PointDistance... and CartesianBox to
support different queried shapes so that the filter need not care about
these details.
This work was presented at LuceneRevolution in Boston on October 8th.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA
administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

2012-03-13 Thread David Smiley (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13228408#comment-13228408
]

David Smiley commented on SOLR-2155:

Harley,
Apparently I haven't been clear because this question does come up often, and
I sympathize with you all because the comments on this issue are ridiculously
long. What I should have done and still can do is add info to the Solr wiki.
Ever since ~September 2011, you no longer need to patch Solr and you can use
any 3x release. The specific comment with further info announcing this is:
https://issues.apache.org/jira/browse/SOLR-2155?focusedCommentId=13117350page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13117350
If you look at the attachments to the issue, you'll notice the latest version
is 1.0.3.

When Solr 3.6 comes out (soon!), my co-author and I will write an online
addendum to discuss the changes in Solr 3.5 Solr 3.6 that affect the content
of the book, or are interesting things that we would have written about if we
were still writing it. I'll add a clarification to the existing info box on
144 mentioning SOLR-2155 that this feature is available in plugin form to 3x
without patching Solr.

Geospatial search using geohash prefixes

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

2012-03-13 Thread Harley Parks (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13228939#comment-13228939
]

Harley Parks commented on SOLR-2155:

I am missing the package solr2155.lucene.spatial.geometry.shape;

Geospatial search using geohash prefixes

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

2012-03-13 Thread Harley Parks (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13228938#comment-13228938
]

Harley Parks commented on SOLR-2155:

I am missing the package solr2155.lucene.spatial.geometry.shape;

Geospatial search using geohash prefixes

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

2012-03-12 Thread Harley Parks (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13228218#comment-13228218
]

Harley Parks commented on SOLR-2155:

So, basic question.. perhaps needs to be posted else where.
I'm working with Solr 3.4, using the GeoHash to store multiple locations for a
document.
if geofilt or geodist doesn't work with the GeoHash, is the only way to add
this patch into solr 3.4?
I'm using a tomcat and solr, and jumping to 4.0 might be a while, even if it's
released soon.

I'm not real clear how to apply the patch, as I would need basically create the
solr.war file from the source... and compile all of the other sources...
painful, but once setup, perhaps rewarding.

Ideally, I would have a jar file from the patch, that I drop into the solr/lib
and make the needed changes to the config files.

So, I'm real interested in getting something stable, so i'm watching the above
mentioned links.

David Smiley, would you be able to amend your book - Apache Solr 3 ESS, which
mentions solr-2155, to include how to implement this patch? or do I need to get
brave, and build the source?

Geospatial search using geohash prefixes

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

2012-02-16 Thread David Smiley (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13209613#comment-13209613
]

David Smiley commented on SOLR-2155:

If someone watching this issue has an interest in this capability winding its
way into Solr out of the box, then I suggest you vote (and maybe watch)
LUCENE-3795. That issue is the first step, the subsequent step is a follow-on
issue that will bring LSP's spatial-solr module which uses spatial-lucene
(LUCENE-3795). I don't intend or support committing SOLR-2155 as is. Spatial
done-right should involve a good framework; SOLR-2155 isn't a framework and
Lucene's existing defunct spatial-contrib module isn't good. That's where LSP
comes in, and LUCENE-3795 is the first step to get it incorporated into
Lucene/Solr.

Geospatial search using geohash prefixes

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

2012-01-14 Thread Bill Bell (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13186295#comment-13186295
]

Bill Bell commented on SOLR-2155:
-

David,

We really need to get this into the trunk. What is left to do? Would really
love to use this, but until it is moved into the trunk most clients won't.

Maybe we can scale it back a bit, add the feature and iterate?

Geospatial search using geohash prefixes

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

2012-01-12 Thread Srikanth Kallurkar (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13185172#comment-13185172
]

Srikanth Kallurkar commented on SOLR-2155:
--

Hi David, Apologies for taking a long time to reply. I incorporated your
suggestions and did see some speedup. I started with geohash length at 12 and
got reasonable speed up for 8. T
But I am bit confused by (1) accuracy associated with length and (2) the query
time operation in this patch.
(1) So, is the accuracy associated with how big/small an enclosing box is?
(2) The entire geohash field is loaded into memory at query time. Is this done
because for the lat-lon comparisons, the patch cannot use lucene string
matching mechanisms. Also, because the whole field is in memory, how are
updates to the index handled. Meaning, if new lat-lons are added, do they get
added to the in-memory geohashes.

Thanks,
Srikanth

Geospatial search using geohash prefixes

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

2012-01-12 Thread David Smiley (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13185224#comment-13185224
]

David Smiley commented on SOLR-2155:

Srikanth,
# According to the table on wikipedia, a geohash length of 8 translates to +-
19 meters error. This means that if you do a filter query, it may match points
that are =19 meters outside of the query shape erroneously, or it may exclude
points = 19 meters within the query shape erroneously. The accuracy is
associated to the indexing of the data, not the query shape.
# At query time, if you only do geospatial filters, then there is *no* RAM
requirement for this field. *If* you sort by distance or influence relevancy
by distance, then the indexed points are all brought into memory inverted so
that the center of the query shape can be compared against all indexed points
for documents matching the query. LatLonType does the same thing for the same
reason, but this code is multiValue aware, returning the closest indexed point
a document has to the query shape. When a commit happens on Solr, this
in-memory cache is unused and garbage collected, and the data is brought into
memory again.

Geospatial search using geohash prefixes

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

2012-01-12 Thread Srikanth Kallurkar (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13185248#comment-13185248
]

Srikanth Kallurkar commented on SOLR-2155:
--

Aha! Thanks for the explaination.

Geospatial search using geohash prefixes

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

2011-12-08 Thread David Smiley (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165396#comment-13165396
]

David Smiley commented on SOLR-2155:

Srikanth: If I were you, I would increase ramBufferSizeMB in solrconfig.xml to
a good amount -- perhaps to as much as 256MB. Remember to do this in
mainIndex, NOT indexDefaults. Secondly and most importantly, I would
configure the geohash length attribute on the field type to have sufficient
search detail for your needs, but no more. Remember, making the geohash shorter
and thus courser granularity doesn't mean you lose the accuracy in your
stored value -- which is retained verbatim as you provided. For a guide on
what geohash length to use, consult the wikipedia page which has a table.
Please let me know how this works out for you.

Geospatial search using geohash prefixes

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

2011-12-08 Thread Mikhail Khludnev

Srikanth,

Have you tried to do a basic profiling or sampling? Just take a few thread
dumps by jstack. If the code is so greedy for CPU, you'll have it in a
stack.

Regards

On Thu, Dec 8, 2011 at 8:57 PM, Srikanth Kallurkar (Commented) (JIRA)
j...@apache.org wrote:

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165326#comment-13165326]

Srikanth Kallurkar commented on SOLR-2155:
--

In my use case, I have a large number of lat-lons for each document - on
the order of about 2K lat-lon pairs. Since the time we started using
geohash prefix filter, the time to index has significantly degraded - by
about 2-3 times. Are there any suggestions for speeding up the indexing
process. I was trying to read the comments here, but am not sure if any
index time caching mechanism is used (or could be used) to lookup geohashes.

Thanks,
Srikanth

Geospatial search using geohash prefixes

Key: SOLR-2155
URL: https://issues.apache.org/jira/browse/SOLR-2155
Project: Solr
Issue Type: Improvement
Reporter: David Smiley
Attachments: GeoHashPrefixFilter.patch,
GeoHashPrefixFilter.patch, GeoHashPrefixFilter.patch,
SOLR-2155_GeoHashPrefixFilter_with_sorting_no_poly.patch,
SOLR.2155.p3.patch, SOLR.2155.p3tests.patch, Solr2155-1.0.2-project.zip,
Solr2155-1.0.3-project.zip, Solr2155-for-1.0.2-3.x-port.patch

There currently isn't a solution in Solr for doing geospatial filtering
on documents that have a variable number of points. This scenario occurs
when there is location extraction (i.e. via a gazateer) occurring on free
text. None, one, or many geospatial locations might be extracted from any
given document and users want to limit their search results to those
occurring in a user-specified area.
I've implemented this by furthering the GeoHash based work in
Lucene/Solr with a geohash prefix based filter. A geohash refers to a
lat-lon box on the earth. Each successive character added further
subdivides the box into a 4x8 (or 8x4 depending on the even/odd length of
the geohash) grid. The first step in this scheme is figuring out which
geohash grid squares cover the user's search query. I've added various
extra methods to GeoHashUtils (and added tests) to assist in this purpose.
The next step is an actual Lucene Filter, GeoHashPrefixFilter, that uses
these geohash prefixes in TermsEnum.seek() to skip to relevant grid squares
in the index. Once a matching geohash grid is found, the points therein
are compared against the user's query to see if it matches. I created an
abstraction GeoShape extended by subclasses named PointDistance... and
CartesianBox to support different queried shapes so that the filter
need not care about these details.
This work was presented at LuceneRevolution in Boston on October 8th.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA
administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

--
Sincerely yours
Mikhail Khludnev
Developer
Grid Dynamics
tel. 1-415-738-8644
Skype: mkhludnev
http://www.griddynamics.com
mkhlud...@griddynamics.com

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

2011-10-17 Thread Olivier Jacquet (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13128980#comment-13128980
]

Olivier Jacquet commented on SOLR-2155:
---

I just wanted to mention another use case for multivalued point fields since
everyone is always talking about this in a location context.

The PointType can also be used to categorize other other stuff. In my case
we're storing qualifications of persons as a tuple of experience, function and
skill (eg. senior, developer, java) which are internally represented by
numerical ids. Now with Solr I would like to be able to do the query: return
everything that is a java developer which would be the same as asking for all
points on a certain line.

Geospatial search using geohash prefixes

Key: SOLR-2155
URL: https://issues.apache.org/jira/browse/SOLR-2155
Project: Solr
Issue Type: Improvement
Reporter: David Smiley
Assignee: Grant Ingersoll
Attachments: GeoHashPrefixFilter.patch, GeoHashPrefixFilter.patch,
GeoHashPrefixFilter.patch,
SOLR-2155_GeoHashPrefixFilter_with_sorting_no_poly.patch, SOLR.2155.p3.patch,
SOLR.2155.p3tests.patch, Solr2155-1.0.2-project.zip,
Solr2155-1.0.3-project.zip, Solr2155-for-1.0.2-3.x-port.patch

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

2011-10-17 Thread David Smiley (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13129002#comment-13129002
]

David Smiley commented on SOLR-2155:

Oliver: Your scenario is interesting but I wouldn't recommend spatial for that.
A key part of spatial is the use of a numerical range. In your case there are
discrete values. Instead, I recommend you experiment with phrase queries, and
if you are expert in Lucene then span queries. As a toy hack example, imagine
indexing each of these values in the form senior developer java (3 words, one
for each part). We assume each value tokenizes as one token. Then search for
the developer java in which the was substituted as a kind of wildcard for
the first position to find java developers in all levels of experience. The
is a stopword and in effect creates a wildcard placeholder. If you search the
solr-user list then you will see information on this topic. I've solved this
problem in a different more difficult way because my values were not single
tokens, but based on the example you present, the solution I present here isn't
bad. If you want to discuss this further I recommend the solr-user list.

Geospatial search using geohash prefixes

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

2011-10-07 Thread David Smiley (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13122801#comment-13122801
]

David Smiley commented on SOLR-2155:

The patch file is against Solr trunk (v4). It is a little out of date but it
should be easy to port it. Perhaps only some import statements would change due
to some moves; I'm not sure. The Solr 4 solution I've been working on with
Chris Ryan is a larger spatial framework called LSP (a temporary name):
http://code.google.com/p/lucene-spatial-playground/ Feel free to email me
about LSP directly for assistance with it. Development on LSP has slowed but
will pick up a lot within a few weeks.

Geospatial search using geohash prefixes

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

2011-10-06 Thread David Smiley (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13121996#comment-13121996
]

David Smiley commented on SOLR-2155:

Frederick, a rough inspection of your problem suggests that the GeoHashField is
declared multiValue=true but the field in your POJO is not correspondingly a
ListString like it should be. If you only need a single value then I suggest
you use LatLonType instead, since it's what comes with Solr.

Geospatial search using geohash prefixes

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

2011-10-06 Thread arin ghazarian (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13122469#comment-13122469
]

arin ghazarian commented on SOLR-2155:
--

Hi David,
i am interested in using this geohash prefix filtering/bbox feature in solr 4.x
with solrcloud, do you have any plans to convert this plugin to a solr 4
compatible one?
Thanks,
arin

Geospatial search using geohash prefixes

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13114585#comment-13114585
]

geert-jan brits commented on SOLR-2155:
---

I have the impression that this code is meant for drawing shapes and to see if
geospatial enriched documents are within this shape. Is that correct?

Perhaps my use-case is also supported, because it's in the 'multi-geopoint
domain' as well.

I envision documents having multiple lat/long points. I would like to query
(sort / filter on) documents by their 'closest point' to a given user-defined
lat/long point. Documents would either contain a bag of lat/long pairs or a
polygon made up out of these lat/long pairs and the query would become: return
the closest distance from a user-defined point to the polygon.

Before delving in the above code or in the LSP-code myself, perhaps someone can
say if this type of querying is supported?

Geospatial search using geohash prefixes

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

2011-09-26 Thread David Smiley (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13114671#comment-13114671
]

David Smiley commented on SOLR-2155:

geert-jan, your impression of the capabilities of the code is correct including
your use-case for the most part. Your use-case describes that the document
might have an indexed polygon -- that is supported in LSP but not this patch.
Sorting by (multi-value) indexed shapes is supported only for points, not yet
other shapes like polygons. But you could basically get this by indexing an
additional multi-point field containing the polygon vertices and center-point,
and then sorting on that. If you have a particular sorting use-case with
nuances that my solution does not cover then please let me know.

Geospatial search using geohash prefixes

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13114686#comment-13114686
]

geert-jan brits commented on SOLR-2155:
---

David, to clarify:
My use-case could be either represented as:
1. a bag of points, in which case I want to be able to return the closest
point to a user-defined point and sort on the distance
2. a polygon made of the points (where the points are the vertices of the
polygon) and return the closest distance from a user-defined point to the
polygon.

Either of the solutions suffices for me, from your answer I can't entirely see
if that was clear.

You mention: Sorting by (multi-value) indexed shapes is supported only for
points.
Does this mean that representation 1.) above is supported? It wasn't entirely
clear for me from your response.

Let me give you the use-case, (and why the sort on center-point / centroid is
not going to work):

Consider a travel application in which walks/itineraries can be defined. Most
of the walks are defined as roundtrips (i.e: beginpoint = endpoint). In my
representation (for now) a walk visits certain Points of interest (poi) (which
each have a lat/long point defined) in a certain order.

A lot of walks can be started at any given Poi. (bc. of the roundtrip nature).
I want a user to be able to request walks that are nearby. (sorted based on
distance). For each walk the distance becomes the closest Poi (thus point)
defined in the walk related to the user-defined point.

Does this make sense?

Geospatial search using geohash prefixes

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

2011-09-26 Thread David Smiley (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13114703#comment-13114703
]

David Smiley commented on SOLR-2155:

bq. You mention: Sorting by (multi-value) indexed shapes is supported only for
points. Does this mean that representation 1.) above is supported? It wasn't
entirely clear for me from your response.

Yes, it does. This patch LSP support sorting documents by a multi-value
point field.

Based on the description of your use-case and my interpretation of what you say
(which I am not 100% sure of), I don't think it's pertinent that the walks
are polygons. What is pertinent is that they are a collection of points (Pois).
So if your walk corresponds to a Solr document, you could have a poi field
that is an indexed multi-value geospatial point field. Then you could sort
walks (documents) according to the pois that are closest to a user-specified
point in the query. I would add a large bounding-box filter which would
improve performance. It's not clear to me there is any need for polygon
support.

FYI I use the well-known JTS library for polygon support.

Geospatial search using geohash prefixes

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13114726#comment-13114726
]

geert-jan brits commented on SOLR-2155:
---

Great thanks, I believe you interpretation of my use-case is correct .
I will go the Multi-point route first, without the polygons.

Just to clarify: I realize I added to the confusion by bringing polygons to the
table where they aren't necessary for the problem I described.
I did this because I thought that perhaps distance of point to polygon' was
implemented in LSP, while 'point to collection of points' was not.

In that case 'transforming the problem space' by representing a 'collection of
points' as a polygon and querying for distance of point to polygon instead
would have given me what I wanted. This is all superfluous now, because doing
'point to collection of points' IS possible.

I will check out the code, thanks again!

Geospatial search using geohash prefixes

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes