Re: Spatial Search on Postal Code

[Manohar Sripada]

Thanks Again Emir! I will try this way.

Thanks David! It looks like building of polygons at index time is better
option than at query time.

Thanks,
Manohar

On Sat, Mar 5, 2016 at 7:54 PM, david.w.smi...@gmail.com <
david.w.smi...@gmail.com> wrote:

> Another path to consider is doing this point-in-zipcode-poly lookup at
> index time and enriching the document with a zipcode field (possibly
> multi-valued if there is doubt).
>
> On Sat, Mar 5, 2016 at 4:05 AM steve shepard <sc_shep...@hotmail.com>
> wrote:
>
> > re: Postal Codes and polygons. I've heard of basic techniques that use
> > Commerce Department (or was it Census within Commerce??) that give the
> > basic points, but the real run is deciding what the "center" of that
> > polygon is. There is likely a commercial solution available, and
> certainly
> > you can buy a spreadsheet with the zipcodes and their guestimated center.
> > Fun project!
> >
> > > Subject: Re: Spatial Search on Postal Code
> > > To: solr-user@lucene.apache.org
> > > From: emir.arnauto...@sematext.com
> > > Date: Fri, 4 Mar 2016 21:18:10 +0100
> > >
> > > Hi Manohar,
> > > I don't think there is such functionality in Solr - you need to do it
> on
> > > client side:
> > > 1. find some postal code polygons (you can use open street map -
> > > http://wiki.openstreetmap.org/wiki/Key:postal_code)
> > > 2. create zip to polygon lookup
> > > 3. create code that will expand zip code polygon by some distance (you
> > > can use JTS buffer api)
> > >
> > > On query time you get zip code and distance:
> > > 1. find polygon for zip
> > > 2. expand polygon
> > > 3. send resulting polygon to Solr and use Intersects function to filter
> > > results
> > >
> > > Regards,
> > > Emir
> > >
> > > On 04.03.2016 19:49, Manohar Sripada wrote:
> > > > Thanks Emir,
> > > >
> > > > Obviously #2 approach is much better. I know its not straight
> forward.
> > But,
> > > > is it really acheivable in Solr? Like building a polygon for a postal
> > code.
> > > > If so, can you throw some light how to do?
> > > >
> > > > Thanks,
> > > > Manohar
> > > >
> > > > On Friday, March 4, 2016, Emir Arnautovic <
> > emir.arnauto...@sematext.com>
> > > > wrote:
> > > >
> > > >> Hi Manohar,
> > > >> This depends on your requirements/usecase. If postal code is
> > interpreted
> > > >> as point than it is expected to have radius that is significantly
> > larger
> > > >> than postal code diameter. In such case you can go with first
> > approach. In
> > > >> order to avoid missing results from postal code in case of small
> > search
> > > >> radius and large postal code, you can reverse geocode records and
> > store
> > > >> postal code with each document.
> > > >> If you need to handle distance from postal code precisely - distance
> > from
> > > >> its border, you have to get postal code polygon, expand it by search
> > > >> distance and use resulting polygon to find matches.
> > > >>
> > > >> HTH,
> > > >> Emir
> > > >>
> > > >> On 04.03.2016 13:09, Manohar Sripada wrote:
> > > >>
> > > >>> Here's my requirement -  User enters postal code and provides the
> > radius.
> > > >>> I
> > > >>> need to find the records with in the radius from the provided
> postal
> > code.
> > > >>>
> > > >>> There are few ways I thought through after going through the
> "Spatial
> > > >>> Search" Solr wiki
> > > >>>
> > > >>> 1. As Latitude and Longitude positions are required for spatial
> > search.
> > > >>> Get
> > > >>> Latitude Longitude position (may be using GeoCoding API) of a
> postal
> > code
> > > >>> and use "LatLonType" field type and query accordingly. As the
> > GeoCoding
> > > >>> API
> > > >>> returns one point and if the postal code area is too big, then I
> may
> > end
> > > >>> up
> > > >>> not getting any results (apart from the records from the same
> postal
> > code)
> > > >>> if the radius provided is small.
> > > >>>
> > > >>> 2. Get the latitude longitude points of the postal code which
> forms a
> > > >>> border (not sure yet on how to get) and build a polygon (using
> RPT).
> > While
> > > >>> querying use this polygon and provide the distance. Can this be
> > achieved?
> > > >>> Or Am I ruminating too much? :(
> > > >>>
> > > >>> Appreciate any help on this.
> > > >>>
> > > >>> Thanks
> > > >>>
> > > >>>
> > > >> --
> > > >> Monitoring * Alerting * Anomaly Detection * Centralized Log
> Management
> > > >> Solr & Elasticsearch Support * http://sematext.com/
> > > >>
> > > >>
> > >
> > > --
> > > Monitoring * Alerting * Anomaly Detection * Centralized Log Management
> > > Solr & Elasticsearch Support * http://sematext.com/
> > >
> >
>
> --
> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
> LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
> http://www.solrenterprisesearchserver.com
>

Re: Spatial Search on Postal Code

[david.w.smi...@gmail.com]

Another path to consider is doing this point-in-zipcode-poly lookup at
index time and enriching the document with a zipcode field (possibly
multi-valued if there is doubt).

On Sat, Mar 5, 2016 at 4:05 AM steve shepard <sc_shep...@hotmail.com> wrote:

> re: Postal Codes and polygons. I've heard of basic techniques that use
> Commerce Department (or was it Census within Commerce??) that give the
> basic points, but the real run is deciding what the "center" of that
> polygon is. There is likely a commercial solution available, and certainly
> you can buy a spreadsheet with the zipcodes and their guestimated center.
> Fun project!
>
> > Subject: Re: Spatial Search on Postal Code
> > To: solr-user@lucene.apache.org
> > From: emir.arnauto...@sematext.com
> > Date: Fri, 4 Mar 2016 21:18:10 +0100
> >
> > Hi Manohar,
> > I don't think there is such functionality in Solr - you need to do it on
> > client side:
> > 1. find some postal code polygons (you can use open street map -
> > http://wiki.openstreetmap.org/wiki/Key:postal_code)
> > 2. create zip to polygon lookup
> > 3. create code that will expand zip code polygon by some distance (you
> > can use JTS buffer api)
> >
> > On query time you get zip code and distance:
> > 1. find polygon for zip
> > 2. expand polygon
> > 3. send resulting polygon to Solr and use Intersects function to filter
> > results
> >
> > Regards,
> > Emir
> >
> > On 04.03.2016 19:49, Manohar Sripada wrote:
> > > Thanks Emir,
> > >
> > > Obviously #2 approach is much better. I know its not straight forward.
> But,
> > > is it really acheivable in Solr? Like building a polygon for a postal
> code.
> > > If so, can you throw some light how to do?
> > >
> > > Thanks,
> > > Manohar
> > >
> > > On Friday, March 4, 2016, Emir Arnautovic <
> emir.arnauto...@sematext.com>
> > > wrote:
> > >
> > >> Hi Manohar,
> > >> This depends on your requirements/usecase. If postal code is
> interpreted
> > >> as point than it is expected to have radius that is significantly
> larger
> > >> than postal code diameter. In such case you can go with first
> approach. In
> > >> order to avoid missing results from postal code in case of small
> search
> > >> radius and large postal code, you can reverse geocode records and
> store
> > >> postal code with each document.
> > >> If you need to handle distance from postal code precisely - distance
> from
> > >> its border, you have to get postal code polygon, expand it by search
> > >> distance and use resulting polygon to find matches.
> > >>
> > >> HTH,
> > >> Emir
> > >>
> > >> On 04.03.2016 13:09, Manohar Sripada wrote:
> > >>
> > >>> Here's my requirement -  User enters postal code and provides the
> radius.
> > >>> I
> > >>> need to find the records with in the radius from the provided postal
> code.
> > >>>
> > >>> There are few ways I thought through after going through the "Spatial
> > >>> Search" Solr wiki
> > >>>
> > >>> 1. As Latitude and Longitude positions are required for spatial
> search.
> > >>> Get
> > >>> Latitude Longitude position (may be using GeoCoding API) of a postal
> code
> > >>> and use "LatLonType" field type and query accordingly. As the
> GeoCoding
> > >>> API
> > >>> returns one point and if the postal code area is too big, then I may
> end
> > >>> up
> > >>> not getting any results (apart from the records from the same postal
> code)
> > >>> if the radius provided is small.
> > >>>
> > >>> 2. Get the latitude longitude points of the postal code which forms a
> > >>> border (not sure yet on how to get) and build a polygon (using RPT).
> While
> > >>> querying use this polygon and provide the distance. Can this be
> achieved?
> > >>> Or Am I ruminating too much? :(
> > >>>
> > >>> Appreciate any help on this.
> > >>>
> > >>> Thanks
> > >>>
> > >>>
> > >> --
> > >> Monitoring * Alerting * Anomaly Detection * Centralized Log Management
> > >> Solr & Elasticsearch Support * http://sematext.com/
> > >>
> > >>
> >
> > --
> > Monitoring * Alerting * Anomaly Detection * Centralized Log Management
> > Solr & Elasticsearch Support * http://sematext.com/
> >
>

-- 
Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
http://www.solrenterprisesearchserver.com

RE: Spatial Search on Postal Code

[steve shepard]

re: Postal Codes and polygons. I've heard of basic techniques that use Commerce 
Department (or was it Census within Commerce??) that give the basic points, but 
the real run is deciding what the "center" of that polygon is. There is likely 
a commercial solution available, and certainly you can buy a spreadsheet with 
the zipcodes and their guestimated center. Fun project!

> Subject: Re: Spatial Search on Postal Code
> To: solr-user@lucene.apache.org
> From: emir.arnauto...@sematext.com
> Date: Fri, 4 Mar 2016 21:18:10 +0100
> 
> Hi Manohar,
> I don't think there is such functionality in Solr - you need to do it on 
> client side:
> 1. find some postal code polygons (you can use open street map - 
> http://wiki.openstreetmap.org/wiki/Key:postal_code)
> 2. create zip to polygon lookup
> 3. create code that will expand zip code polygon by some distance (you 
> can use JTS buffer api)
> 
> On query time you get zip code and distance:
> 1. find polygon for zip
> 2. expand polygon
> 3. send resulting polygon to Solr and use Intersects function to filter 
> results
> 
> Regards,
> Emir
> 
> On 04.03.2016 19:49, Manohar Sripada wrote:
> > Thanks Emir,
> >
> > Obviously #2 approach is much better. I know its not straight forward. But,
> > is it really acheivable in Solr? Like building a polygon for a postal code.
> > If so, can you throw some light how to do?
> >
> > Thanks,
> > Manohar
> >
> > On Friday, March 4, 2016, Emir Arnautovic <emir.arnauto...@sematext.com>
> > wrote:
> >
> >> Hi Manohar,
> >> This depends on your requirements/usecase. If postal code is interpreted
> >> as point than it is expected to have radius that is significantly larger
> >> than postal code diameter. In such case you can go with first approach. In
> >> order to avoid missing results from postal code in case of small search
> >> radius and large postal code, you can reverse geocode records and store
> >> postal code with each document.
> >> If you need to handle distance from postal code precisely - distance from
> >> its border, you have to get postal code polygon, expand it by search
> >> distance and use resulting polygon to find matches.
> >>
> >> HTH,
> >> Emir
> >>
> >> On 04.03.2016 13:09, Manohar Sripada wrote:
> >>
> >>> Here's my requirement -  User enters postal code and provides the radius.
> >>> I
> >>> need to find the records with in the radius from the provided postal code.
> >>>
> >>> There are few ways I thought through after going through the "Spatial
> >>> Search" Solr wiki
> >>>
> >>> 1. As Latitude and Longitude positions are required for spatial search.
> >>> Get
> >>> Latitude Longitude position (may be using GeoCoding API) of a postal code
> >>> and use "LatLonType" field type and query accordingly. As the GeoCoding
> >>> API
> >>> returns one point and if the postal code area is too big, then I may end
> >>> up
> >>> not getting any results (apart from the records from the same postal code)
> >>> if the radius provided is small.
> >>>
> >>> 2. Get the latitude longitude points of the postal code which forms a
> >>> border (not sure yet on how to get) and build a polygon (using RPT). While
> >>> querying use this polygon and provide the distance. Can this be achieved?
> >>> Or Am I ruminating too much? :(
> >>>
> >>> Appreciate any help on this.
> >>>
> >>> Thanks
> >>>
> >>>
> >> --
> >> Monitoring * Alerting * Anomaly Detection * Centralized Log Management
> >> Solr & Elasticsearch Support * http://sematext.com/
> >>
> >>
> 
> -- 
> Monitoring * Alerting * Anomaly Detection * Centralized Log Management
> Solr & Elasticsearch Support * http://sematext.com/
> 
  

Re: Spatial Search on Postal Code

[Emir Arnautovic]

Hi Manohar,
I don't think there is such functionality in Solr - you need to do it on 
client side:
1. find some postal code polygons (you can use open street map - 
http://wiki.openstreetmap.org/wiki/Key:postal_code)

2. create zip to polygon lookup
3. create code that will expand zip code polygon by some distance (you 
can use JTS buffer api)


On query time you get zip code and distance:
1. find polygon for zip
2. expand polygon
3. send resulting polygon to Solr and use Intersects function to filter 
results


Regards,
Emir

On 04.03.2016 19:49, Manohar Sripada wrote:

Thanks Emir,

Obviously #2 approach is much better. I know its not straight forward. But,
is it really acheivable in Solr? Like building a polygon for a postal code.
If so, can you throw some light how to do?

Thanks,
Manohar

On Friday, March 4, 2016, Emir Arnautovic 
wrote:


Hi Manohar,
This depends on your requirements/usecase. If postal code is interpreted
as point than it is expected to have radius that is significantly larger
than postal code diameter. In such case you can go with first approach. In
order to avoid missing results from postal code in case of small search
radius and large postal code, you can reverse geocode records and store
postal code with each document.
If you need to handle distance from postal code precisely - distance from
its border, you have to get postal code polygon, expand it by search
distance and use resulting polygon to find matches.

HTH,
Emir

On 04.03.2016 13:09, Manohar Sripada wrote:


Here's my requirement -  User enters postal code and provides the radius.
I
need to find the records with in the radius from the provided postal code.

There are few ways I thought through after going through the "Spatial
Search" Solr wiki

1. As Latitude and Longitude positions are required for spatial search.
Get
Latitude Longitude position (may be using GeoCoding API) of a postal code
and use "LatLonType" field type and query accordingly. As the GeoCoding
API
returns one point and if the postal code area is too big, then I may end
up
not getting any results (apart from the records from the same postal code)
if the radius provided is small.

2. Get the latitude longitude points of the postal code which forms a
border (not sure yet on how to get) and build a polygon (using RPT). While
querying use this polygon and provide the distance. Can this be achieved?
Or Am I ruminating too much? :(

Appreciate any help on this.

Thanks



--
Monitoring * Alerting * Anomaly Detection * Centralized Log Management
Solr & Elasticsearch Support * http://sematext.com/




--
Monitoring * Alerting * Anomaly Detection * Centralized Log Management
Solr & Elasticsearch Support * http://sematext.com/

Re: Spatial Search on Postal Code

[Manohar Sripada]

Thank Erik,
Does this zip codes index that you created is a one-to-many
mapping from zip code to Lat/Lon points? If so, where did you get this
mapping CSV file?

Thanks,
Manohar

On Friday, March 4, 2016, Erik Hatcher  wrote:

> This is just like an implementation I recently worked on with a customer.
> It’s very much like this sort of thing ;) -
>
> <
> http://nyulangone.org/doctors?query===10019==distance=
> <
> http://nyulangone.org/doctors?query===10019==distance=
> >>
>
> It’s implemented with Solr, leveraging the Lucidworks Fusion query
> pipelines to do these steps:
>
>- =N comes in to the query pipeline
>- a sub-query is made to a separate zipcodes index (built from a CSV
> file of zipcodes and their corresponding representative lat/lon)
>- the matching lat/lon is used to build the appropriate geo-filtering
> and sorting parameters to pass on to the main collection
>
> Straightforward, clean, and effective for the needs.
>
> —
> Erik Hatcher, Senior Solutions Architect
> http://www.lucidworks.com 
>
>
>
> > On Mar 4, 2016, at 7:09 AM, Manohar Sripada  > wrote:
> >
> > Here's my requirement -  User enters postal code and provides the
> radius. I
> > need to find the records with in the radius from the provided postal
> code.
> >
> > There are few ways I thought through after going through the "Spatial
> > Search" Solr wiki
> >
> > 1. As Latitude and Longitude positions are required for spatial search.
> Get
> > Latitude Longitude position (may be using GeoCoding API) of a postal code
> > and use "LatLonType" field type and query accordingly. As the GeoCoding
> API
> > returns one point and if the postal code area is too big, then I may end
> up
> > not getting any results (apart from the records from the same postal
> code)
> > if the radius provided is small.
> >
> > 2. Get the latitude longitude points of the postal code which forms a
> > border (not sure yet on how to get) and build a polygon (using RPT).
> While
> > querying use this polygon and provide the distance. Can this be achieved?
> > Or Am I ruminating too much? :(
> >
> > Appreciate any help on this.
> >
> > Thanks
>
>

Re: Spatial Search on Postal Code

[Manohar Sripada]

Thanks Emir,

Obviously #2 approach is much better. I know its not straight forward. But,
is it really acheivable in Solr? Like building a polygon for a postal code.
If so, can you throw some light how to do?

Thanks,
Manohar

On Friday, March 4, 2016, Emir Arnautovic 
wrote:

> Hi Manohar,
> This depends on your requirements/usecase. If postal code is interpreted
> as point than it is expected to have radius that is significantly larger
> than postal code diameter. In such case you can go with first approach. In
> order to avoid missing results from postal code in case of small search
> radius and large postal code, you can reverse geocode records and store
> postal code with each document.
> If you need to handle distance from postal code precisely - distance from
> its border, you have to get postal code polygon, expand it by search
> distance and use resulting polygon to find matches.
>
> HTH,
> Emir
>
> On 04.03.2016 13:09, Manohar Sripada wrote:
>
>> Here's my requirement -  User enters postal code and provides the radius.
>> I
>> need to find the records with in the radius from the provided postal code.
>>
>> There are few ways I thought through after going through the "Spatial
>> Search" Solr wiki
>>
>> 1. As Latitude and Longitude positions are required for spatial search.
>> Get
>> Latitude Longitude position (may be using GeoCoding API) of a postal code
>> and use "LatLonType" field type and query accordingly. As the GeoCoding
>> API
>> returns one point and if the postal code area is too big, then I may end
>> up
>> not getting any results (apart from the records from the same postal code)
>> if the radius provided is small.
>>
>> 2. Get the latitude longitude points of the postal code which forms a
>> border (not sure yet on how to get) and build a polygon (using RPT). While
>> querying use this polygon and provide the distance. Can this be achieved?
>> Or Am I ruminating too much? :(
>>
>> Appreciate any help on this.
>>
>> Thanks
>>
>>
> --
> Monitoring * Alerting * Anomaly Detection * Centralized Log Management
> Solr & Elasticsearch Support * http://sematext.com/
>
>

Re: Spatial Search on Postal Code

[Erik Hatcher]

This is just like an implementation I recently worked on with a customer.  It’s 
very much like this sort of thing ;) - 


>

It’s implemented with Solr, leveraging the Lucidworks Fusion query pipelines to 
do these steps:

   - =N comes in to the query pipeline
   - a sub-query is made to a separate zipcodes index (built from a CSV file of 
zipcodes and their corresponding representative lat/lon)
   - the matching lat/lon is used to build the appropriate geo-filtering and 
sorting parameters to pass on to the main collection

Straightforward, clean, and effective for the needs.

—
Erik Hatcher, Senior Solutions Architect
http://www.lucidworks.com 



> On Mar 4, 2016, at 7:09 AM, Manohar Sripada  wrote:
> 
> Here's my requirement -  User enters postal code and provides the radius. I
> need to find the records with in the radius from the provided postal code.
> 
> There are few ways I thought through after going through the "Spatial
> Search" Solr wiki
> 
> 1. As Latitude and Longitude positions are required for spatial search. Get
> Latitude Longitude position (may be using GeoCoding API) of a postal code
> and use "LatLonType" field type and query accordingly. As the GeoCoding API
> returns one point and if the postal code area is too big, then I may end up
> not getting any results (apart from the records from the same postal code)
> if the radius provided is small.
> 
> 2. Get the latitude longitude points of the postal code which forms a
> border (not sure yet on how to get) and build a polygon (using RPT). While
> querying use this polygon and provide the distance. Can this be achieved?
> Or Am I ruminating too much? :(
> 
> Appreciate any help on this.
> 
> Thanks

Re: Spatial Search on Postal Code

[Emir Arnautovic]

Hi Manohar,
This depends on your requirements/usecase. If postal code is interpreted 
as point than it is expected to have radius that is significantly larger 
than postal code diameter. In such case you can go with first approach. 
In order to avoid missing results from postal code in case of small 
search radius and large postal code, you can reverse geocode records and 
store postal code with each document.
If you need to handle distance from postal code precisely - distance 
from its border, you have to get postal code polygon, expand it by 
search distance and use resulting polygon to find matches.


HTH,
Emir

On 04.03.2016 13:09, Manohar Sripada wrote:

Here's my requirement -  User enters postal code and provides the radius. I
need to find the records with in the radius from the provided postal code.

There are few ways I thought through after going through the "Spatial
Search" Solr wiki

1. As Latitude and Longitude positions are required for spatial search. Get
Latitude Longitude position (may be using GeoCoding API) of a postal code
and use "LatLonType" field type and query accordingly. As the GeoCoding API
returns one point and if the postal code area is too big, then I may end up
not getting any results (apart from the records from the same postal code)
if the radius provided is small.

2. Get the latitude longitude points of the postal code which forms a
border (not sure yet on how to get) and build a polygon (using RPT). While
querying use this polygon and provide the distance. Can this be achieved?
Or Am I ruminating too much? :(

Appreciate any help on this.

Thanks



--
Monitoring * Alerting * Anomaly Detection * Centralized Log Management
Solr & Elasticsearch Support * http://sematext.com/