Re: Issue with spatial search

2014-03-11 Thread David Smiley (@MITRE.org)
centage? or maybe better put how does it impact perf? steve On Mon, Mar 10, 2014 at 11:17 PM, David Smiley (@MITRE.org) < [hidden email]> wrote: > Correct, Steve. Alternatively you can also put this option in your query > after the end of the last parenthesis, as in this example from t

Re: Issue with spatial search

2014-03-10 Thread David Smiley (@MITRE.org)
Correct, Steve. Alternatively you can also put this option in your query after the end of the last parenthesis, as in this example from the wiki: fq=geo:"IsWithin(POLYGON((-10 30, -40 40, -10 -20, 40 20, 0 0, -10 30))) distErrPct=0" ~ David Steven Bower wrote > Only points in the index.. Am I

Re: Solr spatial search within the polygon

2014-03-10 Thread David Smiley (@MITRE.org)
Lucene has multiple modules, one of which is "spatial". You'll see it in the source tree checkout underneath the lucene directory. Javadocs: http://lucene.apache.org/core/4_7_0/spatial/index.html SpatialExample.java: https://github.com/apache/lucene-solr/blob/trunk/lucene/spatial/src/test/org/apa

Re: Solr spatial search within the polygon

2014-03-10 Thread David Smiley (@MITRE.org)
You're going to have to use the Lucene-spatial module directly then. There's SpatialExample.java to get you started. javinsnc wrote > > David Smiley (@MITRE.org) wrote >> On 3/10/14, 12:56 PM, "javinsnc" < >> javiersangrador@ >> >

Re: Solr is NoSQL database or not?

2014-03-01 Thread David Smiley (@MITRE.org)
+1 Excellent responses, Jack. - Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-is-NoSQL-database-or-not-tp4120554p4120682.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: Adding (spatial) filter query slows down avg response time

2014-03-01 Thread David Smiley (@MITRE.org)
Hi, The reason why it's working faster for you when you put {!geofilt} into your main query is most likely because the particular field type you are using is LatLonType, which works on a per-document basis, and by putting it in the main query it will, by default, leap-frog across the documents wit

Re: Join Scoring

2014-02-11 Thread David Smiley (@MITRE.org)
Hi Anand. Solr's JOIN query, {!join}, constant-scores. It's simpler and faster and more memory efficient (particularly the worse-case memory use) to implement the JOIN query without scoring, so that's why. Of course, you might want it to score and pay whatever penalty is involved. For that you'

Re: Clone (or Restore) Solrcloud

2014-02-02 Thread David Smiley (@MITRE.org)
te that parent shard (since it's not needed anymore; it becomes inactive). I'm not sure why this metadata is recorded because, at least after the split, I can't see why it's pertinent to anything. ~ David David Smiley (@MITRE.org) wrote > Hi, > > I'm attempti

Re: Removing last replica from a SolrCloud collection

2014-02-02 Thread David Smiley (@MITRE.org)
Thanks; that is *exactly* the behavior I'm talking about. I tried to find an existing issue before posting but missed this one somehow. Ramkumar R. Aiyengar wrote > There's already an issue for this, > https://issues.apache.org/jira/browse/SOLR-5209, we were once bitten by > the > same issue, wh

Re: Removing last replica from a SolrCloud collection

2014-02-02 Thread David Smiley (@MITRE.org)
Mark, I appreciate all the context. ... Mark Miller-3 wrote > What do you mean you are hosed? > > All the SolrCores should be gone, so why does it matter so much that the > collection is gone? In the days before the collections api, if you wanted > to then create the collection again, just creat

Clone (or Restore) Solrcloud

2014-01-31 Thread David Smiley (@MITRE.org)
Hi, I'm attempting to come up with a SolrCloud restore / clone process for either recover to a known good state or to clone the environment for experimentation. At the moment my process involves either creating a new zookeeper environment or at least deleting the existing Collection so that I can

Removing last replica from a SolrCloud collection

2014-01-31 Thread David Smiley (@MITRE.org)
Hi, If I issue either a core UNLOAD command, or a collection DELETEREPLICA command, (which both seem pretty much equivalent) it works but if there are no other replicas for the shard, then the metadata for the shard is completely gone in clusterstate.json! That's pretty disconcerting because you

Re: Index JTS Point in Solr/Lucene index

2013-10-28 Thread David Smiley (@MITRE.org)
Just follow-ing up with this thread after a round of emails between Shahbaz and I… David Smiley wrote > Ooooh, I see your confusion. You looked at code in an > UpdateRequestProcessor and expected it to work on the client in SolrJ. It > won't work for the reason that the code in the URP is creat

Re: Solr 4.5 spatial search - distance and score

2013-09-13 Thread David Smiley (@MITRE.org)
Hi Weber, Returning the distance separately from the score is really awkward without being able to use geodist() (which is coming in Solr 4.5 for the RPT spatial field). But as you note in SOLR-4255 it is possible. If you modify the Solr example schema so that the 'store' spatial field is of type

Re: Change the score of a document based on the *value* of a multifield using dismax

2013-09-03 Thread David Smiley (@MITRE.org)
If you want to alter the score in a customized way based on indexed text data on a per-value basis then index Lucene payloads, and use PayloadTermQuery. See the javadocs for PayloadTermQuery in particular and follow the references. This is a bit dated but read this: http://searchhub.org/2009/08/0

Re: Geo spatial clustering of points

2013-08-22 Thread David Smiley (@MITRE.org)
Hi Chris & Jeroen, Tonight I posted some tips on Solr's wiki on this subject: http://wiki.apache.org/solr/SpatialClustering ~ David Chris Atkinson wrote > Did you get any resolution for this? I'm about to implement something > identical. > On 3 Jul 2013 23:03, "Jeroen Steggink" < > jeroen@ >

Re: How to access latitude and longitude with only LatLonType?

2013-08-22 Thread David Smiley (@MITRE.org)
Hi Quan You claim to be using LatLonType, yet the error you posted makes it clear you are in fact using SpatialRecursivePrefixTreeFieldType (RPT). Regardless of which spatial field you use, it's not clear to me what sort of statistics could be useful on a spatial field. The stats component doesn

Re: Distance sort on a multi-value field

2013-08-22 Thread David Smiley (@MITRE.org)
erage response time down from 4 seconds to about > 50ms. > > Very nice! > > > > On 8/20/13 7:37 PM, "David Smiley (@MITRE.org)" < > DSMILEY@ > > wrote: > >>The distance sorting code in SOLR-2155 is roughly equivalent to the code >>that >&g

Re: Distance sort on a multi-value field

2013-08-20 Thread David Smiley (@MITRE.org)
The distance sorting code in SOLR-2155 is roughly equivalent to the code that RPT uses (RPT has its lineage in SOLR-2155 after all). I just reviewed it to double-check. It's possible the behavior is slightly better in SOLR-2155 because the cache (a Solr cache) contains normal hard-references wher

Re: spatial search, geofilt does not work

2013-08-20 Thread David Smiley (@MITRE.org)
trying to figure out the problem for half day. > Probably Solr could use some error msg if the query format is invalid. > > But, THANKS! David, you probably saved me another half day. > > Ming- > > > > On Mon, Aug 19, 2013 at 10:20 PM, David Smiley (@MITRE.org) <

Re: Use case of Spatial search

2013-08-19 Thread David Smiley (@MITRE.org)
Shishir, Use the location_rpt type and index circles of the business and the distance they serve with this syntax: Circle(lat,lon d=degreesRadius) Your query shape is then simply a point; use bbox query parser with d=0. This approach should scale *great* at query time. Erick suggesting using fun

Re: spatial search, geofilt does not work

2013-08-19 Thread David Smiley (@MITRE.org)
Thank goodness for Solr's feature of echo'ing params back in the response as it helps diagnose problems like this. In your case, the filter query that Solr is seeing isn't what you (seemed) to have given on the command line: "fq":"!geofilt sfield=author_geo" Clearly wrong. Try escaping the braces

Re: SOLR4 Spatial sorting and query string

2013-08-19 Thread David Smiley (@MITRE.org)
This is a known limitation. From CHANGES.txt: * SOLR-2345: Enhanced geodist() to work with an RPT field, provided that the field is referenced via 'sfield' and the query point is constant. (David Smiley) The reason why that limitation is there relates to the fact that the function query parse

Re: SOLR4 Spatial sorting and query string

2013-08-15 Thread David Smiley (@MITRE.org)
Hi Roy, You'll have to calculate this client-side. I am aware of this conundrum and I put up a TODO JIRA item for it here months ago: https://issues.apache.org/jira/browse/SOLR-4633It actually shouldn't be that hard to do. ~ David roySolr wrote > Hello David, > > The first months there w

Re: SOLR4 Spatial sorting and query string

2013-08-13 Thread David Smiley (@MITRE.org)
Hi Roy. Using the example schema and data, and copying the "store" field to "store_rpt" indexed with location_rpt field type, try this query: http://localhost:8983/solr/select?indent=true&fl=name,store&q=*:*&sort=query%28{!geofilt%20score=distance%20filter=false%20sfield=store_rpt%20pt=45.15,-93.

Re: Multipoint date ranges with spatial - Invalid Longitude Exception?

2013-08-12 Thread David Smiley (@MITRE.org)
Hi Dan, FYI the main reference page on this technique is here: http://wiki.apache.org/solr/SpatialForTimeDurations (note the slight buffering needed for the query shape). You got the exception you got because you separated the dimensions of your query rectangle using a comma, which got Spatia

Re: Performance question on Spatial Search

2013-08-05 Thread David Smiley (@MITRE.org)
From: "Steven Bower-2 [via Lucene]" mailto:ml-node+s472066n4082569...@n3.nabble.com>> Date: Monday, August 5, 2013 9:14 AM To: "Smiley, David W." mailto:dsmi...@mitre.org>> Subject: Re: Performance question on Spatial Search So after re-feeding our data with a new boolean field that is true when

Re: Ingesting geo data into Solr very slow

2013-07-30 Thread David Smiley (@MITRE.org)
Hi Marta, Presumably you are indexing polygons -- I suspect complex ones. There isn't too much that you can do about this right now other than index them in parallel. I see you are doing this in 2 threads; try 4, or maybe even 6. Also, ensure that maxDistErr is reflective of the smallest distan

Re: Performance question on Spatial Search

2013-07-30 Thread David Smiley (@MITRE.org)
Steve, (1) Can you give a specific example of how your are specifying the spatial query? I'm looking to ensure you are not using "IsWithin", which is not meant for point data. If your query shape is a circle or the bounding box of a circle, you should use the geofilt query parser, otherwise use

Re: custom field type plugin

2013-07-23 Thread David Smiley (@MITRE.org)
s, since we have more than just > human data to deal with. It would get quite messy otherwise. > > > On 7/22/13 11:50 AM, "David Smiley (@MITRE.org)" < > DSMILEY@ > > wrote: > >>Like Hoss said, you're going to have to solve this using >>http

Re: custom field type plugin

2013-07-22 Thread David Smiley (@MITRE.org)
Like Hoss said, you're going to have to solve this using http://wiki.apache.org/solr/SpatialForTimeDurations Using PointType is *not* going to work because your durations are multi-valued per document. It would be useful to create a custom field type that wraps the capability outlined on the wiki

Re: SpatialRecursivePrefixTreeFieldType Spatial Searching

2013-06-04 Thread David Smiley (@MITRE.org)
maxDistErr should be like 0.3 based on earlier parts of this discussion since your data is to one of a couple hours of the day, not whole days. If it was whole days, you would use 1. Changing this requires a re-index. So does changing worldBounds if you do so. distErrPct should be 0. Changing i

Re: SOLR 4.3.0 - How to make fq optional?

2013-05-28 Thread David Smiley (@MITRE.org)
Your client needs to know to submit the proper filter query conditionally. It's not really a spatial issue, and I disagree with the idea to make bbox (and all other query parsers for that matter) do nothing if not given an expected input. ~ David bbarani wrote > I am using the SOLR geospatial c

Re: exact match country

2013-05-26 Thread David Smiley (@MITRE.org)
Hi Bill. So it seems you want an exact match to be first even if it is outside the spatial region, right? Your suggested implementation suggests this. And apparently you want to sort by distance, notwithstanding the exact match being first. Although you don't have to do this as two queries, I t

Re: Restaurant availability from database

2013-05-24 Thread David Smiley (@MITRE.org)
Use this reference: http://wiki.apache.org/solr/SpatialForTimeDurations Alexandre Rafalovitch wrote > On Thu, May 23, 2013 at 6:47 PM, Amit Nithian < > anithian@ > > wrote: >> Hossman did a presentation on something similar to this using spatial >> data >> at a Solr meetup some months ago. >> >

Re: multivalue location_rpt field not indexing with JSON format

2013-05-24 Thread David Smiley (@MITRE.org)
Hi Barbra, Solr needs to see a String for each point value, not a 2-element array. Your doc should look like: [{"id":"054ac6377d6ca4ad387f73b063000910","keywords":["time", "trouble", "exactly"],"description":"a anno is an anno is an anno", "location":["33.44844800999897,-111.98840074003"

Re: How to query docs with an indexed polygon field in java?

2013-05-23 Thread David Smiley (@MITRE.org)
Hi Kevenz, kevenz wrote > ... > String sql = "indexType:219" " AND " > "geo:Contains(POINT(114.078327401257,22.5424866754136))"; > ... > Then I got an error at "java.lang.IllegalArgumentException: missing > parens: Contains". Is there any suggestion? First of all, if your query shape is a point,

Re: How to improve performance of geodist()

2013-05-13 Thread David Smiley (@MITRE.org)
Hi Nicholas, Given that boosting is generally inherently fuzzy / inexact thing, you can likely get away with using simpler calculations. dist() can do the Euclidean distance (i.e. the Pythagorean theorem). If your data is in just one region of the world, you can project your data into a 2-D plan

Re: Multi dimensional spatial search

2013-05-11 Thread David Smiley (@MITRE.org)
Hi Kiran. The often-forgotten PointType field type can be configured to hold a variable number of dimensions. See the "dimension" attribute of the field type's configuration in the example schema. This field type is really just a kind of a macro field type for a configurable number of numeric fi

Re: Easier way to do this?

2013-04-13 Thread David Smiley (@MITRE.org)
Good question. With geofilt it's kilometers. - Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book -- View this message in context: http://lucene.472066.n3.nabble.com/Easier-way-to-do-this-tp4055474p4055784.html Sent from the Solr - User mailing list archive at Nab

Re: Spatial search question

2013-04-12 Thread David Smiley (@MITRE.org)
Yup, Lance is right. But it won't always work if you have multi-valued data since it wouldn't match a document that had a point both in the ring and the hole. Another approach that internally works faster and addresses the multi-value case is to implement a custom Spatial4j Shape. In this case,

Re: Support old syntax including geodist

2013-04-12 Thread David Smiley (@MITRE.org)
Hi Bill, FYI see https://issues.apache.org/jira/browse/SOLR-4242 Billnbell wrote > Since Spatial Lucene 4 does not seem to support geodist(), even sending > d,pt,fq={!geofilt}does not help me = I need to sort. So I end up having to > set up the sortsq. Any other ideas on how to support the old s

Re: Easier way to do this?

2013-04-12 Thread David Smiley (@MITRE.org)
Bill, I responded to the issue you created about this: https://issues.apache.org/jira/browse/SOLR-4704 In summary, use {!geofilt}. ~ David Billnbell wrote > I would love for the SOLR spatial 4 to support pt so that I can run # of > results around a central point easily like in 3.6. How can I

Re: DocValues vs stored fields?

2013-04-01 Thread David Smiley (@MITRE.org)
Otis, DocValues are quite insufficient for true field updates. DocValues is a per-document value storage (hence the name); it's not uninverted/indexed. If you needed to search based on these values (e.g. find all docs that have this value or between these values) then that's not going to work.

Re: Solr 4.1: problems with Spatial Search.

2013-03-09 Thread David Smiley (@MITRE.org)
Rakudten wrote > Hello again! > > Uhm, so if I have understood then if I´m writing/reading from the index at > the same time (in other words, indexing operations are executing at the > same that that other ones are querying) the performance goes down, isn´t > it? Committing is the problem (soft o

Re: InvalidShapeException when using SpatialRecursivePrefixTreeFieldType with custom worldBounds

2013-03-08 Thread David Smiley (@MITRE.org)
Hi Jon. If you're able to trigger an IndexOutOfBoundsException out of the prefix tree then please file a bug (to the Lucene project, not Solr). I'll look into it when I have time. I need to add a Wiki page on the use of spatial for time ranges; there are some tricks to it. Nevertheless you've de

RE: Migrate Solr 3.4 w/ solr-1255 GeoHash to Solr 4

2013-03-08 Thread David Smiley (@MITRE.org)
You're supposed to add geo point data in "latitude, longitude" format, although some other variations work. Is your updating process supplying a geohash instead? If so you could write a simple Solr UpdateRequestProcessor to convert it to the expected format. But that doesn't help the fact that a

RE: Migrate Solr 3.4 w/ solr-1255 GeoHash to Solr 4

2013-03-08 Thread David Smiley (@MITRE.org)
s Parks, GISP > Booz | Allen | Hamilton > Geospatial Visualization Web Developer > > WEB: https://www.apan.org > > USPACOM J73/APAN > Pacific Warfighting Center Ford Island > > p: 808.472.7752 > c: 808.377.0632 > apan: > harley.parks@ > nipr:   > harley.pa

Re: Migrate Solr 3.4 w/ solr-1255 GeoHash to Solr 4

2013-03-06 Thread David Smiley (@MITRE.org)
Hi Harley, See: http://wiki.apache.org/solr/SolrAdaptersForLuceneSpatial4 In SOLR-2155 you had to explicitly specify the prefix encoding length, whereas in Solr 4 you specify how much precision you need and it figures out what the length is that satisfies that. When you first use the field, it'll

Re: Solr 4.1: problems with Spatial Search.

2013-03-06 Thread David Smiley (@MITRE.org)
entation is a little bit confusing, but know It works perfectly. > > Regards, > > - Luis Cappa > > 2013/3/6 David Smiley (@MITRE.org) <[hidden > email]> > > Ah; bingo! >> >> The top error in the log is what Solr reports in the HTTP response

Re: Solr 4.1: problems with Spatial Search.

2013-03-06 Thread David Smiley (@MITRE.org)
s Cappa > > > 2013/3/5 Chris Hostetter < > hossman_lucene@ > > > >> >> 1) which version of solr are you using? >> 2) what is the field & fieldtype for "geolocation" >> 2) can you try changing your query to "q={!func}geodist(

Re: Solr 4.1: problems with Spatial Search.

2013-03-05 Thread David Smiley (@MITRE.org)
Hmm; weird. It looks right. Does it work without the sort? -- i.e. does the filter work? Are there more interesting looking error messages output by Solr? Rakudten wrote > Hello! > > I´m trying to sort by geodist() distance, but it seems that I can´t: > > *The query:* > > http://192.168.1.1

Re: geodist() spatial sorting: sort param could not be parsed as a query, and is not a field that exists in the index: geodist()

2013-02-28 Thread David Smiley (@MITRE.org)
Strange. The code in Solr that has that error string passes an an additional exception that will have its own error message that is more detailed, and you'll see that in the stack trace in the Solr logs; perhaps in your error response too but I'm not sure. If you remove the sorting, are the searc

RE: solr 4.1 spatial with JTS - spatial query withitin a WKT polygon contained within another query ...

2013-02-27 Thread David Smiley (@MITRE.org)
Pires, Guilherme wrote > Hello David, > > Thanks for your response. > I'm working deeply in this and it's fully decided that solr 4.1 + JTS is > going to be supporting the map navigation for a 'public facing' GIS > solution, i.e. will deliver the objects return by a bounding box > intersection. In

Re: Distance Range query

2013-01-22 Thread David Smiley (@MITRE.org)
I'm glad to be of help. This is all possible using Solr 4 without programatic customization. As always remember the docs: http://wiki.apache.org/solr/SolrAdaptersForLuceneSpatial4#IndexingThat has an example of index a circle. Not much to it. The only aspect of the SpatialRecursivePrefixTree

Re: Spatial+Dataimport full import results in OutOfMemory for a rectangle defining a line

2013-01-22 Thread David Smiley (@MITRE.org)
Javier Molina wrote > This very wide rectangle will cause an OutOfMemoryError > > -180 3 180 3.016668 > > While this one, slightly taller will work fine. > > -180 3 180 3.5 Due to the bug, the accuracy computing algorithm believes the width for both of these is 0. That algorithm also l

Re: Spatial+Dataimport full import results in OutOfMemory for a rectangle defining a line

2013-01-21 Thread David Smiley (@MITRE.org)
hat multivalued field that > intersects with the given area? > > Thanks, > Javier > > > > > > On 22 January 2013 05:43, David Smiley (@MITRE.org) < > DSMILEY@ > >wrote: > >> Javier, >> >> Your minX is slightly greater tha

Re: Using Solr Spatial in conjunction with HBASE/Hadoop

2013-01-21 Thread David Smiley (@MITRE.org)
What good is a key-value store in the context of oakstream's question? ~ David - Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book -- View this message in context: http://lucene.472066.n3.nabble.com/Using-Solr-Spatial-in-conjunction-with-HBASE-Hadoop-tp4034307p40

Re: Spatial+Dataimport full import results in OutOfMemory for a rectangle defining a line

2013-01-21 Thread David Smiley (@MITRE.org)
Javier, Your minX is slightly greater than maxX, which is interpreted as a line that wraps nearly the entire globe. Is that what you intended? If this is what you intended, then you got bitten by this unfixed bug: https://issues.apache.org/jira/browse/LUCENE-4550 As a work-around, you could spli

Re: Distance Range query

2013-01-21 Thread David Smiley (@MITRE.org)
Hi Stefano. I answered someone's question on stackoverflow which is basically the same question you have: http://stackoverflow.com/questions/13723891/lucene-4-0-spatial-calculate-max-distance-dynamically-using-indexed-documets/13764793#13764793 Essentially, you should index circles and then searc

Re: Using Solr Spatial in conjunction with HBASE/Hadoop

2013-01-19 Thread David Smiley (@MITRE.org)
oakstream wrote > Thanks guys! > David, > > In general and in your opinion would Lucene Spatial be the way to go to > index hundreds of terabytes of spatial data that continually grows. > Mostly point data, mostly structured, however, could be polygons. The > searches would be within or contains

Re: Using Solr Spatial in conjunction with HBASE/Hadoop

2013-01-17 Thread David Smiley (@MITRE.org)
Hi Oakstream, Coincidentally I've been thinking of porting the geohash prefixtree intersection algorithm in Lucene 4 spatial to Accumulo (another big-table system like HBase). There's a decent chance it'll happen this year, I think. That doesn't help your need right now of course so go with Otis

Re: Spatial filter in solr 4.0 - "Intersects" operation with parameters

2013-01-02 Thread David Smiley (@MITRE.org)
Mladen, FYI I just committed this to 4.x: https://issues.apache.org/jira/browse/SOLR-4230 ~ David mladen micevic wrote > Hi, > I went through example for spatial search in Solr 4.0 > (http://wiki.apache.org/solr/SolrAdaptersForLuceneSpatial4) > Both indexing and searching work fine. > > Examp

Re: Spatial filter in solr 4.0 - "Intersects" operation with parameters

2012-12-26 Thread David Smiley (@MITRE.org)
It's unusual to have Solr be the first point of entry into a service. Usually it's fronted with a web application that has the business logic that knows how to map the request to the search back-end. Given your further questions, almost anything could work without much trouble: * A standard servl

Re: Spatial filter in solr 4.0 - "Intersects" operation with parameters

2012-12-25 Thread David Smiley (@MITRE.org)
Hi Mladen, Despite some similarities at first glance, the Solr 4 spatial fields are not implemented with Solr query parsers, unlike Solr 3 spatial. Everything in quotes is handled by the field type. What you're looking for is for the Solr 3 geospatial functions to be adapted to support the Solr

Re: modeling prices based on daterange using multipoints

2012-12-12 Thread David Smiley (@MITRE.org)
britske wrote >> Ah; ok. But still, my first suggestion is still what I think you could >> do >> except that the algorithm is simpler -- return the first matching 'y' in >> the >> document where the point matches the query. Alternatively, if you're >> confident the number of matching documents (h

Re: modeling prices based on daterange using multipoints

2012-12-11 Thread David Smiley (@MITRE.org)
britske wrote > Hi David, > > Yeah interesting (as well as problematic as far is implementing) use-case > indeed :) > > 1. You mention "there are no special caches / memory requirements inherent > in this.". For a given user-query this would mean all hotels would have to > seach for all point.x e

Re: modeling prices based on daterange using multipoints

2012-12-11 Thread David Smiley (@MITRE.org)
Hi Britske, This is a very interesting question! britske wrote > ... > I realize the new spatial-stuff in Solr 4 is no magic bullet, but I'm > wondering if I could model multiple prices per day as multipoints, > whereas: > > - date*duration*nr of persons*roomtype is modeled as point.x (discr

Re: suggestion howto handle highly repetitive valued field

2012-12-11 Thread David Smiley (@MITRE.org)
The indexed="true" side is quite efficient. The stored="true" side -- not so much, but the strings you have here are pretty small and I wouldn't worry about it. Solr 4.1 (unreleased) does a great job here and compresses all the stored field data across documents. ~ David Jie Sun wrote > Hi - >

Re: Intersect Circle is matching points way outside the radius ( Solr 4 Spatial)

2012-12-10 Thread David Smiley (@MITRE.org)
there is nothing wrong > with Solr and that I didn't mix the concepts, it is just as in many cases > the problem is somewhere else where you would never imagine. > > Thanks for the hint. > > Cheers, > Javier > > > > > > On 11 December 2012 02:47, D

Re: Intersect Circle is matching points way outside the radius ( Solr 4 Spatial)

2012-12-10 Thread David Smiley (@MITRE.org)
Javi, The center point of your query circle and the indexed point is just under 49.9km (just under your query radius); this is why it matched. I plugged in your numbers here: http://www.movable-type.co.uk/scripts/latlong.html Perhaps you are misled by the projection you are using to view the map

RE: Modeling openinghours using multipoints

2012-12-10 Thread David Smiley (@MITRE.org)
Black friday , etc) .. All possible with > this > >> out of the box. Factor in 'offer category' in x and y as well for some > >> extra powerfull querying. > >> > >> Yup im enthousiastic about it , which im sure you can tell :) > >> > >

RE: Modeling openinghours using multipoints

2012-12-10 Thread David Smiley (@MITRE.org)
which im sure you can tell :) >> >> Thanks a lot David, >> >> Cheers, >> Geert-Jan >> >> >> >> Sent from my iPhone >> >> On 9 dec. 2012, at 05:35, "David Smiley (@MITRE.org) [via Lucene]" < >> [hidden email]> wr

Re: Modeling openinghours using multipoints

2012-12-08 Thread David Smiley (@MITRE.org)
Trie numeric fields), 52 should be no problem. I'll have to remember to refer back to this email on the approach if I create a field type that wraps this functionality. ~ David britske wrote > Again, this looks good! > Geert-Jan > > 2012/12/8 David Smiley (@MITRE.org) [via Luce

Re: Modeling openinghours using multipoints

2012-12-08 Thread David Smiley (@MITRE.org)
Hello again Geert-Jan! What you're trying to do is indeed possible with Solr 4 out of the box. Other terminology people use for this is multi-value time duration. This creative solution is a pure application of spatial without the geospatial notion -- we're not using an earth or other sphere mod

RE: Problem with migration to SolrAdaptersForLuceneSpatial4

2012-11-28 Thread David Smiley (@MITRE.org)
type usage. WBR Viacheslav. On 26.11.2012, at 18:52, David Smiley (@MITRE.org) wrote: > Hi Viacheslav, > > 1. You don't need JTS unless you're using polygons or WKT and your examples > uses neither. So you can remove the spatialContext attribute to use the > default, and

Re: Problem with migration to SolrAdaptersForLuceneSpatial4

2012-11-26 Thread David Smiley (@MITRE.org)
Hi Viacheslav, 1. You don't need JTS unless you're using polygons or WKT and your examples uses neither. So you can remove the spatialContext attribute to use the default, and remove the JTS jar. But that shouldn't be related to your reported problem. 2. The units for d= in the circle are in de

Re: PointType multivalued query

2012-11-15 Thread David Smiley (@MITRE.org)
Sorry, you're out of luck. SRPT could be generalized but that's a bit of work. The trickiest part I think would be writing a multi-dimensional SpatialPrefixTree impl. If the # of discrete values at each dimension is pretty small (<100? ish?), then there is a way using term positions and span que

Re: PointType multivalued query

2012-11-15 Thread David Smiley (@MITRE.org)
Borja, Umm, I'm quite confused with the use-case you present. ~ David - Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book -- View this message in context: http://lucene.472066.n3.nabble.com/PointType-multivalued-query-tp4020445p4020609.html Sent from the Solr - Us

Re: PointType multivalued query

2012-11-15 Thread David Smiley (@MITRE.org)
Oh I'm sorry, I should have read your question more clearly. I totally forgot that solr.PointType supports a configurable number of dimensions. If you need more than 2 dimensions as your example shows you do, then you'll have to resort to indexing your spatial data in another Solr core as non-mul

Re: consistency in SolrCloud replication

2012-11-15 Thread David Smiley (@MITRE.org)
Mark Miller-3 wrote > I'm talking about an update request. So if you make an update, when it > returns, your next search will see the update, because it will be on > all replicas. I presume this is only the case if (of course) the client also sent a commit. So you're saying the commit call will n

Re: Solr 4.0 Spatial Search schema.xml and data-config.xml

2012-11-15 Thread David Smiley (@MITRE.org)
The particular JavaScript I referred to is this: function processAdd(cmd) { doc = cmd.solrDoc; // org.apache.solr.common.SolrInputDocument lat = doc.getFieldValue("LATITUDE"); lon = doc.getFieldValue("LONGITUDE"); if (lat != null && lon != null) doc.setField("latLon", lat+","+l

Re: Solr 4.0 Spatial Search schema.xml and data-config.xml

2012-11-14 Thread David Smiley (@MITRE.org)
Tim, Combine them in "lat,lon" format using ScriptUpdateRequestProcessor using JavaScript. I'm doing this already in fact. See a template of an example that comes with Solr in update-script.js referenced by solrconfig.xml. I'd paste it right here if I had it but I have the excerpt for it on an

Re: BM25 model for solr 4?

2012-11-14 Thread David Smiley (@MITRE.org)
See http://wiki.apache.org/solr/SchemaXml#Similarity class="solr.BM25SimilarityFactory" The factories for these have javadocs that document the parameters: http://lucene.apache.org/solr/4_0_0/solr-core/org/apache/solr/search/similarities/package-summary.html I don't know about comparisons betwee

Re: Easy question ? docs with empty geodata field

2012-10-20 Thread David Smiley (@MITRE.org)
That'll probably work. Or with Solr 4's new spatial field types you can do a rectangle query of the whole world: geofieldname:[-90,-180 TO 90,180]. Perhaps it'd be nice to add explicit support for [* TO *]. - Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book -- V

Re: Issue using SpatialRecursivePrefixTreeFieldType

2012-10-17 Thread David Smiley (@MITRE.org)
Nice! On Oct 17, 2012, at 10:50 AM, Eric Khoury [via Lucene] wrote: I'm using the X axis for time availability start and end (total minutes since Jan 2012), each asset can have multiple rectangles (multiple avail start and end). My original design had a bounding rect of 20 years (0 - 10,000,0

Re: Issue using SpatialRecursivePrefixTreeFieldType

2012-10-16 Thread David Smiley (@MITRE.org)
Eric, Can you please elaborate on your workaround? I'm not sure I get your drift. ~ David On Oct 16, 2012, at 12:54 PM, Eric Khoury [via Lucene] wrote: > > Thanks for the help David, makes sense. I found a workaround, creating much > smaller rectangles and updating them more often.Glad to ha

Re: Solr 4 spatial queries, jts and polygons

2012-10-01 Thread David Smiley (@MITRE.org)
Hi Ere, You are using it correctly. The problem is this: https://issues.apache.org/jira/browse/LUCENE- Sadly, this just missed the 4.0 release which appears to be imminent. If the release needs to be "respun" then I'll get this simple fix in it. Yeah this sucks and it's very frustrating to

RE: Using Solr-3304

2012-09-21 Thread David Smiley (@MITRE.org)
When I said "boundary" I meant worldBounds. Oh, and set distErrPct="0" to get precise shapes; the default is non-zero. It'll use more disk space of course, and all the more reason to carefully choose your world bounds carefully. - Author: http://www.packtpub.com/apache-solr-3-enterprise-se

RE: Using Solr-3304

2012-09-21 Thread David Smiley (@MITRE.org)
If you can stick to two dimensions then great. Remember to set the boundary attribute on the field type as I described so that spatial knows the numerical boundaries that all the data must fit in. e.g. boundary="0 0 10 2.5" (substituting whatever appropriate number of time units you need for

Re: Using Solr-3304

2012-09-21 Thread David Smiley (@MITRE.org)
For your use-case of time ranges, set geo="false" (as you've done). At this point you have a quad tree but it doesn't (yet) work properly for the default min & max numbers that a double can store, so you need to specify the boundary rectangle explicitly and to the particular numbers for your us

Re: Using Solr-3304

2012-09-21 Thread David Smiley (@MITRE.org)
http://wiki.apache.org/solr/SolrAdaptersForLuceneSpatial4 Definitely needs some updating; I will try to get to that this weekend. - Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-4-0-Join-

Re: Solr 4.0 - Join performance

2012-09-17 Thread David Smiley (@MITRE.org)
Yes absolutely. Since 4.0 hasn't been released, anything with a fix version to 4.0 basically implies trunk as well. Also notice my comment "Committed to trunk & 4x" which is explicit. ~ David On Sep 17, 2012, at 12:02 PM, Eric Khoury [via Lucene] wrote: Hi David, I see that you committed the

Re: Solr 4.0 - Join performance

2012-08-29 Thread David Smiley (@MITRE.org)
The solr.GeoHashFieldType is useless; I'd like to see it deprecated then removed. You'll need to go with unreleased code and apply patches or wait till Solr 4. ~ David On Aug 29, 2012, at 10:53 AM, Eric Khoury [via Lucene] wrote: Awesome, thanks David. In the meantime, could I potentially u

Re: Solr 4.0 - Join performance

2012-08-29 Thread David Smiley (@MITRE.org)
Solr 4 is certainly the goal. There's a bit of a setback at the moment until some of the Lucene spatial API is re-thought. I'm working heavily on such things this week. ~ David On Aug 28, 2012, at 6:22 PM, Eric Khoury [via Lucene] wrote: David, Solr support for this will come in Solr-3304 I

Re: question(s) re lucene spatial toolkit aka LSP aka spatial4j

2012-08-15 Thread David Smiley (@MITRE.org)
Hey solr-user, are you by chance indexing LineStrings? That is something I never tried with this spatial index. Depending on which iteration of LSP you are using, I figure you'd either end up indexing a vast number of points along the line which would be slow to index and make the index quite big

RE: Solr 4.0 - Join performance

2012-08-15 Thread David Smiley (@MITRE.org)
You would index rectangles of 0 height but that have a left edge 'x' of the start time and a right edge 'x' of your end time. You can index a variable number of these per Solr document and then query by either a point or another rectangle to find documents which intersect your query shape. It can

Documentation on the new updateLog transaction log feature?

2012-08-10 Thread David Smiley (@MITRE.org)
Is there any documentation on the updateLog transaction log feature in Solr 4? I started a quick prototype using Solr 4 alpha with a fairly structured schema; no big text. I disabled auto-commit which came pre-enabled and there's no soft-commit either. With CURL I posted a 1.8GB CSV file. AFter

Re: Tuning caching of geofilt queries

2012-08-10 Thread David Smiley (@MITRE.org)
Yeah it is... I rather like this write-up: https://sites.google.com/site/trescopter/Home/concepts/required-precision-for-gps-calculations#TOC-Precision-of-Float-and-Double -- which also arrives at 2.37m worse case. Aside from RAM savings, I wonder if there is any noticeable performance differenc

Re: Tuning caching of geofilt queries

2012-08-10 Thread David Smiley (@MITRE.org)
Chris's response is quite good, and I have a couple things to add: 1. Since you can tolerate 1km slop, try defining the dynamic field *_coordinate as tfloat instead of tdouble. This will halve your memory requirements, but I'm not sure if it will be any faster -- it's worth a shot since you've al

  1   2   >