Re: Issue with spatial search

2014-03-11 Thread David Smiley (@MITRE.org)
the error percentage? or maybe better put how does it impact perf? steve On Mon, Mar 10, 2014 at 11:17 PM, David Smiley (@MITRE.org) [hidden email]/user/SendEmail.jtp?type=nodenode=4122855i=0 wrote: Correct, Steve. Alternatively you can also put this option in your query after the end

Re: Solr spatial search within the polygon

2014-03-10 Thread David Smiley (@MITRE.org)
You're going to have to use the Lucene-spatial module directly then. There's SpatialExample.java to get you started. javinsnc wrote David Smiley (@MITRE.org) wrote On 3/10/14, 12:56 PM, javinsnc lt; javiersangrador@ gt; wrote: This is indeed the source of the problem. Why do you

Re: Solr spatial search within the polygon

2014-03-10 Thread David Smiley (@MITRE.org)
Lucene has multiple modules, one of which is spatial. You'll see it in the source tree checkout underneath the lucene directory. Javadocs: http://lucene.apache.org/core/4_7_0/spatial/index.html SpatialExample.java:

Re: Issue with spatial search

2014-03-10 Thread David Smiley (@MITRE.org)
Correct, Steve. Alternatively you can also put this option in your query after the end of the last parenthesis, as in this example from the wiki: fq=geo:IsWithin(POLYGON((-10 30, -40 40, -10 -20, 40 20, 0 0, -10 30))) distErrPct=0 ~ David Steven Bower wrote Only points in the index.. Am I

Re: Adding (spatial) filter query slows down avg response time

2014-03-01 Thread David Smiley (@MITRE.org)
Hi, The reason why it's working faster for you when you put {!geofilt} into your main query is most likely because the particular field type you are using is LatLonType, which works on a per-document basis, and by putting it in the main query it will, by default, leap-frog across the documents

Re: Solr is NoSQL database or not?

2014-03-01 Thread David Smiley (@MITRE.org)
+1 Excellent responses, Jack. - Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-is-NoSQL-database-or-not-tp4120554p4120682.html Sent from the Solr - User mailing list archive at

Re: Join Scoring

2014-02-11 Thread David Smiley (@MITRE.org)
Hi Anand. Solr's JOIN query, {!join}, constant-scores. It's simpler and faster and more memory efficient (particularly the worse-case memory use) to implement the JOIN query without scoring, so that's why. Of course, you might want it to score and pay whatever penalty is involved. For that

Re: Removing last replica from a SolrCloud collection

2014-02-02 Thread David Smiley (@MITRE.org)
Mark, I appreciate all the context. ... Mark Miller-3 wrote What do you mean you are hosed? All the SolrCores should be gone, so why does it matter so much that the collection is gone? In the days before the collections api, if you wanted to then create the collection again, just create a

Re: Removing last replica from a SolrCloud collection

2014-02-02 Thread David Smiley (@MITRE.org)
Thanks; that is *exactly* the behavior I'm talking about. I tried to find an existing issue before posting but missed this one somehow. Ramkumar R. Aiyengar wrote There's already an issue for this, https://issues.apache.org/jira/browse/SOLR-5209, we were once bitten by the same issue, when

Re: Clone (or Restore) Solrcloud

2014-02-02 Thread David Smiley (@MITRE.org)
(since it's not needed anymore; it becomes inactive). I'm not sure why this metadata is recorded because, at least after the split, I can't see why it's pertinent to anything. ~ David David Smiley (@MITRE.org) wrote Hi, I'm attempting to come up with a SolrCloud restore / clone process

Removing last replica from a SolrCloud collection

2014-01-31 Thread David Smiley (@MITRE.org)
Hi, If I issue either a core UNLOAD command, or a collection DELETEREPLICA command, (which both seem pretty much equivalent) it works but if there are no other replicas for the shard, then the metadata for the shard is completely gone in clusterstate.json! That's pretty disconcerting because

Clone (or Restore) Solrcloud

2014-01-31 Thread David Smiley (@MITRE.org)
Hi, I'm attempting to come up with a SolrCloud restore / clone process for either recover to a known good state or to clone the environment for experimentation. At the moment my process involves either creating a new zookeeper environment or at least deleting the existing Collection so that I

Re: Index JTS Point in Solr/Lucene index

2013-10-28 Thread David Smiley (@MITRE.org)
Just follow-ing up with this thread after a round of emails between Shahbaz and I… David Smiley wrote Ooooh, I see your confusion. You looked at code in an UpdateRequestProcessor and expected it to work on the client in SolrJ. It won't work for the reason that the code in the URP is

Re: Solr 4.5 spatial search - distance and score

2013-09-13 Thread David Smiley (@MITRE.org)
Hi Weber, Returning the distance separately from the score is really awkward without being able to use geodist() (which is coming in Solr 4.5 for the RPT spatial field). But as you note in SOLR-4255 it is possible. If you modify the Solr example schema so that the 'store' spatial field is of

Re: Change the score of a document based on the *value* of a multifield using dismax

2013-09-03 Thread David Smiley (@MITRE.org)
If you want to alter the score in a customized way based on indexed text data on a per-value basis then index Lucene payloads, and use PayloadTermQuery. See the javadocs for PayloadTermQuery in particular and follow the references. This is a bit dated but read this:

Re: Geo spatial clustering of points

2013-08-23 Thread David Smiley (@MITRE.org)
Hi Chris Jeroen, Tonight I posted some tips on Solr's wiki on this subject: http://wiki.apache.org/solr/SpatialClustering ~ David Chris Atkinson wrote Did you get any resolution for this? I'm about to implement something identical. On 3 Jul 2013 23:03, Jeroen Steggink lt; jeroen@ gt;

Re: Distance sort on a multi-value field

2013-08-22 Thread David Smiley (@MITRE.org)
, David Smiley (@MITRE.org) lt; DSMILEY@ gt; wrote: The distance sorting code in SOLR-2155 is roughly equivalent to the code that RPT uses (RPT has its lineage in SOLR-2155 after all). I just reviewed it to double-check. It's possible the behavior is slightly better in SOLR-2155 because

Re: How to access latitude and longitude with only LatLonType?

2013-08-22 Thread David Smiley (@MITRE.org)
Hi Quan You claim to be using LatLonType, yet the error you posted makes it clear you are in fact using SpatialRecursivePrefixTreeFieldType (RPT). Regardless of which spatial field you use, it's not clear to me what sort of statistics could be useful on a spatial field. The stats component

Re: spatial search, geofilt does not work

2013-08-20 Thread David Smiley (@MITRE.org)
could use some error msg if the query format is invalid. But, THANKS! David, you probably saved me another half day. Ming- On Mon, Aug 19, 2013 at 10:20 PM, David Smiley (@MITRE.org) DSMILEY@ wrote: Thank goodness for Solr's feature of echo'ing params back in the response

Re: Distance sort on a multi-value field

2013-08-20 Thread David Smiley (@MITRE.org)
The distance sorting code in SOLR-2155 is roughly equivalent to the code that RPT uses (RPT has its lineage in SOLR-2155 after all). I just reviewed it to double-check. It's possible the behavior is slightly better in SOLR-2155 because the cache (a Solr cache) contains normal hard-references

Re: SOLR4 Spatial sorting and query string

2013-08-19 Thread David Smiley (@MITRE.org)
This is a known limitation. From CHANGES.txt: * SOLR-2345: Enhanced geodist() to work with an RPT field, provided that the field is referenced via 'sfield' and the query point is constant. (David Smiley) The reason why that limitation is there relates to the fact that the function query

Re: spatial search, geofilt does not work

2013-08-19 Thread David Smiley (@MITRE.org)
Thank goodness for Solr's feature of echo'ing params back in the response as it helps diagnose problems like this. In your case, the filter query that Solr is seeing isn't what you (seemed) to have given on the command line: fq:!geofilt sfield=author_geo Clearly wrong. Try escaping the braces

Re: Use case of Spatial search

2013-08-19 Thread David Smiley (@MITRE.org)
Shishir, Use the location_rpt type and index circles of the business and the distance they serve with this syntax: field name=myfieldNameCircle(lat,lon d=degreesRadius)/field Your query shape is then simply a point; use bbox query parser with d=0. This approach should scale *great* at query

Re: SOLR4 Spatial sorting and query string

2013-08-15 Thread David Smiley (@MITRE.org)
Hi Roy, You'll have to calculate this client-side. I am aware of this conundrum and I put up a TODO JIRA item for it here months ago: https://issues.apache.org/jira/browse/SOLR-4633It actually shouldn't be that hard to do. ~ David roySolr wrote Hello David, The first months there

Re: SOLR4 Spatial sorting and query string

2013-08-13 Thread David Smiley (@MITRE.org)
Hi Roy. Using the example schema and data, and copying the store field to store_rpt indexed with location_rpt field type, try this query:

Re: Multipoint date ranges with spatial - Invalid Longitude Exception?

2013-08-12 Thread David Smiley (@MITRE.org)
Hi Dan, FYI the main reference page on this technique is here: http://wiki.apache.org/solr/SpatialForTimeDurations (note the slight buffering needed for the query shape). You got the exception you got because you separated the dimensions of your query rectangle using a comma, which got

Re: Performance question on Spatial Search

2013-08-05 Thread David Smiley (@MITRE.org)
From: Steven Bower-2 [via Lucene] ml-node+s472066n4082569...@n3.nabble.commailto:ml-node+s472066n4082569...@n3.nabble.com Date: Monday, August 5, 2013 9:14 AM To: Smiley, David W. dsmi...@mitre.orgmailto:dsmi...@mitre.org Subject: Re: Performance question on Spatial Search So after re-feeding

Re: Ingesting geo data into Solr very slow

2013-07-30 Thread David Smiley (@MITRE.org)
Hi Marta, Presumably you are indexing polygons -- I suspect complex ones. There isn't too much that you can do about this right now other than index them in parallel. I see you are doing this in 2 threads; try 4, or maybe even 6. Also, ensure that maxDistErr is reflective of the smallest

Re: custom field type plugin

2013-07-23 Thread David Smiley (@MITRE.org)
, David Smiley (@MITRE.org) lt; DSMILEY@ gt; wrote: Like Hoss said, you're going to have to solve this using http://wiki.apache.org/solr/SpatialForTimeDurations Using PointType is *not* going to work because your durations are multi-valued per document. It would be useful to create a custom field

Re: custom field type plugin

2013-07-22 Thread David Smiley (@MITRE.org)
Like Hoss said, you're going to have to solve this using http://wiki.apache.org/solr/SpatialForTimeDurations Using PointType is *not* going to work because your durations are multi-valued per document. It would be useful to create a custom field type that wraps the capability outlined on the wiki

Re: SpatialRecursivePrefixTreeFieldType Spatial Searching

2013-06-04 Thread David Smiley (@MITRE.org)
maxDistErr should be like 0.3 based on earlier parts of this discussion since your data is to one of a couple hours of the day, not whole days. If it was whole days, you would use 1. Changing this requires a re-index. So does changing worldBounds if you do so. distErrPct should be 0. Changing

Re: SOLR 4.3.0 - How to make fq optional?

2013-05-28 Thread David Smiley (@MITRE.org)
Your client needs to know to submit the proper filter query conditionally. It's not really a spatial issue, and I disagree with the idea to make bbox (and all other query parsers for that matter) do nothing if not given an expected input. ~ David bbarani wrote I am using the SOLR geospatial

Re: exact match country

2013-05-26 Thread David Smiley (@MITRE.org)
Hi Bill. So it seems you want an exact match to be first even if it is outside the spatial region, right? Your suggested implementation suggests this. And apparently you want to sort by distance, notwithstanding the exact match being first. Although you don't have to do this as two queries, I

Re: multivalue location_rpt field not indexing with JSON format

2013-05-24 Thread David Smiley (@MITRE.org)
Hi Barbra, Solr needs to see a String for each point value, not a 2-element array. Your doc should look like: [{id:054ac6377d6ca4ad387f73b063000910,keywords:[time, trouble, exactly],description:a anno is an anno is an anno,

Re: Restaurant availability from database

2013-05-24 Thread David Smiley (@MITRE.org)
Use this reference: http://wiki.apache.org/solr/SpatialForTimeDurations Alexandre Rafalovitch wrote On Thu, May 23, 2013 at 6:47 PM, Amit Nithian lt; anithian@ gt; wrote: Hossman did a presentation on something similar to this using spatial data at a Solr meetup some months ago.

Re: How to query docs with an indexed polygon field in java?

2013-05-23 Thread David Smiley (@MITRE.org)
Hi Kevenz, kevenz wrote ... String sql = indexType:219 AND geo:Contains(POINT(114.078327401257,22.5424866754136)); ... Then I got an error at java.lang.IllegalArgumentException: missing parens: Contains. Is there any suggestion? First of all, if your query shape is a point, then use

Re: How to improve performance of geodist()

2013-05-13 Thread David Smiley (@MITRE.org)
Hi Nicholas, Given that boosting is generally inherently fuzzy / inexact thing, you can likely get away with using simpler calculations. dist() can do the Euclidean distance (i.e. the Pythagorean theorem). If your data is in just one region of the world, you can project your data into a 2-D

Re: Multi dimensional spatial search

2013-05-11 Thread David Smiley (@MITRE.org)
Hi Kiran. The often-forgotten PointType field type can be configured to hold a variable number of dimensions. See the dimension attribute of the field type's configuration in the example schema. This field type is really just a kind of a macro field type for a configurable number of numeric

Re: Easier way to do this?

2013-04-13 Thread David Smiley (@MITRE.org)
Good question. With geofilt it's kilometers. - Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book -- View this message in context: http://lucene.472066.n3.nabble.com/Easier-way-to-do-this-tp4055474p4055784.html Sent from the Solr - User mailing list archive at

Re: Easier way to do this?

2013-04-12 Thread David Smiley (@MITRE.org)
Bill, I responded to the issue you created about this: https://issues.apache.org/jira/browse/SOLR-4704 In summary, use {!geofilt}. ~ David Billnbell wrote I would love for the SOLR spatial 4 to support pt so that I can run # of results around a central point easily like in 3.6. How can I

Re: Support old syntax including geodist

2013-04-12 Thread David Smiley (@MITRE.org)
Hi Bill, FYI see https://issues.apache.org/jira/browse/SOLR-4242 Billnbell wrote Since Spatial Lucene 4 does not seem to support geodist(), even sending d,pt,fq={!geofilt}does not help me = I need to sort. So I end up having to set up the sortsq. Any other ideas on how to support the old

Re: Spatial search question

2013-04-12 Thread David Smiley (@MITRE.org)
Yup, Lance is right. But it won't always work if you have multi-valued data since it wouldn't match a document that had a point both in the ring and the hole. Another approach that internally works faster and addresses the multi-value case is to implement a custom Spatial4j Shape. In this case,

Re: DocValues vs stored fields?

2013-04-01 Thread David Smiley (@MITRE.org)
Otis, DocValues are quite insufficient for true field updates. DocValues is a per-document value storage (hence the name); it's not uninverted/indexed. If you needed to search based on these values (e.g. find all docs that have this value or between these values) then that's not going to work.

Re: Solr 4.1: problems with Spatial Search.

2013-03-09 Thread David Smiley (@MITRE.org)
Rakudten wrote Hello again! Uhm, so if I have understood then if I´m writing/reading from the index at the same time (in other words, indexing operations are executing at the same that that other ones are querying) the performance goes down, isn´t it? Committing is the problem (soft or

RE: Migrate Solr 3.4 w/ solr-1255 GeoHash to Solr 4

2013-03-08 Thread David Smiley (@MITRE.org)
/APAN Pacific Warfighting Center Ford Island p: 808.472.7752 c: 808.377.0632 apan: harley.parks@ nipr:   harley.parks.ctr@ CONTRACTOR: Booz | Allen | Hamilton e: parks_harley@ -Original Message- From: David Smiley (@MITRE.org) [mailto: DSMILEY@ ] Sent

RE: Migrate Solr 3.4 w/ solr-1255 GeoHash to Solr 4

2013-03-08 Thread David Smiley (@MITRE.org)
You're supposed to add geo point data in latitude, longitude format, although some other variations work. Is your updating process supplying a geohash instead? If so you could write a simple Solr UpdateRequestProcessor to convert it to the expected format. But that doesn't help the fact that

Re: InvalidShapeException when using SpatialRecursivePrefixTreeFieldType with custom worldBounds

2013-03-08 Thread David Smiley (@MITRE.org)
Hi Jon. If you're able to trigger an IndexOutOfBoundsException out of the prefix tree then please file a bug (to the Lucene project, not Solr). I'll look into it when I have time. I need to add a Wiki page on the use of spatial for time ranges; there are some tricks to it. Nevertheless you've

Re: Solr 4.1: problems with Spatial Search.

2013-03-06 Thread David Smiley (@MITRE.org)
. : : Without the sort it works perfectly, and there are no more error messages, : just the one I copypasted, :-( : El 05/03/2013 19:05, David Smiley (@MITRE.org) lt; DSMILEY@ gt; : escribió: : : Hmm; weird. It looks right. Does it work without the sort? -- i.e. does : the : filter

Re: Solr 4.1: problems with Spatial Search.

2013-03-06 Thread David Smiley (@MITRE.org)
is a little bit confusing, but know It works perfectly. Regards, - Luis Cappa 2013/3/6 David Smiley (@MITRE.org) [hidden email]/user/SendEmail.jtp?type=nodenode=4045273i=1 Ah; bingo! The top error in the log is what Solr reports in the HTTP response you reported but it's the message

Re: Migrate Solr 3.4 w/ solr-1255 GeoHash to Solr 4

2013-03-06 Thread David Smiley (@MITRE.org)
Hi Harley, See: http://wiki.apache.org/solr/SolrAdaptersForLuceneSpatial4 In SOLR-2155 you had to explicitly specify the prefix encoding length, whereas in Solr 4 you specify how much precision you need and it figures out what the length is that satisfies that. When you first use the field, it'll

Re: Solr 4.1: problems with Spatial Search.

2013-03-05 Thread David Smiley (@MITRE.org)
Hmm; weird. It looks right. Does it work without the sort? -- i.e. does the filter work? Are there more interesting looking error messages output by Solr? Rakudten wrote Hello! I´m trying to sort by geodist() distance, but it seems that I can´t: *The query:*

Re: geodist() spatial sorting: sort param could not be parsed as a query, and is not a field that exists in the index: geodist()

2013-02-28 Thread David Smiley (@MITRE.org)
Strange. The code in Solr that has that error string passes an an additional exception that will have its own error message that is more detailed, and you'll see that in the stack trace in the Solr logs; perhaps in your error response too but I'm not sure. If you remove the sorting, are the

RE: solr 4.1 spatial with JTS - spatial query withitin a WKT polygon contained within another query ...

2013-02-27 Thread David Smiley (@MITRE.org)
Pires, Guilherme wrote Hello David, Thanks for your response. I'm working deeply in this and it's fully decided that solr 4.1 + JTS is going to be supporting the map navigation for a 'public facing' GIS solution, i.e. will deliver the objects return by a bounding box intersection.

Re: Spatial+Dataimport full import results in OutOfMemory for a rectangle defining a line

2013-01-22 Thread David Smiley (@MITRE.org)
Javier Molina wrote This very wide rectangle will cause an OutOfMemoryError -180 3 180 3.016668 While this one, slightly taller will work fine. -180 3 180 3.5 Due to the bug, the accuracy computing algorithm believes the width for both of these is 0. That algorithm also looks

Re: Distance Range query

2013-01-22 Thread David Smiley (@MITRE.org)
I'm glad to be of help. This is all possible using Solr 4 without programatic customization. As always remember the docs: http://wiki.apache.org/solr/SolrAdaptersForLuceneSpatial4#IndexingThat has an example of index a circle. Not much to it. The only aspect of the

Re: Distance Range query

2013-01-21 Thread David Smiley (@MITRE.org)
Hi Stefano. I answered someone's question on stackoverflow which is basically the same question you have: http://stackoverflow.com/questions/13723891/lucene-4-0-spatial-calculate-max-distance-dynamically-using-indexed-documets/13764793#13764793 Essentially, you should index circles and then

Re: Spatial+Dataimport full import results in OutOfMemory for a rectangle defining a line

2013-01-21 Thread David Smiley (@MITRE.org)
Javier, Your minX is slightly greater than maxX, which is interpreted as a line that wraps nearly the entire globe. Is that what you intended? If this is what you intended, then you got bitten by this unfixed bug: https://issues.apache.org/jira/browse/LUCENE-4550 As a work-around, you could

Re: Using Solr Spatial in conjunction with HBASE/Hadoop

2013-01-21 Thread David Smiley (@MITRE.org)
What good is a key-value store in the context of oakstream's question? ~ David - Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book -- View this message in context:

Re: Spatial+Dataimport full import results in OutOfMemory for a rectangle defining a line

2013-01-21 Thread David Smiley (@MITRE.org)
) on a field? With regards to search, will an Intersects function behaviour will be OR, that is match any rectangle in my example on that multivalued field that intersects with the given area? Thanks, Javier On 22 January 2013 05:43, David Smiley (@MITRE.org) lt; DSMILEY@ gt

Re: Using Solr Spatial in conjunction with HBASE/Hadoop

2013-01-19 Thread David Smiley (@MITRE.org)
oakstream wrote Thanks guys! David, In general and in your opinion would Lucene Spatial be the way to go to index hundreds of terabytes of spatial data that continually grows. Mostly point data, mostly structured, however, could be polygons. The searches would be within or contains in a

Re: Using Solr Spatial in conjunction with HBASE/Hadoop

2013-01-17 Thread David Smiley (@MITRE.org)
Hi Oakstream, Coincidentally I've been thinking of porting the geohash prefixtree intersection algorithm in Lucene 4 spatial to Accumulo (another big-table system like HBase). There's a decent chance it'll happen this year, I think. That doesn't help your need right now of course so go with

Re: Spatial filter in solr 4.0 - Intersects operation with parameters

2013-01-02 Thread David Smiley (@MITRE.org)
Mladen, FYI I just committed this to 4.x: https://issues.apache.org/jira/browse/SOLR-4230 ~ David mladen micevic wrote Hi, I went through example for spatial search in Solr 4.0 (http://wiki.apache.org/solr/SolrAdaptersForLuceneSpatial4) Both indexing and searching work fine. Example

Re: Spatial filter in solr 4.0 - Intersects operation with parameters

2012-12-26 Thread David Smiley (@MITRE.org)
It's unusual to have Solr be the first point of entry into a service. Usually it's fronted with a web application that has the business logic that knows how to map the request to the search back-end. Given your further questions, almost anything could work without much trouble: * A standard

Re: Spatial filter in solr 4.0 - Intersects operation with parameters

2012-12-25 Thread David Smiley (@MITRE.org)
Hi Mladen, Despite some similarities at first glance, the Solr 4 spatial fields are not implemented with Solr query parsers, unlike Solr 3 spatial. Everything in quotes is handled by the field type. What you're looking for is for the Solr 3 geospatial functions to be adapted to support the Solr

Re: modeling prices based on daterange using multipoints

2012-12-12 Thread David Smiley (@MITRE.org)
britske wrote Ah; ok. But still, my first suggestion is still what I think you could do except that the algorithm is simpler -- return the first matching 'y' in the document where the point matches the query. Alternatively, if you're confident the number of matching documents (hotels) is

Re: suggestion howto handle highly repetitive valued field

2012-12-11 Thread David Smiley (@MITRE.org)
The indexed=true side is quite efficient. The stored=true side -- not so much, but the strings you have here are pretty small and I wouldn't worry about it. Solr 4.1 (unreleased) does a great job here and compresses all the stored field data across documents. ~ David Jie Sun wrote Hi - our

Re: modeling prices based on daterange using multipoints

2012-12-11 Thread David Smiley (@MITRE.org)
Hi Britske, This is a very interesting question! britske wrote ... I realize the new spatial-stuff in Solr 4 is no magic bullet, but I'm wondering if I could model multiple prices per day as multipoints, whereas: - date*duration*nr of persons*roomtype is modeled as point.x

Re: modeling prices based on daterange using multipoints

2012-12-11 Thread David Smiley (@MITRE.org)
britske wrote Hi David, Yeah interesting (as well as problematic as far is implementing) use-case indeed :) 1. You mention there are no special caches / memory requirements inherent in this.. For a given user-query this would mean all hotels would have to seach for all point.x each time

RE: Modeling openinghours using multipoints

2012-12-10 Thread David Smiley (@MITRE.org)
:35, David Smiley (@MITRE.org) [via Lucene] [hidden email]UrlBlockedError.aspx wrote: britske wrote That's seriously awesome! Some change in the query though: You described: To query for a business that is open during at least some part of a given time duration I want To query

RE: Modeling openinghours using multipoints

2012-12-10 Thread David Smiley (@MITRE.org)
it , which im sure you can tell :) Thanks a lot David, Cheers, Geert-Jan Sent from my iPhone On 9 dec. 2012, at 05:35, David Smiley (@MITRE.org) [via Lucene] [hidden email]UrlBlockedError.aspx wrote: britske wrote That's seriously awesome! Some change in the query

Re: Intersect Circle is matching points way outside the radius ( Solr 4 Spatial)

2012-12-10 Thread David Smiley (@MITRE.org)
Javi, The center point of your query circle and the indexed point is just under 49.9km (just under your query radius); this is why it matched. I plugged in your numbers here: http://www.movable-type.co.uk/scripts/latlong.html Perhaps you are misled by the projection you are using to view the

Re: Intersect Circle is matching points way outside the radius ( Solr 4 Spatial)

2012-12-10 Thread David Smiley (@MITRE.org)
as in many cases the problem is somewhere else where you would never imagine. Thanks for the hint. Cheers, Javier On 11 December 2012 02:47, David Smiley (@MITRE.org) lt; DSMILEY@ gt;wrote: Javi, The center point of your query circle and the indexed point is just under 49.9km

Re: Modeling openinghours using multipoints

2012-12-08 Thread David Smiley (@MITRE.org)
Hello again Geert-Jan! What you're trying to do is indeed possible with Solr 4 out of the box. Other terminology people use for this is multi-value time duration. This creative solution is a pure application of spatial without the geospatial notion -- we're not using an earth or other sphere

Re: Modeling openinghours using multipoints

2012-12-08 Thread David Smiley (@MITRE.org)
this functionality. ~ David britske wrote Again, this looks good! Geert-Jan 2012/12/8 David Smiley (@MITRE.org) [via Lucene] ml-node+s472066n4025359h19@.nabble Hello again Geert-Jan! What you're trying to do is indeed possible with Solr 4 out of the box. Other terminology people use

RE: Problem with migration to SolrAdaptersForLuceneSpatial4

2012-11-28 Thread David Smiley (@MITRE.org)
calculation even without solr.SpatialRecursivePrefixTreeFieldType field type usage. WBR Viacheslav. On 26.11.2012, at 18:52, David Smiley (@MITRE.org) wrote: Hi Viacheslav, 1. You don't need JTS unless you're using polygons or WKT and your examples uses neither. So you can remove

Re: Problem with migration to SolrAdaptersForLuceneSpatial4

2012-11-26 Thread David Smiley (@MITRE.org)
Hi Viacheslav, 1. You don't need JTS unless you're using polygons or WKT and your examples uses neither. So you can remove the spatialContext attribute to use the default, and remove the JTS jar. But that shouldn't be related to your reported problem. 2. The units for d= in the circle are in

Re: Solr 4.0 Spatial Search schema.xml and data-config.xml

2012-11-15 Thread David Smiley (@MITRE.org)
The particular JavaScript I referred to is this: function processAdd(cmd) { doc = cmd.solrDoc; // org.apache.solr.common.SolrInputDocument lat = doc.getFieldValue(LATITUDE); lon = doc.getFieldValue(LONGITUDE); if (lat != null lon != null) doc.setField(latLon, lat+,+lon); }

Re: consistency in SolrCloud replication

2012-11-15 Thread David Smiley (@MITRE.org)
Mark Miller-3 wrote I'm talking about an update request. So if you make an update, when it returns, your next search will see the update, because it will be on all replicas. I presume this is only the case if (of course) the client also sent a commit. So you're saying the commit call will not

Re: PointType multivalued query

2012-11-15 Thread David Smiley (@MITRE.org)
Oh I'm sorry, I should have read your question more clearly. I totally forgot that solr.PointType supports a configurable number of dimensions. If you need more than 2 dimensions as your example shows you do, then you'll have to resort to indexing your spatial data in another Solr core as

Re: PointType multivalued query

2012-11-15 Thread David Smiley (@MITRE.org)
Borja, Umm, I'm quite confused with the use-case you present. ~ David - Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book -- View this message in context: http://lucene.472066.n3.nabble.com/PointType-multivalued-query-tp4020445p4020609.html Sent from the Solr -

Re: PointType multivalued query

2012-11-15 Thread David Smiley (@MITRE.org)
Sorry, you're out of luck. SRPT could be generalized but that's a bit of work. The trickiest part I think would be writing a multi-dimensional SpatialPrefixTree impl. If the # of discrete values at each dimension is pretty small (100? ish?), then there is a way using term positions and span

Re: BM25 model for solr 4?

2012-11-14 Thread David Smiley (@MITRE.org)
See http://wiki.apache.org/solr/SchemaXml#Similarity class=solr.BM25SimilarityFactory The factories for these have javadocs that document the parameters: http://lucene.apache.org/solr/4_0_0/solr-core/org/apache/solr/search/similarities/package-summary.html I don't know about comparisons between

Re: Solr 4.0 Spatial Search schema.xml and data-config.xml

2012-11-14 Thread David Smiley (@MITRE.org)
Tim, Combine them in lat,lon format using ScriptUpdateRequestProcessor using JavaScript. I'm doing this already in fact. See a template of an example that comes with Solr in update-script.js referenced by solrconfig.xml. I'd paste it right here if I had it but I have the excerpt for it on

Re: Easy question ? docs with empty geodata field

2012-10-20 Thread David Smiley (@MITRE.org)
That'll probably work. Or with Solr 4's new spatial field types you can do a rectangle query of the whole world: geofieldname:[-90,-180 TO 90,180]. Perhaps it'd be nice to add explicit support for [* TO *]. - Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book --

Re: Issue using SpatialRecursivePrefixTreeFieldType

2012-10-17 Thread David Smiley (@MITRE.org)
Nice! On Oct 17, 2012, at 10:50 AM, Eric Khoury [via Lucene] wrote: I'm using the X axis for time availability start and end (total minutes since Jan 2012), each asset can have multiple rectangles (multiple avail start and end). My original design had a bounding rect of 20 years (0 -

Re: Issue using SpatialRecursivePrefixTreeFieldType

2012-10-16 Thread David Smiley (@MITRE.org)
Eric, Can you please elaborate on your workaround? I'm not sure I get your drift. ~ David On Oct 16, 2012, at 12:54 PM, Eric Khoury [via Lucene] wrote: Thanks for the help David, makes sense. I found a workaround, creating much smaller rectangles and updating them more often.Glad to have

Re: Solr 4 spatial queries, jts and polygons

2012-10-01 Thread David Smiley (@MITRE.org)
Hi Ere, You are using it correctly. The problem is this: https://issues.apache.org/jira/browse/LUCENE- Sadly, this just missed the 4.0 release which appears to be imminent. If the release needs to be respun then I'll get this simple fix in it. Yeah this sucks and it's very frustrating to

Re: Using Solr-3304

2012-09-21 Thread David Smiley (@MITRE.org)
http://wiki.apache.org/solr/SolrAdaptersForLuceneSpatial4 Definitely needs some updating; I will try to get to that this weekend. - Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book -- View this message in context:

Re: Using Solr-3304

2012-09-21 Thread David Smiley (@MITRE.org)
For your use-case of time ranges, set geo=false (as you've done). At this point you have a quad tree but it doesn't (yet) work properly for the default min max numbers that a double can store, so you need to specify the boundary rectangle explicitly and to the particular numbers for your

RE: Using Solr-3304

2012-09-21 Thread David Smiley (@MITRE.org)
If you can stick to two dimensions then great. Remember to set the boundary attribute on the field type as I described so that spatial knows the numerical boundaries that all the data must fit in. e.g. boundary=0 0 10 2.5 (substituting whatever appropriate number of time units you need for

RE: Using Solr-3304

2012-09-21 Thread David Smiley (@MITRE.org)
When I said boundary I meant worldBounds. Oh, and set distErrPct=0 to get precise shapes; the default is non-zero. It'll use more disk space of course, and all the more reason to carefully choose your world bounds carefully. - Author:

Re: Solr 4.0 - Join performance

2012-09-17 Thread David Smiley (@MITRE.org)
Yes absolutely. Since 4.0 hasn't been released, anything with a fix version to 4.0 basically implies trunk as well. Also notice my comment Committed to trunk 4x which is explicit. ~ David On Sep 17, 2012, at 12:02 PM, Eric Khoury [via Lucene] wrote: Hi David, I see that you committed the

Re: Solr 4.0 - Join performance

2012-08-29 Thread David Smiley (@MITRE.org)
Solr 4 is certainly the goal. There's a bit of a setback at the moment until some of the Lucene spatial API is re-thought. I'm working heavily on such things this week. ~ David On Aug 28, 2012, at 6:22 PM, Eric Khoury [via Lucene] wrote: David, Solr support for this will come in Solr-3304 I

Re: Solr 4.0 - Join performance

2012-08-29 Thread David Smiley (@MITRE.org)
The solr.GeoHashFieldType is useless; I'd like to see it deprecated then removed. You'll need to go with unreleased code and apply patches or wait till Solr 4. ~ David On Aug 29, 2012, at 10:53 AM, Eric Khoury [via Lucene] wrote: Awesome, thanks David. In the meantime, could I potentially

RE: Solr 4.0 - Join performance

2012-08-15 Thread David Smiley (@MITRE.org)
You would index rectangles of 0 height but that have a left edge 'x' of the start time and a right edge 'x' of your end time. You can index a variable number of these per Solr document and then query by either a point or another rectangle to find documents which intersect your query shape. It

Re: question(s) re lucene spatial toolkit aka LSP aka spatial4j

2012-08-15 Thread David Smiley (@MITRE.org)
Hey solr-user, are you by chance indexing LineStrings? That is something I never tried with this spatial index. Depending on which iteration of LSP you are using, I figure you'd either end up indexing a vast number of points along the line which would be slow to index and make the index quite

Re: Tuning caching of geofilt queries

2012-08-10 Thread David Smiley (@MITRE.org)
Chris's response is quite good, and I have a couple things to add: 1. Since you can tolerate 1km slop, try defining the dynamic field *_coordinate as tfloat instead of tdouble. This will halve your memory requirements, but I'm not sure if it will be any faster -- it's worth a shot since you've

Re: Tuning caching of geofilt queries

2012-08-10 Thread David Smiley (@MITRE.org)
Yeah it is... I rather like this write-up: https://sites.google.com/site/trescopter/Home/concepts/required-precision-for-gps-calculations#TOC-Precision-of-Float-and-Double -- which also arrives at 2.37m worse case. Aside from RAM savings, I wonder if there is any noticeable performance

Documentation on the new updateLog transaction log feature?

2012-08-10 Thread David Smiley (@MITRE.org)
Is there any documentation on the updateLog transaction log feature in Solr 4? I started a quick prototype using Solr 4 alpha with a fairly structured schema; no big text. I disabled auto-commit which came pre-enabled and there's no soft-commit either. With CURL I posted a 1.8GB CSV file.

Re: question(s) re lucene spatial toolkit aka LSP aka spatial4j

2012-08-09 Thread David Smiley (@MITRE.org)
solr-user wrote Thanks David. No worries about the delay; am always happy and appreciative when someone responds. I don't understand what you mean by All center points get cached into memory upon first use in a score in question 2 about the Java OOM errors I am seeing. The underlying

  1   2   >