Re: Migrate Solr 3.4 w/ solr-1255 GeoHash to Solr 4

2013-03-06 Thread David Smiley (@MITRE.org)
Hi Harley, See: http://wiki.apache.org/solr/SolrAdaptersForLuceneSpatial4 In SOLR-2155 you had to explicitly specify the prefix encoding length, whereas in Solr 4 you specify how much precision you need and it figures out what the length is that satisfies that. When you first use the field, it'll

Re: Solr 4.1: problems with Spatial Search.

2013-03-06 Thread David Smiley (@MITRE.org)
entation is a little bit confusing, but know It works perfectly. > > Regards, > > - Luis Cappa > > 2013/3/6 David Smiley (@MITRE.org) <[hidden > email]> > > Ah; bingo! >> >> The top error in the log is what Solr reports in the HTTP response

Re: Solr 4.1: problems with Spatial Search.

2013-03-06 Thread David Smiley (@MITRE.org)
s Cappa > > > 2013/3/5 Chris Hostetter < > hossman_lucene@ > > > >> >> 1) which version of solr are you using? >> 2) what is the field & fieldtype for "geolocation" >> 2) can you try changing your query to "q={!func}geodist(

Re: Solr 4.1: problems with Spatial Search.

2013-03-05 Thread David Smiley (@MITRE.org)
Hmm; weird. It looks right. Does it work without the sort? -- i.e. does the filter work? Are there more interesting looking error messages output by Solr? Rakudten wrote > Hello! > > I´m trying to sort by geodist() distance, but it seems that I can´t: > > *The query:* > > http://192.168.1.1

Re: geodist() spatial sorting: sort param could not be parsed as a query, and is not a field that exists in the index: geodist()

2013-02-28 Thread David Smiley (@MITRE.org)
Strange. The code in Solr that has that error string passes an an additional exception that will have its own error message that is more detailed, and you'll see that in the stack trace in the Solr logs; perhaps in your error response too but I'm not sure. If you remove the sorting, are the searc

RE: solr 4.1 spatial with JTS - spatial query withitin a WKT polygon contained within another query ...

2013-02-27 Thread David Smiley (@MITRE.org)
Pires, Guilherme wrote > Hello David, > > Thanks for your response. > I'm working deeply in this and it's fully decided that solr 4.1 + JTS is > going to be supporting the map navigation for a 'public facing' GIS > solution, i.e. will deliver the objects return by a bounding box > intersection. In

Re: Distance Range query

2013-01-22 Thread David Smiley (@MITRE.org)
I'm glad to be of help. This is all possible using Solr 4 without programatic customization. As always remember the docs: http://wiki.apache.org/solr/SolrAdaptersForLuceneSpatial4#IndexingThat has an example of index a circle. Not much to it. The only aspect of the SpatialRecursivePrefixTree

Re: Spatial+Dataimport full import results in OutOfMemory for a rectangle defining a line

2013-01-22 Thread David Smiley (@MITRE.org)
Javier Molina wrote > This very wide rectangle will cause an OutOfMemoryError > > -180 3 180 3.016668 > > While this one, slightly taller will work fine. > > -180 3 180 3.5 Due to the bug, the accuracy computing algorithm believes the width for both of these is 0. That algorithm also l

Re: Spatial+Dataimport full import results in OutOfMemory for a rectangle defining a line

2013-01-21 Thread David Smiley (@MITRE.org)
gt; Is there any other documentation that I am missing? If not could you > please > shed some light on the syntax to index to boxes (rectangles) on a field? > > With regards to search, will an Intersects function behaviour will be OR, > that is match any rectangle in my example on t

Re: Using Solr Spatial in conjunction with HBASE/Hadoop

2013-01-21 Thread David Smiley (@MITRE.org)
What good is a key-value store in the context of oakstream's question? ~ David - Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book -- View this message in context: http://lucene.472066.n3.nabble.com/Using-Solr-Spatial-in-conjunction-with-HBASE-Hadoop-tp4034307p40

Re: Spatial+Dataimport full import results in OutOfMemory for a rectangle defining a line

2013-01-21 Thread David Smiley (@MITRE.org)
Javier, Your minX is slightly greater than maxX, which is interpreted as a line that wraps nearly the entire globe. Is that what you intended? If this is what you intended, then you got bitten by this unfixed bug: https://issues.apache.org/jira/browse/LUCENE-4550 As a work-around, you could spli

Re: Distance Range query

2013-01-21 Thread David Smiley (@MITRE.org)
Hi Stefano. I answered someone's question on stackoverflow which is basically the same question you have: http://stackoverflow.com/questions/13723891/lucene-4-0-spatial-calculate-max-distance-dynamically-using-indexed-documets/13764793#13764793 Essentially, you should index circles and then searc

Re: Using Solr Spatial in conjunction with HBASE/Hadoop

2013-01-19 Thread David Smiley (@MITRE.org)
e so fast over how things worked before, it's for the same reason that Lucene 4 PrefixTree (a synonym of Trie) based fields are fast. I don't think the underlying approach is even possible in the big-table NoSQL based systems without some kind of inverted index. If anyone knows otherwise

Re: Using Solr Spatial in conjunction with HBASE/Hadoop

2013-01-17 Thread David Smiley (@MITRE.org)
of course so go with Otis's advise. ~ David Smiley oakstream wrote > Hello, > I have point data (lat/lon) stored in hbase/hadoop and would like to query > the data spatially with polygons. (If I pass in a few polygons find me > all the records that exist within these polygons. I n

Re: Spatial filter in solr 4.0 - "Intersects" operation with parameters

2013-01-02 Thread David Smiley (@MITRE.org)
Mladen, FYI I just committed this to 4.x: https://issues.apache.org/jira/browse/SOLR-4230 ~ David mladen micevic wrote > Hi, > I went through example for spatial search in Solr 4.0 > (http://wiki.apache.org/solr/SolrAdaptersForLuceneSpatial4) > Both indexing and searching work fine. > > Examp

Re: Spatial filter in solr 4.0 - "Intersects" operation with parameters

2012-12-26 Thread David Smiley (@MITRE.org)
It's unusual to have Solr be the first point of entry into a service. Usually it's fronted with a web application that has the business logic that knows how to map the request to the search back-end. Given your further questions, almost anything could work without much trouble: * A standard servl

Re: Spatial filter in solr 4.0 - "Intersects" operation with parameters

2012-12-25 Thread David Smiley (@MITRE.org)
Hi Mladen, Despite some similarities at first glance, the Solr 4 spatial fields are not implemented with Solr query parsers, unlike Solr 3 spatial. Everything in quotes is handled by the field type. What you're looking for is for the Solr 3 geospatial functions to be adapted to support the Solr

Re: modeling prices based on daterange using multipoints

2012-12-12 Thread David Smiley (@MITRE.org)
britske wrote >> Ah; ok. But still, my first suggestion is still what I think you could >> do >> except that the algorithm is simpler -- return the first matching 'y' in >> the >> document where the point matches the query. Alternatively, if you're >> confident the number of matching documents (h

Re: modeling prices based on daterange using multipoints

2012-12-11 Thread David Smiley (@MITRE.org)
britske wrote > Hi David, > > Yeah interesting (as well as problematic as far is implementing) use-case > indeed :) > > 1. You mention "there are no special caches / memory requirements inherent > in this.". For a given user-query this would mean all hotels would have to > seach for all point.x e

Re: modeling prices based on daterange using multipoints

2012-12-11 Thread David Smiley (@MITRE.org)
Hi Britske, This is a very interesting question! britske wrote > ... > I realize the new spatial-stuff in Solr 4 is no magic bullet, but I'm > wondering if I could model multiple prices per day as multipoints, > whereas: > > - date*duration*nr of persons*roomtype is modeled as point.x (discr

Re: suggestion howto handle highly repetitive valued field

2012-12-11 Thread David Smiley (@MITRE.org)
The indexed="true" side is quite efficient. The stored="true" side -- not so much, but the strings you have here are pretty small and I wouldn't worry about it. Solr 4.1 (unreleased) does a great job here and compresses all the stored field data across documents. ~ David Jie Sun wrote > Hi - >

Re: Intersect Circle is matching points way outside the radius ( Solr 4 Spatial)

2012-12-10 Thread David Smiley (@MITRE.org)
there is nothing wrong > with Solr and that I didn't mix the concepts, it is just as in many cases > the problem is somewhere else where you would never imagine. > > Thanks for the hint. > > Cheers, > Javier > > > > > > On 11 December 2012 02:47, D

Re: Intersect Circle is matching points way outside the radius ( Solr 4 Spatial)

2012-12-10 Thread David Smiley (@MITRE.org)
Javi, The center point of your query circle and the indexed point is just under 49.9km (just under your query radius); this is why it matched. I plugged in your numbers here: http://www.movable-type.co.uk/scripts/latlong.html Perhaps you are misled by the projection you are using to view the map

RE: Modeling openinghours using multipoints

2012-12-10 Thread David Smiley (@MITRE.org)
Black friday , etc) .. All possible with > this > >> out of the box. Factor in 'offer category' in x and y as well for some > >> extra powerfull querying. > >> > >> Yup im enthousiastic about it , which im sure you can tell :) > >> > >

RE: Modeling openinghours using multipoints

2012-12-10 Thread David Smiley (@MITRE.org)
which im sure you can tell :) >> >> Thanks a lot David, >> >> Cheers, >> Geert-Jan >> >> >> >> Sent from my iPhone >> >> On 9 dec. 2012, at 05:35, "David Smiley (@MITRE.org) [via Lucene]" < >> [hidden email]> wr

Re: Modeling openinghours using multipoints

2012-12-08 Thread David Smiley (@MITRE.org)
Trie numeric fields), 52 should be no problem. I'll have to remember to refer back to this email on the approach if I create a field type that wraps this functionality. ~ David britske wrote > Again, this looks good! > Geert-Jan > > 2012/12/8 David Smiley (@MITRE.org) [via Luce

Re: Modeling openinghours using multipoints

2012-12-08 Thread David Smiley (@MITRE.org)
Hello again Geert-Jan! What you're trying to do is indeed possible with Solr 4 out of the box. Other terminology people use for this is multi-value time duration. This creative solution is a pure application of spatial without the geospatial notion -- we're not using an earth or other sphere mod

RE: Problem with migration to SolrAdaptersForLuceneSpatial4

2012-11-28 Thread David Smiley (@MITRE.org)
type usage. WBR Viacheslav. On 26.11.2012, at 18:52, David Smiley (@MITRE.org) wrote: > Hi Viacheslav, > > 1. You don't need JTS unless you're using polygons or WKT and your examples > uses neither. So you can remove the spatialContext attribute to use the > default, and

Re: Problem with migration to SolrAdaptersForLuceneSpatial4

2012-11-26 Thread David Smiley (@MITRE.org)
Hi Viacheslav, 1. You don't need JTS unless you're using polygons or WKT and your examples uses neither. So you can remove the spatialContext attribute to use the default, and remove the JTS jar. But that shouldn't be related to your reported problem. 2. The units for d= in the circle are in de

Re: PointType multivalued query

2012-11-15 Thread David Smiley (@MITRE.org)
Sorry, you're out of luck. SRPT could be generalized but that's a bit of work. The trickiest part I think would be writing a multi-dimensional SpatialPrefixTree impl. If the # of discrete values at each dimension is pretty small (<100? ish?), then there is a way using term positions and span que

Re: PointType multivalued query

2012-11-15 Thread David Smiley (@MITRE.org)
Borja, Umm, I'm quite confused with the use-case you present. ~ David - Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book -- View this message in context: http://lucene.472066.n3.nabble.com/PointType-multivalued-query-tp4020445p4020609.html Sent from the Solr - Us

Re: PointType multivalued query

2012-11-15 Thread David Smiley (@MITRE.org)
Oh I'm sorry, I should have read your question more clearly. I totally forgot that solr.PointType supports a configurable number of dimensions. If you need more than 2 dimensions as your example shows you do, then you'll have to resort to indexing your spatial data in another Solr core as non-mul

Re: consistency in SolrCloud replication

2012-11-15 Thread David Smiley (@MITRE.org)
Mark Miller-3 wrote > I'm talking about an update request. So if you make an update, when it > returns, your next search will see the update, because it will be on > all replicas. I presume this is only the case if (of course) the client also sent a commit. So you're saying the commit call will n

Re: Solr 4.0 Spatial Search schema.xml and data-config.xml

2012-11-15 Thread David Smiley (@MITRE.org)
The particular JavaScript I referred to is this: function processAdd(cmd) { doc = cmd.solrDoc; // org.apache.solr.common.SolrInputDocument lat = doc.getFieldValue("LATITUDE"); lon = doc.getFieldValue("LONGITUDE"); if (lat != null && lon != null) doc.setField("latLon", lat+","+l

Re: Solr 4.0 Spatial Search schema.xml and data-config.xml

2012-11-14 Thread David Smiley (@MITRE.org)
he excerpt for it on another machine. ~ David Smiley - Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-4-0-Spatial-Search-schema-xml-and-data-config-xml-tp4020376p4020413.html Sent from the

Re: BM25 model for solr 4?

2012-11-14 Thread David Smiley (@MITRE.org)
out comparisons between the choices available. I'd love to see one. ~ David Smiley - Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book -- View this message in context: http://lucene.472066.n3.nabble.com/BM25-model-for-solr-4-tp4020400p4020411.html Sent from the

Re: Easy question ? docs with empty geodata field

2012-10-20 Thread David Smiley (@MITRE.org)
That'll probably work. Or with Solr 4's new spatial field types you can do a rectangle query of the whole world: geofieldname:[-90,-180 TO 90,180]. Perhaps it'd be nice to add explicit support for [* TO *]. - Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book -- V

Re: Issue using SpatialRecursivePrefixTreeFieldType

2012-10-17 Thread David Smiley (@MITRE.org)
Nice! On Oct 17, 2012, at 10:50 AM, Eric Khoury [via Lucene] wrote: I'm using the X axis for time availability start and end (total minutes since Jan 2012), each asset can have multiple rectangles (multiple avail start and end). My original design had a bounding rect of 20 years (0 - 10,000,0

Re: Issue using SpatialRecursivePrefixTreeFieldType

2012-10-16 Thread David Smiley (@MITRE.org)
Eric, Can you please elaborate on your workaround? I'm not sure I get your drift. ~ David On Oct 16, 2012, at 12:54 PM, Eric Khoury [via Lucene] wrote: > > Thanks for the help David, makes sense. I found a workaround, creating much > smaller rectangles and updating them more often.Glad to ha

Re: Solr 4 spatial queries, jts and polygons

2012-10-01 Thread David Smiley (@MITRE.org)
Hi Ere, You are using it correctly. The problem is this: https://issues.apache.org/jira/browse/LUCENE- Sadly, this just missed the 4.0 release which appears to be imminent. If the release needs to be "respun" then I'll get this simple fix in it. Yeah this sucks and it's very frustrating to

RE: Using Solr-3304

2012-09-21 Thread David Smiley (@MITRE.org)
When I said "boundary" I meant worldBounds. Oh, and set distErrPct="0" to get precise shapes; the default is non-zero. It'll use more disk space of course, and all the more reason to carefully choose your world bounds carefully. - Author: http://www.packtpub.com/apache-solr-3-enterprise-se

RE: Using Solr-3304

2012-09-21 Thread David Smiley (@MITRE.org)
If you can stick to two dimensions then great. Remember to set the boundary attribute on the field type as I described so that spatial knows the numerical boundaries that all the data must fit in. e.g. boundary="0 0 10 2.5" (substituting whatever appropriate number of time units you need for

Re: Using Solr-3304

2012-09-21 Thread David Smiley (@MITRE.org)
For your use-case of time ranges, set geo="false" (as you've done). At this point you have a quad tree but it doesn't (yet) work properly for the default min & max numbers that a double can store, so you need to specify the boundary rectangle explicitly and to the particular numbers for your us

Re: Using Solr-3304

2012-09-21 Thread David Smiley (@MITRE.org)
http://wiki.apache.org/solr/SolrAdaptersForLuceneSpatial4 Definitely needs some updating; I will try to get to that this weekend. - Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-4-0-Join-

Re: Solr 4.0 - Join performance

2012-09-17 Thread David Smiley (@MITRE.org)
Yes absolutely. Since 4.0 hasn't been released, anything with a fix version to 4.0 basically implies trunk as well. Also notice my comment "Committed to trunk & 4x" which is explicit. ~ David On Sep 17, 2012, at 12:02 PM, Eric Khoury [via Lucene] wrote: Hi David, I see that you committed the

Re: Solr 4.0 - Join performance

2012-08-29 Thread David Smiley (@MITRE.org)
The solr.GeoHashFieldType is useless; I'd like to see it deprecated then removed. You'll need to go with unreleased code and apply patches or wait till Solr 4. ~ David On Aug 29, 2012, at 10:53 AM, Eric Khoury [via Lucene] wrote: Awesome, thanks David. In the meantime, could I potentially u

Re: Solr 4.0 - Join performance

2012-08-29 Thread David Smiley (@MITRE.org)
Solr 4 is certainly the goal. There's a bit of a setback at the moment until some of the Lucene spatial API is re-thought. I'm working heavily on such things this week. ~ David On Aug 28, 2012, at 6:22 PM, Eric Khoury [via Lucene] wrote: David, Solr support for this will come in Solr-3304 I

Re: question(s) re lucene spatial toolkit aka LSP aka spatial4j

2012-08-15 Thread David Smiley (@MITRE.org)
Hey solr-user, are you by chance indexing LineStrings? That is something I never tried with this spatial index. Depending on which iteration of LSP you are using, I figure you'd either end up indexing a vast number of points along the line which would be slow to index and make the index quite big

RE: Solr 4.0 - Join performance

2012-08-15 Thread David Smiley (@MITRE.org)
You would index rectangles of 0 height but that have a left edge 'x' of the start time and a right edge 'x' of your end time. You can index a variable number of these per Solr document and then query by either a point or another rectangle to find documents which intersect your query shape. It can

Documentation on the new updateLog transaction log feature?

2012-08-10 Thread David Smiley (@MITRE.org)
Is there any documentation on the updateLog transaction log feature in Solr 4? I started a quick prototype using Solr 4 alpha with a fairly structured schema; no big text. I disabled auto-commit which came pre-enabled and there's no soft-commit either. With CURL I posted a 1.8GB CSV file. AFter

Re: Tuning caching of geofilt queries

2012-08-10 Thread David Smiley (@MITRE.org)
Yeah it is... I rather like this write-up: https://sites.google.com/site/trescopter/Home/concepts/required-precision-for-gps-calculations#TOC-Precision-of-Float-and-Double -- which also arrives at 2.37m worse case. Aside from RAM savings, I wonder if there is any noticeable performance differenc

Re: Tuning caching of geofilt queries

2012-08-10 Thread David Smiley (@MITRE.org)
:-D. You should be happy to know that this technology is on its way into Solr 4, albeit not quite yet. Cheers, ~ David Smiley - Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book -- View this message in context: http://lucene.472066.n3.nabble.com/Tuning-caching-o

Re: question(s) re lucene spatial toolkit aka LSP aka spatial4j

2012-08-09 Thread David Smiley (@MITRE.org)
On Aug 9, 2012, at 4:16 PM, solr-user [via Lucene] wrote: I didn't know how the cache got triggered and the "needScore=false" now allows some of my problem queries to finally work, and well within 2gb of mem. needScore is an unfortunate hack in the Solr adapter to the Lucene spatial module to w

Re: question(s) re lucene spatial toolkit aka LSP aka spatial4j

2012-08-09 Thread David Smiley (@MITRE.org)
solr-user wrote > > Thanks David. No worries about the delay; am always happy and > appreciative when someone responds. > > I don't understand what you mean by "All center points get cached into > memory upon first use in a score" in question 2 about the Java OOM errors > I am seeing. > The u

Re: question(s) re lucene spatial toolkit aka LSP aka spatial4j

2012-08-08 Thread David Smiley (@MITRE.org)
nType field for sorting, which uses the lucene FieldCache and it'll use much less memory. Consider making it float based to halve your memory requirements further. p.s. I suggest "watching" this JIRA issue: https://issues.apache.org/jira/browse/SOLR-3304 ~ David Smiley - Aut

Re: Solr - Spatial Search for Specif Areas on Map

2012-07-16 Thread David Smiley (@MITRE.org)
Thinking more about this, the way to get a Lucene based system to scale to the maximum extent possible for geospatial queries would be to get a geospatial query to be satisfied by just one (usually) Lucene index segment. It would take quite a bit of customization and work to make this happen. I s

Re: Solr - Spatial Search for Specif Areas on Map

2012-07-16 Thread David Smiley (@MITRE.org)
samabhiK wrote > > David, > > Thanks for such a detailed response. The data volume I mentioned is the > total set of records we have - but we would never ever need to search the > entire base in one query; we would divide the data by region or zip code. > So, in that case I assume that for a sin

Re: Solr - Spatial Search for Specif Areas on Map

2012-07-15 Thread David Smiley (@MITRE.org)
calability vs PostGIS, I think Solr shines in its ability to scale out if you have the resources to do it. ~ David Smiley - Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Spa

Re: How to space between spatial search results? (Declustering)

2012-07-04 Thread David Smiley (@MITRE.org)
Hi mcb You're looking for spatial clustering. I answered this question yesterday on Stack Overflow: http://stackoverflow.com/a/11321723/92186 ~ David Smiley - Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book -- View this message in context: http://lucene.4

Re: Autocomplete using facets

2012-06-28 Thread David Smiley (@MITRE.org)
Ugo, I suggest simply manually filtering out "red" from the facet.prefix results you get back. Not ideal, but it's easy and your problem seems like an infrequent event and a minor nuisance. ~ David Smiley p.s. thanks for buying my book - Author: http://www.packtpub.co

Re: Spatial4j

2012-04-23 Thread David Smiley (@MITRE.org)
Yes, I definitely think so. At a minimum, I expect there will at least be a patch or built jar file for you to get going by 1 June. - Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book -- View this message in context: http://lucene.472066.n3.nabble.com/Spatial4j-tp3

Re: Geospatial search with multivalued field

2012-02-12 Thread David Smiley (@MITRE.org)
Hi Marian. I'm the author of Solr2155. SpatialQueryable is an interface provided by Solr. Where did you put the SOLR-2155 built jar file? Perhaps you put it where it doesn't belong like in your servlet engine's lib directory (varies). You can put this in a variety of places like embedding it

Re: Improving performance for SOLR geo queries?

2012-02-12 Thread David Smiley (@MITRE.org)
Mathias, For what it's worth, someone using LSP (Lucene Spatial Playground)'s RecursivePrefixTreeFieldType (which uses geohash encoding by default) quoted a 2x performance increase over Solr's built-in LatLonType. To boost the performance further, there are a couple parameters you can tweak. One

Re: Pattern: Is there a method of resolving multivalued date ranges into a single document?

2012-01-06 Thread David Smiley (@MITRE.org)
See https://issues.apache.org/jira/browse/SOLR-2155?focusedCommentId=13114839&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13114839 with my response following Geert-Jan's question. ~ David - Author: http://www.packtpub.com/apache-solr-3-enterprise-search-serv

Re: VelocityResponseWriter's future

2011-12-09 Thread David Smiley (@MITRE.org)
aScript template technology like jQuery templates which I've used to great success with AJAX-Solr in place of AJAX-Solr's "Theme" junk. Sorry if this turned into an AJAX-Solr advertisement but it is my opinion that adopting AJAX-Solr for the role /browse has is ulti

Re: multiple dateranges/timeslots per doc: modeling openinghours.

2011-09-26 Thread David Smiley (@MITRE.org)
In case anyone is curious, I responded to him with a solution using either SOLR-2155 (Geohash prefix query filter) or LSP: https://issues.apache.org/jira/browse/SOLR-2155?focusedCommentId=13115244#comment-13115244 ~ David Smiley - Author: https://www.packtpub.com/solr-1-4-enterprise-search

Re: SolrJ & Spatial Search

2011-08-16 Thread David Smiley (@MITRE.org)
solrQuery.setQuery("*:*"); solrQuery.addFilterQuery("{!func}geodist()"); solrQuery.set("sfield", "store"); solrQuery.set("pt", lat + "," + lon); solrQuery.set("sort", "geodist() asc"); //disclaimer: I haven't run this - Author: https://www.packtpub.com/solr-1-4-enterprise

Re: Matching queries on a per-element basis against a multivalued field

2011-08-02 Thread David Smiley (@MITRE.org)
ere are some amazing performance improvements tied to flex that haven't seen the light of day in an official release. It's a shame, really. So it's been so long that, well, after it dawns on everyone that it that the code is 3 friggin years old without a release -- it's time to

Re: Matching queries on a per-element basis against a multivalued field

2011-08-02 Thread David Smiley (@MITRE.org)
y, I can search for documents for the core A > that match some criteria for A, B, and/or C. The Join support works across cores. See the wiki and associated JIRA issue for it. ~ David Smiley - Author: https://www.packtpub.com/solr-1-4-enterprise-search-server/book -- View this me

Re: performance crossover between single index and sharding

2011-08-02 Thread David Smiley (@MITRE.org)
That's a fantastic answer, Shawn. To more directly answer Bernd's question: Bernard, shard your data once you've done reasonable performance optimizations to your single core index setup (see Chapter 9 of my book) and the query response time isn't meeting your requirements in spite of this. Solr

Re: Matching queries on a per-element basis against a multivalued field

2011-08-02 Thread David Smiley (@MITRE.org)
ceptable to me and it can be ameliorated with shingling. Then at search time I used Span queries and their unique ability to positionally query over more than one field. There were some edge conditions that were tricky to debug when I had a null value, but it was at least fixable with a sentinal

Re: Geospatial queries in Solr

2011-07-22 Thread David Smiley (@MITRE.org)
Jamie, You are using the field named "point" which is based on PointFieldType. Keep in mind that just because this field type is named this way, does *not* mean at all that other fields don't hold points, or that this one is especially suited to it. Arguably this one is named poorly. This fiel

Solr Ecosystem / Integration wiki pages

2011-07-10 Thread David Smiley (@MITRE.org)
formation should live on SolrIntegration and so I should move this. Yes? I really liked the idea of naming this "Solr Ecosystem" but I admit that when it comes down to it, it's basically about integrating with Solr. Any thoughts on this from anyone? ~ David Smiley - Author:

Re: SolrJ & Spatial Search

2011-07-10 Thread David Smiley (@MITRE.org)
thers just use solrQuery.setParam(name,value). ~ David Smiley - Author: https://www.packtpub.com/solr-1-4-enterprise-search-server/book -- View this message in context: http://lucene.472066.n3.nabble.com/Spatial-search-with-SolrJ-3-1-How-to-tp2961136p3158134.html Sent from the Solr - User ma

Re: bbox query syntax

2011-07-01 Thread David Smiley (@MITRE.org)
Hi. By the way, your uses of parenthesis are completely superfluous. You can't just plop that "{!" syntax anywhere you please, it only works at the beginning of a query to establish the query parser for the rest of the string and/or to set "local-params". There is a sub-query hacky syntax: ... AN

Re: intersecting map extent with solr spatial documents

2011-06-27 Thread David Smiley (@MITRE.org)
ot; and benchmark it. I have other priorities first. ~ David Smiley - Author: https://www.packtpub.com/solr-1-4-enterprise-search-server/book -- View this message in context: http://lucene.472066.n3.nabble.com/intersecting-map-extent-with-solr-spatial-documents-tp3104098p3114922.html Sent

Re: intersecting map extent with solr spatial documents

2011-06-24 Thread David Smiley (@MITRE.org)
function query is going to be evaluated on every document matching the keyword search. That will probably perform okay; not so sure for large indexes with a *:* query. Again, nice job. Could you please share an example of your ranking query? ~ David Smiley - Author: https://www.packtpub

Re: Pattern: Is there a method of resolving multivalued date ranges into a single document?

2011-06-11 Thread David Smiley (@MITRE.org)
180 TO +180 range. I hope to add better support for this problem to Solr by the end of the year. Good luck. ~ David Smiley - Author: https://www.packtpub.com/solr-1-4-enterprise-search-server/book -- View this message in context: http://lucene.472066.n3.nabble.com/Pattern-Is-there-a

Re: KStemmer for Solr 3.x +

2011-04-08 Thread David Smiley (@MITRE.org)
I see no reason why it would not be compatible. - Author: https://www.packtpub.com/solr-1-4-enterprise-search-server/book -- View this message in context: http://lucene.472066.n3.nabble.com/KStemmer-for-Solr-3-x-tp2796594p2798213.html Sent from the Solr - User mailing list archive at Nabble.

Re: difference between geospatial search from database angle and from solr angle

2011-04-06 Thread David Smiley (@MITRE.org)
one in the Lucene/Solr community. There are even some external spatial providers (JTeam, MetaCarta) and I'm partnering with other individuals to create a new one. Expect to see more in the coming months. If you're looking for some specific geospatial capabilities then let us know. ~ D

Re: MoreLikeThis with document that has not been indexed

2011-03-30 Thread David Smiley (@MITRE.org)
Ben, It's absolutely possible for MLT to find documents similar to another indexed document. That's its primary use case. For externally supplied data, you will need to supply one blob of text. You could derive this by concatenating applicable parts of your structured data before handing to Solr.

Re: noobie question: sorting

2011-03-15 Thread David Smiley (@MITRE.org)
ot supported. Complicated? You bet! But fear not; this is about as complicated as it gets. References: http://wiki.apache.org/solr/SolrQuerySyntax http://wiki.apache.org/solr/CommonQueryParameters#sort http://wiki.apache.org/solr/FunctionQuery#query ~ David Smiley Author: https://www.packtp

Re: Sorting on multiValued fields via function query

2011-03-15 Thread David Smiley (@MITRE.org)
Hi Harish. Did sorting on multiValued fields actually work correctly for you before? I'd be surprised if so. I could be wrong but I think you previously always got the sorting affects of whatever was the last indexed value. It is indeed the case that the FieldCache only supports up to one indexed

Re: Custom request handler/plugin

2011-03-13 Thread David Smiley (@MITRE.org)
eration those roles. You needn't know the set of all super-admins at any time, just wether the current request is one. ~ David Smiley - Author: https://www.packtpub.com/solr-1-4-enterprise-search-server/book -- View this message in context: http://lucene.472066.n3.nabble.com

Re: Custom request handler/plugin

2011-03-13 Thread David Smiley (@MITRE.org)
ement access control. ~ David Smiley - Author: https://www.packtpub.com/solr-1-4-enterprise-search-server/book -- View this message in context: http://lucene.472066.n3.nabble.com/Custom-request-handler-plugin-tp2673822p2674267.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: [ANNOUNCE] Web Crawler

2011-03-01 Thread David Smiley (@MITRE.org)
cific configuration you wanted. Well if that is the case, why didn't you add/enhance such capabilities for an existing crawler? ~ David Smiley - Author: https://www.packtpub.com/solr-1-4-enterprise-search-server/book -- View this message in context: http://lucene.472066.n3.nabble.co

Re: Which version of Solr?

2011-02-14 Thread David Smiley (@MITRE.org)
Wow; I'm glad you figured it out -- sort of. FYI, in the future, don't hijack email threads to talk about a new subject. Start a new thread. ~ David p.s. yes, I'm working on the 2nd edition. - Author: https://www.packtpub.com/solr-1-4-enterprise-search-server/book -- View this message in

Re: Which version of Solr?

2011-02-13 Thread David Smiley (@MITRE.org)
Maven support got a major shot in the arm for releases going forward. If you want to start with 3x, then get the source for that branch and do a build. You should see some ant tasks which build maven artifacts. My guestimate on the 3x release date is 3-4 weeks from now. ~ David Smiley (author

Wither field compresed="true" ?

2010-06-27 Thread David Smiley (@MITRE.org)
I just noticed that field compression (e.g. compressed="true") is no longer in Solr, nor can I find why this was done. Can a committer offer an explanation? If the reason is that it eats up CPU, then I'd rather accept this tradeoff for a big-data yet small query volume use case.

RE: "Solr 1.4 Enterprise Search Server" book examples

2010-05-21 Thread David Smiley (@MITRE.org)
ted on Packt's server. I made a smaller distribution for them (~127MB vs 300-something) and put the data files on MusicBrainz' servers which are downloaded as part of the setup script you should run. ~ David Smiley - Author: https://www.packtpub.com/solr-1-4-enterprise-search-serv

Re: DIH post import event listener for errors

2010-05-21 Thread David Smiley (@MITRE.org)
I'd consider using the logging framework. I do this with Log4j in other apps. Its a generic approach that works for just about any system. ~ David Smiley - Author: https://www.packtpub.com/solr-1-4-enterprise-search-server/book -- View this message in context: http://lucene.4720

Re: Endeca vs Solr?

2010-05-20 Thread David Smiley (@MITRE.org)
d is definitely not a task for Endeca. ~ David Smiley - Author: https://www.packtpub.com/solr-1-4-enterprise-search-server/book -- View this message in context: http://lucene.472066.n3.nabble.com/Endeca-vs-Solr-tp832826p833019.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: Endeca vs Solr?

2010-05-20 Thread David Smiley (@MITRE.org)
odies, then its clearly worthwhile. ~ David Smiley - Author: https://www.packtpub.com/solr-1-4-enterprise-search-server/book -- View this message in context: http://lucene.472066.n3.nabble.com/Endeca-vs-Solr-tp832826p832972.html Sent from the Solr - User mailing list archive at Nabble.com.

RE: One item, multiple fields, and range queries

2010-03-29 Thread David Smiley (@MITRE.org)
ng different queries based on common multi-value field offsets derived from matching term positions? I have no idea how far off the current codebase is to exposing enough information to make such an approach possible. ~ David Smiley From: Steven A Rowe [via Lucene] [mailto:ml-node+684371

RE: One item, multiple fields, and range queries

2010-03-29 Thread David Smiley (@MITRE.org)
make queries like range queries if there are dates involved next to impossible. This "solution" is really a hack workaround to a limitation in Lucene/Solr. I was hoping to start a conversation to a more truer resolution to this problem rather than these workarounds which aren't a

RE: One item, multiple fields, and range queries

2010-03-29 Thread David Smiley (@MITRE.org)
these fields to refer to a known value offset). Following the link you gave is interesting though the general case I'm talking about doesn't have a hierarchy. And I find the use of a single multi-valued field unpalatable for a variety of reasons. ~ David Smiley - Author: https://

Re: One item, multiple fields, and range queries

2010-03-28 Thread David Smiley (@MITRE.org)
ld take some serious hacking and I have no clue how feasible that is. Thoughts? ~ David Smiley -- View this message in context: http://n3.nabble.com/One-item-multiple-fields-and-range-queries-tp475030p682227.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: Features not present in Solr

2010-03-23 Thread David Smiley @MITRE.org
2010, at 4:17 AM, Andrzej Bialecki wrote: > >> On 2010-03-23 06:25, David Smiley @MITRE.org wrote: >>> >>> I use Endeca and Solr. >>> >>> A few notable things in Endeca but not in Solr: >>> 1. Real-time search. >> >> >>&g

Re: Features not present in Solr

2010-03-22 Thread David Smiley @MITRE.org
) but this post would get out of control. It _is_ a capable product, but I'll take Solr over it any day -- at least I understand basically all of what's going on in Solr. Of course I wrote the book on it so I'm biased ;-) ~ David Smiley Author: https://www.packtpub.com/solr-1-4-ente

Re: Boundary match as part of query language?

2010-03-21 Thread David Smiley @MITRE.org
By the way, you'll probably want to shingle or use CommonGrams (with _BEGIN & _END being "common") for acceptable performance. I'm wondering, if Lucene's new payload features might provide an alternative mechanism to mark the first and last term. ~ David Smiley ho

<    1   2   3   4   >