document categorization using solr?

Hi, Does solr have something built in, or recommended add-on that does document categorization? ( I found a thread about a year ago, but not exact same topic) For example, here is a commercial categorization product that will take a website and categorize it

Re: weird sorting behavior

last? thanks Joel On Dec 30, 2009, at 3:11 PM, Joel Nylund wrote: Hi, so this is only available in 1.5? I tried in 1.4 and got : org.apache.solr.common.SolrException: Error loading class 'solr.CollationKeyFilterFactory' Is there a way to do this in 1.4? The link Shalin sent is a 1.5 link I

Re: weird sorting behavior

in your individual field definition instead. The schema.xml examples have additional information that you really should scan at least HTH Erick On Thu, Dec 31, 2009 at 8:53 AM, Joel Nylund jnyl...@yahoo.com wrote: Hi, After some further investigation, it turns out that null fields were

Re: weird sorting behavior

=solr.KeywordTokenizerFactory/ filter class=solr.CollationKeyFilterFactory language= strength=secondary / /analyzer /fieldType On Fri, Dec 25, 2009 at 5:37 AM, Shalin Shekhar Mangar shalinman...@gmail.com wrote: On Thu, Dec 24, 2009 at 11:51 PM, Joel Nylund jnyl...@yahoo.com wrote: update, I tried

weird sorting behavior

I have a field: field name=title type=alphaOnlySort indexed=true stored=true required=false/ fieldType name=alphaOnlySort class=solr.TextField sortMissingLast=true omitNorms=true analyzer !-- KeywordTokenizer does no actual tokenizing, so the entire input

Re: weird sorting behavior

dont work string - sorts nicely for numbers and letters, but special chars like blanks show up first in the list thanks Joel On Dec 24, 2009, at 11:20 AM, Joel Nylund wrote: I have a field: field name=title type=alphaOnlySort indexed=true stored=true required=false/ fieldType name

suggestions for DIH batchSize

Hi, it looks like from looking at the code the default is 500, is the recommended setting for this? Has anyone notice any significant performance/memory tradeoffs by making this much bigger? thanks Joel

Re: Request Assistance with DIH

to the Development console, it does not appear that the connection to Oracle is being made. So if someone could offer some configuration/connection setup directions I would very much appreciate it. Thanks Robbin -Original Message- From: Joel Nylund [mailto:jnyl...@yahoo.com] Sent: Friday

Re: Auto update with deltaimport

windows or unix? unix - make a shell script and call it from cron windows - make a .bat or .cmd file and call it from scheduler within the shell scripts/bat files use wget or curl to call the right import: wget -q -O /dev/null http://localhost:8983/solr/dataimport?command=delta-import

Re: Request Assistance with DIH

add ?command=full-import to your url http://localhost:8983/solr/dataimport?command=full-import thanks Joel On Dec 11, 2009, at 7:45 PM, Robbin wrote: I've been trying to use the DIH with oracle and would love it if someone could give me some pointers. I put the ojdbc14.jar in both the

Re: # in query

you *think* is in your index actually is there. HTH Erick On Mon, Dec 7, 2009 at 3:28 PM, Joel Nylund jnyl...@yahoo.com wrote: ok thanks, sorry my brain wasn't working, but even when I url encode it, I dont get any results, is there something special I have to do for solr? thanks Joel

Re: # in query

of assumptions there any one of which may be violated by your schema. So please post the relevant portions of your schema to make it easier to help. Best Erick On Tue, Dec 8, 2009 at 9:54 AM, Joel Nylund jnyl...@yahoo.com wrote: Thanks Eric, I looked more into this, but still stuck: I have

# in query

Hi, How can I put a # sign in a query, do I need to escape it? For example I want to query books with title that contain # No work so far: http://localhost:8983/solr/select?q=textTitle:#; http://localhost:8983/solr/select?q=textTitle:# http://localhost:8983/solr/select?q=textTitle:\#; Getting

Re: # in query

it as a separator between the URL for the server (on the left) and the fragment identifier (on the right) which is not sent the server. You might want to read about URL-encoding, escaping with backslash is a shell-thing, not a thing for URLs! paul Le 07-déc.-09 à 21:16, Joel Nylund a écrit : Hi

how to get list of unique terms for a field

Hi, lets say I have a field called countryName, is there a way to get a list of all the countries for this field? Trying to figure out a nice way to keep my categories and the solr results in sync, would be nice to get these from solr instead of the database. thanks Joel

weird behavior between 2 enviorments

I have 2 environments one works great for this query: my osx environment: http://localhost:8983/solr/select?q=countryName:%22Bosnia%20and%20Herzegovina%22 - returns 2 results my linux environment: http://localhost:8983/solr/select?q=countryName:%22Bosnia%20and%20Herzegovina%22 -

Re: weird behavior between 2 enviorments

, Joel Nylund jnyl...@yahoo.com wrote: same client, here are the debug results, something interesting is going on, I dont understand solr/lucene well enough to understand, see below not working env (linux) response - lst name=responseHeader int name=status0/int int name=QTime2/int - lst name

debugging javascript DIH

is there a way to print to std out or anything from my javascript DIH transformer? thanks Joel

getting value from parent query in subquery transformer

Hi, I have an entity that has a entity within it that executes a query for each row and calls a transformer. Is there a way to pass a value from the parent query into the transformer? For example, I have an entity called document, and it it has an ID and sometimes it has a category. I

NOT combined with OR is not getting exected results

http://localhost:8983/solr/select?q=%28NOT%20categoryType:%22MEDIATYPE%22%29 :gives 292289 results http://localhost:8983/solr/select?q=fmMediaType:%22text%22 :gives 530 results http://localhost:8983/solr/select?q=%28NOT%20categoryType:%22MEDIATYPE%22%29%20OR%20fmMediaType:%22text%22

Re: NOT combined with OR is not getting exected results

Hi, thanks, but still get 530 results for this new query your proposed. thanks Joel On Dec 2, 2009, at 12:00 PM, AHMET ARSLAN wrote: http://localhost:8983/solr/select?q=%28NOT%20categoryType:%22MEDIATYPE%22%29 :gives 292289 results

Re: NOT combined with OR is not getting exected results

thanks that worked! and yes I have some with no categoryType thanks Joel On Dec 2, 2009, at 2:24 PM, AHMET ARSLAN wrote: Hi, thanks, but still get 530 results for this new query your proposed. May be you have some documents that has empty categoryType field. Can you try this: q = ((*:*

Re: getting total index size last update date/time from query

/LukeRequestHandler Peter -Original Message- From: Joel Nylund [mailto:jnyl...@yahoo.com] Sent: Thursday, November 19, 2009 8:31 AM To: solr-user@lucene.apache.org Subject: getting total index size last update date/time from query Hi, Looking for total number of documents in my index and the last

Re: how to do partial word searches?

queryBest Erick On Tue, Nov 24, 2009 at 7:51 PM, Joel Nylund jnyl...@yahoo.com wrote: Hi, I saw some older postings on this, but didnt see a resolution. I have a field called title, I would like to be able to find partial word matches within the title. For example: http://localhost:8983

Re: solr/jetty not working for anything other than localhost

interfaces netstat -an |grep 8983 You should see tcp0 0 0.0.0.0:8983 0.0.0.0:* LISTEN -Simon On Wed, Nov 25, 2009 at 3:55 PM, Joel Nylund jnyl...@yahoo.com wrote: Hi, if I try to use any other hostname jetty doesnt work, gives a blank page, if I telnet too

Re: solr/jetty not working for anything other than localhost

yes says: 2009-11-25 18:08:59.967::INFO: Started SocketConnector @ 0.0.0.0:8983 running on osx thanks Joel On Nov 25, 2009, at 6:00 PM, simon wrote: On Wed, Nov 25, 2009 at 5:27 PM, Joel Nylund jnyl...@yahoo.com wrote: I see: tcp46 0 0 *.8983

Re: help with dataimport delta query

as ${dataimporter.delta.job_jobs_id} I guess it should be ${dataimporter.delta.id} On Tue, Nov 24, 2009 at 1:19 AM, Joel Nylund jnyl...@yahoo.com wrote: Hi, I have solr all working nicely, except im trying to get deltas to work on my data import handler Here is a simplification of my data import config

how to do partial word searches?

Hi, I saw some older postings on this, but didnt see a resolution. I have a field called title, I would like to be able to find partial word matches within the title. For example: http://localhost:8983/solr/select?q=textTitle:%22*sulli*%22 I would expect it to find: str name=textTitlethe

Re: configure solr

for #1, under example, is there a webapps folder, does it contain solr.war ? are there any errors in your startup log for jetty, does it say anything about setting up solr, and solr home etc. Joel On Nov 24, 2009, at 4:55 PM, Jill Han wrote: Hi, I just downloaded solr -1.4.0 to my

Re: help with dataimport delta query

On Nov 23, 2009, at 2:49 PM, Joel Nylund wrote: Hi, I have solr all working nicely, except im trying to get deltas to work on my data import handler Here is a simplification of my data import config, I have a table called Book which has categories, im doing subquries for the category info

getting total index size last update date/time from query

Hi, Looking for total number of documents in my index and the last updated date/time of the index. Is there a way to get this through the standard query q=? if not, what is the best way to get this info from solr. thanks Joel

Re: deployment questions

better off running solr as a server on its own and using network security? thanks Joel On Nov 9, 2009, at 5:04 PM, Joel Nylund wrote: Hi, I have a java app that is deployed in jboss/tomcat container. I would like to add my solr index to it. I have read about this and it seems fairly

indexing on differnt server

is it possible to index on one server and copy the files over? thanks Joel

deployment questions

Hi, I have a java app that is deployed in jboss/tomcat container. I would like to add my solr index to it. I have read about this and it seems fairly straight forward, but im curious the best way to secure it. I require my users to login to my app to use it, so I want the search

Re: solr query help alpha numeric and not

, The ID is sent back as a string (instead of as an integer) in your example. Could this be the cause? - Jonathan On Nov 4, 2009, at 9:08 AM, Joel Nylund wrote: Hi, I have a field called firstLetterTitle, this field has 1 char, it can be anything, I need help with a few queries on this char

Re: solr query help alpha numeric and not

Avlesh, thanks those worked, for somre reason I never got your mail, found it in one of the list archives though. thanks again Joel On Nov 5, 2009, at 9:08 PM, Avlesh Singh wrote: Didn't the queries in my reply work? Cheers Avlesh On Fri, Nov 6, 2009 at 4:16 AM, Joel Nylund jnyl

solr query help alpha numeric and not

Hi, I have a field called firstLetterTitle, this field has 1 char, it can be anything, I need help with a few queries on this char: 1.) I want all NON ALPHA and NON numbers, so any char that is not A-Z or 0-9 I tried:

how to use ajax-solr - example?

Hi, I looked at the documentation and I have no idea how to get started? Can someone point me to or show me an example of how to send a query to a solr server and paginate through the results using ajax- solr. I would glady write a blog tutorial on how to do this if someone can get me

exact match lookup

Hi, I have a field that I want to do exact match lookups using. (when I say exact match, im looking for equivalent to a sql query where with no like clause so where feedClass = Social News) For example the field is called feedClass and im doing:

Re: exact match lookup

': feedClass:Social News (URI encoded of course). otherwise your request will become (I assume you're using a standard query parser) feedClass:Social defaultField:News . Well that's the idea. It should then work using the type string. Cheers! J. 2009/11/4 Joel Nylund jnyl...@yahoo.com: Hi, I

Re: exact match lookup

that worked, thanks! had to negate the score. thanks Joel On Nov 4, 2009, at 1:57 PM, Jérôme Etévé wrote: If feedClass acts as an identifier, better use string :) use sort=title asc,score desc (not sort:) J. 2009/11/4 Joel Nylund jnyl...@yahoo.com: thank worked for me, changed to: http

Re: how to use ajax-solr - example?

and format the data. I figured this is something I can throw together in a few hours, but I also figured someone would have already done the work. thanks Joel On Nov 4, 2009, at 2:02 PM, Israel Ekpo wrote: On Wed, Nov 4, 2009 at 10:48 AM, Joel Nylund jnyl...@yahoo.com wrote: Hi, I

Re: best way to model 1-N

:55 AM, Joel Nylund jnyl...@yahoo.com wrote: Hi, I have one index so far which contains feeds. I have been able to de-normalize several tables and map this data onto the feed entity. There is one tricky problem that I need help on. Feeds have 1 - many categories. So Lets say we have

Re: best way to model 1-N

out quite easy. Have a look at the thread with the heading from this mailing list (SOLR-USER): DataImportHandler / Import from DB : one data set comes in multiple rows Cheers, Chantal Joel Nylund schrieb: thanks, but im confused how I can aggregate across rows, I dont know of any easy way

Re: best way to model 1-N

Im using apache-solr-1.3.0 I got it to work using javascript function instead. thanks Joel On Oct 30, 2009, at 12:44 PM, Chantal Ackermann wrote: This looks all right to me, but I might be missing something. Which version/build of SOLR are you using? Chantal Joel Nylund schrieb: Thanks

Re: weird problem with letters S and T

(títulos começando em A até N) 4. Read field value for presentation from the original field Cheers! Michel Bottan On Thu, Oct 29, 2009 at 1:23 AM, Norberto Meijome numard...@gmail.comwrote: On Wed, 28 Oct 2009 19:20:37 -0400 Joel Nylund jnyl...@yahoo.com wrote: Well I tried removing

data import with transformer

Hi, I have been reading the solr book and wiki, but I cant find any similar examples to what Im looking for. I have a database field called category, this field needs some text manipulation before it goes in the index here is the java code for what im trying to do: // categories look like

multiple sql queries for one index?

Hi, Its been hurting my brain all day to try to build 1 query for my index (joins upon joins upon joins). Is there a way I can do multiple queries to populate the same index? I have one main table that I can join everything back via ID, it should be theoretically possible If this can

best way to model 1-N

Hi, I have one index so far which contains feeds. I have been able to de- normalize several tables and map this data onto the feed entity. There is one tricky problem that I need help on. Feeds have 1 - many categories. So Lets say we have Category1, Category2 and Category3 Feed 1 - is

weird problem with letters S and T

(I am super new to solr, sorry if this is an easy one) Hi, I want to support an A-Z type view of my data. I have a DataImportHandler that uses sql (my query is complex, but the part that matters is: SELECT f.id, f.title, LEFT(f.title,1) as firstLetterTitle FROM Foo f I can create this

Re: weird problem with letters S and T

a similar issue the other day; in my case the solution turned out to be that the letters were stopwords. Don't know if this is your answer, but worth checking. Bern -Original Message- From: Joel Nylund [mailto:jnyl...@yahoo.com] Sent: Thursday, 29 October 2009 9:17 AM To: solr-user

Re: weird problem with letters S and T

one character per field. There are other text field types that do not have the stop word filter, so give your first letter field that field type. In this way stopword filter analyser is only disabled for searches on the first letter field. Cheers, Martijn 2009/10/28 Joel Nylund jnyl...@yahoo.com