Re: Optimizing to only 1 segment

2010-12-27 Thread Rok Rejc
Okej the same thing happens if i run optimize in java: File file = new File(e:\\myIndex\\index); Directory directory = FSDirectory.open(file); Analyzer analyzer = new StandardAnalyzer(Version.LUCENE_40); IndexWriterConfig config = new

Re: Optimizing to only 1 segment

2010-12-27 Thread Li Li
oh, you mean lucene 4 trunk. LogByteSizeMergePolicy's default size is 2048MB(2GB) I did like this LogByteSizeMergePolicy mp=new LogByteSizeMergePolicy(); mp.setMaxMergeMB(100); 2010/12/27 Rok Rejc rokrej...@gmail.com: Okej the same thing happens if i run optimize in

sort multiple fields using solr not working

2010-12-27 Thread dhanesh
Hi Can anybody help me to sort multiple fields using solr Here is the query $additionalParameters = array( 'facet'='true', 'facet.field'=array('category','subcategory','district'), 'fl'='name,category,package,district,score,city,description,id',

RE: Custom match scoring

2010-12-27 Thread Nelson Branco
Yes, I already looked dismax which I'm using for other purposes, the big deal for this problem is having summed only the best match of each field. In dismax it sum all matches on each field. Thanks anyway. -- Nelson Branco SAPO Mapas/GIS -Original Message- From: Erick Erickson

Re: sort multiple fields using solr not working

2010-12-27 Thread Ahmet Arslan
Here is the query     $additionalParameters     = array(         'facet'='true',         'facet.field'=array('category','subcategory','district'),         'fl'='name,category,package,district,score,city,description,id',        

Re: sort multiple fields using solr not working

2010-12-27 Thread dhanesh
On 12/27/2010 5:09 PM, Ahmet Arslan wrote: Here is the query $additionalParameters = array( 'facet'='true', 'facet.field'=array('category','subcategory','district'), 'fl'='name,category,package,district,score,city,description,id',

Re: sort multiple fields using solr not working

2010-12-27 Thread Ahmet Arslan
id field is the unique field  ( uniqueKeyid/uniqueKey) Okey, but what is its type? There should be something like field name=id type=?... in schema.xml file.

Re: sort multiple fields using solr not working

2010-12-27 Thread dhanesh
On 12/27/2010 5:15 PM, Ahmet Arslan wrote: id field is the unique field ( uniqueKeyid/uniqueKey) Okey, but what is its type? There should be something like field name=id type=?... in schema.xml file. field name=id type=string stored=true required=false /

Re: sort multiple fields using solr not working

2010-12-27 Thread dhanesh
On 12/27/2010 5:22 PM, Ahmet Arslan wrote: field name=id type=tint indexed=true stored=true required=true/ Its an intiger like 1293452485. I changed the solr schema as you specified field name=sortId type=tint indexed=true stored=true required=true/ Restarted the solr server. First sort

Re: sort multiple fields using solr not working

2010-12-27 Thread Ahmet Arslan
Its an intiger like 1293452485. I changed the solr schema as you specified field name=sortId type=tint indexed=true stored=true required=true/ Restarted the solr server. First sort field sortId desc is working but the second sort field packageId desc is not working Then make

Re: sort multiple fields using solr not working

2010-12-27 Thread dhanesh
On 12/27/2010 6:19 PM, Ahmet Arslan wrote: Its an intiger like 1293452485. I changed the solr schema as you specified field name=sortId type=tint indexed=true stored=true required=true/ Restarted the solr server. First sort field sortId desc is working but the second sort field packageId desc

Re: sort multiple fields using solr not working

2010-12-27 Thread Ahmet Arslan
Yes it was sint and now i changed to tint Restarted the solr server Remove the data from the solr server Added fresh data for testing. But not working the sorting. I just noticed, you are supplying multiple sort parameters. That's the problem. There should be only one sort parameter.

Re: exception with xml file processing

2010-12-27 Thread Erick Erickson
This often happens if there is some character at the very beginning of the XML document, outside of any tags, here: character ''' (code 39) in prolog; expected '' at [row,col {unknown-source}]: [1,1] But you indicate that this is happening for every document? If that's the case, it may be an

Re: sort multiple fields using solr not working

2010-12-27 Thread dhanesh
On 12/27/2010 7:03 PM, Ahmet Arslan wrote: f d Yes exactly you said it. It worked Thanks Ahmet Here is the working code 'sort'=array('packageId desc, sortId desc') Thanks dhanesh s.r

DIH and UTF-8

2010-12-27 Thread Mark
Seems like I am missing some configuration when trying to use DIH to import documents with chinese characters. All the documents save crazy nonsense like 这是测试 instead of actual chinese characters. I think its at the JDBC level because if I hardcode one of the fields within

Re: DIH and UTF-8

2010-12-27 Thread Erick Erickson
More data please. Which jdbc driver? Have you tried just printing out the results of using that driver in a simple Java program? Solr should handle UTF-8 just fine, but the servlet container may have to have some settings tweaked, which one of those are you using? What version of Solr? Best

Re: DIH and UTF-8

2010-12-27 Thread Mark
Solr: 1.4.1 JDBC driver: Connector/J 5.1.14 Looks like its the JDBC driver because It doesn't even work with a simple java program. I know this is a little off subject now, but do you have any clues? Thanks again On 12/27/10 1:58 PM, Erick Erickson wrote: More data please. Which jdbc

Re: DIH and UTF-8

2010-12-27 Thread Glen Newton
1 - Verify your mysql is set up using UTF-8 2 - Does your JDBC connect string contain: useUnicode=truecharacterEncoding=UTF-8 See: http://dev.mysql.com/doc/refman/5.0/en/connector-j-reference-charsets.html Glen http://zzzoot.blogspot.com/ On Mon, Dec 27, 2010 at 5:15 PM, Mark

Re: DIH and UTF-8

2010-12-27 Thread Mark
I tried both of those with no such luck. On 12/27/10 2:49 PM, Glen Newton wrote: 1 - Verify your mysql is set up using UTF-8 2 - Does your JDBC connect string contain: useUnicode=truecharacterEncoding=UTF-8 See: http://dev.mysql.com/doc/refman/5.0/en/connector-j-reference-charsets.html Glen

Re: DIH and UTF-8

2010-12-27 Thread Glen Newton
Try this in mysql: mysql SHOW VARIABLES LIKE 'character_set%'; From: http://forums.mysql.com/read.php?45,362257,362257 From the same page: do you have utf8 set in your 'my.cnf' (shouldn't effect JDBC but will effect the server)? [client] default-character-set = utf8 [mysql]

Re: Custom match scoring

2010-12-27 Thread Lance Norskog
The Similarity class in Lucene is the low-level place where documents and fields are rated. You might be able to do this change there. On Mon, Dec 27, 2010 at 3:15 AM, Nelson Branco nelson-bra...@telecom.pt wrote: Yes, I already looked dismax which I'm  using for other purposes, the big deal

Re: exception with xml file processing

2010-12-27 Thread Lance Norskog
Tomcat needs a flag that tells it to use UTF-8. If you don't set that various problems happen, including this one. Look on the solr wiki for Tomcat and UTF-8. Also, there can't be any blank lines at the top of the XML file before the XML header. Can you post a very short XML file that has this

Re: DIH and UTF-8

2010-12-27 Thread Mark
Just like the user of that thread... i have my database, table, columns and system variables all set but it still doesnt work as expected. Server version: 5.0.67 Source distribution Type 'help;' or '\h' for help. Type '\c' to clear the buffer. mysql SHOW VARIABLES LIKE 'collation%';

Re: DIH and UTF-8

2010-12-27 Thread Glen Newton
Is it possible your browser is not set up to properly display the chinese characters? (I am assuming you are looking at things through your browser) Do you have any problems viewing other chinese documents properly in your browser? Using mysql, can you see these characters properly? What happens

Re: DIH and UTF-8

2010-12-27 Thread Lance Norskog
Also if you can show your DIH script. Does the Chinese text come out ok when you do a database query? http://wiki.apache.org/solr/DataImportHandler#Troubleshooting On Mon, Dec 27, 2010 at 4:38 PM, Glen Newton glen.new...@gmail.com wrote: Is it possible your browser is not set up to properly