Re: Highligheter fails using JapaneseAnalyzer

2009-07-02 Thread k.sayama
AM Subject: Re: Highligheter fails using JapaneseAnalyzer On 1 Jul 2009, at 17:39, k.sayama wrote: I could verify Token byte offsets The sytsem outputs aaa:0:3 bbb:0:3 ccc:4:7 That explains the highlighter behaviour. Clearly BBB is not at position 0-3 in the String you supplied Strin

Re: Highligheter fails using JapaneseAnalyzer

2009-07-02 Thread Matthew Hall
Out of curiosity, when you try your other test string "aaa _bbb ccc" what do the token byte offsets show? Matt Mark Harwood wrote: On 1 Jul 2009, at 17:39, k.sayama wrote: I could verify Token byte offsets The sytsem outputs aaa:0:3 bbb:0:3 ccc:4:7 That explains the highlighter behaviou

Re: Highligheter fails using JapaneseAnalyzer

2009-07-01 Thread Mark Harwood
On 1 Jul 2009, at 17:39, k.sayama wrote: I could verify Token byte offsets The sytsem outputs aaa:0:3 bbb:0:3 ccc:4:7 That explains the highlighter behaviour. Clearly BBB is not at position 0-3 in the String you supplied String CONTENTS = "AAA :BBB CCC"; Looks like the Tokenizer need

Re: Highligheter fails using JapaneseAnalyzer

2009-07-01 Thread k.sayama
e.org Sent: Wednesday, 1 July, 2009 16:13:17 Subject: Re: Highligheter fails using JapaneseAnalyzer Sorry I can not verify the Token byte offsets produced by JapaneseAnalyzer How should I verify it? - Original Message - From: "mark harwood" To: Sent: Wednesday, July 01, 2009

Re: Highligheter fails using JapaneseAnalyzer

2009-07-01 Thread mark harwood
day, 1 July, 2009 16:13:17 Subject: Re: Highligheter fails using JapaneseAnalyzer Sorry I can not verify the Token byte offsets produced by JapaneseAnalyzer How should I verify it? - Original Message - From: "mark harwood" To: Sent: Wednesday, July 01, 2009 11:31 PM Subject:

Re: Highligheter fails using JapaneseAnalyzer

2009-07-01 Thread k.sayama
Sorry I can not verify the Token byte offsets produced by JapaneseAnalyzer How should I verify it? - Original Message - From: "mark harwood" To: Sent: Wednesday, July 01, 2009 11:31 PM Subject: Re: Highligheter fails using JapaneseAnalyzer Can you verify the Token by

Re: Highligheter fails using JapaneseAnalyzer

2009-07-01 Thread mark harwood
Can you verify the Token byte offsets produced by this particular analyzer are correct? - Original Message From: k.sayama To: java-user@lucene.apache.org Sent: Wednesday, 1 July, 2009 15:22:37 Subject: Re: Highligheter fails using JapaneseAnalyzer hi I verified it by using

Re: Highligheter fails using JapaneseAnalyzer

2009-07-01 Thread k.sayama
hi I verified it by using SimpleAnalyzer, StandardAnalyzer, and CJKAnalyzer. but, The problem did not happen. I think the problem of JapaneseAnalyzer. Can this problem be solved? Does the same thing happen when you use SimpleAnalyzer, or StandardAnalyzer? I have a sneaking suspicion that the

Re: Highligheter fails using JapaneseAnalyzer

2009-06-30 Thread Matthew Hall
Does the same thing happen when you use SimpleAnalyzer, or StandardAnalyzer? I have a sneaking suspicion that the : in your contents string is what's causing your issue here, as : is a reserved character that denotes a field specification. But I could be wrong. Try swapping analyzers, if you no l

Highligheter fails using JapaneseAnalyzer

2009-06-30 Thread k.sayama
hello. i've tried to highlight string using Highligheter(2.4.1) and JapaneseAnalyzer but the following code extract show the problem String F = "f"; String CONTENTS = "AAA :BBB CCC"; JapaneseAnalyzer analyzer = new JapaneseAnalyzer(); QueryParser qp = new QueryParser( F, analyzer ); Query quer