[jira] [Commented] (LUCENE-6471) Highlighter mergeContiguousFragments shouldn't merge 0-score fragments
[ https://issues.apache.org/jira/browse/LUCENE-6471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16082046#comment-16082046 ] Jim Richards commented on LUCENE-6471: -- [~dsmiley], I'm using [hibernate-search-orm|http://hibernate.org/search/], which is using 5.5.4 of the Lucene libraries and UnifiedHighlighter is from 6.x. I could probably do some magic in {{pom.xml}} to exclude 5.5.4 but my basic test had too many incompatibilities. Jim. > Highlighter mergeContiguousFragments shouldn't merge 0-score fragments > -- > > Key: LUCENE-6471 > URL: https://issues.apache.org/jira/browse/LUCENE-6471 > Project: Lucene - Core > Issue Type: Bug > Components: modules/highlighter >Reporter: David Smiley >Priority: Minor > > Highlighter.mergeContiguousFragments merges adjacent fragments it is given. > But it is given a list of fragments that do not necessarily have embedded > highlights (e.g. have a score of 0), and so it could grow a fragment > needlessly. I never figured out why this old highlighter keeps around such > fragments instead of eagerly tossing them when the fragment completes, which > is what I think it should do. That would address this problem and might make > things faster. I'm not sure if any highlighter user wants the non-scoring > fragments though. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6471) Highlighter mergeContiguousFragments shouldn't merge 0-score fragments
[ https://issues.apache.org/jira/browse/LUCENE-6471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16080328#comment-16080328 ] David Smiley commented on LUCENE-6471: -- [~jrichards22] Have you considered the UnifiedHighlighter? > Highlighter mergeContiguousFragments shouldn't merge 0-score fragments > -- > > Key: LUCENE-6471 > URL: https://issues.apache.org/jira/browse/LUCENE-6471 > Project: Lucene - Core > Issue Type: Bug > Components: modules/highlighter >Reporter: David Smiley >Priority: Minor > > Highlighter.mergeContiguousFragments merges adjacent fragments it is given. > But it is given a list of fragments that do not necessarily have embedded > highlights (e.g. have a score of 0), and so it could grow a fragment > needlessly. I never figured out why this old highlighter keeps around such > fragments instead of eagerly tossing them when the fragment completes, which > is what I think it should do. That would address this problem and might make > things faster. I'm not sure if any highlighter user wants the non-scoring > fragments though. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6471) Highlighter mergeContiguousFragments shouldn't merge 0-score fragments
[ https://issues.apache.org/jira/browse/LUCENE-6471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16080258#comment-16080258 ] Jim Richards commented on LUCENE-6471: -- I ended up writing this to get around this issue {code:java} public final String getBetterFragments(TokenStream tokenStream, String text, int maxNumFragments, String separator) throws IOException, InvalidTokenOffsetsException { maxNumFragments = Math.max(1, maxNumFragments); // sanity check TextFragment[] frag = getBestTextFragments(tokenStream, text, false, maxNumFragments); StringBuilder result = new StringBuilder(); for (int i = 0; i < frag.length; i++) { if (frag[i] != null && frag[i].getScore() > 0) { if (i > 0) { result.append(separator); } result.append(frag[i].toString()); } } return result.toString(); } {code} > Highlighter mergeContiguousFragments shouldn't merge 0-score fragments > -- > > Key: LUCENE-6471 > URL: https://issues.apache.org/jira/browse/LUCENE-6471 > Project: Lucene - Core > Issue Type: Bug > Components: modules/highlighter >Reporter: David Smiley >Priority: Minor > > Highlighter.mergeContiguousFragments merges adjacent fragments it is given. > But it is given a list of fragments that do not necessarily have embedded > highlights (e.g. have a score of 0), and so it could grow a fragment > needlessly. I never figured out why this old highlighter keeps around such > fragments instead of eagerly tossing them when the fragment completes, which > is what I think it should do. That would address this problem and might make > things faster. I'm not sure if any highlighter user wants the non-scoring > fragments though. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org