[jira] Commented: (LUCENE-1796) Speed up repeated TokenStream init

[ https://issues.apache.org/jira/browse/LUCENE-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12742644#action_12742644 ] Uwe Schindler commented on LUCENE-1796: --- I opened LUCENE-1801 for that. A patch is a

[jira] Commented: (LUCENE-1796) Speed up repeated TokenStream init

[ https://issues.apache.org/jira/browse/LUCENE-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12742293#action_12742293 ] Yonik Seeley commented on LUCENE-1796: -- bq. But in principle we could also change the

[jira] Commented: (LUCENE-1796) Speed up repeated TokenStream init

[ https://issues.apache.org/jira/browse/LUCENE-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12741957#action_12741957 ] Uwe Schindler commented on LUCENE-1796: --- bq. I don't know if all of the Tokenizers i

[jira] Commented: (LUCENE-1796) Speed up repeated TokenStream init

[ https://issues.apache.org/jira/browse/LUCENE-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12741952#action_12741952 ] Yonik Seeley commented on LUCENE-1796: -- Token.clear() used to be called by the consum

[jira] Commented: (LUCENE-1796) Speed up repeated TokenStream init

[ https://issues.apache.org/jira/browse/LUCENE-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12741595#action_12741595 ] Michael Busch commented on LUCENE-1796: --- I think Token.reset() wasn't called before

[jira] Commented: (LUCENE-1796) Speed up repeated TokenStream init

[ https://issues.apache.org/jira/browse/LUCENE-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12741594#action_12741594 ] Uwe Schindler commented on LUCENE-1796: --- We had no conclusion on this. I think we sh

[jira] Commented: (LUCENE-1796) Speed up repeated TokenStream init

[ https://issues.apache.org/jira/browse/LUCENE-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12741590#action_12741590 ] Yonik Seeley commented on LUCENE-1796: -- bq. (I only removed the clearAttributes() cal

[jira] Commented: (LUCENE-1796) Speed up repeated TokenStream init

[ https://issues.apache.org/jira/browse/LUCENE-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12741571#action_12741571 ] Michael Busch commented on LUCENE-1796: --- {quote} I think I commit this now and leave

[jira] Commented: (LUCENE-1796) Speed up repeated TokenStream init

[ https://issues.apache.org/jira/browse/LUCENE-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12741567#action_12741567 ] Uwe Schindler commented on LUCENE-1796: --- The shorter the text, the more the construc

[jira] Commented: (LUCENE-1796) Speed up repeated TokenStream init

[ https://issues.apache.org/jira/browse/LUCENE-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12741564#action_12741564 ] Robert Muir commented on LUCENE-1796: - I just want to say I think that 10% test case m

[jira] Commented: (LUCENE-1796) Speed up repeated TokenStream init

[ https://issues.apache.org/jira/browse/LUCENE-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12741545#action_12741545 ] Mark Miller commented on LUCENE-1796: - Just to complete my report: The tests I report

[jira] Commented: (LUCENE-1796) Speed up repeated TokenStream init

[ https://issues.apache.org/jira/browse/LUCENE-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12741532#action_12741532 ] Mark Miller commented on LUCENE-1796: - And mine was a misreport - sorry - a wine progr

[jira] Commented: (LUCENE-1796) Speed up repeated TokenStream init

[ https://issues.apache.org/jira/browse/LUCENE-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12741525#action_12741525 ] Robert Muir commented on LUCENE-1796: - uwe in my case the latest patch performs approx

[jira] Commented: (LUCENE-1796) Speed up repeated TokenStream init

[ https://issues.apache.org/jira/browse/LUCENE-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12741524#action_12741524 ] Mark Miller commented on LUCENE-1796: - I was getting 46-47 with both of the first two

[jira] Commented: (LUCENE-1796) Speed up repeated TokenStream init

[ https://issues.apache.org/jira/browse/LUCENE-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12741523#action_12741523 ] Uwe Schindler commented on LUCENE-1796: --- Hm, and with the termAtt.clear() instead of

[jira] Commented: (LUCENE-1796) Speed up repeated TokenStream init

[ https://issues.apache.org/jira/browse/LUCENE-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12741520#action_12741520 ] Mark Miller commented on LUCENE-1796: - The latest patch appears to hurt the Solr use c

[jira] Commented: (LUCENE-1796) Speed up repeated TokenStream init

[ https://issues.apache.org/jira/browse/LUCENE-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12741514#action_12741514 ] Mark Miller commented on LUCENE-1796: - Nice work Uwe! > Speed up repeated TokenStream

[jira] Commented: (LUCENE-1796) Speed up repeated TokenStream init

[ https://issues.apache.org/jira/browse/LUCENE-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12741459#action_12741459 ] Uwe Schindler commented on LUCENE-1796: --- Ah, you are right! I will try this out. The

[jira] Commented: (LUCENE-1796) Speed up repeated TokenStream init

[ https://issues.apache.org/jira/browse/LUCENE-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12741457#action_12741457 ] Michael Busch commented on LUCENE-1796: --- You don't have to call captureState and clo

[jira] Commented: (LUCENE-1796) Speed up repeated TokenStream init

[ https://issues.apache.org/jira/browse/LUCENE-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12741447#action_12741447 ] Uwe Schindler commented on LUCENE-1796: --- I have another idea: Why not make the Attri

[jira] Commented: (LUCENE-1796) Speed up repeated TokenStream init

[ https://issues.apache.org/jira/browse/LUCENE-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12741445#action_12741445 ] Uwe Schindler commented on LUCENE-1796: --- But if you use the State and there is no st

[jira] Commented: (LUCENE-1796) Speed up repeated TokenStream init

[ https://issues.apache.org/jira/browse/LUCENE-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12741444#action_12741444 ] Michael Busch commented on LUCENE-1796: --- Another good cache, Uwe! :) AttributeSourc

[jira] Commented: (LUCENE-1796) Speed up repeated TokenStream init

[ https://issues.apache.org/jira/browse/LUCENE-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12741430#action_12741430 ] Robert Muir commented on LUCENE-1796: - Uwe, removal of the CharTokenizer clearAttribut

[jira] Commented: (LUCENE-1796) Speed up repeated TokenStream init

[ https://issues.apache.org/jira/browse/LUCENE-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12741416#action_12741416 ] Uwe Schindler commented on LUCENE-1796: --- Mark: The only hotspot could be initTokenWr

[jira] Commented: (LUCENE-1796) Speed up repeated TokenStream init

[ https://issues.apache.org/jira/browse/LUCENE-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12741411#action_12741411 ] Robert Muir commented on LUCENE-1796: - Uwe, your patch seems to help my large doc case

[jira] Commented: (LUCENE-1796) Speed up repeated TokenStream init

[ https://issues.apache.org/jira/browse/LUCENE-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12741408#action_12741408 ] Mark Miller commented on LUCENE-1796: - Yes indeed, its very close now. The filters ar

[jira] Commented: (LUCENE-1796) Speed up repeated TokenStream init

[ https://issues.apache.org/jira/browse/LUCENE-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12741404#action_12741404 ] Uwe Schindler commented on LUCENE-1796: --- If it is still a little bit slower (in my o

[jira] Commented: (LUCENE-1796) Speed up repeated TokenStream init

[ https://issues.apache.org/jira/browse/LUCENE-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12741402#action_12741402 ] Mark Miller commented on LUCENE-1796: - Sorry - to be a bit more clear: Lucene trunk w

[jira] Commented: (LUCENE-1796) Speed up repeated TokenStream init

[ https://issues.apache.org/jira/browse/LUCENE-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12741400#action_12741400 ] Uwe Schindler commented on LUCENE-1796: --- Mark: I do not understand your comment comp

[jira] Commented: (LUCENE-1796) Speed up repeated TokenStream init

[ https://issues.apache.org/jira/browse/LUCENE-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12741397#action_12741397 ] Robert Muir commented on LUCENE-1796: - here are my large doc numbers with the second p

[jira] Commented: (LUCENE-1796) Speed up repeated TokenStream init

[ https://issues.apache.org/jira/browse/LUCENE-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12741395#action_12741395 ] Mark Miller commented on LUCENE-1796: - No noticeable diff in the second patch for me.

[jira] Commented: (LUCENE-1796) Speed up repeated TokenStream init

[ https://issues.apache.org/jira/browse/LUCENE-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12741393#action_12741393 ] Mark Miller commented on LUCENE-1796: - Test on the first patch: Almost brings things

[jira] Commented: (LUCENE-1796) Speed up repeated TokenStream init

[ https://issues.apache.org/jira/browse/LUCENE-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12741391#action_12741391 ] Robert Muir commented on LUCENE-1796: - ok lemme restart my benchmark... the performanc

[jira] Commented: (LUCENE-1796) Speed up repeated TokenStream init

[ https://issues.apache.org/jira/browse/LUCENE-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12741382#action_12741382 ] Uwe Schindler commented on LUCENE-1796: --- bq. is there a way to optimize clear() in a

[jira] Commented: (LUCENE-1796) Speed up repeated TokenStream init

[ https://issues.apache.org/jira/browse/LUCENE-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12741373#action_12741373 ] Robert Muir commented on LUCENE-1796: - unrelated to TokenStream init, but what appears

[jira] Commented: (LUCENE-1796) Speed up repeated TokenStream init

[ https://issues.apache.org/jira/browse/LUCENE-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12741362#action_12741362 ] Robert Muir commented on LUCENE-1796: - me too (not a real benchmark but i think averag

[jira] Commented: (LUCENE-1796) Speed up repeated TokenStream init

[ https://issues.apache.org/jira/browse/LUCENE-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12741360#action_12741360 ] Mark Miller commented on LUCENE-1796: - I've got my test environ all setup, so I'll be