[jira] [Commented] (LUCENE-2834) don't spawn thread statically in FSDirectory on Mac OS X
[ https://issues.apache.org/jira/browse/LUCENE-2834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13039264#comment-13039264 ] Uwe Schindler commented on LUCENE-2834: --- Hi Mike, from my point of view, this seems fine. We broke the MD5 already by forcing UTF-8 instead of the default character set, so we can even change the whole hashing :-) Collisions are not really an issue, this lock file naming is only used for lock files outside index directory to make the chance of file name collisions less. Normal users will store lock files in index directory, so this is not really an issue (lock prefix is null for this case). Alternatively, to prevent collision, instead of hashing we could also collect all characters that are valid for file names (remove all non-ASCII chars and '/'). But this could produce a too-long filename. don't spawn thread statically in FSDirectory on Mac OS X Key: LUCENE-2834 URL: https://issues.apache.org/jira/browse/LUCENE-2834 Project: Lucene - Java Issue Type: Bug Reporter: Robert Muir Fix For: 4.0 Attachments: LUCENE-2834.patch, LUCENE-2834.patch, LUCENE-2834.patch, LUCENE-2834.patch on the Mac, creating the digester starts up a PKCS11 thread. I do not think threads should be created statically (I have this same issue with TimeLimitedCollector and also FilterManager). Uwe discussed removing this md5 digester, I don't care if we remove it or not, just as long as it doesn't create a thread, and just as long as it doesn't use the system default locale. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Commented: (LUCENE-2834) don't spawn thread statically in FSDirectory on Mac OS X
[ https://issues.apache.org/jira/browse/LUCENE-2834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13004179#comment-13004179 ] Michael McCandless commented on LUCENE-2834: I just hit this (got spooky leftover thread warning running test on OS X). I think we should fix it. I like the initial approach: let's not use MessageDigest at all (import our own MD5, that does not spawn threads!!). Sure it's code dup but it's tiny and it mitigates risk so I think it's well worth it. In general Lucene should not use interesting (risky) parts of the JVM/Java if we can avoid it w/o much cost, and this is a really silly reason to be using MessageDigest (similar to our now-gone crazy usage of ManagementFactory just to acquire a test lock). We are a search library! We must use the bare minimum of the OS/filesystem/JVM that we need. In fact in this case... can't we nuke DIGESTER altogether? Lucene now stores lock files in the index dir by default as write.lock. We only need this digest if you change that dir. So, if your app somehow wants to put the lock file elsewhere (unusual), it should be up to you to name it uniquely relative to other IWs storing locks in the same dir (we can do this under a separate issue). And not using SecureRandom to create temp files is a no-brainer -- the builtin File.createTempFile must be secure, by design, but we obviously don't need that here. I've had awful problems in the past w/ SecureRandom (because my machine didn't have enough true randomness!). Again: Lucene should only use what's we really need. I think we can remove the controversial interrupt the weird OS X PKCS11 thread from the patch since serialization is now gone? I'd like to know if this thread suddenly pops up again in our tests... and I agree it's dangerous to interrupt this thread (it could then cause weird failures in subsequent tests, eg if the thread doesn't restart). bq. One thing: I don't like the empty catch blocks /* cannot happen */. Even if this is the case, please throw at least a RuntimException +1 -- I like this idea (I don't do it now but I'll try to going forward). Defensive programming... don't spawn thread statically in FSDirectory on Mac OS X Key: LUCENE-2834 URL: https://issues.apache.org/jira/browse/LUCENE-2834 Project: Lucene - Java Issue Type: Bug Reporter: Robert Muir Fix For: 4.0 Attachments: LUCENE-2834.patch, LUCENE-2834.patch on the Mac, creating the digester starts up a PKCS11 thread. I do not think threads should be created statically (I have this same issue with TimeLimitedCollector and also FilterManager). Uwe discussed removing this md5 digester, I don't care if we remove it or not, just as long as it doesn't create a thread, and just as long as it doesn't use the system default locale. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Commented: (LUCENE-2834) don't spawn thread statically in FSDirectory on Mac OS X
[ https://issues.apache.org/jira/browse/LUCENE-2834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12975228#action_12975228 ] Uwe Schindler commented on LUCENE-2834: --- I forgot to mention: Patch not yet tested, as I first need to boot my Snow Leopard VirtualBOX and install SVN there. Maybe someone else can check this patch. don't spawn thread statically in FSDirectory on Mac OS X Key: LUCENE-2834 URL: https://issues.apache.org/jira/browse/LUCENE-2834 Project: Lucene - Java Issue Type: Bug Reporter: Robert Muir Fix For: 3.1, 4.0 Attachments: LUCENE-2834.patch, LUCENE-2834.patch on the Mac, creating the digester starts up a PKCS11 thread. I do not think threads should be created statically (I have this same issue with TimeLimitedCollector and also FilterManager). Uwe discussed removing this md5 digester, I don't care if we remove it or not, just as long as it doesn't create a thread, and just as long as it doesn't use the system default locale. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Commented: (LUCENE-2834) don't spawn thread statically in FSDirectory on Mac OS X
[ https://issues.apache.org/jira/browse/LUCENE-2834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12975245#action_12975245 ] Uwe Schindler commented on LUCENE-2834: --- This thread is a system thread and has nothing to do with Lucene. Killing it is something you should not do in a running JVM. To remove it from the exclusion list, just get a MessageDigest or somehow else access the PKCS library before you build the system thread map (the static while loop in LTC). Sorry, this time I don't agree with you and will revert all commits like that! don't spawn thread statically in FSDirectory on Mac OS X Key: LUCENE-2834 URL: https://issues.apache.org/jira/browse/LUCENE-2834 Project: Lucene - Java Issue Type: Bug Reporter: Robert Muir Fix For: 3.1, 4.0 Attachments: LUCENE-2834.patch, LUCENE-2834.patch on the Mac, creating the digester starts up a PKCS11 thread. I do not think threads should be created statically (I have this same issue with TimeLimitedCollector and also FilterManager). Uwe discussed removing this md5 digester, I don't care if we remove it or not, just as long as it doesn't create a thread, and just as long as it doesn't use the system default locale. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Commented: (LUCENE-2834) don't spawn thread statically in FSDirectory on Mac OS X
[
https://issues.apache.org/jira/browse/LUCENE-2834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12975251#action_12975251
]
Robert Muir commented on LUCENE-2834:
-
{quote}
One thing: I don't like the empty catch blocks /* cannot happen */. Even if
this is the case, please throw at least a RuntimException. Some user may stiull
use a broken JVM where the charsets.jar file is lost (ok, that should not
happen, but I like it more to have that).
{quote}
Your comment makes no sense, unlike the MD5 case, UTF-8 support is guaranteed
in all versions of java.
Every implementation of the Java platform is required to support the following
standard charsets.
...
UTF-8 Eight-bit UCS Transformation Format
I think you are somehow confused about things that are mandatory to be
supported in java,
and things that only *happen to exist in all sun jvms*.
don't spawn thread statically in FSDirectory on Mac OS X
Key: LUCENE-2834
URL: https://issues.apache.org/jira/browse/LUCENE-2834
Project: Lucene - Java
Issue Type: Bug
Reporter: Robert Muir
Fix For: 2.9.5, 3.0.4, 3.1, 4.0
Attachments: LUCENE-2834.patch, LUCENE-2834.patch
on the Mac, creating the digester starts up a PKCS11 thread.
I do not think threads should be created statically (I have this same issue
with TimeLimitedCollector and also FilterManager).
Uwe discussed removing this md5 digester, I don't care if we remove it or
not, just as long as it doesn't create a thread,
and just as long as it doesn't use the system default locale.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
