[jira] [Updated] (LUCENENET-480) Investigate what needs to happen to make both .NET 3.5 and 4.0 builds possible
[ https://issues.apache.org/jira/browse/LUCENENET-480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prescott Nasser updated LUCENENET-480: -- Attachment: SortedSet.cs Investigate what needs to happen to make both .NET 3.5 and 4.0 builds possible -- Key: LUCENENET-480 URL: https://issues.apache.org/jira/browse/LUCENENET-480 Project: Lucene.Net Issue Type: Task Components: Lucene.Net Contrib, Lucene.Net Core, Lucene.Net Demo, Lucene.Net Test Affects Versions: Lucene.Net 2.9.4, Lucene.Net 2.9.4g, Lucene.Net 3.0.3 Reporter: Christopher Currens Attachments: SortedSet.cs We need to investigate what needs to be done with the source to be able to support both a .NET 3.5 and 4.0 build. There was some concern from at least one member of the community ([see here|http://mail-archives.apache.org/mod_mbox/lucene-lucene-net-dev/201202.mbox/%3C004b01cce111$871f4990$955ddcb0$@fr%3E]) that we've alienated some of our user base by making such a heavy jump from 2.0 to 4.0. There are major benefits to using 4.0 over the 2.0-based runtimes, specifically FAR superior memory management, particularly with the LOH, where Lucene.NET has had major trouble with in the past. Based on what has been done with Lucene.NET 3.0.3, we can't (easily) drop .NET 3.5/3.0 libraries and go back to 2.0. HashSetT and ISetT is used extensively in the code, that would be a major blocker to get done. I suppose it could be possible, but there hasn't been a whole lot of talk about what runtimes we intend to support. The big change between lucene 2.x and 3.x for java was also a change in runtimes, that allowed generics and enums to be used in the base code. We have a similar situation with the API (it's substantially different, with the addition of properties alone) for this next version of Lucene.NET, so I think it's reasonable to at least make a permanent move from 2.0 to 3.5, though that's only my opinion, hasn't been discussed with the committers. It seems that supporting 3.5 and 4.0 should be fairly easy, and is a decent compromise in supporting both a 2.0 and 4.0 runtime. At the very least, we should try our best to support it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (LUCENENET-480) Investigate what needs to happen to make both .NET 3.5 and 4.0 builds possible
[ https://issues.apache.org/jira/browse/LUCENENET-480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13393683#comment-13393683 ] Prescott Nasser commented on LUCENENET-480: --- I agree, I'd prefer to leave ISet, however, if most of hte ISetT x = new HashSetT, I can't modify hashset to inherit from my own ISet Implementation. So I can either, 1. rewrite hashset to implement ISet (and inherit HashSet) or we can change the code from ISet. Attached is my incomplete SortedSet - i found it from another implementation based on sortedlist. Investigate what needs to happen to make both .NET 3.5 and 4.0 builds possible -- Key: LUCENENET-480 URL: https://issues.apache.org/jira/browse/LUCENENET-480 Project: Lucene.Net Issue Type: Task Components: Lucene.Net Contrib, Lucene.Net Core, Lucene.Net Demo, Lucene.Net Test Affects Versions: Lucene.Net 2.9.4, Lucene.Net 2.9.4g, Lucene.Net 3.0.3 Reporter: Christopher Currens Attachments: SortedSet.cs We need to investigate what needs to be done with the source to be able to support both a .NET 3.5 and 4.0 build. There was some concern from at least one member of the community ([see here|http://mail-archives.apache.org/mod_mbox/lucene-lucene-net-dev/201202.mbox/%3C004b01cce111$871f4990$955ddcb0$@fr%3E]) that we've alienated some of our user base by making such a heavy jump from 2.0 to 4.0. There are major benefits to using 4.0 over the 2.0-based runtimes, specifically FAR superior memory management, particularly with the LOH, where Lucene.NET has had major trouble with in the past. Based on what has been done with Lucene.NET 3.0.3, we can't (easily) drop .NET 3.5/3.0 libraries and go back to 2.0. HashSetT and ISetT is used extensively in the code, that would be a major blocker to get done. I suppose it could be possible, but there hasn't been a whole lot of talk about what runtimes we intend to support. The big change between lucene 2.x and 3.x for java was also a change in runtimes, that allowed generics and enums to be used in the base code. We have a similar situation with the API (it's substantially different, with the addition of properties alone) for this next version of Lucene.NET, so I think it's reasonable to at least make a permanent move from 2.0 to 3.5, though that's only my opinion, hasn't been discussed with the committers. It seems that supporting 3.5 and 4.0 should be fairly easy, and is a decent compromise in supporting both a 2.0 and 4.0 runtime. At the very least, we should try our best to support it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (LUCENENET-480) Investigate what needs to happen to make both .NET 3.5 and 4.0 builds possible
[ https://issues.apache.org/jira/browse/LUCENENET-480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13393683#comment-13393683 ] Prescott Nasser edited comment on LUCENENET-480 at 6/18/12 6:30 AM: I agree, I'd prefer to leave ISetT. However, if most of the ISetT x = new HashSetT, I can't modify Hashset to inherit from my own ISet Implementation. So I can either rewrite hashset to implement ISet (and inherit HashSet) or we can change the code from ISet. Attached is my incomplete SortedSet - i found it from another implementation based on sortedlist. was (Author: pnasser): I agree, I'd prefer to leave ISet, however, if most of hte ISetT x = new HashSetT, I can't modify hashset to inherit from my own ISet Implementation. So I can either, 1. rewrite hashset to implement ISet (and inherit HashSet) or we can change the code from ISet. Attached is my incomplete SortedSet - i found it from another implementation based on sortedlist. Investigate what needs to happen to make both .NET 3.5 and 4.0 builds possible -- Key: LUCENENET-480 URL: https://issues.apache.org/jira/browse/LUCENENET-480 Project: Lucene.Net Issue Type: Task Components: Lucene.Net Contrib, Lucene.Net Core, Lucene.Net Demo, Lucene.Net Test Affects Versions: Lucene.Net 2.9.4, Lucene.Net 2.9.4g, Lucene.Net 3.0.3 Reporter: Christopher Currens Attachments: SortedSet.cs We need to investigate what needs to be done with the source to be able to support both a .NET 3.5 and 4.0 build. There was some concern from at least one member of the community ([see here|http://mail-archives.apache.org/mod_mbox/lucene-lucene-net-dev/201202.mbox/%3C004b01cce111$871f4990$955ddcb0$@fr%3E]) that we've alienated some of our user base by making such a heavy jump from 2.0 to 4.0. There are major benefits to using 4.0 over the 2.0-based runtimes, specifically FAR superior memory management, particularly with the LOH, where Lucene.NET has had major trouble with in the past. Based on what has been done with Lucene.NET 3.0.3, we can't (easily) drop .NET 3.5/3.0 libraries and go back to 2.0. HashSetT and ISetT is used extensively in the code, that would be a major blocker to get done. I suppose it could be possible, but there hasn't been a whole lot of talk about what runtimes we intend to support. The big change between lucene 2.x and 3.x for java was also a change in runtimes, that allowed generics and enums to be used in the base code. We have a similar situation with the API (it's substantially different, with the addition of properties alone) for this next version of Lucene.NET, so I think it's reasonable to at least make a permanent move from 2.0 to 3.5, though that's only my opinion, hasn't been discussed with the committers. It seems that supporting 3.5 and 4.0 should be fairly easy, and is a decent compromise in supporting both a 2.0 and 4.0 runtime. At the very least, we should try our best to support it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (LUCENENET-484) Some possibly major tests intermittently fail
[ https://issues.apache.org/jira/browse/LUCENENET-484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13395775#comment-13395775 ] Luc Vanlerberghe commented on LUCENENET-484: TestStressLocks and TestStressLocksNativeFSLockFactory fail intermittently due to an UnauthorizedAccessException in NativeFSLockFactory.Obtain In the code it is actually described that on Windows you can get intermittent Access Denied exceptions and the java version handles them properly. *BUT*: in java, the Access Denied raises an IOException, whereas in .Net this raises an UnauthorizedAccessException So the solution is to add an identical catch block for the UnauthorizedAccessException. This will cause the lock obtain to simply fail and the calling methods to retry. Some possibly major tests intermittently fail -- Key: LUCENENET-484 URL: https://issues.apache.org/jira/browse/LUCENENET-484 Project: Lucene.Net Issue Type: Bug Components: Lucene.Net Core, Lucene.Net Test Affects Versions: Lucene.Net 3.0.3 Environment: All Reporter: Christopher Currens Fix For: Lucene.Net 3.0.3 Attachments: Lucenenet-484-NativeFSLockFactory.patch, Lucenenet-484-WeakDictionary.patch, Lucenenet-484-WeakDictionaryTests.patch These tests will fail intermittently in Debug or Release mode, in the core test suite: # -Lucene.Net.Index:- #- -TestConcurrentMergeScheduler.TestFlushExceptions- # Lucene.Net.Store: #- TestLockFactory.TestStressLocks # Lucene.Net.Search: #- TestSort.TestParallelMultiSort # -Lucene.Net.Util:- #- -TestFieldCacheSanityChecker.TestInsanity1- #- -TestFieldCacheSanityChecker.TestInsanity2- #- -(It's possible all of the insanity tests fail at one point or another)- # -Lucene.Net.Support- #- -TestWeakHashTableMultiThreadAccess.Test- TestWeakHashTableMultiThreadAccess should be fine to remove along with the WeakHashTable in the Support namespace, since it's been replaced with WeakDictionary. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (LUCENENET-484) Some possibly major tests intermittently fail
[ https://issues.apache.org/jira/browse/LUCENENET-484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luc Vanlerberghe updated LUCENENET-484: --- Attachment: Lucenenet-484-NativeFSLockFactory.patch Fixes the intermittent failures in TestStressLocks and TestStressLocksNativeFSLockFactory Some possibly major tests intermittently fail -- Key: LUCENENET-484 URL: https://issues.apache.org/jira/browse/LUCENENET-484 Project: Lucene.Net Issue Type: Bug Components: Lucene.Net Core, Lucene.Net Test Affects Versions: Lucene.Net 3.0.3 Environment: All Reporter: Christopher Currens Fix For: Lucene.Net 3.0.3 Attachments: Lucenenet-484-NativeFSLockFactory.patch, Lucenenet-484-WeakDictionary.patch, Lucenenet-484-WeakDictionaryTests.patch These tests will fail intermittently in Debug or Release mode, in the core test suite: # -Lucene.Net.Index:- #- -TestConcurrentMergeScheduler.TestFlushExceptions- # Lucene.Net.Store: #- TestLockFactory.TestStressLocks # Lucene.Net.Search: #- TestSort.TestParallelMultiSort # -Lucene.Net.Util:- #- -TestFieldCacheSanityChecker.TestInsanity1- #- -TestFieldCacheSanityChecker.TestInsanity2- #- -(It's possible all of the insanity tests fail at one point or another)- # -Lucene.Net.Support- #- -TestWeakHashTableMultiThreadAccess.Test- TestWeakHashTableMultiThreadAccess should be fine to remove along with the WeakHashTable in the Support namespace, since it's been replaced with WeakDictionary. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (LUCENENET-493) Make lucene.net culture insensitive (like the java version)
[ https://issues.apache.org/jira/browse/LUCENENET-493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13395818#comment-13395818 ] Luc Vanlerberghe commented on LUCENENET-493: Perhaps my comment was not sufficiently clear: I didn't solve a problem in the testcases, I solved a problem in Lucene.Net itself I happened to stumble upon the problem because I executed the tests in a non-US locale. The java version tests if Lucene behaves independently of the culture. Even if we can't duplicate the way this is tested, we should still make sure Lucene.net behaves independently of the culture too. When the patch is applied, as a side-effect, the tests also pass when run in another culture. Make lucene.net culture insensitive (like the java version) --- Key: LUCENENET-493 URL: https://issues.apache.org/jira/browse/LUCENENET-493 Project: Lucene.Net Issue Type: Bug Components: Lucene.Net Core, Lucene.Net Test Affects Versions: Lucene.Net 3.0.3 Reporter: Luc Vanlerberghe Labels: patch Fix For: Lucene.Net 3.0.3 Attachments: Lucenenet-493.patch In Java, conversion of the basic types to and from strings is locale (culture) independent. For localized input/output one needs to use the classes in the java.text package. In .Net, conversion of the basic types to and from strings depends on the default Culture. Otherwise you have to specify CultureInfo.InvariantCulture explicitly. Some of the testcases in lucene.net fail if they are not run on a machine with culture set to US. In the current version of lucene.net there are patches here and there that try to correct for some specific cases by using string replacement (like System.Double.Parse(s.Replace(., CultureInfo.CurrentCulture.NumberFormat.NumberDecimalSeparator)), but that seems really ugly. I submit a patch here that removes the old workarounds and replaces them by calls to classes in the Lucene.Net.Support namespace that try to handle the conversions in a compatible way. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (LUCENENET-495) Use of DateTime.Now causes huge amount of System.Globalization.DaylightTime object allocations
[
https://issues.apache.org/jira/browse/LUCENENET-495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Christopher Currens resolved LUCENENET-495.
---
Resolution: Fixed
Use of DateTime.Now causes huge amount of System.Globalization.DaylightTime
object allocations
--
Key: LUCENENET-495
URL: https://issues.apache.org/jira/browse/LUCENENET-495
Project: Lucene.Net
Issue Type: Bug
Components: Lucene.Net Core
Affects Versions: Lucene.Net 2.9.4, Lucene.Net 3.0.3
Reporter: Christopher Currens
Assignee: Christopher Currens
Priority: Critical
Fix For: Lucene.Net 3.0.3
This issue mostly just affects RAMDirectory. However, RAMFile and
RAMOutputStream are used in other (all?) directory implementations, including
FSDirectory types.
In RAMOutputStream, the file last modified property for the RAMFile is
updated when the stream is flushed. It's calculated using
{{DateTime.Now.Ticks / TimeSpan.TicksPerMillisecond}}. I've read before that
Microsoft has regretted making DateTime.Now a property instead of a method,
and after seeing what it's doing, I'm starting to understand why.
DateTime.Now is returning local time. In order for it to calculate that, it
has to get the utf offset for the machine, which requires the creation of a
_class_, System.Globalization.DaylightTime. This is bad for performance.
Using code to write 10,000 small documents to an index (4kb sizes), it
created 1,570,157 of these DaylightTime classes, a total of 62MB of extra
memory...clearly RAMOutputStream.Flush() is called a lot.
A fix I'd like to propose is to change the RAMFile from storing the
LastModified date to UTC instead of local. DateTime.UtcNow doesn't create
any additional objects and is very fast. For this small benchmark, the
performance increase is 31%.
I've set it to convert to local-time, when {{RAMDirectory.LastModified(string
name)}} is called to make sure it has the same behavior (tests fail
otherwise). Are there any other side-effects to making this change?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (LUCENENET-493) Make lucene.net culture insensitive (like the java version)
[
https://issues.apache.org/jira/browse/LUCENENET-493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Christopher Currens updated LUCENENET-493:
--
Attachment: UpdatedLocalizedTestCase.patch
I've updated LocalizedTestCase, so that it will actually run against all
installed cultures. The workaround is unfortunate...there is now only 1 test
that does all localization checks. It's done more or less the same way that
java does it, however, instead of being able to override the {{runBare()}}
method, I've created a test method that will run all methods in all installed
cultures. If a method were to fail, it would list the method name, and the
culture it failed in. I've attached a patch that shows the solution, so if
anyone has a better solution, we can discuss that and possibly use it instead.
Interestingly enough, the tests that Java Lucene has set to test, don't
actually fail when using the older code that doesn't have localization changes
in it. However, when I added {{TestBoost}} to the list in {{TestQueryParser}},
that one did fail before the push that Simon did. So, it concerns me that we
don't have enough tests written that actually will cause it to fail, when run
as a localized test.
So, what I propose we do before we apply Luc's patch, is to write tests that
*will fail* when using as a LocalizedTestCase and then make sure his patch
makes all of the tests pass.
Make lucene.net culture insensitive (like the java version)
---
Key: LUCENENET-493
URL: https://issues.apache.org/jira/browse/LUCENENET-493
Project: Lucene.Net
Issue Type: Bug
Components: Lucene.Net Core, Lucene.Net Test
Affects Versions: Lucene.Net 3.0.3
Reporter: Luc Vanlerberghe
Labels: patch
Fix For: Lucene.Net 3.0.3
Attachments: Lucenenet-493.patch, UpdatedLocalizedTestCase.patch
In Java, conversion of the basic types to and from strings is locale
(culture) independent. For localized input/output one needs to use the
classes in the java.text package.
In .Net, conversion of the basic types to and from strings depends on the
default Culture. Otherwise you have to specify CultureInfo.InvariantCulture
explicitly.
Some of the testcases in lucene.net fail if they are not run on a machine
with culture set to US.
In the current version of lucene.net there are patches here and there that
try to correct for some specific cases by using string replacement (like
System.Double.Parse(s.Replace(.,
CultureInfo.CurrentCulture.NumberFormat.NumberDecimalSeparator)), but that
seems really ugly.
I submit a patch here that removes the old workarounds and replaces them by
calls to classes in the Lucene.Net.Support namespace that try to handle the
conversions in a compatible way.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
