NightOwl888 commented on code in PR #1058: URL: https://github.com/apache/lucenenet/pull/1058#discussion_r1912481597
########## src/Lucene.Net.Memory/MemoryIndex.cs: ########## @@ -275,23 +284,45 @@ public virtual TokenStream KeywordTokenStream<T>(ICollection<T> keywords) return new TokenStreamAnonymousClass<T>(keywords); } + /// <summary> + /// An anonymous implementation of <see cref="TokenStream"/> for + /// <see cref="KeywordTokenStream{T}(ICollection{T})"/>. Review Comment: Please change the name of the method to `GetKeywordTokenStream()`. This can be done in a separate PR, if preferred. ########## src/Lucene.Net.Memory/MemoryIndex.cs: ########## @@ -242,17 +242,26 @@ public virtual void AddField(string fieldName, string text, Analyzer analyzer) throw new ArgumentNullException(nameof(analyzer), "analyzer must not be null"); // LUCENENET specific - changed from IllegalArgumentException to ArgumentNullException (.NET convention) } - TokenStream stream; + // LUCENENET specific: dispose of the TokenStream when done here, instead of in AddField Review Comment: Since the `AddField(string fieldName, TokenStream stream, float boost, int positionIncrementGap, int offsetGap)` overload starts the `try` block on the very first line, even before guard clauses, and calls `Close()` in a finally block, this is redundant and will always call `Close()` twice. If `GetTokenStream()` fails in any way, an exception is raised before the `TokenStream` is assigned, so there is nothing to close. But I see the reasoning, though. I think it would make sense to only run the guard clauses in `AddField(string fieldName, TokenStream stream, float boost, int positionIncrementGap, int offsetGap)` and to add an `AddFieldImpl()` method that does the real work that is called from both methods. Then calls would only go through the guard clauses once and if a guard clause throws, it wouldn't get wrapped into a `LuceneSystemException`, which could be confusing to .NET users. It doesn't seem logical that one overload would allow `ArgumentNullException` to be thrown, and another to wrap it in `LuceneSystemException`, so I don't think this was done intentionally. ########## src/Lucene.Net.Tests.Analysis.Common/Analysis/Core/TestBugInSomething.cs: ########## @@ -261,13 +261,10 @@ public override void End() Console.WriteLine(m_input.GetType().Name + ".end()"); } - protected override void Dispose(bool disposing) + public override void Close() { - base.Dispose(disposing); - if (disposing) - { - Console.WriteLine(m_input.GetType().Name + ".close()"); - } + base.Close(); + Console.WriteLine(m_input.GetType().Name + ".close()"); Review Comment: Please change the casing of `.Close()` in the output. ########## src/Lucene.Net.Memory/MemoryIndex.cs: ########## @@ -275,23 +284,45 @@ public virtual TokenStream KeywordTokenStream<T>(ICollection<T> keywords) return new TokenStreamAnonymousClass<T>(keywords); } + /// <summary> + /// An anonymous implementation of <see cref="TokenStream"/> for + /// <see cref="KeywordTokenStream{T}(ICollection{T})"/>. + /// </summary> + /// <typeparam name="T">The type of item in the collection.</typeparam> + /// <remarks> + /// LUCENENET specific - This class originally got an enumerator in the constructor and stored it to a field + /// that was never reset, which meant that it could not be reused (since most IEnumerator implementations can + /// only be iterated once and throw on <see cref="System.Collections.IEnumerator.Reset()"/>). This class has Review Comment: Where did you find that info? I checked the implementations of `HashSet<T>` and `List<T>` and both of the enumerators will reset to the beginning rather than throw. It seems like if we are going to change this, we should do it in the true spirit of reuse - that is, don't reallocate an enumerator instance upon `Reset()` unless it is absolutely required. ### Alternative 1 Only support collections that support `Reset()`. The user can wrap the implementation if it doesn't support `Reset()` and decide how to implement it themselves. ### Alternative 2 Add a parameter to the `KeywordTokenStream()` method that allows the user to specify whether to allocate a new enumerator or cascade the call to `iter.Reset()`. ### Alternative 3 On the first call to `Reset()`, attempt to call `iter.Reset()` and if it throws, set a flag and allocate on each call to `Reset()`. If the flag is not set, always call `iter.Reset()` instead of allocating. This could fail if we are passed an `IEnumerator<T>` with an empty implementation of `Reset()`, but would cover most cases. ########## src/Lucene.Net.Tests.Analysis.Stempel/Pl/TestPolishAnalyzer.cs: ########## @@ -71,13 +72,13 @@ public void TestRandomStrings() } /// <summary> - /// LUCENENET specific. The original Java implementation relied on String.subSequence(int, int) to throw an IndexOutOfBoundsException - /// (in .NET, it would be string.SubString(int, int) and an ArgumentOutOfRangeException). - /// However, the logic was corrected for .NET to test when the argument is negative and not + /// LUCENENET specific. The original Java implementation relied on String.subSequence(int, int) to throw an IndexOutOfBoundsException + /// (in .NET, it would be string.SubString(int, int) and an ArgumentOutOfRangeException). + /// However, the logic was corrected for .NET to test when the argument is negative and not /// throw an exception, since exceptions are expensive and not meant for "normal" /// behavior in .NET. This test case was made trying to figure out that issue (since initially an IndexOutOfRangeException, - /// rather than ArgumentOutOfRangeException, was in the catch block which made the TestRandomStrings test fail). - /// It will trigger the behavior that cause the second substring argument to be negative + /// rather than ArgumentOutOfRangeException, was in the catch block which made the TestRandomStrings test fail). + /// It will trigger the behavior that cause the second substring argument to be negative /// (although that behavior no longer throws an exception). /// </summary> [Test] Review Comment: Please add a `LuceneNetSpecific` attribute here. ########## src/Lucene.Net.Tests.Highlighter/Highlight/HighlighterTest.cs: ########## @@ -1305,6 +1305,7 @@ public void TestGetTextFragments() for (int i = 0; i < hits.TotalHits; i++) { String text = searcher.Doc(hits.ScoreDocs[i].Doc).Get(FIELD_NAME); + // TODO: #271 review disposable Review Comment: Since this is the fix for #271, this comment seems like it wasn't intentional. But if it was, please make it `LUCENENET TODO`. ########## src/Lucene.Net.Spatial/Prefix/PrefixTreeStrategy.cs: ########## @@ -157,17 +158,28 @@ public virtual Field[] CreateIndexableFields(IShape? shape, double distErr) IndexOptions = IndexOptions.DOCS_ONLY }.Freeze(); - /// <summary>Outputs the tokenString of a cell, and if its a leaf, outputs it again with the leaf byte.</summary> + /// <summary> + /// Outputs the tokenString of a cell, and if its a leaf, outputs it again with the leaf byte. + /// </summary> + /// <remarks> + /// LUCENENET specific - This class originally took an enumerator, which meant that it could not + /// be reused (since most IEnumerator implementations can only be iterated once and throw on Review Comment: See my comments in `MemoryIndex`. Whatever option we end up with should probably be the same used in each of these cases. ########## src/Lucene.Net.TestFramework/Analysis/CrankyTokenFilter.cs: ########## @@ -69,15 +69,12 @@ public override void Reset() } } - protected override void Dispose(bool disposing) + public override void Close() { - base.Dispose(disposing); - if (disposing) + base.Close(); + if (thingToDo == 3 && random.nextBoolean()) { - if (thingToDo == 3 && random.nextBoolean()) - { - throw new IOException("Fake IOException from TokenStream.Dispose(bool)"); - } + throw new IOException("Fake IOException from TokenStream.DoClose()"); Review Comment: Maybe we should use string interpolation here to ensure the method name stays up to date when it is changed. `DoClose()` has been removed, so this should stay current with `TokenStream` (and changed to `Close()`). ########## src/Lucene.Net.Tests.Highlighter/Highlight/HighlighterTest.cs: ########## @@ -2159,21 +2160,12 @@ public override void End() } - protected override void Dispose(bool disposing) + public override void Close() { - try - { - if (disposing) - { - this.realStream.Dispose(); - this.st?.Dispose(); - this.st = null; - } - } - finally - { - base.Dispose(disposing); - } + this.realStream.Close(); + this.st?.Dispose(); + this.st = null; + base.Close(); Review Comment: In the upstream code, `realStream` is closed after `base.close()`. Not sure whether that matters, but I suspect adding the try/finally block was the only reason why it was moved. https://github.com/apache/lucene/blob/releases/lucene-solr/4.8.1/lucene/highlighter/src/test/org/apache/lucene/search/highlight/HighlighterTest.java#L2049-L2053 ########## src/Lucene.Net/Util/IOUtils.cs: ########## @@ -82,66 +140,107 @@ public static class IOUtils // LUCENENET specific - made static /// </para> /// </summary> /// <param name="priorException"> <c>null</c> or an exception that will be rethrown after method completion. </param> - /// <param name="objects"> Objects to call <see cref="IDisposable.Dispose()"/> on. </param> - [Obsolete("Use DisposeWhileHandlingException(Exception, params IDisposable[]) instead.")] - public static void CloseWhileHandlingException(Exception priorException, params IDisposable[] objects) + /// <param name="objects"> Objects to call <see cref="ICloseable.Close()"/> on. </param> + public static void CloseWhileHandlingException(Exception priorException, params ICloseable[] objects) { - DisposeWhileHandlingException(priorException, objects); - } + Exception th = null; - /// <summary> - /// Disposes all given <see cref="IDisposable"/>s, suppressing all thrown exceptions. </summary> - /// <seealso cref="DisposeWhileHandlingException(Exception, IDisposable[])"/> - [Obsolete("Use DisposeWhileHandlingException(Exception, IEnumerable<IDisposable>) instead.")] - public static void CloseWhileHandlingException(Exception priorException, IEnumerable<IDisposable> objects) - { - DisposeWhileHandlingException(priorException, objects); + foreach (ICloseable @object in objects) + { + try + { + @object?.Close(); + } + catch (Exception t) when (t.IsThrowable()) + { + AddSuppressed(priorException ?? th, t); + if (th is null) + { + th = t; + } + } + } + + if (priorException != null) + { + ExceptionDispatchInfo.Capture(priorException).Throw(); // LUCENENET: Rethrow to preserve stack details from the original throw + } + else + { + ReThrow(th); + } } /// <summary> - /// Disposes all given <see cref="IDisposable"/>s. Some of the - /// <see cref="IDisposable"/>s may be <c>null</c>; they are - /// ignored. After everything is closed, the method either - /// throws the first exception it hit while closing, or - /// completes normally if there were no exceptions. + /// Closes all given <see cref="ICloseable"/>s, suppressing all thrown exceptions. /// </summary> - /// <param name="objects"> - /// Objects to call <see cref="IDisposable.Dispose()"/> on </param> - [Obsolete("Use Dispose(params IDisposable[]) instead.")] - public static void Close(params IDisposable[] objects) + /// <seealso cref="CloseWhileHandlingException(Exception, ICloseable[])"/> + public static void CloseWhileHandlingException(Exception priorException, IEnumerable<ICloseable> objects) { - Dispose(objects); - } + Exception th = null; - /// <summary> - /// Disposes all given <see cref="IDisposable"/>s. </summary> - /// <seealso cref="Dispose(IDisposable[])"/> - [Obsolete("Use Dispose(IEnumerable<IDisposable>) instead.")] - public static void Close(IEnumerable<IDisposable> objects) - { - Dispose(objects); + foreach (ICloseable @object in objects) + { + try + { + @object?.Close(); + } + catch (Exception t) when (t.IsThrowable()) + { + AddSuppressed(priorException ?? th, t); + if (th is null) + { + th = t; + } + } + } + + if (priorException != null) + { + ExceptionDispatchInfo.Capture(priorException).Throw(); // LUCENENET: Rethrow to preserve stack details from the original throw + } + else + { + ReThrow(th); + } } /// <summary> - /// Disposes all given <see cref="IDisposable"/>s, suppressing all thrown exceptions. - /// Some of the <see cref="IDisposable"/>s may be <c>null</c>, they are ignored. + /// Closes all given <see cref="ICloseable"/>s, suppressing all thrown exceptions. + /// Some of the <see cref="ICloseable"/>s may be <c>null</c>, they are ignored. /// </summary> - /// <param name="objects"> - /// Objects to call <see cref="IDisposable.Dispose()"/> on </param> - [Obsolete("Use DisposeWhileHandlingException(params IDisposable[]) instead.")] - public static void CloseWhileHandlingException(params IDisposable[] objects) + /// <param name="objects">Objects to call <see cref="ICloseable.Close()"/> on</param> + public static void CloseWhileHandlingException(params ICloseable[] objects) { - DisposeWhileHandlingException(objects); + foreach (ICloseable @object in objects) + { + try + { + @object?.Close(); + } + catch (Exception t) when (t.IsThrowable()) + { + } + } } /// <summary> - /// Disposes all given <see cref="IDisposable"/>s, suppressing all thrown exceptions. </summary> - /// <seealso cref="DisposeWhileHandlingException(IEnumerable{IDisposable})"/> - /// <seealso cref="DisposeWhileHandlingException(IDisposable[])"/> - [Obsolete("Use DisposeWhileHandlingException(IEnumerable<IDisposable>) instead.")] - public static void CloseWhileHandlingException(IEnumerable<IDisposable> objects) + /// Closes all given <see cref="ICloseable"/>s, suppressing all thrown exceptions. + /// </summary> + /// <seealso cref="CloseWhileHandlingException(IEnumerable{ICloseable})"/> + /// <seealso cref="CloseWhileHandlingException(ICloseable[])"/> + public static void CloseWhileHandlingException(IEnumerable<ICloseable> objects) { - DisposeWhileHandlingException(objects); + foreach (ICloseable @object in objects) + { + try + { + @object?.Close(); + } + catch (Exception t) when (t.IsThrowable()) + { + } Review Comment: Sonarcloud complains when we leave an empty set of brackets like this. Please use the comment from the original method: ```c# // eat it ``` ########## src/Lucene.Net.Tests.Analysis.Common/Analysis/Core/TestRandomChains.cs: ########## @@ -1150,7 +1150,7 @@ internal class CharFilterSpec } [Test] - [AwaitsFix(BugUrl = "https://github.com/apache/lucenenet/issues/271#issuecomment-973005744")] // LUCENENET TODO: this test occasionally fails + //[AwaitsFix(BugUrl = "https://github.com/apache/lucenenet/issues/271#issuecomment-973005744")] // LUCENENET TODO: this test occasionally fails Review Comment: Let's put AwaitsFix back, since the failure has been moved to #1072 ########## src/Lucene.Net/Analysis/Tokenizer.cs: ########## @@ -131,7 +120,7 @@ private sealed class ReaderAnonymousClass : TextReader { public override int Read(char[] cbuf, int off, int len) { - throw IllegalStateException.Create("TokenStream contract violation: Reset()/Dispose() call missing, " + throw IllegalStateException.Create("TokenStream contract violation: Reset()/Dispose() call missing, " Review Comment: Please change the string here to `Close()` instead of `Dispose()`, as the latter has nothing to do with the token stream contract now. ########## src/Lucene.Net/Util/IOUtils.cs: ########## @@ -82,66 +140,107 @@ public static class IOUtils // LUCENENET specific - made static /// </para> /// </summary> /// <param name="priorException"> <c>null</c> or an exception that will be rethrown after method completion. </param> - /// <param name="objects"> Objects to call <see cref="IDisposable.Dispose()"/> on. </param> - [Obsolete("Use DisposeWhileHandlingException(Exception, params IDisposable[]) instead.")] - public static void CloseWhileHandlingException(Exception priorException, params IDisposable[] objects) + /// <param name="objects"> Objects to call <see cref="ICloseable.Close()"/> on. </param> + public static void CloseWhileHandlingException(Exception priorException, params ICloseable[] objects) { - DisposeWhileHandlingException(priorException, objects); - } + Exception th = null; - /// <summary> - /// Disposes all given <see cref="IDisposable"/>s, suppressing all thrown exceptions. </summary> - /// <seealso cref="DisposeWhileHandlingException(Exception, IDisposable[])"/> - [Obsolete("Use DisposeWhileHandlingException(Exception, IEnumerable<IDisposable>) instead.")] - public static void CloseWhileHandlingException(Exception priorException, IEnumerable<IDisposable> objects) - { - DisposeWhileHandlingException(priorException, objects); + foreach (ICloseable @object in objects) + { + try + { + @object?.Close(); + } + catch (Exception t) when (t.IsThrowable()) + { + AddSuppressed(priorException ?? th, t); + if (th is null) + { + th = t; + } + } + } + + if (priorException != null) + { + ExceptionDispatchInfo.Capture(priorException).Throw(); // LUCENENET: Rethrow to preserve stack details from the original throw + } + else + { + ReThrow(th); + } } /// <summary> - /// Disposes all given <see cref="IDisposable"/>s. Some of the - /// <see cref="IDisposable"/>s may be <c>null</c>; they are - /// ignored. After everything is closed, the method either - /// throws the first exception it hit while closing, or - /// completes normally if there were no exceptions. + /// Closes all given <see cref="ICloseable"/>s, suppressing all thrown exceptions. /// </summary> - /// <param name="objects"> - /// Objects to call <see cref="IDisposable.Dispose()"/> on </param> - [Obsolete("Use Dispose(params IDisposable[]) instead.")] - public static void Close(params IDisposable[] objects) + /// <seealso cref="CloseWhileHandlingException(Exception, ICloseable[])"/> + public static void CloseWhileHandlingException(Exception priorException, IEnumerable<ICloseable> objects) { - Dispose(objects); - } + Exception th = null; - /// <summary> - /// Disposes all given <see cref="IDisposable"/>s. </summary> - /// <seealso cref="Dispose(IDisposable[])"/> - [Obsolete("Use Dispose(IEnumerable<IDisposable>) instead.")] - public static void Close(IEnumerable<IDisposable> objects) - { - Dispose(objects); + foreach (ICloseable @object in objects) + { + try + { + @object?.Close(); + } + catch (Exception t) when (t.IsThrowable()) + { + AddSuppressed(priorException ?? th, t); + if (th is null) + { + th = t; + } + } + } + + if (priorException != null) + { + ExceptionDispatchInfo.Capture(priorException).Throw(); // LUCENENET: Rethrow to preserve stack details from the original throw + } + else + { + ReThrow(th); + } } /// <summary> - /// Disposes all given <see cref="IDisposable"/>s, suppressing all thrown exceptions. - /// Some of the <see cref="IDisposable"/>s may be <c>null</c>, they are ignored. + /// Closes all given <see cref="ICloseable"/>s, suppressing all thrown exceptions. + /// Some of the <see cref="ICloseable"/>s may be <c>null</c>, they are ignored. /// </summary> - /// <param name="objects"> - /// Objects to call <see cref="IDisposable.Dispose()"/> on </param> - [Obsolete("Use DisposeWhileHandlingException(params IDisposable[]) instead.")] - public static void CloseWhileHandlingException(params IDisposable[] objects) + /// <param name="objects">Objects to call <see cref="ICloseable.Close()"/> on</param> + public static void CloseWhileHandlingException(params ICloseable[] objects) { - DisposeWhileHandlingException(objects); + foreach (ICloseable @object in objects) + { + try + { + @object?.Close(); + } + catch (Exception t) when (t.IsThrowable()) + { + } Review Comment: Sonarcloud complains when we leave an empty set of brackets like this. Please use the comment from the original method: ```c# //eat it ``` ########## src/Lucene.Net/Analysis/TokenStream.cs: ########## @@ -36,13 +37,13 @@ namespace Lucene.Net.Analysis /// A new <see cref="TokenStream"/> API has been introduced with Lucene 2.9. this API /// has moved from being <see cref="Token"/>-based to <see cref="Util.IAttribute"/>-based. While /// <see cref="Token"/> still exists in 2.9 as a convenience class, the preferred way - /// to store the information of a <see cref="Token"/> is to use <see cref="Attribute"/>s. + /// to store the information of a <see cref="Token"/> is to use <see cref="System.Attribute"/>s. /// <para/> /// <see cref="TokenStream"/> now extends <see cref="AttributeSource"/>, which provides /// access to all of the token <see cref="Util.IAttribute"/>s for the <see cref="TokenStream"/>. - /// Note that only one instance per <see cref="Attribute"/> is created and reused + /// Note that only one instance per <see cref="System.Attribute"/> is created and reused Review Comment: This reference is now incorrect. It should be `Lucene.Net.Util.Attribute`. ########## src/Lucene.Net/Analysis/TokenStream.cs: ########## @@ -36,13 +37,13 @@ namespace Lucene.Net.Analysis /// A new <see cref="TokenStream"/> API has been introduced with Lucene 2.9. this API /// has moved from being <see cref="Token"/>-based to <see cref="Util.IAttribute"/>-based. While /// <see cref="Token"/> still exists in 2.9 as a convenience class, the preferred way - /// to store the information of a <see cref="Token"/> is to use <see cref="Attribute"/>s. + /// to store the information of a <see cref="Token"/> is to use <see cref="System.Attribute"/>s. Review Comment: This reference is now incorrect. It should be `Lucene.Net.Util.Attribute`. ########## src/Lucene.Net.Benchmark/ByTask/Tasks/ReadTokensTask.cs: ########## @@ -87,6 +87,7 @@ field is SingleField || tokenCount++; } stream.End(); + stream.Close(); Review Comment: Let's use a finally block here so we will clean up whether there is an exception or not. ########## src/Lucene.Net.Tests.Analysis.Common/Analysis/Core/TestRandomChains.cs: ########## @@ -1177,7 +1177,7 @@ public void TestRandomChains_() // we might regret this decision... [Test] - [AwaitsFix(BugUrl = "https://github.com/apache/lucenenet/issues/271#issuecomment-973005744")] // LUCENENET TODO: this test occasionally fails + //[AwaitsFix(BugUrl = "https://github.com/apache/lucenenet/issues/271#issuecomment-973005744")] // LUCENENET TODO: this test occasionally fails Review Comment: Let's put AwaitsFix back, since the failure has been moved to #1072 ########## src/Lucene.Net/Analysis/TokenStream.cs: ########## @@ -36,13 +37,13 @@ namespace Lucene.Net.Analysis /// A new <see cref="TokenStream"/> API has been introduced with Lucene 2.9. this API /// has moved from being <see cref="Token"/>-based to <see cref="Util.IAttribute"/>-based. While /// <see cref="Token"/> still exists in 2.9 as a convenience class, the preferred way - /// to store the information of a <see cref="Token"/> is to use <see cref="Attribute"/>s. + /// to store the information of a <see cref="Token"/> is to use <see cref="System.Attribute"/>s. /// <para/> /// <see cref="TokenStream"/> now extends <see cref="AttributeSource"/>, which provides /// access to all of the token <see cref="Util.IAttribute"/>s for the <see cref="TokenStream"/>. - /// Note that only one instance per <see cref="Attribute"/> is created and reused + /// Note that only one instance per <see cref="System.Attribute"/> is created and reused /// for every token. This approach reduces object creation and allows local - /// caching of references to the <see cref="Attribute"/>s. See + /// caching of references to the <see cref="System.Attribute"/>s. See Review Comment: This reference is now incorrect. It should be `Lucene.Net.Util.Attribute`. ########## src/Lucene.Net/Analysis/Tokenizer.cs: ########## @@ -57,29 +59,16 @@ protected Tokenizer(AttributeFactory factory, TextReader input) } /// <summary> - /// Releases resources associated with this stream. - /// <para/> - /// If you override this method, always call <c>base.Dispose(disposing)</c>, otherwise - /// some internal state will not be correctly reset (e.g., <see cref="Tokenizer"/> will - /// throw <see cref="InvalidOperationException"/> on reuse). + /// <inheritdoc cref="TokenStream.Close()"/> /// </summary> - /// <remarks> - /// <b>NOTE:</b> Review Comment: Please add this note back, as was the case upstream. https://github.com/apache/lucene/blob/releases/lucene-solr/4.8.0/lucene/core/src/java/org/apache/lucene/analysis/Tokenizer.java#L60-L62 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@lucenenet.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org