NightOwl888 commented on issue #271: URL: https://github.com/apache/lucenenet/issues/271#issuecomment-2519228615
> I don't have a solution to that failing test As [pointed out above](https://github.com/apache/lucenenet/issues/271#issuecomment-973005744), the backported test failed in Java as well. > And after translating it to Java, **it still fails** However, IIRC there were still some problems with managing the state of the `TokenStream` where the execution paths diverge between .NET and Java. when stepping through. I think it had something to do with whether `WordDelimiterFlags.CATENATE_NUMBERS` is included in the test, which was controlled by another condition (this may have been fixed in a later version of Lucene). > I discovered this is primarily due to the fact that some flags are not supported during indexing: https://stackoverflow.com/a/66478863/, and confirmed that removing the `WordDelimiterFlags.CATENATE_NUMBERS` flag makes the test pass. > > In `TestRandomChains`, `WordDelimiterFilter` is added to the `brokenOffsetsConstructors` dictionary. This causes [`offsetsAreCorrect` to be set to `true`](https://github.com/apache/lucenenet/blob/03bc03bb5bcb74d12524cc9c0d85e501473a4445/src/Lucene.Net.Tests.Analysis.Common/Analysis/Core/TestRandomChains.cs#L1028), which [`BaseTokenStreamTestCase.CheckAnalysisConsistency`](https://github.com/apache/lucenenet/blob/03bc03bb5bcb74d12524cc9c0d85e501473a4445/src/Lucene.Net.TestFramework/Analysis/BaseTokenStreamTestCase.cs#L1014) is supposed to handle. `BaseTokenStreamTestCase` has been changed from its original implementation to include try-finally and using blocks (both in [`CheckAnalysisConsistency()`](https://github.com/apache/lucenenet/blob/03bc03bb5bcb74d12524cc9c0d85e501473a4445/src/Lucene.Net.TestFramework/Analysis/BaseTokenStreamTestCase.cs#L1038) and in [`AssertTokenStreamContents()`](https://github.com/apache/lucenenet/blob/03bc03bb5bcb74d12524cc9c0d85e501473a4445/src/Lucene .Net.TestFramework/Analysis/BaseTokenStreamTestCase.cs#L144-L148) and that is where we are running into trouble. > > Long story short, this circles back to the fact that we converted the `Closable` interface in Java to `IDisposable` in .NET on `TokenStream`. But `TokenStream`'s `close()` method has preconditions that must be met or it will throw an exception. The exception doesn't play well when you are trying to close open `TextReader`s successfully. Before these finally blocks were put into place to call `Dispose()` one failing analysis test would cause nearly all of them to fail. There is some context that I failed to document between the last two paragraphs, but it shouldn't be too hard to figure out why I thought this was failing due to the addition of the try-finally blocks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@lucenenet.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
