[I] ArrayIndexOutOfBoundsException in ByteBlockPool [lucenenet]

[via GitHub]

hidingmyname opened a new issue, #1003:
URL: https://github.com/apache/lucenenet/issues/1003

   ### Is there an existing issue for this?
   
   - [X] I have searched the existing issues
   
   ### Describe the bug
   
   A field with a very large number of small tokens can cause 
ArrayIndexOutOfBoundsException in ByteBlockPool due to an arithmetic overflow.
   
   The issue was originally reported in 
Lucene(https://issues.apache.org/jira/browse/LUCENE-8614 and 
https://issues.apache.org/jira/browse/LUCENE-10441), where an arithmetic 
overflow occurs in the byteOffset calculation when BytesBlockPool advances to 
the next buffer on the last line of the 
[nextBuffer()](https://github.com/apache/lucenenet/blob/7c368026c40ddd30c674739296bb63085b112fa6/src/Lucene.Net/Util/ByteBlockPool.cs#L255)
 method.
   
   Although the statuses of the two issue reports from Lucene remain open, the 
developers have, in fact, resolved this issue through 
[PR](https://github.com/apache/lucene/pull/12392/files).
   
   The resolution in Lucene involves using `Math.addExact` to throw an 
ArithmeticException when the offset overflows in a ByteBlockPool. The fix code 
in ByteBlockPool as below:
   ```
   - byteOffset += BYTE_BLOCK_SIZE;
   + byteOffset = Math.addExact(byteOffset, BYTE_BLOCK_SIZE);
   ```
   
   A [test 
case](https://github.com/apache/lucene/blob/937432acd8d0483604fc144bc97e423dd2df7156/lucene/core/src/test/org/apache/lucene/util/TestByteBlockPool.java#L140)
 is presented in the Lucene repo. I have migrated this test case to Lucene.Net 
version and the test fails. See **Steps To Reproduce**.
   
   ### Expected Behavior
   
   Throw an ArithmeticException when the offset overflows in a ByteBlockPool.
   
   ### Steps To Reproduce
   
   The migrated test case is provided as below:
   ```
   using System;
   using System.Collections.Generic;
   using System.Linq;
   using System.Threading.Tasks;
   using Lucene.Net.Analysis;
   using Lucene.Net.Analysis.Core;
   using Lucene.Net.Analysis.Standard;
   using Lucene.Net.QueryParsers.Simple;
   using Lucene.Net.Search;
   using Lucene.Net.Util;
   using NUnit.Framework;
   
   namespace TestProject1
   {
       [TestFixture]
       public class Test_1
       {
           [Test]
           public void TestTooManyAllocs()
           {
               // Use a mock allocator that doesn't waste memory
               ByteBlockPool pool = new ByteBlockPool(new MockAllocator(0));
               pool.NextBuffer();
   
               bool throwsException = false;
               int maxIterations = int.MaxValue / ByteBlockPool.BYTE_BLOCK_SIZE 
+ 1;
   
               for (int i = 0; i < maxIterations; i++)
               {
                   try
                   {
                       pool.NextBuffer();
                   }
                   catch (OverflowException)
                   {
                       // The offset overflows on the last attempt to call 
NextBuffer()
                       throwsException = true;
                       break;
                   }
               }
   
               Assert.That(throwsException, Is.True);
               Assert.That(pool.ByteOffset + ByteBlockPool.BYTE_BLOCK_SIZE < 
pool.ByteOffset, Is.True);
           }
   
           private class MockAllocator : ByteBlockPool.Allocator
           {
               private readonly byte[] buffer;
   
               public MockAllocator(int blockSize) : base(blockSize)
               {
                   buffer = Array.Empty<byte>();
               }
   
               public override void RecycleByteBlocks(byte[][] blocks, int 
start, int end)
               {
                   // No-op
               }
   
               public override byte[] GetByteBlock()
               {
                   return buffer;
               }
           }
       }
   }
   ```
   
   ### Exceptions (if any)
   
   Assert.That(throwsException, Is.True)
     Expected: True
     But was:  False
   
   ### Lucene.NET Version
   
   4.8.0-beta00016
   
   ### .NET Version
   
   8.0.403
   
   ### Operating System
   
   Windows 10
   
   ### Anything else?
   
   _No response_


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@lucenenet.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org