https://bz.apache.org/ooo/show_bug.cgi?id=128579

--- Comment #3 from dam...@apache.org ---
The bug has to do with how integer overflow behaves with different integer type
sizes.

Patching main/editeng/source/editeng/editdoc2.cxx to print out its parameters
and the computer "nEnd" value shows that, when we have:

static sal_uInt16 FastGetPos( const VoidPtr *pPtrArray, sal_uInt16
nPtrArrayLen,
                          VoidPtr pPtr, sal_uInt16 &rLastPos )

this is printed:

FastGetPos(0x899aca8d0, 1, 0x899c65710, 22)
  nEnd = 1

but when we have:

static sal_uInt16 FastGetPos( const VoidPtr *pPtrArray, sal_uInt32
nPtrArrayLen,
                          VoidPtr pPtr, sal_uInt16 &rLastPos )

this is printed instead:

FastGetPos(0x8951ef290, 1, 0x8951d6bf0, 22)
  nEnd = 24

Why? Because in FastGetPos() we do this:

---snip---
static sal_uInt16 FastGetPos( const VoidPtr *pPtrArray, sal_uInt32
nPtrArrayLen,
                          VoidPtr pPtr, sal_uInt16 &rLastPos )
{
  // Through certain filter code-paths we do a lot of appends, which in
  // turn call GetPos - creating some N^2 nightmares. If we have a
  // non-trivially large list, do a few checks from the end first.
  if( rLastPos > 16 )
    {
      sal_uInt16 nEnd;
      if (rLastPos > nPtrArrayLen - 2)
        nEnd = nPtrArrayLen;
      else
        nEnd = rLastPos + 2;

      for( sal_uInt16 nIdx = rLastPos - 2; nIdx < nEnd; nIdx++ )
        {
          if( pPtrArray[ nIdx ] == pPtr )
            {
              rLastPos = nIdx;
              return nIdx;
            }
        }
    }
  // The world's lamest linear search from svarray ...
  for( sal_uInt16 nIdx = 0; nIdx < nPtrArrayLen; nIdx++ )
    if (pPtrArray[ nIdx ] == pPtr )
      return rLastPos = nIdx;
  return USHRT_MAX;
}
---snip---

In both those tests, nPtrArrayLen was 1, but in the 16 bit nPtrArrayLen case,
nEnd == 1, which means the "if (rLastPos > nPtrArrayLen - 2)" was true, whereas
in the 32 bit nPtrArrayLen case, nEnd == 24, which means that "if" was false.

Even in that "for" loop later, rLastPos - 2 could wrap around to the maximum
integer value, if rLastPos is 0 or 1. And this was a problem even in the old
code!

That whole FastGetPos() function sucks and should be rewritten.

-- 
You are receiving this mail because:
You are the assignee for the issue.

Reply via email to