sv_setpvn buffer size limit???

[Zoltan Magyar]

Hi,

I'm working on a binary encryption library for perl. The library receives a
few parameters, does the encryption and returns the encrypted data. Before
the encryption takes place the required buffer size for the encrypted data
is calculated (calling EncryptData with NULL output pointer). When this
value (nOutDataLen) is calculated sv_setpvn is used to allocate the buffer
for the result. This call seems to be failing somehow. The context is

unsigned int
EncryptData(pRemoteKey,pLightData,nLightDataLen,pHeavyData,nHeavyDataLen)
 char*    pRemoteKey
 unsigned char* pLightData
 int    nLightDataLen
 unsigned char* pHeavyData
 int    nHeavyDataLen
 PPCODE:
 unsigned int nOutDataLen = 0;
 SV*    svOutData = newSVsv(&sv_undef);
 unsigned char* pOutData = NULL;
 unsigned int bRet = 0;

 /* make an initial call with a NULL buffer to get the output length */
 bRet = EncryptData(pRemoteKey,
       pLightData,
       nLightDataLen,
       pHeavyData,
       nHeavyDataLen,
       NULL,
       &nOutDataLen);

 /* calling with a NULL output buffer should always fail */
 if(bRet == 0)
  {
  /* set up our output buffer */
  sv_setpvn(svOutData, "", nOutDataLen);

  /* get the pointer to the buffer */
  pOutData = (unsigned char*)SvPV(svOutData, nOutDataLen);

  /* now try and create the blob */
  bRet = EncryptData(pRemoteKey,
        pLightData,
        nLightDataLen,
        pHeavyData,
        nHeavyDataLen,
        pOutData,
        &nOutDataLen);
  }
 /* push the return values onto the stack */
.
..
... etc


This code worked fine so far but it turned out recently that if the
calculated result length is too long the code is failing. I did some sanity
check and found that if nOutDataLen >= 7361 then the library crashes but
working fine if nOutDatalength <= 7360. According to the debug session
sv_setpvn is failing.

Interestingly the code is working fine when the compiler optimisations (used
/O1) are turned off.

I have used Win32 for the investigation and debugging but the library has
similar problems on Linux (I'm not sure the problem is exactly the same on
Linux as I haven't debugged it, but the symptoms are the same).

I did quite long search on the net about this issue but I haven't found any
similar (but found this list :-).

Any thoughts?

TIA

Zoltan Magyar