Re: [Firebird-devel] The IBatch Interface and Buffer Overflow

2021-09-20 Thread Alex Peshkoff via Firebird-devel
On 8/23/21 3:44 PM, Tony Whyman wrote: On 23/08/2021 13:03, Alex Peshkoff via Firebird-devel wrote: The minor bug is that if TAG_BUFFER_BYTES_SIZE is set to > 256MB no error status is reported on a call to createBatch. Instead, you only know you have a problem when an error is returned from IBa

Re: [Firebird-devel] The IBatch Interface and Buffer Overflow

2021-08-23 Thread Tony Whyman
On 23/08/2021 13:03, Alex Peshkoff via Firebird-devel wrote: The minor bug is that if TAG_BUFFER_BYTES_SIZE is set to > 256MB no error status is reported on a call to createBatch. Instead, you only know you have a problem when an error is returned from IBatch::add - looks like the buffer size i

Re: [Firebird-devel] The IBatch Interface and Buffer Overflow

2021-08-23 Thread Alex Peshkoff via Firebird-devel
On 8/21/21 6:26 PM, Tony Whyman wrote: The good news is that I believe I have found a way around the problem. The bad news is that I think I have found another minor bug. The workaround for variable message sizes is to compute the "used" buffer size after each call to IBatch::add by ensuring t

Re: [Firebird-devel] The IBatch Interface and Buffer Overflow

2021-08-21 Thread Tony Whyman
The good news is that I believe I have found a way around the problem. The bad news is that I think I have found another minor bug. The workaround for variable message sizes is to compute the "used" buffer size after each call to IBatch::add by ensuring that getAlignedLength() is refreshed for

Re: [Firebird-devel] The IBatch Interface and Buffer Overflow

2021-08-21 Thread Tony Whyman
On 20/08/2021 16:18, Alex Peshkoff via Firebird-devel wrote: On 8/20/21 4:01 PM, Tony Whyman wrote: You can specify such limit - use IBatch::TAG_BUFFER_BYTES_SIZE parameter in batch parameters block when creating a batch, default is 16Mb, hard limit - 256Mb. Ywo such buffers will be availab

Re: [Firebird-devel] The IBatch Interface and Buffer Overflow

2021-08-20 Thread Alex Peshkoff via Firebird-devel
On 8/20/21 4:01 PM, Tony Whyman wrote: .. With my test data, the buffer overflow occurs after several thousand calls to IBatch::add. The number of message buffers lost is of the order of hundreds. I don't want to have to keep a copy of every message buffer as that could result in other ou

[Firebird-devel] The IBatch Interface and Buffer Overflow

2021-08-20 Thread Tony Whyman
After adding support for the IBatch interface to IBX and then stress testing the implementation, I found that IBatch, when used with a remote server was silently losing data when the buffer size was exceeded. That issue has now been fixed. See - https://github.com/FirebirdSQL/firebird/issues/6