Thanks for taking the time to read my dirty code.. I apologize if that
is too hard to read for you ... I'm not a good programmer yet... sorry!
So I ignored the zero und -1 returned by the Receive(), and it worked :)
But the "TempSize" in my code was never changed. it was set as to a
constant value en the constructor. Anyhow, as I was told, I want to
decrease the GetMem(), Free() as much as possible.. so I think I'll do
it before the first request, and free it after the last request... but
for reassembling the original stream, what do you think ? "consecutive
Realloc()" (First scenario) or "Array of pointer" (Second scenario)?
First scenario ------------------------------------------------------------
Line 11: in > txt 1561
// 1561 means 1561 widechars => 3123 bytes
// then GetMemTempBuffer, GetMem TotalReceiveBuffer, Let
sizeReceived=0, Let sizeLeft=3123
Line 12: out> sendtxt
Line 13: (Written 15 bytes)
Line 14: Preparing Temp Buffer...
Line 15: Received 1460 bytes, Remaining 3123 bytes
// GetMem TotalReceiveBuffer, TEMPSIZE(2048bytes)
// thisPacketSize = Receive(TempBuffer)
// MemCpy TempBuffer -> TotalReceiveBuffer
// sizeReceived = sizeReceived + thisPacketSize
Line 16: Received 1460 bytes, Remaining 1663 bytes
// Zero TempBuffer
// Let thisPacketSize = Receive(TempBuffer)
// Realloc(TotalReceiveBuffer, sizeReceived+thisPacketSize)
// MemCpy TempBuffer -> TotalReceiveBuffer+sizeReceived
// sizeReceived = sizeReceived + thisPacketSize
Line 17: Received 203 bytes, Remaining 203 bytes
// Zero TempBuffer
// Let thisPacketSize = Receive(TempBuffer)
// Realloc(TotalReceiveBuffer, sizeReceived+thisPacketSize)
// MemCpy TempBuffer -> TotalReceiveBuffer+sizeReceived
// sizeReceived = sizeReceived + thisPacketSize
Line 18: All packets finished.
// TriggerOnFinish(TotalReceiveBuffer, DONE_FLAG) or etc
Line 19: Redirecting Text Message...
Line 20: out> msg AliasName and DriverName a...(too long to display)
Line 21: (!!) Command: msg AliasName and DriverName are m...(too long to
display)
// then some more "big text reception"... until the other says say
Line n : in > nomore_msg
// FreeMem(TempBuffer)
second scenario ------------------------------------------------------------
Line 11: in > txt 1561
// 1561 means 1561 widechars => 3123 bytes
// Let packDone=0 Let sizeLeft=3123
// BufferArray = nil
// SizeArray = nil
Line 12: out> sendtxt
Line 13: (Written 15 bytes)
Line 14: Preparing Temp Buffer...
Line 15: Received 1460 bytes, Remaining 3123 bytes
// GetMem(ptrThisPack, PACK_MAXSIZE);
// intThisPackSize = Receive(ptrThisPack);
// packDone = packDone + 1;
// SetLength(BufferArray, packDone);
// SetLength(SizeArray, packDone);
// BufferArray[packDone] = ptrThisPack;
// SizeArray[packDone] = intThisPackSize;
Line 16: Received 1460 bytes, Remaining 1663 bytes
// GetMem(ptrThisPack, PACK_MAXSIZE);
// intThisPackSize = Receive(ptrThisPack);
// packDone = packDone + 1;
// SetLength(BufferArray, packDone);
// SetLength(SizeArray, packDone);
// BufferArray[packDone] = ptrThisPack;
// SizeArray[packDone] = intThisPackSize;
Line 17: Received 203 bytes, Remaining 203 bytes
// GetMem(ptrThisPack, PACK_MAXSIZE);
// intThisPackSize = Receive(ptrThisPack);
// packDone = packDone + 1;
// SetLength(BufferArray, packDone);
// SetLength(SizeArray, packDone);
// BufferArray[packDone] = ptrThisPack;
// SizeArray[packDone] = intThisPackSize;
Line 18: All packets finished.
// Getmem(ptrAltogether, intOriginalSize);
// for each BufferArray
// Append BufferArray[i] into ptrAltogether
// Free BufferArray[1], SizeArray[i]
// TriggerOnFinishCallback(ptrAltogether, DONE_FLAG) or etc
Line 19: Redirecting Text Message...
Line 20: out> msg AliasName and DriverName a...(too long to display)
Line 21: (!!) Command: msg AliasName and DriverName are m...(too long to
display)
Line n : in > nomore_msg
Thanks for reading this long posting!! But please tell me what you think, which
scenario is better?
David
Wilfried Mestdagh wrote:
>Hello David,
>
>
>
>> GetMem(TempBuffer, TempSize); // first packet
>>
>>
> <snip>
>
>
>> bytesReceived := Self.Receive(TempBuffer, ReceiveSize);
>>
>>
>
>In the part where you allocate mem for first packet ReceiveSize should
>be set to TempSize.
>
>Wy do you allocate anyway. Wy not put your buffer as an array on the
>stack ? Now you allocate / free data for every packet received. Not a
>good idea.
>
>You also clear all things when received count < 0. Not needed, when
>count <= 0 just exit the procedure.
>
>It is difficult to read because it is too complex. Please divide it into
>a few logic blocks. Also naming conventiosn. ReceiveStr is a method of
>TWSocket, please dont use these names for other things, specially if you
>are in an event of TWSocket. This make it very confusing.
>
>I try to check it later, but please try to simplify it yourself. In
>OnDataAvailable, jsut receive in a buffer on the stack. Then eahter pass
>a pointer to the buffer on a function called CommandInterpreter or
>ReceiveBinData or something (depending on the mode you are in).
>
>---
>Rgds, Wilfried [TeamICS]
>http://www.overbyte.be/eng/overbyte/teamics.html
>http://www.mestdagh.biz
>
>Tuesday, November 1, 2005, 10:32, Kei wrote:
>
>
>
>>I'm Sorry! I thought the code was too big to attach.. anyway here it is
>>My component "TSession" is a descendent from the TWCliSocket.
>>WaitStatus is a public property, that states what "state" the class is
>>currently at.
>>
>>
>
>
>
>>David
>>
>>
>
>
>
>>------------------------
>>procedure TSession.MyDataAvailable(Sender: TObject; ErrCode: Word);
>>var
>> bytesReceived: integer;
>> response, strReceived: WideString;
>> binReceive: pointer; // the pointer that will receive the buffer and
>>passed back to the user
>> chrReceived: array[0..MAX_BUF-1] of WideChar;
>> i: integer;
>>
>>
>
>
>
>> tempPtr: pointer;
>>
>>
>
>
>
>>begin
>> with (Sender as TSession) do
>> { note: for WaitSendFile, WaitSendChunk, we do not process them here.
>> they are to be done in ProcessCommand }
>> if(WaitStatus=wsGetChunk)or(WaitStatus=wsGetMsg)then
>> begin
>> { new version; solves the "multiple packet" problem }
>> if(TempBuffer=nil) then
>> begin
>>{$IFDEF VERBOSE}
>> consoleOut('Preparing Temp Buffer...');
>>{$ENDIF}
>> GetMem(TempBuffer, TempSize); // first packet
>> if(not assigned(ReceiveStr)) then ReceiveStr :=
>>TMemoryStream.Create;
>> ReceiveStr.Clear;
>> end;
>>
>>
>
>
>
>> ZeroMemory(TempBuffer, TempSize);
>>
>>
>
>
>
>> bytesReceived := Self.Receive(TempBuffer, ReceiveSize);
>>
>>
>
>
>
>>{$IFDEF VERBOSE}
>> consoleOut('Received '+inttostr(bytesReceived)+' bytes,
>>Remaining '+inttostr(ReceiveSize)+' bytes');
>>{$ENDIF}
>>
>>
>
>
>
>> if(bytesReceived>0) then
>> begin
>> { given that the transfer was a successfull one... }
>> dec(ReceiveSize, bytesReceived);
>> ReceiveStr.Write(TempBuffer^, bytesReceived);
>> { we can test whether we have finally done. }
>> {
>>------------------------------------------------------------------------
>> if(ReceiveSize<=0) then
>> begin
>>{$IFDEF VERBOSE}
>> consoleOut('All packets finished.');
>>{$ENDIF}
>> { we just finished the last packet }
>> FreeMem(TempBuffer);
>> TempBuffer := nil;
>> if(WaitStatus=wsGetChunk) then
>> begin
>> { we can now notify the user about it }
>> ReceiveStr.Seek(0, soFromBeginning);
>> tempPtr := ReceiveStr.Memory;
>> if(WideCompareStr(ReceiveHash, '0')<>0) then
>> begin
>> { verify checksum }
>> if( GetMD5(ReceiveStr.Memory,
>>ReceiveStr.Size) = ReceiveHash ) then
>> TriggerOnChunkDone(tempPtr,
>>ReceiveTag, Success) else
>> TriggerOnChunkDone(tempPtr,
>>ReceiveTag, Fail);
>>
>>
>
>
>
>> end else
>> TriggerOnChunkDone(tempPtr, ReceiveTag,
>>Success);
>>
>>
>
>
>
>> WaitStatus := wsNone;
>> end
>> else
>> if(WaitStatus=wsGetMsg) then
>> begin
>>{$IFDEF VERBOSE}
>> consoleOut('Redirecting Text Message...');
>>{$ENDIF}
>> { now we've got the complete string. let's
>>put it in ReceiveLine and let
>> ProcessCommand() handle it }
>> //DumpStream;
>> ReceiveStr.position := 0;
>> GetMem(tempPtr, ReceiveStr.Size);
>> ZeroMemory(tempPtr, ReceiveStr.Size);
>> ReceiveStr.ReadBuffer(tempPtr^,
>>ReceiveStr.Size);
>> ReceiveLine := PWideChar(tempPtr);
>> WaitStatus := wsNone;
>> FreeMem(tempPtr);
>>
>>
>
>
>
>> ProcessCommand;
>> end;
>>
>>
>
>
>
>> ReceiveStr.Clear;
>> ReceiveStr.Free;
>> ReceiveStr := nil;
>>
>>
>
>
>
>> end;
>> {
>>-------------------------------------------------------------------------
>> end else
>> begin
>> consoleOut('WARNING! Transfer went wrong! Error
>>'+inttostr(GetLastError));
>> { receiveSize is -1 }
>> { something terrible happened. no more file transfer
>>will occur! so we better dispose of what we have }
>> ReceiveStr.Clear;
>> ReceiveStr.Free;
>> ReceiveStr := nil;
>> WaitStatus := wsNone;
>> Self.Flush;
>> SendText('fail');
>> end;
>>
>>
>
>
>
>
>
>> end
>> else
>> begin
>> // we got a "normal length" response.
>> //Application.ProcessMessages;
>> ZeroMemory(@chrReceived, sizeof(chrReceived));
>> bytesReceived := Self.Receive(@chrReceived,
>>sizeof(chrReceived));
>> ReceiveLine := WideString(chrReceived);
>> bytesReceived := bytesReceived + 1;
>> consoleOut('(received '+inttostr(bytesReceived)+'
>>bytes): '+ ReceiveLine);
>> ProcessCommand;
>> end;
>>
>>
>
>
>
>>end; // end procedure
>>
>>
>
>
>
>>------------------------
>>
>>
>
>
>
>
>>Francois PIETTE wrote:
>>
>>
>
>
>
>>>>I asked about a method to recognize the "splitted packets" and to join
>>>>them upon arrival of the last one. So far my protocol works fine, and
>>>>
>>>>
>>>>
>>>>
>>>This is really the most common task when dealing with TCP transmission. You
>>>have such "packet reassembly" in almost all ICS components. TWSocket itself
>>>has a LineMode that does this reassembly for lines. Lines are not necessary
>>>text line. It is just a convenient name to specify any variable length data
>>>with end of data marker.
>>>
>>>
>>>
>>>
>>>
>>>>Please would anybody give me a clue what's going wrong here?
>>>>
>>>>
>>>>
>>>>
>>>How would you receive help when we don't know anything about your code !
>>>It is likely that the problem is at the receiving part.
>>>
>>>--
>>>Contribute to the SSL Effort. Visit
>>>http://www.overbyte.be/eng/ssl.html
>>>--
>>>[EMAIL PROTECTED]
>>>http://www.overbyte.be
>>>
>>>
>>>----- Original Message -----
>>>From: "Kei" <[EMAIL PROTECTED]>
>>>To: "ICS support mailing" <[email protected]>
>>>Sent: Tuesday, November 01, 2005 9:45 AM
>>>Subject: [twsocket] transmission stress test
>>>
>>>
>>>
>>>
>>>
>>>
>>>>Hi, this is David.
>>>>
>>>>I asked about a method to recognize the "splitted packets" and to join
>>>>them upon arrival of the last one. So far my protocol works fine, and
>>>>all the data is assembled in an orderly manner. For large texts or
>>>>binary data, the clients knows how to handle it as follows:
>>>>The "Redirecting Text MEssage" just means, the text buffer is going to
>>>>be further processed in a function called "HandleResponse()".
>>>>
>>>>sender side-----------------------------------------
>>>>Line 7: out> txt 1561
>>>>Line 8: (Written 17 bytes)
>>>>Line 9: (received 16 bytes): sendtxt
>>>>Line 10: out> sendtxt
>>>>Line 11: out> msg AliasName and DriverName a...(too long to display)
>>>>
>>>>recipient side---------------------------------------------
>>>>Line 10: (received 18 bytes): txt 1561
>>>>Line 11: out> txt 1561
>>>>Line 12: out> sendtxt
>>>>Line 13: (Written 15 bytes)
>>>>Line 14: Preparing Temp Buffer...
>>>>Line 15: Received 1460 bytes, Remaining 3123 bytes
>>>>Line 16: Received 1460 bytes, Remaining 1663 bytes
>>>>Line 17: Received 203 bytes, Remaining 203 bytes
>>>>Line 18: All packets finished.
>>>>Line 19: Redirecting Text Message...
>>>>Line 20: out> msg AliasName and DriverName a...(too long to display)
>>>>Line 21: (!!) Command: msg AliasName and DriverName are m...(too long
>>>>to display)
>>>>------------
>>>>
>>>>The "sendtxt" is a command through which the recipient tells the sender
>>>>"hey I'm ready to get the long buffer".
>>>>This mechanism works very fine if I enter the command one by one. But
>>>>today I wanted to know if the recipient can handle successive "long msg"
>>>>requests by one single sender. So I tried a "stress test" to send the
>>>>1561 widechar long text on an interval of 10ms, to the recipient. There
>>>>are totally 200 requests to be sent consecutively.
>>>>
>>>>Sometimes it works fine, but sometimes .. the following happens
>>>>
>>>>Sender side-----------------------------------------
>>>>Line 1720: out> txt 1561
>>>>Line 1721: (Written 17 bytes)
>>>>Line 1722: Warning! Session is busy!
>>>>Line 1723: (received 25 bytes): sendtxt<JUNK>
>>>>Line 1724: out> sendtxt<JUNK>
>>>>Line 1725: Warning! Session is busy!
>>>>Line 1726: Warning! Session is busy!
>>>>Line 1727: Warning! Session is busy!
>>>>:
>>>>:
>>>>
>>>><JUNK> = some strange characters
>>>>
>>>>Recipent side---------------------------------------
>>>>Line 4112: (received 18 bytes): txt 1561
>>>>Line 4113: out> txt 1561
>>>>Line 4114: out> sendtxt
>>>>Line 4115: (Written 15 bytes)
>>>>Line 4116: Preparing Temp Buffer...
>>>>Line 4117: Received -1 bytes, Remaining 3123 bytes
>>>>Line 4118: WARNING! Transfer went wrong! Error 0
>>>>Line 4119: out> fail
>>>>Line 4120: (Written 9 bytes)
>>>>
>>>>Although the recipient sent 15 bytes of command "sendtxt" (line 4114),
>>>>the sender receives 25 bytes, with 10 bytes of junk following it.
>>>>Supposingly, the recipient should say "fail" to the sender and the
>>>>sender shoud receive it. But in this case, since the "sender" is kept
>>>>asking for more send (triggered by Timer, where interval=10ms), so there
>>>>are many "Warning! Session is busy!" messages appearing (because the
>>>>previous send-long-text action is still pending). Although the "fail"
>>>>message was sent from the other party, it seems that the "Sender" never
>>>>received this message.
>>>>
>>>>Also, out of so many times of test I've done, the "junk" fter the
>>>>"sendtxt" is ALWAYS THE SAME junk. So.. I think there might be some
>>>>problem in my code... perhaps is it something to do with multithreading?
>>>>
>>>>This problem only occur when I repeatly do the test. As in above you can
>>>>see the line number gets very big because I haven't got a problem from
>>>>line0... till now.. but I got to test the protocol under stress..
>>>>because it's going to be some sort of "database server."
>>>>
>>>>Please would anybody give me a clue what's going wrong here?
>>>>
>>>>Thanks!
>>>>
>>>>David
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>--
>>>>To unsubscribe or change your settings for TWSocket mailing list
>>>>please goto http://www.elists.org/mailman/listinfo/twsocket
>>>>Visit our website at http://www.overbyte.be
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>>
>
>
>
>
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://www.elists.org/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be
