I have some new info, which unfortunately only makes things less logical for me :p Since at the moment the focus, for me, is moved from my code to ICS code, I've switched back to a simple HTTP server I made in stead of the demo. This server app is much smaller then the demo, thus easier to debug, and shows exactly identical results to the demo app.

The complete source of my small server app is at the bottom of this post.

Here is some real logging from both client and server side, when the problem occurs:

client:
08:30:56.149 - starting machine gun
08:30:56.164 - done. errcode: 0 (No Error) statuscode: 200
08:30:56.240 - done. errcode: 0 (No Error) statuscode: 200
08:30:56.325 - done. errcode: 0 (No Error) statuscode: 200
08:31:06.424 - problem occured, stopping machine gun
08:31:06.432 - machine gun stopped
08:31:06.448 - done. errcode: 8 (Not a Winsocket error) statuscode: 404


server:
08:30:56.155 - post document application/binary
08:30:56.159 - rx 1059 / 1059
08:30:56.226 - post document application/binary
08:30:56.235 - rx 4550 / 4550
08:30:56.300 - post document application/binary
08:30:56.310 - rx 4550 / 4550
08:30:56.385 - post document application/binary
08:30:56.393 - rx 1630 / 4550
08:30:56.400 - rx 2920 / 4550


The interesting part is this: if I uncomment the Log() call in TForm1.OnClientConnect, the problem does not occur anymore. It doesn't matter what is logged, as long as something is added to Form1.MemoLog.Lines, the problem does not occur. Looking into TMemo, nothing much seems to happen besides some messages being sent:

procedure TMemoStrings.Insert(Index: Integer; const S: string);
var
  SelStart, LineLen: Integer;
  Line: string;
begin
  if Index >= 0 then
  begin
    SelStart := SendMessage(Memo.Handle, EM_LINEINDEX, Index, 0);
    if SelStart >= 0 then Line := S + #13#10 else
    begin
      SelStart := SendMessage(Memo.Handle, EM_LINEINDEX, Index - 1, 0);
      if SelStart < 0 then Exit;
      LineLen := SendMessage(Memo.Handle, EM_LINELENGTH, SelStart, 0);
      if LineLen = 0 then Exit;
      Inc(SelStart, LineLen);
      Line := #13#10 + s;
    end;
    SendMessage(Memo.Handle, EM_SETSEL, SelStart, SelStart);
    SendMessage(Memo.Handle, EM_REPLACESEL, 0, Longint(PChar(Line)));
  end;
end;


Sending these messages in TForm1.OnClientConnect instead of adding something to the memo makes the problem come back again.

Hoping for some insight.

regards,

Merijn






unit main;

interface

uses
Windows, Messages, SysUtils, Variants, Classes, Graphics, Controls, Forms,
  Dialogs, StdCtrls, OverbyteIcsHttpSrv, OverByteIcsWSocketS;

type
  TMyHTTPConnection = class(THttpConnection)
  private
   fData: string;
   fAllReceived: boolean;

   procedure InternalOnPostedData(Sender: TObject; ErrCode: word);
  public
   procedure Init;
  end;

  TMyHTTPServer = class(THttpServer)
  protected
procedure WSocketServerClientConnect(Sender: TObject; Client: TWSocketClient; Error: Word); override; procedure TriggerPostDocument(Sender: TObject; var Flags: THttpGetFlag); override;
  end;

  TForm1 = class(TForm)
   MemoLog: TMemo;
   procedure FormCreate(Sender: TObject);
   procedure FormDestroy(Sender: TObject);
  private
   fMyHTTPServer: TMyHTTPServer;
procedure OnClientConnect(Sender: TObject; Client: TObject; Error: word);
  public
   procedure Log(st: string);
  end;

var
  Form1: TForm1;

implementation

{$R *.dfm}

{ TMyHTTPConnection }

procedure TMyHTTPConnection.Init;
begin
 fData := '';
 fAllReceived := false;

 Self.OnPostedData := InternalOnPostedData;
end;

procedure TMyHTTPConnection.InternalOnPostedData(Sender: TObject; ErrCode: word);
var Client: TMyHTTPConnection;
    Data: string;
    Flags: THttpGetFlag;
begin
 Client := TMyHTTPConnection(Sender);
 Data := Client.ReceiveStr();

 if fAllReceived then
 begin
  sleep(0);
  exit;
 end;

 fData := fData + Data;

Form1.Log(Format('rx %d / %d', [Length(fData), Client.RequestContentLength]));

 if (Length(fData) = Client.RequestContentLength) then
 begin
  fAllReceived := true;
  PostedDataReceived();
  AnswerString(Flags, '', '', '', 'empty');
 end;
end;

{ TMyHTTPServer }

procedure TMyHTTPServer.TriggerPostDocument(Sender: TObject; var Flags: THttpGetFlag);
begin
 inherited;

form1.Log('post document ' + TMyHTTPConnection(Sender).RequestContentType);

 TMyHTTPConnection(Sender).LineMode := false;
 TMyHTTPConnection(Sender).fData := '';
 Flags := hgAcceptData;
end;

procedure TMyHTTPServer.WSocketServerClientConnect(Sender: TObject; Client: TWSocketClient; Error: Word);
begin
 inherited;

 TMyHTTPConnection(Client).Init();
end;

procedure TForm1.FormCreate(Sender: TObject);
begin
 fMyHTTPServer := TMyHTTPServer.Create(nil);
 fMyHTTPServer.OnClientConnect := OnClientConnect;
 fMyHTTPServer.ClientClass := TMyHTTPConnection;
 fMyHTTPServer.DocDir := '';

 fMyHTTPServer.Port := '80';
 fMyHTTPServer.Addr := '0.0.0.0';
 fMyHTTPServer.Start();
end;

procedure TForm1.FormDestroy(Sender: TObject);
begin
 fMyHTTPServer.Free();
end;

procedure TForm1.Log(st: string);
begin
 MemoLog.Lines.Add(FormatDateTime('hh:nn:ss.zzz - ', Now()) + st);
end;

procedure TForm1.OnClientConnect(Sender, Client: TObject; Error: word);
begin
// Log('client connected');
end;

end.






On 03/09/2014 23:36, Merijn Bosma wrote:
I just noticed I can't attach a file through the mailing list (makes sense), so I've put the zip file with HTTPPoster sources online:
http://www.xs4all.nl/~bosma/HTTPPoster.zip

regards,

Merijn

On 03/09/2014 23:34, Merijn Bosma wrote:
Hi all,

For a current project I need to build an HTTP server to which data can be POSTed, so I build one using OverbyteIcsWebServ.dpr as an example. I ran into stability problems, and starting slimming my code down to isolate the problem. I'm now at a point where it seems to be an issue in ICS code, though I haven't found the cause yet. Currently the issue is reproducible using the OverbyteIcsWebServ.dpr which comes with the ICS code as a demo.

The issue is as follows:
when posting a file to the OverbyteIcsWebServ, sometimes the post fails, on the client side this manifests as getting back a 408, on the server side it manifests as not receiving data at all, or missing the last block (in case the data is split up in blocks). Looking at OverbyteIcsWebServ.dpr, this means HttpServer1PostDocument() is called always and HttpServer1PostedData() is not called at all, or not for the last expected block.

If you add the following line:
Display(Format('received %d bytes, total received %d, total expecting %d', [Len, ClientCnx.FDataLen, ClientCnx.RequestContentLength])); In procedure TWebServForm.HttpServer1PostedData() in OverbyteIcsWebServ1.pas, right after Inc(ClientCnx.FDataLen, Len);

So it looks like this:

Len := ClientCnx.Receive(ClientCnx.FPostedRawData + ClientCnx.FDataLen, Remains);
    { Sometimes, winsock doesn't wants to givve any data... }
    if Len <= 0 then
        Exit;

    { Add received length to our count }
    Inc(ClientCnx.FDataLen, Len);

Display(Format('received %d bytes, total received %d, total expecting %d', [Len, ClientCnx.FDataLen, ClientCnx.RequestContentLength]));

    { Check maximum length }
    if ClientCnx.FDataLen > MAX_UPLOAD_SIZE then begin

When the fault situation happens, you will see log like:

received 255 bytes, total received 255, total expecting 1420
received 255 bytes, total received 512, total expecting 1420
received 255 bytes, total received 768, total expecting 1420
received 255 bytes, total received 1024, total expecting 1420
received 255 bytes, total received 1280, total expecting 1420

Then it stops (as if the last block was never received), and timeout (error 408) occurs. Looking with wireshark shows that the last packet was on the wire, just not received in the application.

To reproduce I've attached a small app which can be tested against OverbyteIcsWebServ.dpr.This application will show all files found in subdir 'postdata', you can double click one to post it to given URL. Sometimes it takes a long time to reproduce, sometimes it happens fast (first file you post). The 'machine gun' button in the app will post random files from the postdata subdir fast, to help reproduce.
So, exact steps to reproduce are:
- compile OverbyteIcsV8Gold.zip\Samples\Delphi\WebDemos\OverbyteIcsWebServ.dpr and run it
- unzip HTTPPoster with the postdata subdir and run it
- change the IP in HTTPPoster if required
- hit the 'machine gun' button

Log in HTTPPoster will show when the timeout occured (and the 'machine gun' will be shut off automatically).

I'll try to dig into ICS code later, but first looks give me the idea that won't be really easy to debug, so I'm hoping for some help from the guru's

thanks in advance,

Merijn


--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Reply via email to