[twsocket] [PATCH] HttpSrv: Fix in THttpRangeStream for incomplete reads
Hi, when using the HTTP server component with some custom TStream descendant together with partial GET requests I noticed that the Read method of THttpRangeStream incorrectly handles the situation where Read returns less bytes that requested. It switches to the next stream member in the list without checking if end-of-stream has been reached. The included patch fixes this bug. It would be great if you could consider including the patch in the upstream version of ICS. Regards, Tobias (begin patch) Index: OverbyteIcsHttpSrv.pas === --- OverbyteIcsHttpSrv.pas (revision 1075) +++ OverbyteIcsHttpSrv.pas (working copy) @@ -6236,9 +6236,12 @@ ActSize := min(Count - DataRead, Rec.Size - (ActOffset)); Rec.Stream.Position := ActOffset + Rec.StartPos; SizeRead := Rec.Stream.Read(Pointer(DWORD(@Buffer) + DWORD(DataRead))^, ActSize); -Inc(Index); Inc(DataRead, SizeRead); Inc(FPosition, SizeRead); +if (Rec.Offset + Rec.Size) FPosition then +Break +else +Inc(Index); end; Result := DataRead; Exit; (end patch) -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] [PATCH] HttpSrv: Fix in THttpRangeStream for incomplete reads
Angus Robertson - Magenta Systems Ltd wrote: Thanks, the V8 repository will be updated this morning. Nice. Any chance the patch could also be applied to V7 (trunk)? Regards, Tobias -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] [PATCH] HttpSrv: Fix in THttpRangeStream for incomplete reads
Angus Robertson - Magenta Systems Ltd wrote: OK, both V7 and V8 are updated with your fix, and the 'nightly' zips updated early. I also added another web server fix that previously was only in V8. Great. Thanks, Tobias -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] Using Unicode in HdrTo of TSmtpCli component
Hi! I am currently using the TSmtpCli component to send email notifications to users. I want to change the HdrTo property value to additionaly contain the user name instead of just the plain email address. As my user name is retrieved from a database as WideString I wonder how I handle non-ASCII characters in the user name. My application is compiled using Delphi 2006. I see in my mail client that such non-ASCII string parts are escaped like =?ISO-8859-1?Q?H=E4llo?= world Is there some way to produce this string escape sequence when setting the HdrTo property? I have looked at the ICS sources but could not find the appropriate helper function. Regards, Tobias -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Using Unicode in HdrTo of TSmtpCli component
Thanks Arno for the help. Would it also work if I would set Allow8BitChars to TRUE, Charset to UTF-8 and manually encode all string properties like HdrTo, HdrSubject and MailMessage.Text using Utf8Encode()? My assumption was that the header lines should not contain non-ASCII characters, but maybe it is legal to use UTF-8 if the header also declares something like Content-Type: text/plain; charset=UTF-8 What is your opinion? Regards, Tobias -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Using Unicode in HdrTo of TSmtpCli component
Angus Robertson - Magenta Systems Ltd wrote: The headers may have different charset to the content, indeed they are usually ASCII. Never use the body charset to decode the headers. OK, I see. I guess I have all necessary parts together now. Thanks! Regards, Tobias -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] [PATCH] THttpCli: add support for DELETE requests
OK, just checked in your patch rev. #896 Thanks! Regards, Tobias -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] [PATCH] THttpCli: add support for DELETE requests
Arno Garrels wrote: Thanks, however the method name conflicts with System.Delete. The new method should be IMO renamed to either Del, Dele or Delete_. TList, TStrings, TDataSet and more objects have a Delete method so I see no real problem here. However if this is the only thing preventing my changes to be merged with upstream ICS I would choose Del :-) Regards, Tobias -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Support for HEAD in HTTP server component
Arno Garrels wrote: Feb 07, 2012 V7.44 Arno - The HEAD method *MUST NOT* return a message-body in the response. Do not skip compression on HEAD requests, we need to send the correct size. Method SendDocument simplified and added two overloads. AnswerStreamAcceptRange got an overload too. Your fix has worked well in my tests so far. Thanks! Regards, Tobias -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] [PATCH] THttpCli: add support for DELETE requests
Hi,
in order to support HTTP DELETE requests [1] in the THttpCli component I
have made some small additions (see patch below). It would be great if you
could consider including the patch in the upstream version of ICS.
Note: I first considered adding the new request type by overloading the
component but could not find a good way to achieve that.
Regards,
Tobias
Links:
[1] https://tools.ietf.org/html/rfc2616#section-9.7
(start patch)
Index: OverbyteIcsHttpProt.pas
===
--- OverbyteIcsHttpProt.pas (revision 894)
+++ OverbyteIcsHttpProt.pas (working copy)
@@ -585,7 +585,7 @@
THttpEncoding= (encUUEncode, encBase64, encMime);
THttpRequest = (httpABORT, httpGET, httpPOST, httpPUT,
-httpHEAD, httpCLOSE);
+httpHEAD, httpDELETE, httpCLOSE);
THttpState = (httpReady, httpNotConnected, httpConnected,
httpDnsLookup, httpDnsLookupDone,
httpWaitingHeader, httpWaitingBody, httpBodyReceived,
@@ -923,12 +923,14 @@
procedure Post; { Synchronous blocking Post}
procedure Put;{ Synchronous blocking Put }
procedure Head; { Synchronous blocking Head}
+procedure Delete; { Synchronous blocking Delete }
procedure Close; { Synchronous blocking Close }
procedure Abort; { Synchrounous blocking Abort }
procedure GetASync; { Asynchronous, non-blocking Get }
procedure PostASync; { Asynchronous, non-blocking Post }
procedure PutASync; { Asynchronous, non-blocking Put }
procedure HeadASync; { Asynchronous, non-blocking Head }
+procedure DeleteASync;{ Asynchronous, non-blocking Delete}
procedure CloseAsync; { Asynchronous, non-blocking Close }
procedure ThreadAttach; override;
procedure ThreadDetach; override;
@@ -2333,6 +2335,10 @@
SocketDataSent(FCtrlSocket, 0);
{$ENDIF}
end;
+httpDELETE:
+begin
+SendRequest('DELETE', FRequestVer);
+end;
httpHEAD:
begin
SendRequest('HEAD', FRequestVer);
@@ -4548,6 +4554,10 @@
SocketDataSent(FCtrlSocket, 0);
{$ENDIF}
end;
+httpDELETE:
+begin
+SendRequest('DELETE', FRequestVer);
+end;
httpHEAD:
begin
SendRequest('HEAD', FRequestVer);
@@ -4621,6 +4631,15 @@
{* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *}
+{ This will start the Delete process and wait until terminated (blocking) }
+procedure THttpCli.Delete;
+begin
+FLocationChangeCurCount := 0 ;
+DoRequestSync(httpDELETE);
+end;
+
+
+{* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *}
{ This will start the Post process and wait until terminated (blocking) }
procedure THttpCli.Post;
begin
@@ -4665,6 +4684,15 @@
{* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *}
+{ This will start the delete process and returns immediately (non blocking) }
+procedure THttpCli.DeleteAsync;
+begin
+FLocationChangeCurCount := 0 ;
+DoRequestASync(httpDELETE);
+end;
+
+
+{* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *}
{ This will start the post process and returns immediately (non blocking) }
procedure THttpCli.PostAsync;
begin
(end patch)
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be
[twsocket] Support for HEAD in HTTP server component
Hi,
I am currently debugging some problems in my application using the
THttpServer/THttpConnection components regarding the support for HEAD
requests. As far as I understand the specs no response body should be
returned for HEAD but it seems that THttpConnection does send response
bodies in procedure ProcessPost() in case of 400/404/etc. answers. This
probably confuses the client.
Do you think it is meaningful to update the AnswerXXX() procedures like the
following diff example:
-procedure THttpConnection.Answer404;
+procedure THttpConnection.Answer404(SendType : THttpSendType = httpSendDoc);
var
Body : String;
begin
@@ -3223,7 +3223,8 @@
GetKeepAliveHdrLines +
#13#10);
FAnswerStatus := 404; { V7.19 }
-SendStr(Body);
+if SendType httpSendHead then
+SendStr(Body);
end;
and call Answer404(httpSendHead) within ProcessHead()?
Regards,
Tobias
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be
Re: [twsocket] Support for HEAD in HTTP server component
I wrote: [...] As far as I understand the specs no response body should be returned for HEAD but it seems that THttpConnection does send response bodies in procedure ProcessPost() in case of 400/404/etc. answers. ^---^ I wanted to say ProcessHead() here. Regards, Tobias -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] [PATCH] THttpSrv: add support for returning partialstreams
Arno Garrels wrote (in a response to RTT): Problem is that AnswerStreamAcceptRange can only be used for OK-responses. Yes, that is right and like you said I only use it for some of the responses where it makes sense for the client (to allow seeking within streamed audio files). I just checked in Tobias' change as SVN rev. #890 However renamed to AnswerStreamAcceptRange and added a parameter Header: String. That sounds good! Regards, Tobias -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] HTTP server component with WebDav support
Hi! I would like to extend my existing ICS-based HTTP server to allow upload and removal of files on the server. From reading the HTTP specs at [1] it seems that the natural commands for these actions would be PUT [2] and DELETE [3]. Unfortunately the THttpServer/THttpConnection components do not support these actions yet as they seem only common for HTTP servers supporting WebDAV (at least you need to enable WebDAV in apache in order to support these HTTP requests). I have seen that I can set an OnUnknownRequestMethod event handler in current versions of ICS but am unsure how to fetch the request body transmitted by the client for PUT requests. Has anybody done such a WebDAV-like extension of the ICS components yet? Regards, Tobias Links: [1] http://tools.ietf.org/html/rfc2616 [2] http://tools.ietf.org/html/rfc2616#section-9.6 [3] http://tools.ietf.org/html/rfc2616#section-9.7 -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] [PATCH] THttpSrv: add support for returning partial streams
Hi! I have added a new function AnswerStreamPart in component THttpConnection to allow answering partial retrieve requests using any tStream descendant (see attached patch file). It would be nice if the patch could be considered for inclusion in the upstream ICS sources and I am open for suggestions regarding possible improvements. Kind regards, Tobias -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] [PATCH] THttpSrv: add support for returning partial streams
Index: OverbyteIcsHttpSrv.pas
===
--- OverbyteIcsHttpSrv.pas (revision 887)
+++ OverbyteIcsHttpSrv.pas (working copy)
@@ -807,6 +807,9 @@
const Status : String;
const ContType : String;
const Header : String); virtual;
+procedure AnswerStreamPart(var Flags : THttpGetFlag;
+ const ContType : String;
+ LastModified : TDateTime = 0); virtual; {
Added by TR 2012-01-30 }
procedure AnswerString(var Flags: THttpGetFlag;
const Status : String;
const ContType : String;
@@ -2862,6 +2865,129 @@
{* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *}
+{ANDREAS Byte-range-separator (use the same as IIS) }
+const
+ByteRangeSeparator = '[lka9uw3et5vxybtp87ghq23dpu7djv84nhls9p]';
+
+
+{* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *}
+{ANDREAS Helperfunction to create the HTTP-Header }
+function CreateHttpHeader(
+Version : String;
+ProtoNumber : Integer;
+AnswerContentType : String;
+RangeList : THttpRangeList;
+DocSize : THttpRangeInt;
+CompleteDocSize : THttpRangeInt): String;
+begin
+if ProtoNumber = 200 then
+Result := Version + ' 200 OK' + #13#10 +
+ 'Content-Type: ' + AnswerContentType + #13#10 +
+ 'Content-Length: ' + _IntToStr(DocSize) + #13#10 +
+ 'Accept-Ranges: bytes' + #13#10
+{else if ProtoNumber = 416 then
+Result := Version + ' 416 Request range not satisfiable' + #13#10}
+else if ProtoNumber = 206 then begin
+if RangeList.Count = 1 then begin
+Result := Version + ' 206 Partial Content' + #13#10 +
+ 'Content-Type: ' + AnswerContentType + #13#10 +
+ 'Content-Length: ' + _IntToStr(DocSize) + #13#10 +
+ 'Content-Range: bytes ' +
+
RangeList.Items[0].GetContentRangeString(CompleteDocSize) +
+ #13#10;
+end
+else begin
+Result := Version + ' 206 Partial Content' + #13#10 +
+ 'Content-Type: multipart/byteranges; boundary=' +
+ ByteRangeSeparator + #13#10 +
+ 'Content-Length: ' + _IntToStr(DocSize) + #13#10;
+end;
+end
+else
+raise Exception.Create('Unexpected ProtoNumber in CreateHttpHeader');
+end;
+
+
+{* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *}
+procedure THttpConnection.AnswerStreamPart(
+var Flags: THttpGetFlag;
+const ContType : String; { if emtpy, default to text/html }
+LastModified : TDateTime = 0);
+var
+NewDocStream: TStream;
+ProtoNumber : Integer;
+CompleteDocSize : THttpRangeInt;
+SyntaxError : Boolean;
+ContEncoderHdr : String;
+ContStatusHdr : String;
+begin
+Flags := hgWillSendMySelf;
+ProtoNumber := 200;
+ContEncoderHdr := '';
+if ContType '' then
+FAnswerContentType := ContType
+else
+FAnswerContentType := 'text/html';
+FLastModified := LastModified;
+
+CompleteDocSize := FDocStream.Size;
+{ANDREAS Create the virtual 'byte-range-doc-stream', if we are ask for
ranges}
+if RequestRangeValues.Valid then begin
+{ NewDocStream will now be the owner of FDocStream - don't free
FDocStream }
+NewDocStream := RequestRangeValues.CreateRangeStream(FDocStream,
+ FAnswerContentType, CompleteDocSize, SyntaxError);
+if Assigned(NewDocStream) then begin
+FDocStream := NewDocStream;
+FDocStream.Position := 0;
+ProtoNumber := 206;
+end
+else begin
+if SyntaxError then
+{ Ignore the content range header and send entire document in case
}
+{ of syntactically invalid byte-range-set
}
+FDocStream.Position := 0
+else begin
+{ Answer 416 Request range not satisfiable }
+FDocStream.Free;
+FDocStream := nil;
+if not FKeepAlive then
+PrepareGraceFullShutDown;
+Answer416;
+Exit;
+end;
+end;
+end;
+
+FDataSent := 0; { will be incremented after each send part of data }
+FDocSize := FDocStream.Size;
+OnDataSent := ConnectionDataSent;
+
+{ V7.21 are we allowed to compress content }
+if CheckContentEncoding(FAnswerContentType) then begin
+ContEncoderHdr := DoContentEncoding; { V7.21 do it, returning new
header }
+
Re: [twsocket] ICS and Delphi XE2
Angus Robertson - Magenta Systems Ltd wrote: ICS does not currently support FireMonkey or Mac OS-X, it needs a lot of work since Linux is not event driven like Windows, and makes use of threads to prevent blocking. ICS did support Kylix for Linux several years ago, so some work has been done. Both other VCL-like cross-platform libraries I'm aware of (GLib/GTK+ and Qt) include an abstraction of the underlying event system ([1] and [2]). I don't understand why Embarcado should not use a similar approach and includes a cross-platform event-system wrapper in FireMonkey. It would make the life easier not only for ICS developers, I suppose. BTW: I am not 100% sure but I think that at least GLib doesn't build its event system on threads but on I/O signals for better performance. Regards, Tobias Links: [1] http://developer.gnome.org/glib/stable/glib-The-Main-Event-Loop.html [2] http://doc.qt.nokia.com/4.7/eventsandfilters.html -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] ICS and Delphi XE2
Angus Robertson - Magenta Systems Ltd wrote: There may be such an event system in FireMonkey, don't actually know. But supporting ICS with FireMonkey and Mac OS-X is not simply a case of a few bug fixes and conditional compilation changes which is effectively all that is needed for 64-bit VCL support (and a massive effort from Arno testing it all!). Yes, I absolutely understand that. I'm as curious as you on what level event handling will be supported by FireMonkey :-) BTW: Thanks to the ICS team for their work on 64-bit VCL support! Regards, Tobias -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] [OT] 64-bit Delphi preview
Angus Robertson wrote: Maybe it's time to check all our projects for 64-bit compatibility? :) Only if your ICS project needs to access more than 2 gigs of memory (or is it 4), or is a DLL called by other 64-bit applications. When working with Windows messages I often use a pattern like type tMyMsgData = record Value1, Value2, Value3 : string; end; pMyMsgData = ^tMyMsgData; var Msg : pMyMsgData; begin New(Msg); Msg^.Value1 := 'hello'; Msg^.Value1 := 'world'; ... PostMessage(Handle, WM_MY_MSG, integer(Msg), 0); ... end; I guess that won't work with 64bit-Delphi because I cannot cast a pointer into an integer. Will I have to split the pointer into Int64Rec.Lo/Hi? Any ideas? Regards, Tobias -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] [OT] 64-bit Delphi preview
Arno Garrels wrote: No, WPARAM and LPARAM are 64-bit in WIN64 so just cast the pointer to WParam in your sample. PostMessage(Handle, WM_MY_MSG, WParam(Msg), 0); Oh, that's fine! NativeInt and NativeUInt are Delphi's integer types that always have the size of a Pointer. Integer will _not change of course. Just noticed that even my Delphi 2006 supports NativeInt. Will use it for pointer/handle casting related code whenever I stumble over it now. Regards, Tobias -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] [OT] Re: SSL broken?
I guess the centralized trust model of SSL has been a known problem for ages. Don't understand why they try to make so much noise about it now. IMO the problem of the alternative model (web of trust) is that it lacks the cash cow properties and thus is less appealing to certificate authorities. Regards, Tobias -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Why ICS prefers non-blocking sockets?
It is all much better explained here why blocking is better: http://www.ararat.cz/synapse/doku.php/about I would not say that blocking is better. Programming with synchronous functions needs less callbacks and thus gives more compact code at the cost of performance. If this is OK depends on the requirements of your application. That said I think that the real question should be: Why doesn't Delphi improve support for asynchronous programming? because there are other programming languages that allow you to write asynchronous code without explicit callbacks by supporting coroutines (see http://en.wikipedia.org/wiki/Coroutine). Vala is my current favourite language in that direction, all you have to do there is to add a yield in front of calling asynchronous functions and the compiler takes care of the rest (callback, state machine). Code examples: http://live.gnome.org/Vala/GIOSamples#Asynchronous_Stream_Reading http://live.gnome.org/Vala/GIONetworkingSample#line-156 How I wish there would be a yield keyword in Delphi... Regards, Tobias -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer: client not removed from server list on disconnect
Hello Anton! destructor TSrvClient.Destroy; begin FreeAndNil(FList); // *** set breakpoint here to ensure client is destroyed *** end; This should be: destructor TSrvClient.Destroy; begin FreeAndNil(FList); inherited; // -- added call to ancestor end; Regards, Tobias -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Tool for analyzing network throughput issues
Arno Garrels wrote: If I were in your shoes I would at first search my application for bottlenecks, lengthy blocking tasks, too small data chunks sent/received etc. When I search for issues I usually try to divide the area where the problem possibly exists into two parts and check both sides. If the problem can be excluded from one part I focus on the other part and try to divide it again. So in my current situation I want to find out if the problem comes from the sender side (data takes too long to generate) or receiver side (data takes too long to consume). Thus I hope to be able to take a look at the network buffer fill sizes to see if data is queued there or not. Do you know some other technique to sepearate the issue between server and client? Regards, Tobias -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] Tool for analyzing network throughput issues
Hello, I am experiencing throughput problems with my ICS-based network transfer application and want to ask if anybody knows a tool to debug those kind of issues (visualize maybe the current network latency, TCP send/receive buffer fill size, etc.). Regards, Tobias -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Is the SVN server online?
Hello SZ, With the Tortoise CVS client when I click checkout and paste the URL :ext:i...@svn.overbyte.be/ics/branches/icsipv6 it says: end of file from server I guess you confused TortoiseCVS - TortoiseSVN. The GUI client for Subversion (SVN) can be downloaded at http://tortoisesvn.net/downloads . Tobias -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Installing ICS within C++ Builder 2006
Hello Arno, You should no longer use ICSv6 but ICSv7 which installs fine with BCB. I know that the latest version is preffered but I need to use a library built on top of ICS with BCB so I need the old version (V5 in fact). In order to install ICS into both personalities you just have to change the Delphi package options to: Linking | Linker output | Generate all C++ Builder files (including package libs) and rebuild all. Thanks for the hint. I tried it and it worked in a way that now the necessary *.bpi file is created (along with some *.hpp files) but still the components are only available in the palette at design time when I start a Delphi project, not when I start a C++ project. Do I have to completely remove the package and all packages depending on it before re-running the install? It was installed before and I just opened the IcsDel100 package file and run Install from the menu. Regards, Tobias -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Installing ICS within C++ Builder 2006
Hello again, In order to install ICS into both personalities you just have to change the Delphi package options to: Linking | Linker output | Generate all C++ Builder files (including package libs) and rebuild all. Thanks for the hint. I tried it and it worked in a way that now the necessary *.bpi file is created (along with some *.hpp files) but still the components are only available in the palette at design time when I start a Delphi project, not when I start a C++ project. Do I have to completely remove the package and all packages depending on it before re-running the install? To answer my own question: Installation worked when - removing the existing ICS component package within BDS - removing the remaining *.bpl, *.dcp, *.bpi package output files - restarting BDS - installing ICS package with General all C++ Builder files linker option set as described by Arno - restarting BDS Thanks for the help, Tobias -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] Installing ICS within C++ Builder 2006
Hello, I have ICS package installed within the Delphi personality of BDS2006 but now I want to use it within the C++ Builder personality. After adding some missing *.pas files to IcsBcb100.bdsproj BDS complains: Remove WSocketE, WSockBuf, WSocket, WSocketS, ... were found in required package IcsDel100. followed by: Unable to find package import: IcsDel100.bpi How do I generate the missing *.bpi file? Or does anybody know another way of making a package installed under Delphi available for C++ Builder? Regards, Tobias -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] New DLL hijacking vulnerability KB 2269637
Arno Garrels wrote:
The DLL names are globally writable typed constants, set their values
before the OpenSSL libraries are loaded.
Is there any advantage to use writable typed constants like
const GSSLEAY_DLL_Name : String = 'SSLEAY32.DLL';
instead of
var GSSLEAY_DLL_Name : String = 'SSLEAY32.DLL';
?
Both seems to work but I try to avoid the first one whenever possible because
it can break multi-threading when not used properly. For example I had a
commercial component that contained code like this to save some code lines:
procedure SomeProc();
{$J+} // enable writeable consts
const
Buffer : array[0..31] of byte = (0, 0, ..., 0);
{$J-} // disable writeable consts
begin
// read and write to Buffer like it's a variable
end;
which caused problems when the procedure is used by different threads
simultaneously because all instances read and write to the same block in
memory.
I know this is not a problem with ICS but I wonder why writable consts are
used/preferred at all.
Regards,
Tobias
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be
Re: [twsocket] [OT] EMB NDA Briefing
Fastream Technologies wrote: Please let us know about their plans for 64-bit compiler support! https://forums.embarcadero.com/message.jspa?messageID=257995#257995 The plan seems to be clear, only the expected time frame is secret (needs an NDA). Regards, Tobias -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] ICS documentation
Markus Humm wrote: What is so hard in documenting just a single property or work on those property overview pages which aren't complete yet? Copying the table of a already started property list to a not yet started one and modifying it so that the correct properties are listed is not hard. My very personal reason is that I don't like APIs documented in a Wiki as they are incomplete most of the time (e.g. lack of type information for properties and events) and the required effort to keep this in sync with the code is very high. As I am already browsing the code in 95% of the lookup cases I tend to document my personal code in-line instead of separated from the code and use a tool to output fancy HTML directly from the sources (http://www.naturaldocs.org). Writing tutorials and how-tos is a different thing. In my opinion a Wiki makes sense here. Regards, Tobias -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Serious bug in TWSocket(AVorbufferoverflow)Pleasehelp !
I have found some similar code in function TCustomLineWSocket.DoRecv() from
ICS-V5:
===
--- D:/DelphiComponents/Ics/Delphi/Vc32/WSocket.pas (revision 279)
+++ D:/DelphiComponents/Ics/Delphi/Vc32/WSocket.pas (working copy)
@@ -7874,13 +7874,13 @@
{ We are in line mode an a line is received }
if FLineLength = BufferSize then begin
{ User buffer is greater than received data, copy all and clear }
-Move(FRcvdPtr^, Buffer, FLineLength);
+Move(FRcvdPtr^, Buffer^, FLineLength);
Result := FLineLength;
FLineLength := 0;
Exit;
end;
{ User buffer is smaller, copy as much as possible }
-Move(FRcvdPtr^, Buffer, BufferSize);
+Move(FRcvdPtr^, Buffer^, BufferSize);
Result := BufferSize;
{ Move the end of line to beginning of buffer to be read the next time
}
Move(FRcvdPtr[BufferSize], FRcvdPtr^, FLineLength - BufferSize);
@@ -7897,13 +7897,13 @@
{ We already have received data into our internal buffer }
if FRcvdCnt = BufferSize then begin
{ User buffer is greater than received data, copy all and clear }
-Move(FRcvdPtr^, Buffer, FRcvdCnt);
+Move(FRcvdPtr^, Buffer^, FRcvdCnt);
Result := FRcvdCnt;
FRcvdCnt := 0;
Exit;
end;
{ User buffer is smaller, copy as much as possible }
-Move(FRcvdPtr^, Buffer, BufferSize);
+Move(FRcvdPtr^, Buffer^, BufferSize);
Result := BufferSize;
{ Then move remaining data to front og buffer 16/10/99 }
Move(FRcvdPtr[BufferSize], FRcvdPtr^, FRcvdCnt - BufferSize + 1);
Regards,
Tobias
Max Terentiev wrote:
Hi Arno,
I think all Move() calls in ICS code should be checked...
Bugs maybe not only in this place
---
With best regards, Max Terentiev.
Business Software Products.
AMS Development Team.
supp...@bspdev.com
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be
Re: [twsocket] Serious bug in TWSocket(AVorbufferoverflow)Pleasehelp !
Arno Garrels wrote: I don't think it is a bug in ICS-V5, since Buffer mostly was of no type (if memory serves well). This changed in ICS V6 where Buffer is of type TWSocketData which maps to Pointer in Win32. That's true. As a programmer coming from C language the typeless var Buffer thing often buffles me :-) Tobias -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Package Count / Wireshark
Alper Albayrak wrote: Monitoring packages arrived, I see that one of the packages (505 byte) is divided into 3 packages. What i want to ask; is this the way tcp protocol transfers data over network, or package count is because of the method client uses to send data..? I once have also seen this some long time ago when tracing the network traffic. Don't remember how I reacted but maybe you can check the size of data blocks passed to tWSocket send functions and check the tWSocket.BufSize property. Optimizing the data throughput with Windows sockets is still a big mystery to me. If anybody else has some ideas, please share. Regards, Tobias -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] FileMD5 : Different Hash Result - same File ondifferent workstation
ineic...@triple-m.ch wrote: I wrote this tool to check FileMD5 with ICS_V6, ICS_V7 and INDY_10. It display's the result in a memo with time measurement. E.g. a 44MB Mpeg File displays: Host: [WSBIERI] File: [D:\_BuProcVideoData\19052008noch eine aufzeichung395876456777894.mpg] MD5 ICS_V6 :[E48FFC453C495B2F2EA2F33ACBC5586B] in [696] msec MD5 ICS_V7 :[E48FFC453C495B2F2EA2F33ACBC5586B] in [680] msec MD5 Indy_10:[E48FFC453C495B2F2EA2F33ACBC5586B] in [1488] msec If your intent is to benchmark the speed of different MD5 implementations I suggest that you add MD5 routines from the DcpCrypt package [1] into your comparison list. When I compared it to ICS v5 on my machine some time ago it was faster than ICS. Also it would be great if your tool would output both processing time (wall clock time) and used CPU time (kernel + user time, see [2]). Just my personal suggestions. Tobias [1] http://www.cityinthesky.co.uk/cryptography.html [2] http://msdn.microsoft.com/en-us/library/ms683223(VS.85).aspx -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] THttpCli digest and basic authentication
Maurizio Lotauro wrote: Sooner or later I must install (and use) a svn client :-) Any suggestion? When using Windows there is the great TortoiseSVN client (http://tortoisesvn.tigris.org/) that integrates into the Windows explorer. Have not found a better tool in the last years - and I really miss it when working on GNU/Linux. Regards, Tobias -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] Fixed small bug in HttpSrv digest auth
Hi! I have found a small bug in ICS V5 HttpSrv.pas within procedure tHttpConnection.Answer401: Header := FVersion + ' 401 Unauthorized' + #13#10 + 'WWW-Authenticate: ' + //'Digest realm=' + FAuthRealm +// original 'Digest realm=' + FAuthRealm + '' + // fixed ', qop=' + AuthString + '' + ', nonce=' + FAuthDigestServerNonce + '' + ', opaque='+ FAuthDigestServerOpaque + ''; if FAuthDigestStale then Header := Header + ', stale=true'; I must admit that I don't know which version conforms to the standards. It's just the case that in the original code version the auth realm is displayed wrong in Mozilla Firefox. Regards, Tobias -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] Advice on using tHttpServer with throttling and threads
Hello, I am currently trying to implement a HTTP file streaming server using tHttpServer and I'd like to know if anybody has some advice regarding the use of threads. I know that it's basically possible to use tHttpServer without threads but I suppose that threads are needed in my case because I'd like to - deliver multiple files of 10-200MB size towards the clients simultaneously - limit the throughput for some type of file content (throttling) So what is the best way to implement a HTTP server like this? Should I start a thread for each client generally (like the ICS example ThrdSrv.dpr does) or is it meaningful to start a thread inside of the HTTP GET event handler only if some content type is detected? How can I implement throughput throttling? Is it more or less done by putting a call to Sleep() into ConnectionDataSent()? Any hints and shared experiences are welcome. Regards, Tobias -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] FTP client supports Utf-8 in ICSv7 now
Francois Piette wrote: Angus and I added UTF-8 support to the TFtpCli V7. I think Angus and Arno deserve a BIG thank from all of us for the huge work they've done supporting internationalisation in FTP component. Thanks guys ! Yes, it's great to hear that! Thanks for all the work that is done to provide ICS Unicode support. Tobias -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Socket throughput optimizations
Olivier Sannier wrote: Here are a few ideas to get high performances: 4) Enlarge winsock buffers 5) Enlarge TWSocket send buffer (BufSize property) to mach actual network packet size. By default BufSize is the size of a standard ethernet packet. What values would you recommend for a 802.11 100Mbps network? In my testings I had good results when enlarging the winsock buffers to 256kB. I use this relatively large buffer size as my application has a rather limited amount of clients (approx. 10-20) and is running in a network with high round-trip-times (approx. 8ms). Regards, Tobias -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Socket throughput optimizations
Hello Francois, thanks for your long and descriptive reply. Using the same client or server, at the same time in the day, are you able to check for network thruput using other tools ? [...] Yes, I've tried that and got 12MB/s for standard Windows file copy (CIFS), 4MB/s for FTP file copy using Internet Explorer and 3MB/s for FTP file copy using the ICS demo client. I have also tested these client using different round-trip-times (I have a GNU/Linux router in between in my test setup where I can adjust RTT) and realized that they degrade quite different when increasing the RTT (2ms, 4ms) so I asked myself how to handle this. You may also write two very simple client and server applications. The client sending 256KB of data comming from memory (to avoid disk I/O slowness) and the server just reading the data and throwing it away (do not write to disk or allocate memory to store data). You'll have an idea abour the maximum thruput you can have. The max throughput I got on my 1Gbps network was by using the ICS FTP demo and changing the tWSocket.BufSize to 256kB and using WSocket_setsockopt(FHSocket, SOL_SOCKET, SO_SNDBUF, 256kB). This gives a throughput of 35MB/s (the FTP server is FileZilla running on Windows 2003 server). Using some very simple ICS-based client and server applications I get around 20MB/s when setting the client to the same buffer sizes. Don't know what to optimize on server side... Anyway will try to follow the list of your suggestions, especially the one about separating the disk-writing by putting it into a separate thread. Regards, Tobias -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] Socket throughput optimizations
Hi! I'd like to know if anybody of you has some suggestions about how to tune a tWSocket-based application for data throughput. Do you change some socket buffer sizes? Like in WSocket_setsockopt(FHSocket, SOL_SOCKET, SO_SNDBUF, ...); WSocket_setsockopt(FHSocket, SOL_SOCKET, SO_RCVBUF, ...); What else do you suggest? Maybe using a second internal buffer so one writes the retrieved data in OnDataAvailable into that buffer and returns immediately? Using multiple threads? Other ideas? /Tobias -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Socket throughput optimizations
Francois Piette wrote: Which protocol ? Well, I'm basically using TCP. My client inherits from tWSocket and uses it's own XML-based protocol to deliver data blocks of around 256kB to the server (which is also a self-written Delphi ICS application). When connecting to the server app over a 1Gbps network cable with 2ms RTT I get only around 3-5MB/s. Processing each data block takes 5ms and I expected to gain at least 20MB/s of throughput. /Tobias -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] FTP and SSH
Arno Garrels wrote: SSH (Secure Shell) is available on nix platforms. Well, it is typically used on Unix-like platforms but there is software available for Windows most notably the popular Putty and WinSCP tools. Due to the lack of SSL SSH has been used in the past to tunnel unsecure FTP on those platforms. Today however most FTP server even in the Unix wourld do support SSL as well. Hmm, do you talk about SFTP? AFAIK SFTP is not just FTP tunneled over SSH but an own protocol (see [1]). If you talk about tunneling using SSH in general then I would say that this feature is still popular today as it does not need a full-fledged public-key infrastructure (see [2]) and is very easy to setup. The fact that SSL support is implemented in FTP servers by now might be more related to the fact that SSH is considered too powerful to be enabled on a public server :-) (see [3]). IMO the effort to implement an antiquated protocol, that is specific to the Unix platform, doesn't make much sense. IMHO it would make sense to implement such a feature-rich multi-purpose protocol in ICS but of course this wouldn't be an easy task. Regards, Tobias [1] http://en.wikipedia.org/wiki/SSH_file_transfer_protocol [2] http://en.wikipedia.org/wiki/Secure_Shell#Uses_of_SSH [3] http://en.wikipedia.org/wiki/Secure_Shell#Security_cautions -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] FTP and SSH
Hi! Does the ICS FTP Client support SSH ? No, SSH is not supported by ICS. I don't know of any SSH implementation based on ICS (or any other Delphi library) so most likely you will have to implement it on your own. We want to use ICS ftp client in a secure mode. And the secure mode must be SSH (not SSL). If it's ok, how can I do or does a sample exist ? I would suggest to do port forwarding using Putty (see [1]) on the machine where the ftp client is running. Then you could use the standard ICS FTP client inside of your application by connecting to the forwarded port on the local machine instead of the remote port. Regards, Tobias [1] http://www.cs.uu.nl/technical/services/ssh/putty/puttyfw.html -- NOA Audio Solutions Vertriebsges. m.b.H. Tel: +43-1-5452700 Johannagasse 42/4 Fax: +43-1-545270014 A - 1050 Wien Www: http://www.noa-audio.com -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] HTTP...
Hi! DZ-Jay wrote: A way to create a multi-part response is to encapsulate it as a MIME 822 message. Here's some basic information on this: http://www.motobit.com/tips/detpg_multiple-files-one-request/ I stumbled over the following lines on that page: | The situation is even worse if you need authentication for each image | preview - you must do tenth of authentication requests against user | database, separated for each http request. Is that really true? I thought that in the meanwhile it is common behavior to establish and keep one TCP connection for all HTTP requests until the page is loaded. And that the authentication is remembered for a established TCP connection. /Tobias -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] THttpCli: default webpage name
DZ-Jay wrote: [EMAIL PROTECTED] wrote: If THttpCli is used to get a webpage with no document name specified, only directory (www.codegear.com/products) - is it possible to find out the default document name? Whether it be index.html or main.html or just some custom name specified on the webserver? This is a feature of the HTTP server: When a document is not specified, it decides which is the default resource. There is no way for the client to discover this. Well in the case where the server performs a redirection it might be possible, e.g. http://en.wikipedia.org/wiki/ - http://en.wikipedia.org/wiki/Main_Page /Tobias -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] UDP Newbie...
If I call listen and then try to send I get an error 10049 (Bind Address not available) Are you sure you filled in the correct IP? As an additional note: You can probably set the IP string of the server socket to 0.0.0.0 which will listen on all available interfaces. So you don't have to find or configure the appropriate local interface address. Allowing the user of the program to change that interface address is helpful, though, as someone might have two network cards built-in connected to two different networks and wants to attach your application only to one of the networks. Regards, Tobias @@VON Tobias Rapp t.rapp~~noa-audio.com @@ -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] Always set MultiThreaded flag TRUE in tWSocket?
Hi! I am using a tWSocket derived class inside a DLL. One of the users of the DLL had problems with using the class and it turned out that I had forgotten to set MultiThreaded to TRUE (as the user of the DLL has created the class in a separate thread). My question now is: Is it safe to always set MultiThreaded to TRUE - even when there are no multiple threads? As far as I can see the code should also work in that case. I'm using ICS v5. Kind regards, Tobias -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] Problem with ICS package and Delphi 2006
Hello everybody! I have a problem related with the ICS package 'IcsDel100.dpk'. I have it installed fine with Delphi 2006. Now I have a self-made package that depends on tWSocket and thus has 'IcsDel100' in the packages requires clause. When I try to compile the package I get the following error: [Pascal Error] NoaComm_D2006.dpk(41): E1026 File not found: 'IcsDel100.drf' I have no idea what a .drf file is and where Delphi should find it. When I uncomment 'IcsDel100' in the requires clause the compile is running through with the warning: [Pascal Warning] NoaComm_D2006.dpk(77): W1033 Unit 'WSocket' implicitly imported into package 'NoaComm_D2006' Maybe the problem is not ICS-specific (although it seems that other packages are installed fine). Has anybody a hint what to do here? /Tobias -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] FTPClient Put problem
Fias Norbert István wrote: The problem is that Indy sends a large amount TCP segments as the received window increasing and as no drop occured while ICS FTP client does not increase the number of sent segments, so the unacknowledged amount of bytes is not increasing. I have experienced similar issues when comparing ICS and Indy components. The problem decreased when setting the BufSize property of tWSocket to values between 256k and 1M. Maybe the FTP client of ICS is can be changed to use a larger BufSize values by default (as Indy does, I suppose). /Tobias -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Using SourceForge for ICS ?
Hello, The first decision is should I select CVS or SVN ? Any advice appreciated. I'm using SVN together with TortoiseSvn each day in my work for more than a year now. Did not have any serious problems since (the only thing that I stumbled over is the filename case-sensitivity issue - but that is the same for CVS AFAIK). One big advantage of SVN in my opinion is the fact that it does not require a server process. This comes handy for smaller projects, e.g. if I am the only developer or all developers are in the same LAN. This would not be the case with ICS on SourceForge, of course :-) And SVN comes with very good documentation. /Tobias -- NOA Audio Solutions Vertriebsges. m.b.H. Tel: +43-1-5452700 Johannagasse 42/4 Fax: +43-1-545270014 A - 1050 Wien Www: http://www.noa-audio.com -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Delphi2007,UDP, 2x packets
How do you manage missed/double/wrong ordered packets? Each packet got some sequence ID and alive packets were sent from time to time. Also there was no special acknowledge packet - each received packet contained the last successful received ID of the sender. If one side detects that an incoming packet's ID is LastID+1 it sends out an request for the missing packets. Wrong ordered packets were no big problem in my scenario because the client is supposed to be within the same subnet. Double received packets are just thrown away. Actually in some situations all packets are sent out twice just to be sure. /Tobias -- NOA Audio Solutions Vertriebsges. m.b.H. Tel: +43-1-5452700 Johannagasse 42/4 Fax: +43-1-545270014 A - 1050 Wien Www: http://www.noa-audio.com -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocket for .Net and C#?
Anyway, I am doing some Visual Studio 2005 projects and I want to have a good async-component. Basically TWSocket but that it can be used in C# and .net. Does such exist or is it somehow possible to use TWSocket in such projects? I was even thinking of doing an ActiveX wrapper around TWSocket and use that. Any ideas? I have not tested it but since the new Delphi versions include a Delphi for .NET it should be able to compile the ICS units to managed code. So a .NET library could be created and then used within your Visual Studio project. /Tobias -- NOA Audio Solutions Vertriebsges. m.b.H. Tel: +43-1-5452700 Johannagasse 42/4 Fax: +43-1-545270014 A - 1050 Wien Www: http://www.noa-audio.com -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TSmtpCli Attach files without real files
I have not tried it but from looking at the sources I would say that it might work if you would change the OnAfterFileOpen event so that it passes the FStream as a var variable. Then you could initialize that stream variable with a memory- or stringstream in your event handler. Maybe you will have to do Base64-transcoding in your event handler also but there are routines for that in the MimeUtil unit. /Tobias Francois PIETTE wrote: Out of my mind without checking the source code: You may override the function which read the data frm file. Contribute to the SSL Effort. Visit http://www.overbyte.be/eng/ssl.html -- [EMAIL PROTECTED] The author of the freeware multi-tier middleware MidWare The author of the freeware Internet Component Suite (ICS) http://www.overbyte.be - Original Message - From: Wilfried Mestdagh [EMAIL PROTECTED] To: ICS support mailing twsocket@elists.org Sent: Tuesday, May 22, 2007 6:29 PM Subject: [twsocket] TSmtpCli Attach files without real files Hello, I want to send mail with TSmtpCli with a few attached file. But I really do not want to create files on the disk unless absolutely nececarry. But I'm a little stuck on how to attach files because it seems to be designed to only get them from disk or is there something I do not see ? To be clear: forget the word 'file'. I have data and I want this data attached as a file with a name. For example 'test.txt' as it is a 'test.txt' file attached. It is a *.csv file and a *.txt file, no binary stuff, so maybe this could be very simple ? -- Rgds, Wilfried http://www.mestdagh.biz -- NOA Audio Solutions Vertriebsges .m.b.H. Tel: +43-1-5452700 Johannagasse 42/4 Fax: +43-1-545270014 A - 1050 Wien Www: http://www.noa-audio.com -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Architectural question II
Something like attached below? It creates its window thread-save, but I prefer a timer that was able to use the hidden window(s) of V6, or may be we think about a windowless timer (Thread, signals, and WaitForMultipleObjects)? Just for interest: What is the drawback of using a own window handle for the timer? Does it use that many resources or has it an big impact on message processing (e.g. when using thousands of tHttpCli's)? /Tobias -- NOA Audio Solutions Vertriebsges.m.b.H.Tel: +43-1-5452700 Johannagasse 42/4 Fax: +43-1-545270014 A - 1050 Wien Www: http://www.noa-audio.com -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Project proposal
I would be very interested if you start this project as my company also thinks that in the future it might be interesting to port some of our Windows service applications to GNU/Linux. When evaluating what way to go I thought that either Kylix could be used (but is that product still active?) or we port the code (including ICS) to C/C++. Maybe the glib library could be used to provide the Unix alternative to messages and threads: http://www.gtk.org/api/2.6/glib/index.html @Francois: I think you are right with your statement that GNU/Linux users want free software when it comes to desktop users. But IMHO there is a market for commercial server software. Many of our customers are already running their Oracle database on a Unix machine. /Tobias Fastream Technologies wrote: Dear Francois, Before detailing the project, I want to learn if principally you can do the following and when can you start: - We plan to port our reverse proxy/HTTP/FTP servers to Linux 2.6 Kernel. It is specifically 2.6 because according to my research it includes advanced async I/O similar to Windows. IOW, we need both messages and threads so that we will be handling 10k connections with 330 threads (32 connections/thread) as we now can do with Windows. We want to be able to use the gcc compiler and Eclipse and donate the base component code to you so that it can be open source. Unfortunately Fastream has little UNIX coding experience but change is inevitable as we see reluctance in the server market to buy Windows based software. Best Regards, SZ www.fastream.com -- NOA Audio Solutions Vertriebsges.m.b.H.Tel: +43-1-5452700 Johannagasse 42/4 Fax: +43-1-545270014 A - 1050 Wien Www: http://www.noa-audio.com -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Throughput problem with TWSocket over a long line
Hello Arno, I would play with sligtly increased SocketSndBufSize and a BufSize set to something below like 16/8 and 32/16 kb. TWSocket's property BufSize actually is the block size and the amount of data copied into the winsock buffer in a single (winsock) send call. Unfortunately the connection link is busy now and for the next days but I will make more tests whenever I find time. Increasing SocketSndBufSize however also decreases the maximum posible sockets in the system (no problem if you don't need hundreds of sockets at the same time). Basically I have dedicated machines for both server and client side of my file transfer application so using some system resources is possible if better throughput is gained. However as far as my first test shows it does not make much difference if I also increase the SocketSndBufSize in that scenario. Increasing BufSize makes the biggest change. /Tobias -- NOA Audio Solutions Vertriebsges.m.b.H.Tel: +43-1-5452700 Johannagasse 42/4 Fax: +43-1-545270014 A - 1050 Wien Www: http://www.noa-audio.com -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Throughput problem with TWSocket over a long line
Hi Wilfried, You find on my site a few programs to test performance with TWSocket. I have found the PerformanceTest project and tested it. In the first run it also gave the well-known value of ~1.2MB/s. Then I digged into TWSocket's Send() function and found the BufSize property. After changing the BufSize to 1MB (I have read the warning about setting it larger than SocketSndBufSize in the comments but wanted to try myself) I got rates of ~7.7MB/s - what an improvement! Looks like http://support.microsoft.com/default.aspx?scid=kb;en-us;823764 is wrong in my special case or the system is automagically increasing the socket's send buffer behind the scenes. When I read the socket send buffer size with getsockopt() while the transfer is running it remains at 8192 bytes, though. So shall I explicitly add a call setsockopt() that increases the socket send buffer dramatically (1MB)? What is your opinion about that? /Tobias -- NOA Audio Solutions Vertriebsges.m.b.H.Tel: +43-1-5452700 Johannagasse 42/4 Fax: +43-1-545270014 A - 1050 Wien Www: http://www.noa-audio.com -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Throughput problem with TWSocket over a long line
Hi SZ, I think your problem should be in the message pump. What do you use for that in your thread? When I switched to GetMessage from PeekMessage, I was WOW! I use TWSocket's MessageLoop() function in my threaded application which uses GetMessage() internally. And Wilfried's test programm uses no threads at all. But the basic thing was not that I cannot reach more throughput than 1.2MB/s at all. If I use two computers in my intranet I easily gain a throughput of more than 20MB/s (disc IO included). It was a special case that I had a problem with: When TCP/IP connection is going through an ATM line (high round trip times). /Tobias -- NOA Audio Solutions Vertriebsges.m.b.H.Tel: +43-1-5452700 Johannagasse 42/4 Fax: +43-1-545270014 A - 1050 Wien Www: http://www.noa-audio.com -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Throughput problem with TWSocket over a long line
Hello Wilfried, Looks like http://support.microsoft.com/default.aspx?scid=kb;en-us;823764 is wrong in my special case or the system is automagically increasing the Can you resent the URL please ? It seems dead here. http://support.microsoft.com/default.aspx?scid=kb;en-us;823764 It's the link to the MSDN page titled Slow performance occurs when you copy data to a TCP server by using a Windows Sockets API program that is referred to in the WSocket.pas unit (see comment line that starts with Jun 03, 2005 V5.18). /Tobias -- NOA Audio Solutions Vertriebsges.m.b.H.Tel: +43-1-5452700 Johannagasse 42/4 Fax: +43-1-545270014 A - 1050 Wien Www: http://www.noa-audio.com -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Throughput problem with TWSocket over a long line
Hello Arno, By default winsock's send buffer size is 8 kb which is much bigger than TFtpCli's constant BLOCK_SIZE of 1460 bytes. So one condition described in the article is not true since a single send call in TFtpCli does not fill the whole underlying winsock send buffer. I realy wonder why disabling Nagle doesn't help? Same applies to WSocket's internal send buffer that has a block size of 1460 as well. So if you haven't increased BufSize of TWSocket a single send call does never fill the whole underlying winsock send buffer. That's exactly the thing I'm doing at the moment: I increase the BufSize of TWSocket from 1460 bytes to 1MB. The throughput increases from 1.2MB/s to more than 7MB/s. As far as I understand Nagle=off only would help me if I would like to send small packets ( MCU) at a very high rate. But that is not the case: I want to deliver large files ( 3GB). When I look at the IP traffic with Wireshark I see something like this in the case where TWSocket.BufSize=1460 : Client-Server 1448 bytes Client-Server 12 bytes Server-Client ACK Client-Server 1448 bytes Client-Server 12 bytes Server-Client ACK And in the case where TWSocket.BufSize=1MB : Client-Server 1448 bytes Client-Server 1448 bytes Client-Server 1448 bytes // previous line repeated 10-30 times Server-Client ACK Client-Server 1448 bytes Client-Server 1448 bytes Client-Server 1448 bytes // previous line repeated 10-30 times Server-Client ACK So shall I explicitly add a call setsockopt() that increases the socket send buffer dramatically (1MB)? Use WSocket option SocketSndBufSize instead, it will the call setsockopt() internally. I think that 1 MB is much too big, since this memory is allocated from kernel memory which is not an endless resource. SocketSndBufSize property is read-only, at least in my version of WSocket.pas (V5.25). /Tobias -- NOA Audio Solutions Vertriebsges.m.b.H.Tel: +43-1-5452700 Johannagasse 42/4 Fax: +43-1-545270014 A - 1050 Wien Www: http://www.noa-audio.com -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Throughput problem with TWSocket over a long line
To improve performance you can make your packets equal to the MTU, or switch Nagle off. There are some notes on Microsoft site about this. If you search for Nagle on MSDN you will find a lots information about that. They dont encourage to switch it off but in some cases (maybe yours) it is better. Note that you have to switch it of at both ends. I tested it with Nagle=off but without singnificant changes. Then I had the idea to capture the IP traffic that occurs when delivering the file with the ICS/Indy FTP servers. There is one big difference: - When delivering the file with ICS FTP server the server sends 2 IP packets (1448 + 600 bytes data) and then seems to wait for ACK. Note that I did not switch Nagle off in that test. - When delivering the file with Indy10 FTP server the server sends many IP packets (up to 60 packets of 1448 bytes each) until the first ACKs arrive. Are there any socket options regarding that behaviour? The ICS demo programs are not specifically written to improve speed. They are written to give a as mutch as possible clear for newcomers. A PII300 CPU can easy saturate a 100 mbps network with TWSocket. My data delivery application uses some specific XML protocol for talking with the client which is implemented in a tWSocket-derived component. I just wanted to make sure it is not something in my socket that slows down data throughput significantly and thus made a test with the 'plain' FTP server component as found in the ICS FTP demo server project. /Tobias -- NOA Audio Solutions Vertriebsges.m.b.H.Tel: +43-1-5452700 Johannagasse 42/4 Fax: +43-1-545270014 A - 1050 Wien Www: http://www.noa-audio.com -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] Throughput problem with TWSocket over a long line
Hi! I have a throughput/performance problem using the latest stable version of the ICS package. When using a TWSocket derived application over an TCP/IP connection with high round trip times and small packet sizes (it is an 155 MBit ATM line) the throughput is very low. To check, if there is a general problem in my application I also checked the throughput with the FTP Server example and further with an FTP Server example from the Indy10 component package. The results for delivering a file over the line was: ~ 1.18 MB/s when using my application ~ 1.2 MB/s when using the ICS FTP demo server ~ 6.5 MB/s when using the Indy10 FTP demo server What can I do to improve the throughput in my application? Are there any known tweaks? I'd like to avoid changing to Indy because I like the clear and simple structured sources of ICS. Please share your thoughts, Tobias -- NOA Audio Solutions Vertriebsges.m.b.H.Tel: +43-1-5452700 Johannagasse 42/4 Fax: +43-1-545270014 A - 1050 Wien Www: http://www.noa-audio.com -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Throughput problem with TWSocket over a long line
Hello Dod, Are you using ICS in sync or async mode ? I'm using ICS in async (event driven) mode in my application. The file is sent in fragments of 256KB in the OnDataSent event handler. Do yo use same buffer and write-to-disk as Indy ? When comparing ICS and Indy I did not use my application but the standard FTP server examples that came with both component packages and thus I'm not sure what buffering technique is used inside the Indy FTP server component (it seems to be quite complex when looking at it's source). The FTP client used for comparing both applications was the standard MS commandline FTP client on Windows2003 (BTW: the serving application was also run on Windows2003 in all cases). Do you use multi-threading or not ? I use multi-threading (each client has a separate thread) in my application but I know that the ICS FTP server does not use multi-threading. Is it binary or text file (I guess it is text file as you said long line in the subject ? It is a binary file of 128MB random data that I used for delivery. The long line in the subject is bad translation and should be corrected to long cable, I think. Tobias -- NOA Audio Solutions Vertriebsges.m.b.H.Tel: +43-1-5452700 Johannagasse 42/4 Fax: +43-1-545270014 A - 1050 Wien Www: http://www.noa-audio.com -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Throughput problem with TWSocket over a long cable
Hello Dod, Do you see any CPU overload server side during transfert ? CPU load varies from 2% to 5% while the FTP file transfer is running. /Tobias -- NOA Audio Solutions Vertriebsges.m.b.H.Tel: +43-1-5452700 Johannagasse 42/4 Fax: +43-1-545270014 A - 1050 Wien Www: http://www.noa-audio.com -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
