Re: [fpc-pascal] lnet for TCP daemon
On Thursday, September 12, 2013 7:14 AM, Michael Schnell mschn...@lumino.de wrote: On 09/12/2013 12:47 PM, Mark Morgan Lloyd wrote: I've concluded that using a thread is, in fact, preferable True. It's a pity that Synapse and friend (especially SynaSer) does does not implement internal threads that throw appropriate events in the Main Thread when something comes in (or an error appears). This is a very common request, and since the fpc-Team some time ago enabled TThread.Queue in the RTL, this is doable in a straight forward, fpc-ish way (portable but Delphi compatible). would this hamper or cause problems with normal non-gui programs using the library? ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] lnet for TCP daemon
On 09/13/2013 06:06 AM, wkitt...@windstream.net wrote: would this hamper or cause problems with normal non-gui programs using the library? 1) This feature of course should only be provided additionally to the functions we know and love. 2) Other than the work-alike Application.QueueAsyncCall which is provided by Lazarus long since, the rather new TThread.Queue is not only Delphi-compatible, but it also is located in the RTL and thus can be used in LCL based nongui applications and even without linking to the LCL at all. You need to do calls to a synchronize function which the RTL provides, to pull the event queue. This _can_ be done in a close loop (e.g. containing a sleep() call), which of course increases latency and processor overhead. Better it is done by a decent triggering mechanism (e.g. using a semaphore or self-piping) the loop waits for and each queue push triggers. This is another improvement I hope for: enhancing the synchronize and TThread.Queue/TThread.Synchronize features in the RTL in an OS-depending way that (optionally) automatically pulls the queue for the main thread. -Michael ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
[fpc-pascal] Get empty memory quickly or... an empty file
I'm working on implementing zip64 support and trying to test things by
compressing large files or streams using FPC's zip code.
Currently using this to get a huge memorystream filled with the same
data - so it should compress very well:
...
ContentStream:=TMemoryStream.Create();
ContentStream.SetSize(1+$); //4 bytes+1
ContentStream.Position:=0;
Buffer:='A'; //pchar
for i:=0 to ContentStream.Size-1 do
begin
ContentStream.Write(Buffer,1);
end;
ContentStream.Position:=0;
writeln('Buffer created');
try
Zipper.Entries.AddFileEntry(ContentStream, ArchiveFile);
writeln('entry added');
Zipper.ZipAllFiles;
finally
ContentStream.Free;
...
However, I'm sure there must be quicker ways (e.g. less newbie code;
using OS functions to quickly create empty memor, or alternatively
create a sparse file and zip the file instead of the stream).
Suggestions?
Thanks,
Reinier
___
fpc-pascal maillist - fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Get empty memory quickly or... an empty file
Once upon a time, Reinier Olislagers said: However, I'm sure there must be quicker ways (e.g. less newbie code; using OS functions to quickly create empty memor, or alternatively create a sparse file and zip the file instead of the stream). Have you tried FillChar(), FillDWord(), ... I not, have a look at http://www.freepascal.org/docs-html/rtl/system/fillchar.html -- Ewald ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Get empty memory quickly or... an empty file
On 9/13/13, Reinier Olislagers reinierolislag...@gmail.com wrote:
I'm working on implementing zip64 support and trying to test things by
compressing large files or streams using FPC's zip code.
Currently using this to get a huge memorystream filled with the same
data - so it should compress very well:
...
ContentStream:=TMemoryStream.Create();
ContentStream.SetSize(1+$); //4 bytes+1
Actually 4 GB : 4294967295 + 1 = 4294967296 = 4 * 1024 * 1024 * 1024
ContentStream.Position:=0;
Buffer:='A'; //pchar
for i:=0 to ContentStream.Size-1 do
begin
ContentStream.Write(Buffer,1);
end;
ContentStream.Position:=0;
However, I'm sure there must be quicker ways (e.g. less newbie code;
using OS functions to quickly create empty memor, or alternatively
create a sparse file and zip the file instead of the stream).
Now you have 4G write calls, resulting in 4G System.Move calls.
Maybe it is faster to create a 1GB datastructure (StringOfChar('A',
1024*1024*1024) and do 4 Write's?
(Untested code!)
Bart
___
fpc-pascal maillist - fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Get empty memory quickly or... an empty file
In our previous episode, Reinier Olislagers said: However, I'm sure there must be quicker ways (e.g. less newbie code; using OS functions to quickly create empty memor, or alternatively create a sparse file and zip the file instead of the stream). Do not create the data, but generate it in an overriden tstream read() call? If you can define a function that can generate count bytes of data at position n (in this case the function is constant), then you don't have to write it out. Ken in mind that TStream is an abstraction, not necessarily a block of (RAM or disk) memory. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Get empty memory quickly or... an empty file
On 13/09/2013 15:19, Bart wrote:
On 9/13/13, Reinier Olislagers reinierolislag...@gmail.com wrote:
ContentStream.SetSize(1+$); //4 bytes+1
Actually 4 GB : 4294967295 + 1 = 4294967296 = 4 * 1024 * 1024 * 1024
Yes... but I had trouble counting Fs, so that reminder makes more sense
to me ;)
Now you have 4G write calls, resulting in 4G System.Move calls.
Maybe it is faster to create a 1GB datastructure (StringOfChar('A',
1024*1024*1024) and do 4 Write's?
Thanks, I'll look into it.
Meanwhile, it turns out (surprisingly ;) ) that the creation time of the
4GB memorystream pales in comparison the compression code takes...
Ah well.
___
fpc-pascal maillist - fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Get empty memory quickly or... an empty file
On 13/09/2013 15:38, Marco van de Voort wrote: In our previous episode, Reinier Olislagers said: However, I'm sure there must be quicker ways (e.g. less newbie code; using OS functions to quickly create empty memor, or alternatively create a sparse file and zip the file instead of the stream). Do not create the data, but generate it in an overriden tstream read() call? If you can define a function that can generate count bytes of data at position n (in this case the function is constant), then you don't have to write it out. Ken in mind that TStream is an abstraction, not necessarily a block of (RAM or disk) memory. Mmm, yes, that sounds very smart, thanks! ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Get empty memory quickly or... an empty file
On 13/09/2013 15:12, Ewald wrote: Once upon a time, Reinier Olislagers said: However, I'm sure there must be quicker ways (e.g. less newbie code; using OS functions to quickly create empty memor, or alternatively create a sparse file and zip the file instead of the stream). Have you tried FillChar(), FillDWord(), ... I not, have a look at http://www.freepascal.org/docs-html/rtl/system/fillchar.html Arggh. Thanks! Have even used FillChar yesterday or the day before. Just getting a bit too numb I suppose. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Get empty memory quickly or... an empty file
On Fri, 13 Sep 2013, Reinier Olislagers wrote:
On 13/09/2013 15:38, Marco van de Voort wrote:
In our previous episode, Reinier Olislagers said:
However, I'm sure there must be quicker ways (e.g. less newbie code;
using OS functions to quickly create empty memor, or alternatively
create a sparse file and zip the file instead of the stream).
Do not create the data, but generate it in an overriden tstream read() call?
If you can define a function that can generate count bytes of data at
position n (in this case the function is constant), then you don't have to
write it out.
Ken in mind that TStream is an abstraction, not necessarily a block of
(RAM or disk) memory.
Mmm, yes, that sounds very smart, thanks!
Attached a NullStream implementation.
Michael.{
This file is part of the Free Component Library (FCL)
Copyright (c) 1999-2000 by Michael Van Canneyt and Florian Klaempfl
See the file COPYING.FPC, included in this distribution,
for details about the copyright.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
**}
{$mode objfpc}
unit nullstream;
interface
uses Classes;
type
ENullStreamError = class(EStreamError);
{ TNullStream }
TNullStream = class(THandleStream)
private
FPos : Int64;
protected
function GetPosition: Int64; override;
procedure InvalidSeek; override;
public
function Read(var Buffer; Count : LongInt) : Longint; override;
function Write(const Buffer; Count : LongInt) : LongInt; override;
function Seek(const Offset: int64; Origin: TSeekOrigin): int64; override;
end;
implementation
const
SInvalidOperation = 'Cannot perform this operation on a IOStream.';
function TNullStream.GetPosition: Int64;
begin
Result:=FPos;
end;
procedure TNullStream.InvalidSeek;
begin
raise ENullStreamError.Create(SInvalidOperation);
end;
function TNullStream.Read(var Buffer; Count : LongInt) : Longint;
begin
FillChar(Buffer,Count,0);
Result:=Count;
Inc(FPos,Count);
end;
function TNullStream.Write(const Buffer; Count : LongInt) : LongInt;
begin
Inc(FPos,Count);
end;
function TNullStream.Seek(const Offset: int64; Origin: TSeekOrigin): int64;
begin
if (Origin=soCurrent) and (Offset=0) then
Result:=FPos
else
FakeSeekForward(Offset,Origin,FPos);
end;
end.
___
fpc-pascal maillist - fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Get empty memory quickly or... an empty file
On 13/09/2013 16:17, Michael Van Canneyt wrote: Attached a NullStream implementation. Thanks ;) ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
[fpc-pascal] Re: returning a generic object?
Oops, sorry *must be because of my sleepiness :p -- View this message in context: http://free-pascal-general.1045716.n5.nabble.com/returning-a-generic-object-tp5716433p5716472.html Sent from the Free Pascal - General mailing list archive at Nabble.com. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] lnet for TCP daemon
Am 13.09.2013 10:47 schrieb Michael Schnell mschn...@lumino.de: On 09/13/2013 06:06 AM, wkitt...@windstream.net wrote: would this hamper or cause problems with normal non-gui programs using the library? 1) This feature of course should only be provided additionally to the functions we know and love. 2) Other than the work-alike Application.QueueAsyncCall which is provided by Lazarus long since, the rather new TThread.Queue is not only Delphi-compatible, but it also is located in the RTL and thus can be used in LCL based nongui applications and even without linking to the LCL at all. You need to do calls to a synchronize function which the RTL provides, to pull the event queue. This _can_ be done in a close loop (e.g. containing a sleep() call), which of course increases latency and processor overhead. Better it is done by a decent triggering mechanism (e.g. using a semaphore or self-piping) the loop waits for and each queue push triggers. This is another improvement I hope for: enhancing the synchronize and TThread.Queue/TThread.Synchronize features in the RTL in an OS-depending way that (optionally) automatically pulls the queue for the main thread. As we have already written in some previous mails to there is a global event procedure to wake up the main thread that is triggered when something is queued. The LCL uses this already and other programs could use a TEvent or whatever. Regards, Sven ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
