Re: [fpc-pascal] help with synapse mime routines...
On Fri, February 18, 2011 02:24, waldo kitty wrote: On 2/17/2011 18:16, Kevin Jesshope wrote: . . my next problem is to get it compiling on my OS/2 box with FPC 2.4.2 and so far, that is not working... complaints of missing units and include files that i just don't know where to tell the compiler to look for... my development has . . on my OS/2 box, i've tried targets of OS2 and OS2 with EMX but both have failed all kinds of ways... i suspect that it is simply to do with the paths to the units and include files... i have been able to get something going somewhat but then run into masses of duplicate identifier errors... I'm willing to have a look at your current version if you can upload it somewhere. Synapse has probably not been ported to the OS/2 target and it possibly uses various platform specific APIs but it should be possible to port it with reasonable effort (especially stuff like processing MIME messages should be fairly generic). First of all, you should check whether the missing units are supposed to be platform independent or not. If not, it is surely not just the matter of paths, etc. Also, if you mention concrete error messages, people may be able to advise. i'm just not sure where to turn now so out of frustration, i'm now pulling down the dos242full.zip of FPC and will be installing that to another directory on the OS/2 box... then it should be a simple matter of copying the project to another directory in there and giving it a kick in the arse to see what happens... hopefully i'll end up with a workable executable and my project with be, basically, at the end :) I'd recommend using the OS/2 target directly in your case; there are surely areas / functionalities which are better supported on the GO32v2 target (unit graph coming to my mind as one particular example), but I don't think that your project should really need those. Tomas ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
[fpc-pascal] assign code to a method
While waiting for a 64 bits Delphi, I am doing experimentation with
fpc-Lazarus 64 bits. And this is my problem:
In most of my programs I use my own library for parsing and evaluating
floating point functions like
'x sin(y z)'
passed by the user as strings. The basic organization is as follows:
{---simplified code}
TFunctionEvaluator= class(TObject)
...
code: AnsiString;
...
public
{call the parser}
function ParseAFunctText(t, vars: AnsiString): AnsiString;
{call the evaluator}
V: function(X: array of Double): Double;
end;
{--end code}
Suppose that theF is an already created instance of
TFunctionEvaluator and suppose that a program calls
msg:= theF.ParseAFunctText('pi', 'xyz');
Here, 'pi' is the text of the constant funcion that must give 3.14...
as result, 'xyz' contains the chars in the function text that the
parser should take as independent variables if they appear in t. The
result will be an error message msg if something fails. Then
ParseAFunctText builds code and puts
@V:= PChar(code);
Suppose that a program calls for example
r:= theF.V([1, 2, 3]);
Then Lazarus (fpc 2.2.4, lazarus 0.9.28.2 beta, target windows 64
bits, mode Delphi) compiles, but it raises an
'External:SIGSEGV' error.
In the debugger the exception is raised before the effective call of
the function code. If then I press F7, then the debugger jumps to the
following assemblies
push%rbp
mov %esp,%ebp
fldpi {loads pi to the fpu}
ftspl -0x18(%rbp) {copy pi to stack}
movsd -0x18(%rbp),%xmm0 {copies pi to XMM0, so caller
{may find also there the result}
leaveq
retq
I have tried all possible combinations like V:= PChar(code), V:=
@code[1], etc. to no avail. Thus I think something is wrong in my
understanding of the whole business in 64 bits or in fpc-Lazarus,
because this is not a problem in Delphi nor it was in an old version
of fpc-Lazarus 32 bits. Please give me a tip.
Many thanks in advance.
--
montesin at uv dot es
___
fpc-pascal maillist - fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] assign code to a method
On 18 Feb 2011, at 09:14, Angel Montesinos wrote: I have tried all possible combinations like V:= PChar(code), V:= @code[1], etc. to no avail. Thus I think something is wrong in my understanding of the whole business in 64 bits or in fpc-Lazarus, because this is not a problem in Delphi nor it was in an old version of fpc-Lazarus 32 bits. Please give me a tip. Many thanks in advance. You probably have to mark the memory in which you write the code as executable first (this has to be done using an OS-specific function; I know next to nothing about Windows programming, but you should be able to find the necessary information in MSDN). In fact, you should also do this on 32 bit platforms, since more and more systems mark memory by default as not executable for security purposes. Jonas PS: you should mention that you are not subscribed to the mailing list in all messages you send, because otherwise people won't know that they have to CC you when replying to your messages (I know only because I'm the list admin and let your message through when it was stuck in the moderation queue). ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] assign code to a method
In our previous episode, Jonas Maebe said: because this is not a problem in Delphi nor it was in an old version of fpc-Lazarus 32 bits. Please give me a tip. Many thanks in advance. You probably have to mark the memory in which you write the code as executable first (this has to be done using an OS-specific function; I know next to nothing about Windows programming, but you should be able to find the necessary information in MSDN). In fact, you should also do this on 32 bit platforms, since more and more systems mark memory by default as not executable for security purposes. This is called DEP (Data Execution Protection, which is an amalgam of resp AMD and Intel hardware protection and some software forms). IIRC it is always on for 64-bit, and for 32-bit server versions that use a PAE kernel. (client OSes don't have PAE version on windows afaik) Afaik it can only be turned on before starting a binary. An executable can't disable it while running (since then it would be useless, as a virus could simply disable it) See e.g. http://techblissonline.com/enable-disable-dep-in-windows-xp-vista/ Searching for disable dep or so will probably yield more results. Of course, this is not healthy and supportable application behaviour, but I assume this is already known. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] assign code to a method
Am 18.02.2011 12:16, schrieb Marco van de Voort: In our previous episode, Jonas Maebe said: because this is not a problem in Delphi nor it was in an old version of fpc-Lazarus 32 bits. Please give me a tip. Many thanks in advance. You probably have to mark the memory in which you write the code as executable first (this has to be done using an OS-specific function; I know next to nothing about Windows programming, but you should be able to find the necessary information in MSDN). In fact, you should also do this on 32 bit platforms, since more and more systems mark memory by default as not executable for security purposes. This is called DEP (Data Execution Protection, which is an amalgam of resp AMD and Intel hardware protection and some software forms). IIRC it is always on for 64-bit, and for 32-bit server versions that use a PAE kernel. (client OSes don't have PAE version on windows afaik) This is not entirely correct. Quote from Wikipedia regarding PAE: ===quote begin=== Windows XP SP2 and later, by default, on processors with the no-execute (NX) or execute-disable (XD) feature, runs in PAE mode in order to allow NX. ===quote end=== Afaik it can only be turned on before starting a binary. An executable can't disable it while running (since then it would be useless, as a virus could simply disable it) See e.g. http://techblissonline.com/enable-disable-dep-in-windows-xp-vista/ Searching for disable dep or so will probably yield more results. Of course, this is not healthy and supportable application behaviour, but I assume this is already known. I personally would say that mapping a page with the Execute flag set and storing the to-be-executed content there should be enough. After all JIT compilers must do that as well. ;) Regards, Sven ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] assign code to a method
Am 18.02.2011 13:46, schrieb Sven Barth: Am 18.02.2011 12:16, schrieb Marco van de Voort: In our previous episode, Jonas Maebe said: because this is not a problem in Delphi nor it was in an old version of fpc-Lazarus 32 bits. Please give me a tip. Many thanks in advance. You probably have to mark the memory in which you write the code as executable first (this has to be done using an OS-specific function; I know next to nothing about Windows programming, but you should be able to find the necessary information in MSDN). In fact, you should also do this on 32 bit platforms, since more and more systems mark memory by default as not executable for security purposes. This is called DEP (Data Execution Protection, which is an amalgam of resp AMD and Intel hardware protection and some software forms). IIRC it is always on for 64-bit, and for 32-bit server versions that use a PAE kernel. (client OSes don't have PAE version on windows afaik) This is not entirely correct. Quote from Wikipedia regarding PAE: ===quote begin=== Windows XP SP2 and later, by default, on processors with the no-execute (NX) or execute-disable (XD) feature, runs in PAE mode in order to allow NX. ===quote end=== Afaik it can only be turned on before starting a binary. An executable can't disable it while running (since then it would be useless, as a virus could simply disable it) See e.g. http://techblissonline.com/enable-disable-dep-in-windows-xp-vista/ Searching for disable dep or so will probably yield more results. Of course, this is not healthy and supportable application behaviour, but I assume this is already known. I personally would say that mapping a page with the Execute flag set and storing the to-be-executed content there should be enough. After all JIT compilers must do that as well. ;) Also according to here: http://en.wikipedia.org/wiki/Data_Execution_Prevention#Configuration DEP is not enabled by default for all applications. Regards, Sven ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] assign code to a method
In our previous episode, Sven Barth said: I personally would say that mapping a page with the Execute flag set and storing the to-be-executed content there should be enough. After all JIT compilers must do that as well. ;) Also according to here: http://en.wikipedia.org/wiki/Data_Execution_Prevention#Configuration DEP is not enabled by default for all applications. See the exception of OptIn. Both are before W7 released, so I assume it goes for W7/64 too. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] assign code to a method
On 02/18/11 03:14, Angel Montesinos wrote:
While waiting for a 64 bits Delphi, I am doing experimentation with
fpc-Lazarus 64 bits. And this is my problem:
In most of my programs I use my own library for parsing and evaluating
floating point functions like
'x sin(y z)'
passed by the user as strings. The basic organization is as follows:
In the debugger the exception is raised before the effective call of the
function code. If then I press F7, then the debugger jumps to the
following assemblies
push%rbp
mov %esp,%ebp
fldpi {loads pi to the fpu}
ftspl -0x18(%rbp) {copy pi to stack}
movsd -0x18(%rbp),%xmm0 {copies pi to XMM0, so caller
{may find also there the result}
leaveq
retq
I have tried all possible combinations like V:= PChar(code), V:=
@code[1], etc. to no avail. Thus I think something is wrong in my
understanding of the whole business in 64 bits or in fpc-Lazarus,
because this is not a problem in Delphi nor it was in an old version of
fpc-Lazarus 32 bits. Please give me a tip.
Many thanks in advance.
From the other comments it seems like you are writing some assembly to
memory at runtime then calling that code? If so then maybe the following
can help you.
I made some trampoline procedures where I wrote some executable code at
runtime and I had to, both on Window and Linux, allocate some memory
that was marked as executable. Here's a snippet of how I allocated the
memory. The code worked on Windows 7 x64/32 and Gentoo Linux x64/32
PTrampolineBlock = ^TTrampolineBlock;
TTrampolineBlock = record
NextBlock: Pointer;
Cursor: DWord;
Size: DWord;
Code: PtrUint; // this is actually the start of the remainder of the
allocated size
end;
procedure TTrampolineManager.AllocateBlock;
const
AllocSize = $1000;
var
NewBlock: PTrampolineBlock;
begin
{$IFDEF UNIX}
NewBlock := fpmmap(nil, AllocSize, PROT_EXEC or PROT_READ or
PROT_WRITE, MAP_ANONYMOUS or MAP_PRIVATE, -1, 0);
{$ENDIF}
{$IFDEF MSWINDOWS}
NewBlock := VirtualAlloc(nil,AllocSize,MEM_COMMIT,PAGE_EXECUTE_READWRITE);
{$ENDIF}
NewBlock^.Cursor:=PtrUint(@NewBlock^.Code)-PtrUint(@NewBlock^.NextBlock);
NewBlock^.Size:=AllocSize;
NewBlock^.NextBlock:=Block;
Block := NewBlock;
end;
procedure TTrampolineManager.FreeBlock(var ABlock: PTrampolineBlock);
begin
if ABlock = nil then
Exit;
FreeBlock(ABlock^.NextBlock);
{$IFDEF UNIX}
Fpmunmap(ABlock, ABlock^.Size);
{$ENDIF}
{$IFDEF MSWINDOWS}
VirtualFree(ABlock, 0, MEM_RELEASE);
{$ENDIF}
ABlock := nil;
end;
procedure TTrampolineManager.WriteData(const AData; ASize: Byte);
begin
if Block = nil then AllocateBlock;
//WriteLn('Writing: ', hexStr(PtrUint(AData), ASize*2));
if Block^.Cursor + ASize = Block^.Size then
raise TrampolineBlockFullException.Create('');
Move(AData, PByte(@Block^.Code)[Block^.Cursor], ASize);
Inc(Block^.Cursor, ASize);
end;
so the usage would be like so
function TTrampolineManager.GenerateCode(args: ): Pointer;
begin
try
Result := CurrentBlock.Position; // = @Block + Block.Cursor
repeat
WriteData(your_data, size_of_data);
until done;
except
on e: TrampolineBlockFullException do
begin
TrampolineManager.AllocateBlock;
Result := GenerateCode(args);
end;
end;
end;
Hope this helps :)
Regards,
Andrew Haines
___
fpc-pascal maillist - fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
[fpc-pascal] Fingerprinting operation of a program
I need to trace through a program, running on two different systems which might or might not have the same OS, in order to find out where it's failing. Before I start giving myself a gdb refresher course or manually inserting trace messages, is there any way that I can extract a trace of procedure entry/exits, preferably identified by name rather than by address, using e.g. profiling hooks? -- Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues] ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] help with synapse mime routines...
On 2/18/2011 04:47, Tomas Hajny wrote: On Fri, February 18, 2011 02:24, waldo kitty wrote: on my OS/2 box, i've tried targets of OS2 and OS2 with EMX but both have failed all kinds of ways... i suspect that it is simply to do with the paths to the units and include files... i have been able to get something going somewhat but then run into masses of duplicate identifier errors... I'm willing to have a look at your current version if you can upload it somewhere. my code is pretty small... it should easily fit the attachment limitations of this list :) i still have a few more tweaks to add to it as a few things are still specific to certain types of messages but overall, it is pretty straight forward ;) Synapse has probably not been ported to the OS/2 target and it possibly uses various platform specific APIs but it should be possible to port it with reasonable effort (especially stuff like processing MIME messages should be fairly generic). that is/was my thinking as well... First of all, you should check whether the missing units are supposed to be platform independent or not. yes, part of what i'm running into is platform dependency problems... i forget which unit it is that's triggering the platform stuff but where it is failing is when trying to bring in platform stuff like loading libraries and there's nothing in my code that needs any libraries to be loaded for any reason that i'm aware of... it is possible that WalkParts might want something loaded from a lib but i do not need anything like that for my immediate purposes since i'm only looking for the text/plain section(s) and extracting them to an ASCII text file... If not, it is surely not just the matter of paths, etc. Also, if you mention concrete error messages, people may be able to advise. i shall try to grab these in the future... one thing that may also be causing waves is that my OS/2 and DOS FPCs are 2.4.2 releases whereas on my development machine(s), i'm pulling FPC and lazarus from their SVNs and building them from the ground up... both of my development machines started with nothing on them until i pulled the code from SVN and the bootstrap compiler to kick it in the arse and build working environments... i'm just not sure where to turn now so out of frustration, i'm now pulling down the dos242full.zip of FPC and will be installing that to another directory on the OS/2 box... then it should be a simple matter of copying the project to another directory in there and giving it a kick in the arse to see what happens... hopefully i'll end up with a workable executable and my project with be, basically, at the end :) yeah, the above failed miserably... i didn't realize that LFNs were necessary and the DOS version on OS/2 simply cannot access OS/2 LFNs in any way shape or fashion at all... I'd recommend using the OS/2 target directly in your case; OS/2 straight? not the OS/2 w/ EMX?? it really doesn't matter to me as long as i end up with a working executable for the task at hand... there are surely areas / functionalities which are better supported on the GO32v2 target (unit graph coming to my mind as one particular example), but I don't think that your project should really need those. you are absolutely 1000+% correct on that... there's nothing visual... the tool is fired from a console command line... there's little to no output (unless i decide to add some for logging purposes)... and, lastly, the tool is doing nothing more than extracting a/the text/plain MIME section(s) from messages saved as text files... the only other thing the tool is doing is converting characters like the trademark symbol from their unicode(???) character back to the old DOS 8bit/7bit old school alternative... ie: the trademark symbol is located as the = 9 9 (spaced to prevent conversion) MIME text in the message body and converted to (tm) which any old DOS 3.x can understand :) let me fix up a few things and i'll post another message, in the list or privately, containing my current code... i'll also try to bang up the error messages from the attempts to compile to the OS/2 (native) target... thanks for your interest and assistance! ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] help with synapse mime routines...
On 2/18/2011 04:47, Tomas Hajny wrote:
On Fri, February 18, 2011 02:24, waldo kitty wrote:
I'm willing to have a look at your current version if you can upload it
somewhere. Synapse has probably not been ported to the OS/2 target and it
possibly uses various platform specific APIs but it should be possible to
port it with reasonable effort (especially stuff like processing MIME
messages should be fairly generic). First of all, you should check whether
the missing units are supposed to be platform independent or not. If not,
it is surely not just the matter of paths, etc. Also, if you mention
concrete error messages, people may be able to advise.
ok, as i wrote before, i would attempt to provide more information... here's
what i'm getting with the attached source code when attempting to compile a
GO32V2 (native) target on my win2k development box... this environment is from
the DOS242FULL.ZIP archive with the same last synapse public release code as
previously used in my i386-win32 console compilations... this is the same/only
public release code available from the synapse site... i have NOT (yet)
attempted to pull the SVN synapse code... yes, this synapse code appears to be
ancient in that it is dated 2009 (IIRC!)...
Fatal: Can't find unit dynlibs used by synafpc
my sources, yet unmodified for more general processing, are attached... yes, it
is ugly and i need to move at least one section to another procedure or two
but here it is in all its ugliness ;)
i /THINK/ this is the same initial error i get when attempting to compile on
OS/2 from the OS2242FULL.ZIP archive... on each, they are set up as a separate
and distinct entity and the exact same synapse sources are copied to each
distinct base installation's projects directory... the only thing i've done in
each case is to add the synapse sources' directory to the unit and include files
sections in the FPC IDE...
i hope this is clear and understandable... i will provide another post with the
necessary information for the OS/2 native side if necessary... just let me know,
eh? :P
{$MODE DELPHI}
Program ArnewsMimeExtractor;
uses
mimepart,
synachar,
synautil,
classes;
type
Tc = class(TObject)
public
class procedure ph(const Sender: TMimePart);
end;
class procedure Tc.ph(const Sender: TMimePart);
begin
// Sender.Charset := 'ISO_8859_1';
// Sender.DefaultCharset := 'ISO_8859_1';
// Sender.TargetCharset := GetCPFromID('ISO_8859_1');
Sender.ConvertCharset := FALSE;
Sender.DecodePart;
Sender.EncodePart;
end;
var
LineList: tstringlist;
MsgIn: tmimepart;
PartCount: integer;
MsgWork: tmimepart;
MyPart: integer;
MyPartCount: integer;
i: integer;
cpos: integer;
llen: integer;
ArrayCount: integer;
FindChars: array of ansistring;
ReplaceChars: array of ansistring;
begin
// set up arrays for character string replacements
ArrayCount := 14;
setlength(FindChars,ArrayCount);
setlength(ReplaceChars,ArrayCount);
FindChars[0]:= '=20'; ReplaceChars[0] := ' ';
FindChars[1]:= '=3D'; ReplaceChars[1] := '=';
FindChars[2]:= '=85'; ReplaceChars[2] := '...';
FindChars[3]:= '=96'; ReplaceChars[3] := '-';
FindChars[4]:= '=99'; ReplaceChars[4] := '(tm)';
FindChars[5]:= '=A0'; ReplaceChars[5] := ' ';
FindChars[6]:= '=A7'; ReplaceChars[6] := '§';
FindChars[7]:= '=AD'; ReplaceChars[7] := ' ';
FindChars[8]:= '=AE'; ReplaceChars[8] := '(r)';
FindChars[9]:= '=B7'; ReplaceChars[9] := '·';
FindChars[10] := '=D8'; ReplaceChars[10] := 'Ø';
FindChars[11] := '=E1'; ReplaceChars[11] := 'á';
FindChars[12] := '=F6'; ReplaceChars[12] := '';
FindChars[13] := ' '; ReplaceChars[13] := ' ';
MsgIn := tmimepart.create;
LineList := TStringList.create;
writeln('Processing file ',paramstr(1),'...');
try
MsgIn.OnWalkPart:=tc.ph;
MsgIn.Lines.LoadFromFile(paramstr(1));
writeln(' Working with ',MsgIn.Lines.Count,' lines in message file...');
MsgIn.Charset := 'ISO_8859_1';
MsgIn.DefaultCharset := 'ISO_8859_1';
MsgIn.TargetCharset := GetCPFromID('ISO_8859_1');
MsgIn.ConvertCharset := FALSE;
MsgIn.DecomposeParts;
MsgIn.Charset := 'ISO_8859_1';
MsgIn.DefaultCharset := 'ISO_8859_1';
MsgIn.TargetCharset := GetCPFromID('ISO_8859_1');
MsgIn.ConvertCharset := FALSE;
MsgIn.WalkPart;
PartCount := MsgIn.GetSubPartCount;
MsgIn.Charset := 'ISO_8859_1';
MsgIn.DefaultCharset := 'ISO_8859_1';
MsgIn.TargetCharset := GetCPFromID('ISO_8859_1');
MsgIn.ConvertCharset := FALSE;
//writeln('MsgIn.GetSubPartCount: ',PartCount);
//writeln('MsgIn.Primary: ',MsgIn.Primary);
//writeln('MsgIn.Encoding : ',MsgIn.Encoding);
//writeln('MsgIn.Charset: ',MsgIn.Charset);
//writeln('MsgIn.DefaultCharset : ',MsgIn.DefaultCharset);
//writeln('MsgIn.PrimaryCode: ',MsgIn.PrimaryCode);
//writeln('MsgIn.EncodingCode :
[fpc-pascal] Handling recursive symlinks in Linux
I'm trying to convert a Delphi console-mode program which needs to walk directory trees to FP **and Linux** - is there a standard way to detect and handle recursive symlinks, if the user has been daft enough to create them? Thanks, Brian. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
