Re: [fpc-devel] refcounting in fcl-passrc
On Sun, 22 Aug 2010, Marco van de Voort wrote: I still get errors on deallocation in fpdoc from time to time. The most recent one is related to TPasFunctionType. I searched a bit, and since the type is not used that much, I suspect TPasTreeContainer.CreateFunctionType. Since in both cases the resultparent is an already used reference, shouldn't it be increfed ? Probably. Till you told me, I wasn't even aware that there was a refcounting mechanism. I see no reason why there should be one, so I'd be in favour of removing it alltogether. Michael. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] refcounting in fcl-passrc
In our previous episode, Michael Van Canneyt said: > > Since in both cases the resultparent is an already used reference, shouldn't > > it be increfed ? > > Probably. > > Till you told me, I wasn't even aware that there was a refcounting mechanism. > I see no reason why there should be one, so I'd be in favour of removing it > alltogether. Then you need some global list to add them to. (to be able to deallocate). It might be the various TPasModule sections already double as something like this, but that has to be validated. Dmitry, what do you think? ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] refcounting in fcl-passrc
On Sun, 22 Aug 2010, Marco van de Voort wrote: In our previous episode, Michael Van Canneyt said: Since in both cases the resultparent is an already used reference, shouldn't it be increfed ? Probably. Till you told me, I wasn't even aware that there was a refcounting mechanism. I see no reason why there should be one, so I'd be in favour of removing it alltogether. Then you need some global list to add them to. (to be able to deallocate). Why ? It might be the various TPasModule sections already double as something like this, but that has to be validated. Each element should deallocate it's "children". Since an element tree is always a strict tree, this is unambiguous. Refcounting is not needed at all. I'm not in favour of a global list and a 'garbage collection' style of memory management. It makes manipulating the tree a pain. This will take some reworking, apparently (each destructor must be filled with code to deallocate children). For the time being, try the incref. Michael. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] refcounting in fcl-passrc
In our previous episode, Michael Van Canneyt said: > > It might be the various TPasModule sections already double as something like > > this, but that has to be validated. > > Each element should deallocate it's "children". > Since an element tree is always a strict tree, this is unambiguous. I don't really see this, since in many of passrc's routines objects are created if they cannot be found? This means that there is a chance that such object has no other refernece than the one returned. E.g. when searched for a reference that is not strictly a child like ancestor. That would need to be refactored too then. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
[fpc-devel] [patch] pscanner: differentiate between EOL & Tab characters from general Whitespace
Hi, In the following bug report, I attached a minor patch for the pascal scanner in fcl-passrc. http://bugs.freepascal.org/view.php?id=17237 I introduced two new tokens. Without this it is not possible to rebuild a source code unit from a token list. -- Regards, - Graeme - ___ fpGUI - a cross-platform Free Pascal GUI toolkit http://opensoft.homeip.net/fpgui/ ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] refcounting in fcl-passrc
If ref-counting mechanism is already implemented, i'd prefer to keep and debug it. thanks, dmitry ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] [patch] pscanner: differentiate between EOL & Tab characters from general Whitespace
On Sun, 22 Aug 2010, Graeme Geldenhuys wrote: Hi, In the following bug report, I attached a minor patch for the pascal scanner in fcl-passrc. http://bugs.freepascal.org/view.php?id=17237 I introduced two new tokens. Without this it is not possible to rebuild a source code unit from a token list. I cannot accept this patch as it is. It breaks the parser, more specifically NextToken. The parser ignores whitespace. It should also ignore the new tokens. Other than that, I doubt the usefulness of this patch. You are on the wrong track with what you attempt to do. Michael ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] refcounting in fcl-passrc
On Sun, 22 Aug 2010, dmitry boyarintsev wrote: If ref-counting mechanism is already implemented, i'd prefer to keep and debug it. And why do you want this ? refcounting is always a pain, and should be avoided when possible. Michael. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] [patch] pscanner: differentiate between EOL & Tab characters from general Whitespace
In our previous episode, Michael Van Canneyt said: > I cannot accept this patch as it is. > > It breaks the parser, more specifically NextToken. > The parser ignores whitespace. It should also ignore the new tokens. > Other than that, I doubt the usefulness of this patch. > You are on the wrong track with what you attempt to do. He wants the source as a kind of DOM model. That is IMHO a valid use, but I doubt it is unifiable with the current parser. This is typically one of the reasons why the "one parser to rule them all" approach is flawed. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] refcounting in fcl-passrc
On Sun, Aug 22, 2010 at 11:32 AM, Michael Van Canneyt wrote: > And why do you want this ? refcounting is always a pain, and should be > avoided when possible. Rewriting code might produce even more bugs. I prefer to make as less changes as possible. thanks, dmitry ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] [patch] pscanner: differentiate between EOL & Tab characters from general Whitespace
On 22 August 2010 13:31, Michael Van Canneyt wrote: > > I cannot accept this patch as it is. > > It breaks the parser, more specifically NextToken. > The parser ignores whitespace. It should also ignore the new tokens. Oh, I'll take another look. Let me explain what I'm doing - maybe it makes more sense then. I'm using the tokenizer/scanner and generating a list of all tokens. Then search for specific areas in the code using the tokens (uses clauses, begin..end pares of procedures or methods), then insert timing code, generate the units with newly included code, do a compile so I can get profiling/timing information. So far this works just fine for me, and already found some "hot spots" in my DocView code. After all this, I Googled profilers to see what everything they check for. I searching, I stumbled accross the fpprofiler which is exactly what I was trying to do (don't you just hate that - after you wrote some code). Funny enough, the fpprofile also only uses the pascal scanner. I tried using the parser as you suggested in a private email yesterday, but that made my profiler 2x slower at processing my source code (I used EpikTimer to get high precision timing results). The scanner seems good enough for what I want. But without differentiating between general whitespace and EOL (required) and Tab (optional), I can't regenerate my source code units. Anyway, I abandoned my profiling code and started working with fpprofiler (it was a bit further along than my own attempt), but it wasn't touched for over 2 years, so didn't compile. It was also riddled with memory leaks (sorry Darius Blaszyk). I fixed a lot of things including memory leaks, removed the custom parser to rather use the one included in FCL etc.. But the final step is tokenizing the EOL character correctly - which fcl-passrc doesn't do. BTW: fcl-passrc can't parse program units either, but I'm already working on a patch for that. > Other than that, I doubt the usefulness of this patch. You are on the wrong > track with what you attempt to do. Based on the information above, and what I'm trying to do, do you still think I'm on the wrong track? Otherwise, what do you suggest? I think all I need to do, is add the LineEnding token to the parser to make sure I didn't break it (sorry, I never tested the parser before I generated the patch). -- Regards, - Graeme - ___ fpGUI - a cross-platform Free Pascal GUI toolkit http://opensoft.homeip.net/fpgui/ ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] [patch] pscanner: differentiate between EOL & Tab characters from general Whitespace
On 22 August 2010 13:35, Marco van de Voort wrote: > > He wants the source as a kind of DOM model. That is IMHO a valid use, but I > doubt it is unifiable with the current parser. Not nearly as complicated as a DOM model really. I just want to insert some timing calls and a new entry to the uses clause. See my reply to Michael to understand what I am trying to accomplish. > This is typically one of the reasons why the "one parser to rule them all" > approach is flawed. For fpGUI IDE, I hope to do more code modification/generation using fcl-passrc, so then I'll can answer that question. :) -- Regards, - Graeme - ___ fpGUI - a cross-platform Free Pascal GUI toolkit http://opensoft.homeip.net/fpgui/ ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] [patch] pscanner: differentiate between EOL & Tab characters from general Whitespace
On 22 August 2010 13:31, Michael Van Canneyt wrote: > > It breaks the parser, more specifically NextToken. > The parser ignores whitespace. It should also ignore the new tokens. I attached a parser patch to that bug report in Mantis. I tested the scanner and parser over various units of mine, and it seems to work fine now. -- Regards, - Graeme - ___ fpGUI - a cross-platform Free Pascal GUI toolkit http://opensoft.homeip.net/fpgui/ ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] [patch] pscanner: differentiate between EOL & Tab characters from general Whitespace
On Sun, 22 Aug 2010, Graeme Geldenhuys wrote: On 22 August 2010 13:31, Michael Van Canneyt wrote: I cannot accept this patch as it is. It breaks the parser, more specifically NextToken. The parser ignores whitespace. It should also ignore the new tokens. Oh, I'll take another look. Let me explain what I'm doing - maybe it makes more sense then. I'm using the tokenizer/scanner and generating a list of all tokens. Then search for specific areas in the code using the tokens (uses clauses, begin..end pares of procedures or methods), then insert timing code, generate the units with newly included code, do a compile so I can get profiling/timing information. So far this works just fine for me, and already found some "hot spots" in my DocView code. After all this, I Googled profilers to see what everything they check for. I searching, I stumbled accross the fpprofiler which is exactly what I was trying to do (don't you just hate that - after you wrote some code). Funny enough, the fpprofile also only uses the pascal scanner. I tried using the parser as you suggested in a private email yesterday, but that made my profiler 2x slower at processing my source code (I used EpikTimer to get high precision timing results). The scanner seems good enough for what I want. But without differentiating between general whitespace and EOL (required) and Tab (optional), I can't regenerate my source code units. Anyway, I abandoned my profiling code and started working with fpprofiler (it was a bit further along than my own attempt), but it wasn't touched for over 2 years, so didn't compile. It was also riddled with memory leaks (sorry Darius Blaszyk). I fixed a lot of things including memory leaks, removed the custom parser to rather use the one included in FCL etc.. But the final step is tokenizing the EOL character correctly - which fcl-passrc doesn't do. Why ? To the compiler there is no difference between EOL and whitespace. If you want to generate it with the exact layout, then, yes. But for profiling I don't see the need. BTW: fcl-passrc can't parse program units either, but I'm already working on a patch for that. This is a known shortcoming. Other than that, I doubt the usefulness of this patch. You are on the wrong track with what you attempt to do. Based on the information above, and what I'm trying to do, do you still think I'm on the wrong track? Otherwise, what do you suggest? I think all I need to do, is add the LineEnding token to the parser to make sure I didn't break it (sorry, I never tested the parser before I generated the patch). Well, I still fail to see why you need the tab and EOL. Michael. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] refcounting in fcl-passrc
On Sun, 22 Aug 2010, dmitry boyarintsev wrote: On Sun, Aug 22, 2010 at 11:32 AM, Michael Van Canneyt wrote: And why do you want this ? refcounting is always a pain, and should be avoided when possible. Rewriting code might produce even more bugs. I prefer to make as less changes as possible. We would not be rewriting the parser; merely the TPasElement nodes. I'll do it myself, this is not a problem. Michael. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] refcounting in fcl-passrc
On Sun, Aug 22, 2010 at 4:27 PM, Michael Van Canneyt wrote: > We would not be rewriting the parser; merely the TPasElement nodes. > > I'll do it myself, this is not a problem. Ok then. No more refcounting. So TPasExpr classes doesn't require changes. Currently they free sub expressions instead of releasing them. thanks, dmitry ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] [patch] pscanner: differentiate between EOL & Tab characters from general Whitespace
On 22 August 2010 18:25, Michael Van Canneyt wrote:
>
> Why ? To the compiler there is no difference between EOL and whitespace.
>
> If you want to generate it with the exact layout, then, yes.
> But for profiling I don't see the need.
Maybe to the compiler it doesn't matter, but to a human trying to
debug their profiler to see if they inserted code in the correct
place, or to see why code doesn't compile any more after the profiler
modified the code - such "human readable code" is rather important.
:)
If no NewLine characters are inserted into the newly generated units,
a compiler error on line 1 doesn't help the developer much. Line 1
could be 50k bytes or more long.
Lazarus can't open the file (error message says it doesn't look like a
text file), other editor show one line, but clips text because they
can't display such a long single line of text. The profiler becomes a
near perfect obfuscater. :-)
>
> Well, I still fail to see why you need the tab and EOL.
Well, does it hurt having them tokenized?
If anybody other than me comes up with a new idea/way of using the
fcl-passrc parser on tokenizer - maybe they would like to rebuild
source code at some point. Without tokens telling you where EOL used
to be or TAB characters used to be (though the latter is not to
important), it does limit it's functionality.
Attached is a file processed with the profiler and fcl-passrc that
doesn't use tkLineEnding or tkTab. It's rather hard to read and fix
compiler errors introduced by the profile inserting code.
I don't see why introducing such new tokens is bad? It can only make
the tokenizer and parser more useful in the long term.
--
Regards,
- Graeme -
___
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/
unit fpputils; interface uses fpprof, pscanner, PasTree, SysUtils, Classes, FPPWriter; const FPPROF_EXT = '.fpprof'; type TPasToken = record token: TToken; value: string; end; PPasToken = ^TPasToken; TPasTokenList = class(TObject) FFileName: string; private FList: TFPList; function GetList(index: integer): TPasToken; procedure SetFileName(const AValue: string); procedure SetList(AIndex: integer; const AValue: TPasToken); public constructor Create; destructor Destroy; override; function Count: integer; function ParseSource(AFileName: string): boolean; procedure Add(AToken: TToken; AValue: string); procedure Clear; procedure Insert(APos: integer; AToken: TToken; AValue: string); procedure SaveToFile(const AFileName: string); property FileName: string read FFileName write SetFileName; property List[index: integer]: TPasToken read GetList write SetList; default; end; TModTokenProc = procedure(AFileName: string; tokenlist: TPasTokenList); procedure FileSearch(SearchDir: string; ExtensionMask: string; var FileList: TStrings; Recursive: boolean = false); procedure InsertProfilingCode(FileList: TStrings; ModTokenProc: TModTokenProc); procedure RemoveProfilingCodeFromFile(const FileName: string); procedure RemoveProfilingCode(FileList: TStrings); implementation procedure FileSearch(SearchDir: string; ExtensionMask: string; var FileList: TStrings; Recursive: boolean = false); var Info : TSearchRec; ExtensionList: TStrings; begin fpprof_entry_profile;
SearchDir := IncludeTrailingPathDelimiter(SearchDir); ExtensionList := TStringList.Create; ExtensionList.Delimiter := ';'; ExtensionList.DelimitedText := ExtensionMask; if FindFirst(SearchDir+AllFilesMask, faAnyFile and faDirectory, Info) = 0 then begin repeat if Recursive then if ((Info.Attr and faDirectory) = faDirectory) and (Info.Name <> '.') and (Info.Name <> '..')then FileSearch(SearchDir + Info.Name, ExtensionMask, FileList, Recursive); if ExtensionList.IndexOf(ExtractFileExt(Info.Name)) <> -1 then begin if FileList.IndexOf(SearchDir + Info.Name) = -1 then FileList.Add(SearchDir + Info.Name); end; until FindNext(Info)<>0; end; FindClose(Info); ExtensionList.Free; fpprof_exit_profile;
end; procedure InsertProfilingCode(FileList: TStrings; ModTokenProc: TModTokenProc); var i: integer; PasTokenList: TPasTokenList; Success: boolean; writer: TFPPWriter; begin fpprof_entry_profile;
FileList.SaveToFile('./file_processed_list.txt'); PasTokenList := TPasTokenList.Create; writer := TFPPWriter.Create; writer.CreateIgnored; for i := 0 to FileList.Count - 1 do begin write('insert: ', FileList[i]); try Success := PasTokenList.ParseSource(FileList[i]); Success := Success and RenameFile(FileList[i], FileList[i] + FPPROF_EXT); if Assigned(ModTokenProc) then ModTokenProc(FileList[i], PasTokenList); PasTokenList.SaveToFile(FileList[i]);if Success then writeln(' .. OK') except writeln(' .. FAIL'); writer.AddIgnoredFile(FileList[i]); end; PasTokenList.Clear; end; writer.Save; writer.Free; PasTokenList.Free; FileList.Free; fpprof_exit_profile;
end; procedure Remov
Re: [fpc-devel] Building and running FPC natively on ARM under Debian "Lenny"
On 21 Aug 2010, at 00:19, Mark Morgan Lloyd wrote: > Jonas Maebe wrote: >> On 20 Aug 2010, at 22:38, Mark Morgan Lloyd wrote: >>> What should I be doing here- using both -Cf and -d? >> Yes, that should solve the problem. For the resulting EABI compiler, -Cfsoft >> will be the default. > > Thanks Jonas, work continues and I'll report back. > > Is there a way of getting FPC to report its build parameters? For example, if > I say fpc -i to 2.4.0 I get > > .. > Supported ABI targets: > DEFAULT > .. > > but it doesn't actually say what the default is. To check whether or not you have an ARM EABI compiler, compile any file with -vei and verify that the output contains the line: Target OS: Linux for ARMEL (rather than "Linux for ARM") Jonas___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
[fpc-devel] CHM help snapshot
Hello, I've generated a new help snapshot. (chm files) The archive still is at http://www.stack.nl/~marcov/doc-chm.zip and contains rtl,fcl,lcl ,fpdoc,ref,user,prog.chm This release benefits from several improvements in the fpdoc generation, specially links between packages, and links in class "inheritance" descriptions. As a result of this, the size also increased considerably, specially the LCL one. Greetings Marco ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] [patch] pscanner: differentiate between EOL & Tab characters from general Whitespace
- "Graeme Geldenhuys" schreef: > On 22 August 2010 13:35, Marco van de Voort wrote: > > > > He wants the source as a kind of DOM model. That is IMHO a valid > use, but I > > doubt it is unifiable with the current parser. > > Not nearly as complicated as a DOM model really. I just want to > insert > some timing calls and a new entry to the uses clause. See my reply to > Michael to understand what I am trying to accomplish. > > > > This is typically one of the reasons why the "one parser to rule > them all" > > approach is flawed. > > For fpGUI IDE, I hope to do more code modification/generation using > fcl-passrc, so then I'll can answer that question. :) > mmm, looks to me that you are trying to implement some kind of AOP (Aspect Oriented Programming) for this. (which IS interesting) The way most profilers that I've worked with did it was adding those timing-stuff (and counting) on a link-phase. (or through a custom exe-loader). Every PIC/method-table or VMT was replaced by a stubb. Or is it something else you want? (custom attributes, the .net way, with AOP methods and a method sink?) Anyway, if so, would this "profiling" stuff be optional, on/off all or on/off for a specific method/class/unit/package/program? kind regards, Dimitri Smits ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Building and running FPC natively on ARM under Debian "Lenny"
Jonas Maebe wrote: Supported ABI targets: DEFAULT .. but it doesn't actually say what the default is. To check whether or not you have an ARM EABI compiler, compile any file with -vei and verify that the output contains the line: Target OS: Linux for ARMEL (rather than "Linux for ARM") Thanks Jonas, noted. Build still running- significant memory overcommit. I'll report back when done. -- Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues] ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] [patch] pscanner: differentiate between EOL & Tab characters from general Whitespace
On 22 August 2010 20:50, Dimitri Smits wrote: > > Anyway, if so, would this "profiling" stuff be optional, on/off all or on/off > for a specific method/class/unit/package/program? fpprofiler (which I now switched to instead of my own) is optional. Profiling is applied per unit, and then "attaches" to every procedure/function or method in that unit. fpprofiler can be told to ignore certain units. How it currently works, is that you pass the exact same compiler parameters to the profiler, as you would to compile your project. It inserts the profiling code, compilers your project with the profiling code, then reverts all profiling changes. You end up with your "normal" code, but with a "fat" executable which does profile logging for you. Reporting wise, it does timing, call count and generates a call graph. Output formats are Plain Text, LaTeX and PNG images for the call graph. What I like is that it is really easy to get started, and the profiling information is nice and easy to understand and interpret. Valgrind, callgrind, cachegrind, coffee-grinder etc... just seems very complex to interpret. This will scare of any developer new to profiling. Another BIG plus point for fpprofiler, is that it only requires FPC - no other dependency. Something I always like - low dependencies! :) I'm planning on integrating fpprofiler (or a similar profiler) into fpGUI IDE (my sideline project), so you can enable it by a toggling a checkbox (just like you currently can for Valgrind or Gprof via a command line parameter). What I want to add extra into fpGUI IDE, is more visual screen that interpret the profiling data. Anyway, I'm getting way off-topic here... I'll wait an see what Michael says regarding my last email and the new patch to fix the pparser unit. -- Regards, - Graeme - ___ fpGUI - a cross-platform Free Pascal GUI toolkit http://opensoft.homeip.net/fpgui/ ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] [patch] pscanner: differentiate between EOL & Tab characters from general Whitespace
Graeme Geldenhuys schrieb: If no NewLine characters are inserted into the newly generated units, a compiler error on line 1 doesn't help the developer much. Line 1 could be 50k bytes or more long. The FPC scanner uses a flag (EolFound?), to signal an EOL to the preprocessor. DoDi ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] [patch] pscanner: differentiate between EOL & Tab characters from general Whitespace
On Sun, 22 Aug 2010, Graeme Geldenhuys wrote: Well, does it hurt having them tokenized? Of course. It slows the code. Marginally, I agree, but nevertheless. If anybody other than me comes up with a new idea/way of using the fcl-passrc parser on tokenizer - maybe they would like to rebuild source code at some point. Without tokens telling you where EOL used to be or TAB characters used to be (though the latter is not to important), it does limit it's functionality. Attached is a file processed with the profiler and fcl-passrc that doesn't use tkLineEnding or tkTab. It's rather hard to read and fix compiler errors introduced by the profile inserting code. I don't see why introducing such new tokens is bad? It can only make the tokenizer and parser more useful in the long term. I never said it is bad; I just don't see why you need it. You could perfectly do it differently, e.g. insert a newline after certain tokens: then begin ; end else var type const That would perfectly be enough for your purposes, which is, to debug your profiler. And once that is done, it is no longer needed at all. Don't worry, I'll apply the patch, but you have not convinced me it is a needed patch, even for your purposes. I'm surprised you didn't think of the above ways to do what you need. Michael ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] [patch] pscanner: differentiate between EOL & Tab characters from general Whitespace
2010/8/23 Michael Van Canneyt : >> I don't see why introducing such new tokens is bad? It can only make >> the tokenizer and parser more useful in the long term. > > I never said it is bad; I just don't see why you need it. I guess one wants the line numbers between the orginal source to be the same, for example to have the same line info in case of a backtrace when you compare before and after the changes. > > You could perfectly do it differently, e.g. insert a newline after certain > tokens: > then > begin > ; > end > else > var > type > const > So, programs compiled with profile info generate different backtraces with line info as programs without them. That is not nice. > That would perfectly be enough for your purposes, which is, to debug your > profiler. > And once that is done, it is no longer needed at all. > Why is it not longer needed? > Don't worry, I'll apply the patch, but you have not convinced me it is a > needed patch, even for your purposes. I'm surprised you didn't think of the > above ways to do what you need. I think with the new tokens a profiler can do a better job. Vincent ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] [patch] pscanner: differentiate between EOL & Tab characters from general Whitespace
Op 2010-08-23 01:17, Michael Van Canneyt het geskryf: >> Well, does it hurt having them tokenized? > > Of course. > It slows the code. Marginally, I agree, but nevertheless. Sorry, but I disagree here. Before it got tokenized as a tkWhitespace; now it gets tokenized as tkLineEnding or tkTab. It's still being assigned a token, just one that describes it better. As for the parsers NextToken method now having two extra tokens it needs to ignore. I profiled that, and the difference was too small to notice. > I never said it is bad; I just don't see why you need it. > > You could perfectly do it differently, e.g. insert a newline after certain > tokens: > then > begin > ; Vincent hit the nail on the head. Now the compiled code with profiling has very different line numbers (now much more than before) to the real code. So line number info reported by LineInfo unit is now even more useless. And considering that under 64-bit systems, method names are not reported by LineInfo for some reason, the line number info is rather important. > needed patch, even for your purposes. I'm surprised you didn't think of > the above ways to do what you need. See two paragraphs back. ;-) Regards, - Graeme - -- fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal http://opensoft.homeip.net/fpgui/ ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
