Re: [fpc-devel] FPDoc imports
michael.vancann...@wisa.be schrieb: In each case: If you want to file bug reports, please use the official sources only to reproduce the problem. FPDoc (trunk) exits with an AV, when I want to make the trunk RTL docs on Win7: make rtl.chk Is there any mandatory option missing from the commandline? Creating a project (--write-project) succeeds. Adding -v reports: FPDoc - Free Pascal Documentation Tool Version 2.7.1 [2012/02/03] (c) 2000 - 2003 Areca Systems GmbH / Sebastian Guenther, s...@freepascal.org Opening source file "../rtl/win32/system.pp". Exception at 0046F463: EAccessViolation: Access violation. What's wrong? DoDi ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] FPDoc imports
On Thu, 2 Feb 2012, Hans-Peter Diettrich wrote: michael.vancann...@wisa.be schrieb: One reason may be the different assumptions about the content of the FProcessedUnits list, where either the input file part or the entire input specification is stored. But this seems not to be the only bug :-( The entire input spec is stored. Only in HandleOnParseUnit, not in CreateDocumentation! Indeed. Fixed in rev. 20216. I just store the unit filename. In each case: Connecting this mail to your other mail reports of packages automatically appearing etc, I'm inclined to think that the automatisms you added in your private version are somehow interfering with the regular workings. But that is a wild guess. You're right, for some reason SVN duplicated just the line where the module is added, and did not find it as a modification later :-( Having fixed beforementioned bug, I couldn't find any more errors :-) In each case: If you want to file bug reports, please use the official sources only to reproduce the problem. Currently I don't want to file bug reports, when your modifications are in alpha state. Discussing issues in Mantis is so slow ;-) BTW, did you test MakeSkel, too? Not yet. But I don't think makeskel needs this recursive algorithm. Michael. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] FPDoc imports
michael.vancann...@wisa.be schrieb: One reason may be the different assumptions about the content of the FProcessedUnits list, where either the input file part or the entire input specification is stored. But this seems not to be the only bug :-( The entire input spec is stored. Only in HandleOnParseUnit, not in CreateDocumentation! In each case: Connecting this mail to your other mail reports of packages automatically appearing etc, I'm inclined to think that the automatisms you added in your private version are somehow interfering with the regular workings. But that is a wild guess. You're right, for some reason SVN duplicated just the line where the module is added, and did not find it as a modification later :-( Having fixed beforementioned bug, I couldn't find any more errors :-) In each case: If you want to file bug reports, please use the official sources only to reproduce the problem. Currently I don't want to file bug reports, when your modifications are in alpha state. Discussing issues in Mantis is so slow ;-) BTW, did you test MakeSkel, too? DoDi ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] FPDoc imports
On Thu, 2 Feb 2012, Hans-Peter Diettrich wrote: Michael Van Canneyt schrieb: Well, It took me about 15 lines to implement it in dglobals.pas of fpdoc. I moved one function out of fpdocoptsxml to mkfpdoc to split the input line. First tests work fine. See revision 20213. A fine solution :-) I happen to get an AV during the Engine destruction, when the packages are released (in TPasElement.Release). It seems to occur when the current package is relased, not for imported packages. In one test I found each module duplicated in the modules list, so that the release of the first entry will invalidate the second one. One reason may be the different assumptions about the content of the FProcessedUnits list, where either the input file part or the entire input specification is stored. But this seems not to be the only bug :-( The entire input spec is stored. I could change this, but I don't think it is the problem. (eventually I probably will, but for other reasons) In each case: Connecting this mail to your other mail reports of packages automatically appearing etc, I'm inclined to think that the automatisms you added in your private version are somehow interfering with the regular workings. But that is a wild guess. In each case: If you want to file bug reports, please use the official sources only to reproduce the problem. Michael. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] FPDoc imports
Michael Van Canneyt schrieb: Well, It took me about 15 lines to implement it in dglobals.pas of fpdoc. I moved one function out of fpdocoptsxml to mkfpdoc to split the input line. First tests work fine. See revision 20213. A fine solution :-) I happen to get an AV during the Engine destruction, when the packages are released (in TPasElement.Release). It seems to occur when the current package is relased, not for imported packages. In one test I found each module duplicated in the modules list, so that the release of the first entry will invalidate the second one. One reason may be the different assumptions about the content of the FProcessedUnits list, where either the input file part or the entire input specification is stored. But this seems not to be the only bug :-( DoDi ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] FPDoc imports
On Wed, 1 Feb 2012, Hans-Peter Diettrich wrote: michael.vancann...@wisa.be schrieb: Why do you need this ? You make it quite complicated. All you need is a FindModule(Const AName,AFileHint : String) : TPasModule call in TPasTreeContainer which must be overridden in a descendent (such as TFPDocEngine). The default implementation returns nil. This call can search the module list if it has one. But it does not have such a list :-( It does not need it. A descendent that needs recursive parsing needs it. Likewise, the FindElement returns nil. Only the FPDoc engine implementation contains a search. If it doesn't find it, it can then decide to parse it anyway and return the just-parsed instance. This requires a list of the Inputs (compiler parameters), which only exists in TFPDocCreator. It also requires a list of units being or having been parsed. When a parser is started for a new unit, it doesn't return the created TPasModule until finished, so that TPasTreeContainer cannot tell whether FindModule addresses a module that has not been parsed yet, or one that already is being parsed, but not finished. Correct. This needs to be done by descendants. TPasTreeContainer itself and the parser class just need the hook. All the parser needs to do is call FindModule whenever it encounters a unit in a uses clause; The rest will happen then automagically. I found it much more complicated, after a look at the existing code. Well, It took me about 15 lines to implement it in dglobals.pas of fpdoc. I moved one function out of fpdocoptsxml to mkfpdoc to split the input line. First tests work fine. See revision 20213. Michael. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] FPDoc imports
michael.vancann...@wisa.be schrieb: Why do you need this ? You make it quite complicated. All you need is a FindModule(Const AName,AFileHint : String) : TPasModule call in TPasTreeContainer which must be overridden in a descendent (such as TFPDocEngine). The default implementation returns nil. This call can search the module list if it has one. But it does not have such a list :-( If it doesn't find it, it can then decide to parse it anyway and return the just-parsed instance. This requires a list of the Inputs (compiler parameters), which only exists in TFPDocCreator. It also requires a list of units being or having been parsed. When a parser is started for a new unit, it doesn't return the created TPasModule until finished, so that TPasTreeContainer cannot tell whether FindModule addresses a module that has not been parsed yet, or one that already is being parsed, but not finished. All the parser needs to do is call FindModule whenever it encounters a unit in a uses clause; The rest will happen then automagically. I found it much more complicated, after a look at the existing code. Then you can create a TPasTreeContainer descendent called TModulesPasTreeContainer which keeps a list of modules and which implements FindModule with the above described recursive mechanism. Then I'll follow this route, as outlined in the last paragraph of my message. DoDi ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] FPDoc imports
On Wed, 1 Feb 2012, Hans-Peter Diettrich wrote: Sven Barth schrieb: Am 31.01.2012 14:42, schrieb Hans-Peter Diettrich: I still wonder why the number of units in the imported RTL package increases over time. Initially it contains 7 modules, and 38 modules in the end, where the RTL currently consists of 46 units. 6 of the initial modules contain classes, but I couldn't find the 7th module "iostream" anywhere, which is the first module in the list. The only "iostream" unit I'm aware of is in packages/fcl-base/src. Right, and it somehow found its way into the rtl package. I'll know more soon. In the meantime I found a solution for the input files order, by recursive compilation. Unfortunately I couldn't test it right now, for several reasons. Most important is the large number of affected units, which may make some people reject such an patch immediatly. So let me explain first what I implemented so far, and what remains to be done. First the storage format of the fpdoc Inputs is not really handy, every commandline has to be parsed for the input filename, from which the unit name can be extracted. It would help when the storage format would be modified, so that the entries can be found with IndexOfName, as implemented in FPDocManager. This is what I didn't implement yet. Next problem is the separation between parser, engine and the fpdoc creators, which are involved in finding the used units in the current package. Herfore I added a method AddModule to the engine, which adds the newly created module to the Modules list, and also notifies fpdoc of the addition by an OnAddModule callback. Then the creator can put the TPasModule into Inputs.Objects, to signify that it is or has been parsed. Why do you need this ? You make it quite complicated. All you need is a FindModule(Const AName,AFileHint : String) : TPasModule call in TPasTreeContainer which must be overridden in a descendent (such as TFPDocEngine). The default implementation returns nil. This call can search the module list if it has one. If it doesn't find it, it can then decide to parse it anyway and return the just-parsed instance. All the parser needs to do is call FindModule whenever it encounters a unit in a uses clause; The rest will happen then automagically. Then you can create a TPasTreeContainer descendent called TModulesPasTreeContainer which keeps a list of modules and which implements FindModule with the above described recursive mechanism. The FPDoc engine can then be a descendent of that class. The above is what I was planning to implement. It is simple and will do the job, plus, for circumstances where there is no need for the recursive mechanism, nothing changes. Michael. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] FPDoc imports
Sven Barth schrieb: Am 31.01.2012 14:42, schrieb Hans-Peter Diettrich: I still wonder why the number of units in the imported RTL package increases over time. Initially it contains 7 modules, and 38 modules in the end, where the RTL currently consists of 46 units. 6 of the initial modules contain classes, but I couldn't find the 7th module "iostream" anywhere, which is the first module in the list. The only "iostream" unit I'm aware of is in packages/fcl-base/src. Right, and it somehow found its way into the rtl package. I'll know more soon. In the meantime I found a solution for the input files order, by recursive compilation. Unfortunately I couldn't test it right now, for several reasons. Most important is the large number of affected units, which may make some people reject such an patch immediatly. So let me explain first what I implemented so far, and what remains to be done. First the storage format of the fpdoc Inputs is not really handy, every commandline has to be parsed for the input filename, from which the unit name can be extracted. It would help when the storage format would be modified, so that the entries can be found with IndexOfName, as implemented in FPDocManager. This is what I didn't implement yet. Next problem is the separation between parser, engine and the fpdoc creators, which are involved in finding the used units in the current package. Herfore I added a method AddModule to the engine, which adds the newly created module to the Modules list, and also notifies fpdoc of the addition by an OnAddModule callback. Then the creator can put the TPasModule into Inputs.Objects, to signify that it is or has been parsed. When a uses list is parsed, the parser asks the engine to search for an alredy parsed unit (FindModule). Since the engine doesn't know about package Inputs, it uses another callback to get the unit from the creator out of Inputs.Objects. When no module has been parsed, the creator starts another parser for this unit, like in CreateDocumentation(). This routine currently parses all Inputs sequentially, and has to be modified to skip those entries which already have an TPasModule unit assigned. Much work could be avoided when TPasPackage uses a TStringList for its Modules. Then this list could be copied from the creator Inputs before the units are parsed, and all searching for the units in a package can be done inside the package, without references back to the engine and creator. This solution also would replace beforementioned callbacks, so that the parser still can be used for other tasks, too. Please let me know what you think about my ideas. Once I know how to proceed, I can present a solution soon. DoDi ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] FPDoc imports
Sven Barth schrieb: Am 31.01.2012 14:42, schrieb Hans-Peter Diettrich: I still wonder why the number of units in the imported RTL package increases over time. Initially it contains 7 modules, and 38 modules in the end, where the RTL currently consists of 46 units. 6 of the initial modules contain classes, but I couldn't find the 7th module "iostream" anywhere, which is the first module in the list. The only "iostream" unit I'm aware of is in packages/fcl-base/src. How could that unit slip into package RTL, as the first unit before those containing classes? I couldn't find "iostream" in all the rtl units. Debugging reveals that only 6 modules are added during import of the content file, so that "iostream" must be added somewhere else, and inserted as the first module of the package. The following warnings are reported during import of rtl.xct: >> Warning : ancestor class #rtl.System.IUnknown of class IClassFactory could not be resolved Warning : ancestor class #rtl.System.IUnknown of class ISequentialStream could not be resolved Warning : ancestor class Exception of class EPropertyError could not be resolved Warning : ancestor class Exception of class EPropertyConvertError could not be resolved Warning : ancestor class #rtl.System.TObject of class tport could not be resolved Warning : ancestor class #rtl.System.TObject of class tportw could not be resolved Warning : ancestor class #rtl.System.TObject of class tportl could not be resolved Warning : ancestor class ematherror of class EInvalidArgument could not be resolved << These ancestors really are missing in :classes, but links and heritage seem to be resolved properly in the docs. DoDi ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] FPDoc imports
Am 31.01.2012 14:42, schrieb Hans-Peter Diettrich: Would it help if I try to create packages from the imports? No. Seems like I misinterpreted the many "unit not found" :-( I still wonder why the number of units in the imported RTL package increases over time. Initially it contains 7 modules, and 38 modules in the end, where the RTL currently consists of 46 units. 6 of the initial modules contain classes, but I couldn't find the 7th module "iostream" anywhere, which is the first module in the list. The only "iostream" unit I'm aware of is in packages/fcl-base/src. Regards, Sven ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] FPDoc imports
On Tue, 31 Jan 2012, Hans-Peter Diettrich wrote: Michael Van Canneyt schrieb: On Tue, 31 Jan 2012, Hans-Peter Diettrich wrote: In an attempt to help improve fpdoc I'm blocked in understanding the handling of imported modules. The content files are imported into the link tree, but this tree seems not to be used in finding external identifiers? The content files are converted to 2 kinds of trees: - A TPasElement tree. - A link tree. The link tree is used in TFPDocEngine.ResolveLink and THTMLWriter.ResolveLinkID The paselement tree is used by the parser to find identifiers (classes mostly, for inheritance trees), so there are less TPasUnsolvedTypeRef instances. This would explain why the System unit never is found - it seems not to contain any classes. Would it help if I try to create packages from the imports? No. Seems like I misinterpreted the many "unit not found" :-( Yes. I still wonder why the number of units in the imported RTL package increases over time. Initially it contains 7 modules, and 38 modules in the end, where the RTL currently consists of 46 units. 6 of the initial modules contain classes, but I couldn't find the 7th module "iostream" anywhere, which is the first module in the list. For the input file order: as long as the user has no idea how to order the files, fpdoc should do that for him. Or, in Unix tradition, supply an app that sorts the units ;-) Nono, the parser will need to take care of this. fpdoc is not the only application that uses fcl-passrc. It's planned. Michael. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] FPDoc imports
Michael Van Canneyt schrieb: On Tue, 31 Jan 2012, Hans-Peter Diettrich wrote: In an attempt to help improve fpdoc I'm blocked in understanding the handling of imported modules. The content files are imported into the link tree, but this tree seems not to be used in finding external identifiers? The content files are converted to 2 kinds of trees: - A TPasElement tree. - A link tree. The link tree is used in TFPDocEngine.ResolveLink and THTMLWriter.ResolveLinkID The paselement tree is used by the parser to find identifiers (classes mostly, for inheritance trees), so there are less TPasUnsolvedTypeRef instances. This would explain why the System unit never is found - it seems not to contain any classes. Would it help if I try to create packages from the imports? No. Seems like I misinterpreted the many "unit not found" :-( I still wonder why the number of units in the imported RTL package increases over time. Initially it contains 7 modules, and 38 modules in the end, where the RTL currently consists of 46 units. 6 of the initial modules contain classes, but I couldn't find the 7th module "iostream" anywhere, which is the first module in the list. For the input file order: as long as the user has no idea how to order the files, fpdoc should do that for him. Or, in Unix tradition, supply an app that sorts the units ;-) DoDi ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] FPDoc imports
On Tue, 31 Jan 2012, Hans-Peter Diettrich wrote: In an attempt to help improve fpdoc I'm blocked in understanding the handling of imported modules. The content files are imported into the link tree, but this tree seems not to be used in finding external identifiers? The content files are converted to 2 kinds of trees: - A TPasElement tree. - A link tree. The link tree is used in TFPDocEngine.ResolveLink and THTMLWriter.ResolveLinkID The paselement tree is used by the parser to find identifiers (classes mostly, for inheritance trees), so there are less TPasUnsolvedTypeRef instances. Instead packages are searched, where I could not yet figure out how these are constructed. When the FCL documentation is created, which imports package (content file) RTL, a package RTL seems to be created, somehow, but its content is not related to the content file. In detail unit System never becomes part of the RTL package? Can somebody explain the creation and population of external packages a bit? What's the purpose of these packages at all, when the essential information about identifiers is contained in the content files? The purpose is to try and let the parser resolver find identifiers as if they had been parsed. (compare it to the info in a .ppu file) Would it help if I try to create packages from the imports? No. Michael. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
[fpc-devel] FPDoc imports
In an attempt to help improve fpdoc I'm blocked in understanding the handling of imported modules. The content files are imported into the link tree, but this tree seems not to be used in finding external identifiers? Instead packages are searched, where I could not yet figure out how these are constructed. When the FCL documentation is created, which imports package (content file) RTL, a package RTL seems to be created, somehow, but its content is not related to the content file. In detail unit System never becomes part of the RTL package? Can somebody explain the creation and population of external packages a bit? What's the purpose of these packages at all, when the essential information about identifiers is contained in the content files? Would it help if I try to create packages from the imports? DoDi ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
