Re: Where can I find the doxyfile?
>> I have found it very strange that a tool like doxygen which can create >> all sorts of call graphs, just ignores some comments. The comments >> above function are very important. I agree with you . I hated doxygen for decades because of the irritating annotations it required. When I discovered IDEs were creating doxygen-like popups from conventional comments, I tried to figure out what they were doing. In the process, I discovered filters, and I created a filter to match what I thought the IDEs were doing. (As it turns out, IDEs implement their own rendering independent of doxygen.) I find it ironic I’ve gone from being a long-time hater of doxygen to being its advocate. * John Morris Tiny tidbit of history. Back in the 70’s, I created a comment extractor for the language Ratfor. We used it to maintain an alphabetical index of functions and to display pseudo-code. We drew our call graphs by hand and saved them in manila folders. Hard to imagine now.
Re: Where can I find the doxyfile?
On Mon, Feb 5, 2024 at 05:29:08PM +, John Morris wrote: > >> It seems something you want to keep for your personal use. > > >> I think this filter is the kind of genious idea that it's OK if you > > >> can do it at home > > I don’t agree with your characterization. > > The purpose of the filter is to bring existing Postgres comments into the > doxygen output. While I haven’t done a full survey, the majority of Postgres > code has comments describing functions, globals, macros and structure fields. > > Currently, those comments are thrown away. They do not appear in the existing > Doxygen output. I have found it very strange that a tool like doxygen which can create all sorts of call graphs, just ignores some comments. The comments above function are very important. -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com Only you can decide what is important to you.
Re: Where can I find the doxyfile?
Update: another patch with 1) suggested changes, 2) delete old html before generating new, and 3) added flex names for the more complex regular expressions. Exercising the “ninja doxygen” command, brought up some issues. 1. The generated html directory contained old as well as new pages. The build script now deletes old pages before building new ones. 2. Using a line of dashes to suppress text formatting is not implemented. Some dashed comments are harder to read, especially around code samples. 3. Comments at the end of “#define” are considered part of the define and not annotated as doxygen comments. The first issue was very confusing, so it has been fixed. My preference is to make the other two issues part of a future enhancement. Thoughts? I’ve also been scanning the generated pages looking for anomalies. 1. About 1/3 of pages examined. 2. In most cases, the filter works as expected. 3. Many places are missing information, so the output has blank fields (as expected). 4. A few places have obvious nonsense. (eg. a group comment applied to one element) 5. No situations where the output would be misleading. 6. In all cases, the source code is “a click away”. While I had planned to look at *every* page, I’ll probably stop at the 1/3 sample – unless someone wants to help scan through the pages with me. I also heard back from Jetbrains about incorporating custom Doxyfiles. They do their own rendering and do not invoke the doxygen command. Custom Doxyfiles are not going to happen. (It’s probably the same for VS.) On my Mac M1, generating doxygen takes about 20 seconds. When graphs are added, it takes 5 minutes. doxygen_v5.patch Description: doxygen_v5.patch
Re: Where can I find the doxyfile?
>> We also have a handful of .cpp files. Fortunately, our .cpp files are simple and do not include C++ specific features. We shouldn’t have any problem adding .cpp to the filter list. At some point it will be necessary to support general C++, but I wasn’t planning to do it yet. >> This file isn't really following postgres coding style - any reason not to >> do so? I’ll update it to Postgres commenting. The filter started as a standalone project. >> What do you mean with "build time error messages" specifically? Why would we >> want to output anything at build time (rather than configure time)? “build time error messages” occur after typing “ninja”. Documentation is usually produced long after meson setup messages have been forgotten. It is much more useful to see “Please install the doxygen program” after typing “ninja doxygen” than to see “target not supported”, or to have to look through the setup log. It’s personal preference, but I think the sgml docs should display a similar message. >> I'd add "native: true", it'd not work to find the program in the target >> environment of a cross build. Got it. Thanks.
Re: Where can I find the doxyfile?
Hi,
On 2024-02-08 18:30:01 +, John Morris wrote:
> +FILTER_PATTERNS= *.c *.h
We also have a handful of .cpp files.
> +/*
> + * Custom banner character to be removed from comments.
> + * We'll hardcode it to suit postgreSQL, but it should be set through a
> command line arg.
> + */
> +char customBanner = '-';
> +
> +/*
> +A simple program which reads a file, updates the comments, and writes to
> stdout.
> +This is intended to be used as a doxygen filter, converting existing
> comments to doxygen comments.
> +**/
> +int main(int argc, char**argv) {
> +
> +// Verify we have a single argument.
This file isn't really following postgres coding style - any reason not to do
so?
> diff --git a/doc/doxygen/meson.build b/doc/doxygen/meson.build
> new file mode 100644
> index 00..e5539b7854
> --- /dev/null
> +++ b/doc/doxygen/meson.build
> @@ -0,0 +1,73 @@
> +# Generate doxygen pages for PostgreSQL using "ninja doxygen"
> +#
> +# Doxygen pages are optional. Nothing in this script should
> +# cause PostgreSQL builds to fail.
> +#
> +# Currently there are no explicit error messages
> +# - If doxygen is not found, the doxygen target will not be defined.
> +# - If dot is not found, no graphs will be generated.
> +# - flex is already required, so we don't check for it.
> +#
> +# As a future enhancement, display meaningful error messages
> +# when doxygen or dot are not found. Meson does not
> +# support build time error messages, but they can be displayed
> +# using a python custom target.
What do you mean with "build time error messages" specifically? Why would we
want to output anything at build time (rather than configure time)?
> +# Find the doxygen command. If not found, stop and don't define the target.
> +doxygen_cmd = find_program('doxygen', required: false)
I'd add "native: true", it'd not work to find the program in the target
environment of a cross build.
> +
> +# Find the dot command. If not found, no graphs will be generated.
> +dot_cmd = find_program('dot', required: false)
Dito.
Greetings,
Andres Freund
Re: Where can I find the doxyfile?
>> I think all the explanatory messages in doc/doxygen/meson.build >> are a bit much. I think it's enough to just not define the target Here is a patch with an updated meson.build as you suggested. I agree the messages were a bit much. On the other hand, I would like to see clear error messages when dot or doxygen are not installed, but I’ll leave that for a future discussion. * John - doxygen-v4.patch Description: doxygen-v4.patch meson.build Description: meson.build
Re: Where can I find the doxyfile?
>> Maybe this is something that can be tweaked on the doxygen side? I’ll look into that idea. Doxygen is a separate project with its own developers to persuade. I could imagine a special “C” or “C++” mode. I wanted to keep things simple, and the filter mechanism is how they extend doxygen to support other languages. One alternative is to add the filter to doxygen’s webpage. They currently list a number of filters, none of which address this particular issue. The filter would become a build dependency like doxygen itself. In the meantime, I’d like to address Alvaro’s concern about generating artifacts. It’s a bit tedious, but a visual survey of the output may add confidence or show areas needing improvement.
Re: Where can I find the doxyfile?
>> I think all the explanatory messages in doc/doxygen/meson.build >> are a bit much. I think it's enough to just not define the target >> when the required conditions (dependencies, options) are not there. >> Maybe something like docs_pdf can serve as an example. Makes sense, and it is simpler. Let’s continue after I take a look at docs_pdf. Thanks, John
Re: Where can I find the doxyfile?
On 02.02.24 01:19, John Morris wrote: Here is the updated patch. It should fix the meson issue when no doxygen is present. I think all the explanatory messages in doc/doxygen/meson.build are a bit much. I think it's enough to just not define the target when the required conditions (dependencies, options) are not there. Maybe something like docs_pdf can serve as an example.
Re: Where can I find the doxyfile?
On 05.02.24 18:29, John Morris wrote: The purpose of the filter is to bring existing Postgres comments into the doxygen output. While I haven’t done a full survey, the majority of Postgres code has comments describing functions, globals, macros and structure fields. Currently, those comments are thrown away. They do not appear in the existing Doxygen output. Maybe this is something that can be tweaked on the doxygen side? For example, clangd can also process doxygen-style comments. But it can also process non-decorated comments, because it knows that the comment just before a declaration is probably the comment describing the thing. Maybe doxygen could have that functionality as well.
Re: Where can I find the doxyfile?
>> It seems something you want to keep for your personal use. >> I think this filter is the kind of genious idea that it's OK if you >> can do it at home I don’t agree with your characterization. The purpose of the filter is to bring existing Postgres comments into the doxygen output. While I haven’t done a full survey, the majority of Postgres code has comments describing functions, globals, macros and structure fields. Currently, those comments are thrown away. They do not appear in the existing Doxygen output. The filter itself is not rocket science. It is a straightforward scanner built using the flex tool. I understand many people are more comfortable with perl or python, but flex is uniquely appropriate for creating small, efficient scanners. The resulting output is not perfect. However, it is a major step forward from what we are currently producing. Over time, we can gradually update Postgres comments to fill in missing information. Eventually, doxygen output could become a “first look” tool for looking at code. Will it be my “first look” tool. Not likely. Like you, I am comfortable with Intellij and VsCode. I have my build environments already set up. I do not see this as a high-risk endeavor. It is “off to the side” of the main Postgres code. It takes an existing tool, doxygen, which is mostly useless, and starts to make it usable. As an aside, I have contacted Intellij about allowing custom Doxygen files for hover information. No reply yet, but it would allow us to tailor our IDEs to display the information we need most.
Re: Where can I find the doxyfile?
>> I wish you'd stop proposing the lex filter so that we can discuss the >> Doxyfile.in file and the accompanying Meson rule. I see no obstacle to discussing Doxyfile.in and the meson.build file. Your suggestions are welcome. If you can find the original Doxyfile, it would make sense to incorporate it. >> but having something that purports to display our source code and >> then show something that is*not* our source tree sounds insane. The filter enables doxygen to interpret Postgres comments. Any code displayed is the original code, not the filtered code. >> Speaking from my personal point of view, I make very little use >> of our Doxygen pages, Same for me. The pages are missing critical information, such as function descriptions and descriptions of structure fields. This filter is an attempt at fixing those problems. Of course, nothing will fix doxygen when there are no comments to begin with. If we want comprehensive doxygen output, we need to add comments. However, we don’t need to add those irritating extra symbols.
Re: Where can I find the doxyfile?
On 2024-Feb-02, John Morris wrote: > Here is the updated patch. It should fix the meson issue when no > doxygen is present. I wish you'd stop proposing the lex filter so that we can discuss the Doxyfile.in file and the accompanying Meson rule. I think there's pretty much zero chance that we'd accept the doxy_filter.l thingy in our source tree. It seems something you want to keep for your personal use. I think this filter is the kind of genious idea that it's OK if you can do it at home, but having something that purports to display our source code and then show something that is *not* our source tree sounds insane. Who knows what artifacts are going to show up in the doxygen output, and then we'll be on the hook to fix those. I propose to add the file to the wiki, maybe link to it from the Developer_FAQ page. Also, I'm not sure about the use case of people who wants to study the Postgres source code but doesn't have time to download it into VSCode or whatever. In short: -1 from me. I see the Doxy thing as roughly parallel to the coverage build rules ... but the coverage rules seem more much closely intertwined with the tree in a way that the Doxygen processing is not. Speaking from my personal point of view, I make very little use of our Doxygen pages, but I do use them --- and very occassionally I would like to see what the changes would be with some patch applied. However, again, maybe having the .in file in the wiki is enough, where you can download it and have your own script to run it, and tweak the prefs however you want them and so on? -- Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
Re: Where can I find the doxyfile?
Here is the updated patch. It should fix the meson issue when no doxygen is present. doxygen-v3.patch Description: doxygen-v3.patch
Re: Where can I find the doxyfile?
On Thu, 18 Jan 2024 at 06:39, vignesh C wrote: > > On Tue, 7 Nov 2023 at 12:23, John Morris wrote: > > > > Another update, this time with an abbreviated Doxyfile. Rather than > > including the full Doxyfile, this updated version only includes modified > > settings. It is more compact and more likely to survive across future > > doxygen versions. > > Meson build is failing in CFbot at [1] and [2] with: > Message: Install doxygen if you wish to create Doxygen documentation > Program dot found: NO > Configuring Doxyfile using configuration > > doc/doxygen/meson.build:52:0: ERROR: Tried to use not-found external > program in "command" > With no update to the thread and the build still failing I'm marking this as returned with feedback. Please feel free to resubmit to the next CF when there is a new version of the patch. Regards, Vignesh
Re: Where can I find the doxyfile?
On Tue, 7 Nov 2023 at 12:23, John Morris wrote: > > Another update, this time with an abbreviated Doxyfile. Rather than including > the full Doxyfile, this updated version only includes modified settings. It > is more compact and more likely to survive across future doxygen versions. Meson build is failing in CFbot at [1] and [2] with: Message: Install doxygen if you wish to create Doxygen documentation Program dot found: NO Configuring Doxyfile using configuration doc/doxygen/meson.build:52:0: ERROR: Tried to use not-found external program in "command" [1] - https://cirrus-ci.com/task/5823573617541120 [2] - https://api.cirrus-ci.com/v1/artifact/task/5823573617541120/meson_log/build/meson-logs/meson-log.txt Regards, Vignesh
Re: Where can I find the doxyfile?
Another update, this time with an abbreviated Doxyfile. Rather than including the full Doxyfile, this updated version only includes modified settings. It is more compact and more likely to survive across future doxygen versions. * John Morris doxygen_v2.patch Description: doxygen_v2.patch
Re: Where can I find the doxyfile?
On 14.10.23 21:54, John Morris wrote: Thank you for trying the patch out and commenting on it. Thank you! I even didn't know such filters are possible, and after that googling filter for C/C++ didn't gave good results. I'm starting to think of it as a project. Here's a quick project statement. The purpose is to generate improved Doxygen output while making maximal use of how Postgres currently does program comments. Yes, it shouldn't alter existing workflow. * Provide options for other (non-html) output. (Which ones?) I guess for now it is not needed, there were only HTML doxygen. * Mention it in developer guidelines and provide sample code showing a "desired" commenting style. I think no changes are needed in guidelines at this point, just allow existing comments to be collected. In future some documentation from Doxygen style can be copied to describe how to document function's parameter, etc. Does that list seem complete? I don't want people to think we're imposing new standards or legislating new commenting styles. It's more a matter of describing what we currently do, maybe with some minor suggestions for improving. Yes, that's all I can think about at first stage, at least. Except last bullet is too much. Better to not impose any new standards at this point. Just get as much comments as possible from existing code-base. And allow people build it locally. Later on we can find problematic places where comments that should not be pulled are pulled, and add some syntax into filter to exclude such comments. This are places, like section comments, like: "all following fields are ***" But at first step it's just good to have as much as possible comments collected, even if some are misplaced. So done is better than perfect. Best regards, Bohdan Mart. OpenPGP_0xD72F9AC47D08E8B6.asc Description: OpenPGP public key OpenPGP_signature.asc Description: OpenPGP digital signature
Re: Where can I find the doxyfile?
Thank you for trying the patch out and commenting on it. I'm starting to think of it as a project. Here's a quick project statement. The purpose is to generate improved Doxygen output while making maximal use of how Postgres currently does program comments. Thinking in terms of specific steps, and adding to what you have mentioned: * Configure Doxyfile so it produces output similar to previous output. * Only build Doxygen output if requested * Don't compile the filter or configure the Doxyfile if they aren't needed * Include contrib in the sources to document * Provide options for other (non-html) output. (Which ones?) * Update Postgres documentation to include instructions for creating Doxygen output. * Mention it in developer guidelines and provide sample code showing a "desired" commenting style. Does that list seem complete? I don't want people to think we're imposing new standards or legislating new commenting styles. It's more a matter of describing what we currently do, maybe with some minor suggestions for improving. * John Morris
Re: Where can I find the doxyfile?
On 07.10.23 00:23, Bohdan Mart wrote: On 07.10.23 00:29, postg...@coyotebush.net wrote: Sometime earlier, I created a filter to annotate regular C comments as doxy comments. I'll attach it along with a sample doxyfile for running it. Just in case it looks useful. I've never been a big fan of Doxygen, but it seems to have gotten better over time. Now that some IDEs display doxy comments on hover, I'm beginning to appreciate it. Thank you for providing this `flex` filter! It is actually useful. I've tested it on one file from postures source base, and it actually converted regular comments to doc-comments! If this filter could be used by official Doxygen generation of Postgres, it would highly increase quality of online documentation of Postgres! I have not actually looked at the code of the filter itself but personally I'm not super happy with doing a C-based filter as part of our doxygen documentation generation step - that seems like an additional maintainance burden infrastrukcture wise - is there nothing more lightweight available(assuming we even want that)? Stefan
Re: Where can I find the doxyfile?
On 07.10.23 00:29, postg...@coyotebush.net wrote: Sometime earlier, I created a filter to annotate regular C comments as doxy comments. I'll attach it along with a sample doxyfile for running it. Just in case it looks useful. I've never been a big fan of Doxygen, but it seems to have gotten better over time. Now that some IDEs display doxy comments on hover, I'm beginning to appreciate it. Thank you for providing this `flex` filter! It is actually useful. I've tested it on one file from postures source base, and it actually converted regular comments to doc-comments! If this filter could be used by official Doxygen generation of Postgres, it would highly increase quality of online documentation of Postgres! My initial idea was to create patches to upstream with changing comments to documented comments, because current online dock is lacking comments, but with this filter it would be unneeded, as documentation would be able to be generated from current sources! To illustrate the point: ~~~ This function has Doxy style comment ``` src/interfaces/ecpg/compatlib/informix.c 672:/** 673- * initialize the struct, which holds the different forms 674- * of the long value 675- */ 676-static int 677-initValue(long lng_val) ``` And it is rendered properly: https://doxygen.postgresql.org/informix_8c.html#a0dad4c2ee52a831d3aa1bf1133e0e17d But function like this ``` src/backend/foreign/foreign.c /* * get_foreign_server_oid - given a server name, look up the OID * * If missing_ok is false, throw an error if name not found. If true, just * return InvalidOid. */ Oid get_foreign_server_oid(const char *servername, bool missing_ok) ``` https://doxygen.postgresql.org/foreign_8c.html#a7959c56969be440c25b62c3c98ce2a78 doesn't have rendered documentation. P.S. Was this original message from postg...@coyotebush.net intended to be sent on John's Morris behalf?
RE: Where can I find the doxyfile?
Sometime earlier, I created a filter to annotate regular C comments as doxy comments. I'll attach it along with a sample doxyfile for running it. Just in case it looks useful. I've never been a big fan of Doxygen, but it seems to have gotten better over time. Now that some editors are displaying doxygen comments on hover, I'm beginning to appreciate it. d DoxygenFilter.l Description: Binary data Doxyfile Description: Binary data
RE: Where can I find the doxyfile?
Sometime earlier, I created a filter to annotate regular C comments as doxy comments. I'll attach it along with a sample doxyfile for running it. Just in case it looks useful. I've never been a big fan of Doxygen, but it seems to have gotten better over time. Now that some IDEs display doxy comments on hover, I'm beginning to appreciate it. DoxygenFilter.l Description: Binary data Doxyfile Description: Binary data
Re: Where can I find the doxyfile?
On 07.10.23 00:06, Bruce Momjian wrote: On Fri, Oct 6, 2023 at 10:50:25PM +0300, Bogdan Mart wrote: > On 2008-06-02 18:33:42 Stefan Kaltenbrunner wrote: >> >> Zdenek Kotala wrote: >>> Xin Wang napsal(a): >>>> Hi, >>>> I don't know where I can find the doxyfile which generate >>>> "doxygen.postgresql.org" web site. I found that when reading code the >>>> doxygen source code is quite helpful. However, I want to generate an >>>> off-line copy of doxygen docs myself, but I can't find the doxyfile in >>>> the lastest source release. >>> >>> I think it is good idea. Stefan, what's about put it on the wiki? >> >> don't think the wiki is a good place (we would have to maintain it in >> addition to the main file) so I simply added a link on >> http:/doxygen.postgresql.org that contains the current copy of the >> configuration file that was used to generate a particular set of docs. >> However some of the things there are specific to our install so a bit >> (like some of the paths) of manual change will be required anyway. > > Hello, sorry for resurrecting this old discussion, and necroposting, > but this was relevant. > > It turns out that way back in 2008 it was possible to download Doxyfile > from https://doxygen.postgresql.org/ , which can be seen at: > https://web.archive.org/web/20080915132924/https://doxygen.postgresql.org/ > > I was able to download this Doxyfile from the web archive, and it actually > worked for me, after I changed the path to postgres sources and removed > header template. It even produced same result as on current web-site. > > It would be nice if this doxygen file would be placed somewhere > on the current site. > > P.S. I've tried to run doxygen on postgres sources without config, > and it didn't find any source file, this archived thread helped. > > Perhaps doxygen file can be just placed into postgres git repository? We do link to a Doxygen build from our website: https://www.postgresql.org/developer/coding/ Yes, there is link to deployed documentation generated by Doxygen (unless I'm missing something), but I am talking about config file for Doxygen to generate same documentation locally. There are no entries on Wiki regarding doxygen, and not in build instructions. I was able to locate needed config file in Web Archive: https://web.archive.org/web/20081210081808if_/http://doxygen.postgresql.org:80/pgsql.conf But it would be nice if it was readily available if anybody other would like to build docs locally.
Re: Where can I find the doxyfile?
On Fri, Oct 6, 2023 at 10:50:25PM +0300, Bogdan Mart wrote: > On 2008-06-02 18:33:42 Stefan Kaltenbrunner wrote: > > > > Zdenek Kotala wrote: > > > Xin Wang napsal(a): > > >> Hi, > > >> I don't know where I can find the doxyfile which generate > > >> "doxygen.postgresql.org" web site. I found that when reading code the > > >> doxygen source code is quite helpful. However, I want to generate an > > >> off-line copy of doxygen docs myself, but I can't find the doxyfile in > > >> the lastest source release. > > > > > > I think it is good idea. Stefan, what's about put it on the wiki? > > > > don't think the wiki is a good place (we would have to maintain it in > > addition to the main file) so I simply added a link on > > http:/doxygen.postgresql.org that contains the current copy of the > > configuration file that was used to generate a particular set of docs. > > However some of the things there are specific to our install so a bit > > (like some of the paths) of manual change will be required anyway. > > Hello, sorry for resurrecting this old discussion, and necroposting, > but this was relevant. > > It turns out that way back in 2008 it was possible to download Doxyfile > from https://doxygen.postgresql.org/ , which can be seen at: > https://web.archive.org/web/20080915132924/https://doxygen.postgresql.org/ > > I was able to download this Doxyfile from the web archive, and it actually > worked for me, after I changed the path to postgres sources and removed > header template. It even produced same result as on current web-site. > > It would be nice if this doxygen file would be placed somewhere > on the current site. > > P.S. I've tried to run doxygen on postgres sources without config, > and it didn't find any source file, this archived thread helped. > > Perhaps doxygen file can be just placed into postgres git repository? We do link to a Doxygen build from our website: https://www.postgresql.org/developer/coding/ -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com Only you can decide what is important to you.
Re: Where can I find the doxyfile?
On 2008-06-02 18:33:42 Stefan Kaltenbrunner wrote: > > Zdenek Kotala wrote: > > Xin Wang napsal(a): > >> Hi, > >> I don't know where I can find the doxyfile which generate > >> "doxygen.postgresql.org" web site. I found that when reading code the > >> doxygen source code is quite helpful. However, I want to generate an > >> off-line copy of doxygen docs myself, but I can't find the doxyfile in > >> the lastest source release. > > > > I think it is good idea. Stefan, what's about put it on the wiki? > > don't think the wiki is a good place (we would have to maintain it in > addition to the main file) so I simply added a link on > http:/doxygen.postgresql.org that contains the current copy of the > configuration file that was used to generate a particular set of docs. > However some of the things there are specific to our install so a bit > (like some of the paths) of manual change will be required anyway. Hello, sorry for resurrecting this old discussion, and necroposting, but this was relevant. It turns out that way back in 2008 it was possible to download Doxyfile from https://doxygen.postgresql.org/ , which can be seen at: https://web.archive.org/web/20080915132924/https://doxygen.postgresql.org/ I was able to download this Doxyfile from the web archive, and it actually worked for me, after I changed the path to postgres sources and removed header template. It even produced same result as on current web-site. It would be nice if this doxygen file would be placed somewhere on the current site. P.S. I've tried to run doxygen on postgres sources without config, and it didn't find any source file, this archived thread helped. Perhaps doxygen file can be just placed into postgres git repository? Best regards, Bohdan Mart.
