On 25 July 2010 20:29, Johan Hake <[email protected]> wrote: > On Saturday July 24 2010 09:48:44 Kristian Ølgaard wrote: >> On 22 July 2010 20:10, Johan Hake <[email protected]> wrote: >> > On Monday July 19 2010 12:54:44 Kristian Oelgaard wrote: >> >> On 23 June 2010 15:35, Kristian Oelgaard <[email protected]> wrote: >> >> > On 23 June 2010 10:55, Kristian Oelgaard <[email protected]> > wrote: >> >> >> On 22 June 2010 19:03, Johan Hake <[email protected]> wrote: >> >> >>> On Tuesday June 22 2010 08:28:37 Kristian Oelgaard wrote: >> >> >>>> I've started writing the programmer's reference for FEniCS. >> >> >>>> One of the features that we decided on was that doc-strings for >> >> >>>> PyDOLFIN should be written and maintained as part of the >> >> >>>> documentation project and then added to the dolfin module on >> >> >>>> import. >> >> >>>> >> >> >>>> I thought about doing this in the following way: >> >> >>>> >> >> >>>> 1) Create a pseudo module 'dolfin-doc' which is a copy of the >> >> >>>> classes and functions in the 'real' dolfin module only it contains >> >> >>>> no code at all, just doc-strings. (This approach will also make it >> >> >>>> easy to create a script to check if all functions are documented >> >> >>>> or if any docs are obsolete). >> >> >>> >> >> >>> Sounds good. I first thought of a structure (other than a dummy >> >> >>> class) that just mimics the class hierarchy, but in some way that >> >> >>> is what you actually suggests and it is probably as easy as >> >> >>> anything else. >> >> >>> >> >> >>>> 2) Use the autodoc functionality of Sphinx to create parts of the >> >> >>>> documentation for functions and classes >> >> >>>> >> >> >>>> 3) Manually add additional information (in the reST file) and links >> >> >>>> to other parts of the documentation like demos etc. This will not >> >> >>>> be available using help() in the Python interpreter. >> >> >>>> >> >> >>>> 4) In the dolfin.__init__.py function import the 'dolfin-doc' >> >> >>>> module and copy the doc-strings from all classes and functions to >> >> >>>> the classes and functions in the real dolfin module as was >> >> >>>> suggested by Johan Hake. >> >> >>>> >> >> >>>> The problem with this approach is that assigning to __doc__ is not >> >> >>>> permitted for objects of 'type' type. >> >> >>>> >> >> >>> :( >> >> >>> >> >> >>> I did not anticipate this. Not sure why this is. I have got the >> >> >>> impression that numpy get around this. They use numpydoc to >> >> >>> dynamically add their documentation. It makes heavy use of sphinx, >> >> >>> but I couldn't figure how they get around that __doc__ is >> >> >>> read-only. >> >> >> >> >> >> To me it looks like numpydoc is a Sphinx extension that translates >> >> >> the Numpy docstrings into something that Sphinx can understand, not >> >> >> the other way around which is what we want. >> >> >> >> >> >> http://projects.scipy.org/numpy/browser/trunk/doc/sphinxext/README.tx >> >> >> t >> >> >> >> >> >> So I think our best bet is to proceed with your suggestions below. >> >> >> >> >> >> Kristian >> >> >> >> >> >>> While it might be cool to look into what NumPy have done, (they also >> >> >>> define a pseudo classes, which they populate with docstrings, (look >> >> >>> into phantom_import.py), and they also define some nice format for >> >> >>> the reST used in the docstrings), I suggest two things we can do: >> >> >>> >> >> >>> 1) SWIG can generate docstrings. We do that allready using parsed >> >> >>> doxygen documentation. All of this is gathered in docstrings.i. I >> >> >>> suggest generating such a file from our documentation. We need to >> >> >>> turn of %feature("autodoc","1") in dolfin.i to get rid of the long >> >> >>> and sometimes faulty generated signatures. >> >> > >> >> > I turns out that it's only the __doc__ of the class I can't assign to, >> >> > not the __doc__ of member functions (and regular functions). >> >> > A simpler solution (at least for me) is to parse the cpp.py module >> >> > once generated and substitute all docstrings of classes with the >> >> > docstrings from the dolfindoc module rather than creating the >> >> > docstrings.i file. >> >> > >> >> > Then for the classes that we manually add we use the method you >> >> > described below, but only for class.__doc__ . >> >> > >> >> > class Foo(object): >> >> > __doc__ = dolfindoc.Foo.__doc__ >> >> > def bar(self): >> >> > "this doc string will be substituted with the >> >> > dolfindoc.Foo.__dict__["bar"].__doc__." >> >> > pass >> >> >> >> Unfortunately it turned out I was too quick to make this conclusion. >> >> For instance, >> >> for the Swig generated class Mesh in the module cpp I can't assign to >> >> >> >> Mesh.__doc__ >> >> >> >> because it is a 'type' object as pointed our earlier, I can assign to: >> >> >> >> Mesh.__dict__["__init__"].__doc__ >> >> >> >> but not >> >> >> >> Mesh.__dict__["size"].__doc__ >> >> >> >> along with practically all other member functions. >> >> >> >> The reason is that Swig, after the class definitions does this: >> >> >> >> Mesh.size = new_instancemethod(_cpp.Mesh_size,None,Mesh) >> >> >> >> to which no assignment is possible. >> >> 'AttributeError: attribute '__doc__' of 'instancemethod' objects is >> >> not writable' >> >> The code in Mesh.size is already >> >> >> >> def size(self, *args): >> >> return _cpp.Mesh_size(self, *args) >> >> >> >> so why Swig overrides the method I don't know. If it didn't I could >> >> assign to the docstring. >> >> >> >> Recall that my initial plan was to simply parse the cpp.py file and >> >> substitute the docstrings of all classes with what the pseudo module >> >> dolfindoc would define. However, that doesn't seem so practical >> >> anymore since I would need to also comment out all lines where >> >> instancemethods are being assigned to class members such that on >> >> import (in dolfin/__init__.py) I can loop classes and assign to >> >> function docstrings. This is of course possible but seems a bit >> >> awkward and I don't know what implications the commenting out of >> >> instancemethod assignment will have. >> >> As I see it, two solutions remain. >> >> >> >> 1) Take the pseudo module dolfindoc from the FEniCS documentation and >> >> create a docstring.i generator. The script >> >> dolfin/dolfin/swig/generator.py should then try to import the >> >> dolfindoc module, if successful use that to create docstring.i, >> >> otherwise use the Doxy2SWIG generator. >> > >> > I thought that you landed on this alternative. The point is to generate a >> > dosctrings.i file from the FEniCS documentation. Then for the extended >> > Python layer we can use a generated Python module to assign the >> > docstrings: >> > >> > class VariationalProblem(...): >> > ... >> > __doc__ generated_docstring_module.VariationalProblem.__doc__ >> >> This approach is getting more and more complicated; it turns out we >> also need to generate all the >> _post.i files to get the docstrings correct. > > Do yo meen the extended python functions that resides in these files?
Yes, they will need a docstring too. But since Swig doesn't mess with these particular (member-) functions, it is actually possible to assign to __doc__ dynamically (on import) so maybe that would be an easier option. >> At this point it seems a >> lot easier to me to simply extend the Python layer with everything we >> want to have detailed documentation for and distribute the >> dolfindocstrings module with DOLFIN so we can do: >> >> class VariationalProblem(...): >> ... >> __doc__ generated_docstring_module.VariationalProblem.__doc__ >> >> this way we handle the docstrings in a uniform way, else we need to >> generate docstrings.i for some functions, *_post.i files for other >> functions and STILL do the assign to __doc__ trick. >> >> For functions that we don't extend in the Python layer we still have >> the oneliner docs from the header files extended with some random >> output from Swig. > > How do we handle the documentation of the pure cpp classes, for example > cpp.Mesh? For all functions/classes we put a docstring in the docstringsmodule and distribute this module with DOLFIN. The current approach that we're pursuing is: For the classes in cpp, we generate the docstrings.i, For add on functions in *_post.i we need to either generate *_post.i or assign to __doc__ of those particular functions on import. For classes functions in the Python layer we just assign to __doc__ in the definition of classes/functions. or as I suggested: Extend the Python layer with whatever classes/functions that we want to have documented in detail and assign to __doc__ in the definitions. Kristian > Johan > >> Kristian >> >> >> One question w.r.t this approach, when is docstrings.i being >> >> generated? I see it is distributed with DOLFIN but shouldn't it be >> >> generated everytime DOLFIN is rebuild with enablePython, or does >> >> enableDocs have to be switched on as well? >> > >> > It is generated by running dolfin/swig/generate.py. It takes quite a long >> > time so I think it is good to pre-generate this in the distribution. >> > >> >> 2) Extend our Python layer with all functions and classes that we want >> >> to use the dolfindoc docstrings for. >> >> These should just be empty and redirect calls to classes and functions >> >> of the cpp. In dolfin/mesh/mesh.py: >> >> >> >> class Mesh(cpp.Mesh): >> >> def size(self): >> >> try: >> >> import dolfindoc.mesh.mesh >> >> __doc__ = dolfindoc.mesh.mesh.Mesh.__doc__ >> >> except: >> >> __doc__ = cpp.Mesh.__doc__ >> >> return cpp.Mesh.size() >> >> >> >> An additional benefit of this approach is that the module structure >> >> can be identical to what we have in the _real_ DOLFIN, not as it is >> >> now where everything is dumped in the dolfin.cpp module. >> >> I don't know how much overhead this will create, alternatively we can >> >> skip the try/except clause and simply have the documentation as a >> >> dependency, or not add the docstrings for memberfunctions and add them >> >> later on import as was the original idea. >> > >> > This sounds cumbersome and in the example above will the try: except >> > clause be called everytime size is called. >> > >> > Johan >> > >> >> Suggestions and comments are more than welcome! >> >> >> >> Kristian >> >> >> >> > then in dolfin/__init__.py we load the classes as we do now from the >> >> > dolfin module, and then iterate over all functions and member >> >> > functions and substitute docstrings from the dolfindoc module. >> >> > >> >> > Kristian >> >> > >> >> >>> 2) The added python classes and methods can be documented using your >> >> >>> suggested approach, but instead of adding the docstring after class >> >> >>> creation, do it during class (method or function) creation, a la: >> >> >>> >> >> >>> class Foo(object): >> >> >>> __doc__ = docstrings.Foo.__doc__ >> >> >>> ... >> >> >>> >> >> >>> where docstrings is the generated module containing the docstrings. >> >> >>> >> >> >>> Johan >> >> >>> >> >> >>>> In other words we can't assign to the __doc__ of >> >> >>>> >> >> >>>> class Foo(object): >> >> >>>> "Foo doc" >> >> >>>> pass >> >> >>>> >> >> >>>> Which is a new-style class and found in UFL and the SWIG code in >> >> >>>> DOLFIN. >> >> >>>> >> >> >>>> It works fine for >> >> >>>> >> >> >>>> def some_function(v): >> >> >>>> "function doc" >> >> >>>> return 2*v >> >> >>>> >> >> >>>> and >> >> >>>> >> >> >>>> class Bar: >> >> >>>> "Bar doc" >> >> >>>> pass >> >> >>>> >> >> >>>> which is the old-style class often found in FFC. >> >> >>>> >> >> >>>> Does anyone have a solution or comments to the above approach, or >> >> >>>> maybe we can do it in a completely different way. >> >> >>>> >> >> >>>> I read about some workaround for the 'assign to __doc__' problem, >> >> >>>> but it doesn't seem that nice and it might be a problem to >> >> >>>> incorporate in the SWIG generated code? >> >> >>>> >> >> >>>> http://stackoverflow.com/questions/71817/using-the-docstring-from-o >> >> >>>> ne- metho d-to-automatically-overwrite-that-of-another-me >> >> >>>> >> >> >>>> >> >> >>>> Kristian >> >> >>>> >> >> >>>> _______________________________________________ >> >> >>>> Mailing list: https://launchpad.net/~fenics >> >> >>>> Post to : [email protected] >> >> >>>> Unsubscribe : https://launchpad.net/~fenics >> >> >>>> More help : https://help.launchpad.net/ListHelp >> >> >> >> _______________________________________________ >> >> Mailing list: https://launchpad.net/~fenics >> >> Post to : [email protected] >> >> Unsubscribe : https://launchpad.net/~fenics >> >> More help : https://help.launchpad.net/ListHelp >> >> _______________________________________________ >> Mailing list: https://launchpad.net/~fenics >> Post to : [email protected] >> Unsubscribe : https://launchpad.net/~fenics >> More help : https://help.launchpad.net/ListHelp > _______________________________________________ Mailing list: https://launchpad.net/~fenics Post to : [email protected] Unsubscribe : https://launchpad.net/~fenics More help : https://help.launchpad.net/ListHelp
