On 22 July 2010 20:10, Johan Hake <[email protected]> wrote: > On Monday July 19 2010 12:54:44 Kristian Oelgaard wrote: >> On 23 June 2010 15:35, Kristian Oelgaard <[email protected]> wrote: >> > On 23 June 2010 10:55, Kristian Oelgaard <[email protected]> wrote: >> >> On 22 June 2010 19:03, Johan Hake <[email protected]> wrote: >> >>> On Tuesday June 22 2010 08:28:37 Kristian Oelgaard wrote: >> >>>> I've started writing the programmer's reference for FEniCS. >> >>>> One of the features that we decided on was that doc-strings for >> >>>> PyDOLFIN should be written and maintained as part of the documentation >> >>>> project and then added to the dolfin module on import. >> >>>> >> >>>> I thought about doing this in the following way: >> >>>> >> >>>> 1) Create a pseudo module 'dolfin-doc' which is a copy of the classes >> >>>> and functions in the 'real' dolfin module only it contains no code at >> >>>> all, just doc-strings. (This approach will also make it easy to create >> >>>> a script to check if all functions are documented or if any docs are >> >>>> obsolete). >> >>> >> >>> Sounds good. I first thought of a structure (other than a dummy class) >> >>> that just mimics the class hierarchy, but in some way that is what you >> >>> actually suggests and it is probably as easy as anything else. >> >>> >> >>>> 2) Use the autodoc functionality of Sphinx to create parts of the >> >>>> documentation for functions and classes >> >>>> >> >>>> 3) Manually add additional information (in the reST file) and links to >> >>>> other parts of the documentation like demos etc. This will not be >> >>>> available using help() in the Python interpreter. >> >>>> >> >>>> 4) In the dolfin.__init__.py function import the 'dolfin-doc' module >> >>>> and copy the doc-strings from all classes and functions to the classes >> >>>> and functions in the real dolfin module as was suggested by Johan >> >>>> Hake. >> >>>> >> >>>> The problem with this approach is that assigning to __doc__ is not >> >>>> permitted for objects of 'type' type. >> >>>> >> >>> :( >> >>> >> >>> I did not anticipate this. Not sure why this is. I have got the >> >>> impression that numpy get around this. They use numpydoc to >> >>> dynamically add their documentation. It makes heavy use of sphinx, but >> >>> I couldn't figure how they get around that __doc__ is read-only. >> >> >> >> To me it looks like numpydoc is a Sphinx extension that translates the >> >> Numpy docstrings into something that Sphinx can understand, not the >> >> other way around which is what we want. >> >> >> >> http://projects.scipy.org/numpy/browser/trunk/doc/sphinxext/README.txt >> >> >> >> So I think our best bet is to proceed with your suggestions below. >> >> >> >> Kristian >> >> >> >>> While it might be cool to look into what NumPy have done, (they also >> >>> define a pseudo classes, which they populate with docstrings, (look >> >>> into phantom_import.py), and they also define some nice format for the >> >>> reST used in the docstrings), I suggest two things we can do: >> >>> >> >>> 1) SWIG can generate docstrings. We do that allready using parsed >> >>> doxygen documentation. All of this is gathered in docstrings.i. I >> >>> suggest generating such a file from our documentation. We need to turn >> >>> of %feature("autodoc","1") in dolfin.i to get rid of the long and >> >>> sometimes faulty generated signatures. >> > >> > I turns out that it's only the __doc__ of the class I can't assign to, >> > not the __doc__ of member functions (and regular functions). >> > A simpler solution (at least for me) is to parse the cpp.py module >> > once generated and substitute all docstrings of classes with the >> > docstrings from the dolfindoc module rather than creating the >> > docstrings.i file. >> > >> > Then for the classes that we manually add we use the method you >> > described below, but only for class.__doc__ . >> > >> > class Foo(object): >> > __doc__ = dolfindoc.Foo.__doc__ >> > def bar(self): >> > "this doc string will be substituted with the >> > dolfindoc.Foo.__dict__["bar"].__doc__." >> > pass >> >> Unfortunately it turned out I was too quick to make this conclusion. >> For instance, >> for the Swig generated class Mesh in the module cpp I can't assign to >> >> Mesh.__doc__ >> >> because it is a 'type' object as pointed our earlier, I can assign to: >> >> Mesh.__dict__["__init__"].__doc__ >> >> but not >> >> Mesh.__dict__["size"].__doc__ >> >> along with practically all other member functions. >> >> The reason is that Swig, after the class definitions does this: >> >> Mesh.size = new_instancemethod(_cpp.Mesh_size,None,Mesh) >> >> to which no assignment is possible. >> 'AttributeError: attribute '__doc__' of 'instancemethod' objects is >> not writable' >> The code in Mesh.size is already >> >> def size(self, *args): >> return _cpp.Mesh_size(self, *args) >> >> so why Swig overrides the method I don't know. If it didn't I could >> assign to the docstring. >> >> Recall that my initial plan was to simply parse the cpp.py file and >> substitute the docstrings of all classes with what the pseudo module >> dolfindoc would define. However, that doesn't seem so practical >> anymore since I would need to also comment out all lines where >> instancemethods are being assigned to class members such that on >> import (in dolfin/__init__.py) I can loop classes and assign to >> function docstrings. This is of course possible but seems a bit >> awkward and I don't know what implications the commenting out of >> instancemethod assignment will have. >> As I see it, two solutions remain. >> >> 1) Take the pseudo module dolfindoc from the FEniCS documentation and >> create a docstring.i generator. The script >> dolfin/dolfin/swig/generator.py should then try to import the >> dolfindoc module, if successful use that to create docstring.i, >> otherwise use the Doxy2SWIG generator. > > I thought that you landed on this alternative. The point is to generate a > dosctrings.i file from the FEniCS documentation. Then for the extended Python > layer we can use a generated Python module to assign the docstrings:
I landed on a slightly modified version of that yes. OK, the docstrings.i part should be relatively easy to generate and probably very fast too since I don't have to generate the html pages and parse. > class VariationalProblem(...): > ... > __doc__ generated_docstring_module.VariationalProblem.__doc__ This I don't understand, if we generate a module which we will distribute with DOLFIN why not simply distribute the module from the FEniCS documentation? >> One question w.r.t this approach, when is docstrings.i being >> generated? I see it is distributed with DOLFIN but shouldn't it be >> generated everytime DOLFIN is rebuild with enablePython, or does >> enableDocs have to be switched on as well? > > It is generated by running dolfin/swig/generate.py. It takes quite a long time > so I think it is good to pre-generate this in the distribution. OK, so if a developer actually happens to update or even write docstrings, the docstrings.i should be updated before commit. (at least in theory ;)) >> 2) Extend our Python layer with all functions and classes that we want >> to use the dolfindoc docstrings for. >> These should just be empty and redirect calls to classes and functions >> of the cpp. In dolfin/mesh/mesh.py: >> >> class Mesh(cpp.Mesh): >> def size(self): >> try: >> import dolfindoc.mesh.mesh >> __doc__ = dolfindoc.mesh.mesh.Mesh.__doc__ >> except: >> __doc__ = cpp.Mesh.__doc__ >> return cpp.Mesh.size() >> >> An additional benefit of this approach is that the module structure >> can be identical to what we have in the _real_ DOLFIN, not as it is >> now where everything is dumped in the dolfin.cpp module. >> I don't know how much overhead this will create, alternatively we can >> skip the try/except clause and simply have the documentation as a >> dependency, or not add the docstrings for memberfunctions and add them >> later on import as was the original idea. > > This sounds cumbersome and in the example above will the try: except clause be > called everytime size is called. Yes, I thought about having no docstrings as default (just need them for the class docstring) and the dynamically add them on import for all member functions. I know it's a bit more work, but we will have the same layout of modules as we have for the directories in DOLFIN. Kristian > Johan > >> Suggestions and comments are more than welcome! >> >> Kristian >> >> > then in dolfin/__init__.py we load the classes as we do now from the >> > dolfin module, and then iterate over all functions and member >> > functions and substitute docstrings from the dolfindoc module. >> > >> > Kristian >> > >> >>> 2) The added python classes and methods can be documented using your >> >>> suggested approach, but instead of adding the docstring after class >> >>> creation, do it during class (method or function) creation, a la: >> >>> >> >>> class Foo(object): >> >>> __doc__ = docstrings.Foo.__doc__ >> >>> ... >> >>> >> >>> where docstrings is the generated module containing the docstrings. >> >>> >> >>> Johan >> >>> >> >>>> In other words we can't assign to the __doc__ of >> >>>> >> >>>> class Foo(object): >> >>>> "Foo doc" >> >>>> pass >> >>>> >> >>>> Which is a new-style class and found in UFL and the SWIG code in >> >>>> DOLFIN. >> >>>> >> >>>> It works fine for >> >>>> >> >>>> def some_function(v): >> >>>> "function doc" >> >>>> return 2*v >> >>>> >> >>>> and >> >>>> >> >>>> class Bar: >> >>>> "Bar doc" >> >>>> pass >> >>>> >> >>>> which is the old-style class often found in FFC. >> >>>> >> >>>> Does anyone have a solution or comments to the above approach, or >> >>>> maybe we can do it in a completely different way. >> >>>> >> >>>> I read about some workaround for the 'assign to __doc__' problem, but >> >>>> it doesn't seem that nice and it might be a problem to incorporate in >> >>>> the SWIG generated code? >> >>>> >> >>>> http://stackoverflow.com/questions/71817/using-the-docstring-from-one- >> >>>> metho d-to-automatically-overwrite-that-of-another-me >> >>>> >> >>>> >> >>>> Kristian >> >>>> >> >>>> _______________________________________________ >> >>>> Mailing list: https://launchpad.net/~fenics >> >>>> Post to : [email protected] >> >>>> Unsubscribe : https://launchpad.net/~fenics >> >>>> More help : https://help.launchpad.net/ListHelp >> >> _______________________________________________ >> Mailing list: https://launchpad.net/~fenics >> Post to : [email protected] >> Unsubscribe : https://launchpad.net/~fenics >> More help : https://help.launchpad.net/ListHelp > _______________________________________________ Mailing list: https://launchpad.net/~fenics Post to : [email protected] Unsubscribe : https://launchpad.net/~fenics More help : https://help.launchpad.net/ListHelp
