[sphinx-users] Is sphinx-build extremely dangerous? Or is it just me?
I'm new to Sphinx, but a seasoned Python programmer. I'm working through the Sphinx tutorials, and I may have by sheer luck barely avoided a disaster. Please: someone tell me if I'm being over dramatic, and calm this Sphinx-noob down. I'm so new, I'm not even really sure which part of Sphinx calls which, and which is exhibiting the [what I consider] extremely dangerous behavior: sphinx-build? sphinx-apidoc? make? So in my ignorance, I'll just attribute all to sphinx-build. It seems that sphinx-build imports every Python file it finds in the directory it is pointed to. But of course, it can't distinguish a true module (reusable code that performs no side-effects until functions are invoked) from a script (code invoke to do something). It can't do this because that distinction is purely in the developer's head, not in Python or Python files themselves. Therein is -- what I consider -- the extreme danger. In our system, we occasionally have maintenance scripts sitting in the directories with the modules. Script that do minor stuff like, oh, delete important system logs, kick off long-running (multi-hour) table generation routines, or alter production tables, or even drop entire databases. Imagine my panic when running sphinx-build for the first time, and I realize by the output that EVERYTHING is being imported... which means everything is being executed. Did I leave any scripts in a state where they are deleting important files or dropping databases? Are any of those configured to execute against our production system??? Apparently, by sheer luck, no script was configured so as to produce an irrecoverable side effect, and I think my system escaped unscathed. But it could easily have happened. Once my blood pressure returned to normal and my panic subsided, I went back to the tutorial docs, looking for an explanation that ALL files would actually be imported/executed. I didn't see that. I looked for a warning: if any of your modules or scripts in the tree perform side effects, put all that code in an if __name___=='__main__' block, or put ::sphinx-ignore-this-file as a comment somewhere in the file.. I didn't see that. Did I miss that? Has anyone considered how dangerous it is to execute EVERY PYTHON FILE in a large directory without proper warning (from Sphinx) and thorough code review (by the developer)? I guess I expected Sphinx would do its work by a simple lexical analysis of the Python files... and not actually import/execute them. I admit: the docs (the tutorials that I've read so far) don't SAY that. But they also don't say all will be imported/executed... which should be a VERY BIG caveat/warning in the tutorials. I love how the Sphinx docs look, and I'd like to use the system for my project and my team... But I need to have a rock solid way to make sure that the documentation build doesn't start trashing my operational system by executing scripts that had no intention of running or even have anything useful to be documented in them. -- You received this message because you are subscribed to the Google Groups sphinx-users group. To unsubscribe from this group and stop receiving emails from it, send an email to sphinx-users+unsubscr...@googlegroups.com. To post to this group, send email to sphinx-users@googlegroups.com. Visit this group at http://groups.google.com/group/sphinx-users. For more options, visit https://groups.google.com/groups/opt_out.
Re: [sphinx-users] Is sphinx-build extremely dangerous? Or is it just me?
Thanks for reporting this. To be more precise, it is `sphinx-apidoc http://sphinx-doc.org/invocation.html#invocation-of-sphinx-apidoc`_ who scans the filesystem and finds every Python module, writing Sphinx source files. You must then run sphinx-build with `sphinx.ext.autodoc http://sphinx-doc.org/ext/autodoc.html`_ enabled to actually import all those modules. I suggest to add a warning about this possible pitfall on both documentation pages. Luc On 04/03/14 16:09, Dan Harasty wrote: I'm new to Sphinx, but a seasoned Python programmer. I'm working through the Sphinx tutorials, and I may have by sheer luck barely avoided a disaster. Please: someone tell me if I'm being over dramatic, and calm this Sphinx-noob down. I'm so new, I'm not even really sure which part of Sphinx calls which, and which is exhibiting the [what I consider] extremely dangerous behavior: sphinx-build? sphinx-apidoc? make? So in my ignorance, I'll just attribute all to sphinx-build. It seems that sphinx-build imports every Python file it finds in the directory it is pointed to. But of course, it can't distinguish a true module (reusable code that performs no side-effects until functions are invoked) from a script (code invoke to do something). It can't do this because that distinction is purely in the developer's head, not in Python or Python files themselves. Therein is -- what I consider -- the extreme danger. In our system, we occasionally have maintenance scripts sitting in the directories with the modules. Script that do minor stuff like, oh, delete important system logs, kick off long-running (multi-hour) table generation routines, or alter production tables, or even drop entire databases. Imagine my panic when running sphinx-build for the first time, and I realize by the output that EVERYTHING is being imported... which means everything is being executed. Did I leave any scripts in a state where they are deleting important files or dropping databases? Are any of those configured to execute against our production system??? Apparently, by sheer luck, no script was configured so as to produce an irrecoverable side effect, and I think my system escaped unscathed. But it could easily have happened. Once my blood pressure returned to normal and my panic subsided, I went back to the tutorial docs, looking for an explanation that ALL files would actually be imported/executed. I didn't see that. I looked for a warning: if any of your modules or scripts in the tree perform side effects, put all that code in an if __name___=='__main__' block, or put ::sphinx-ignore-this-file as a comment somewhere in the file.. I didn't see that. Did I miss that? Has anyone considered how dangerous it is to execute EVERY PYTHON FILE in a large directory without proper warning (from Sphinx) and thorough code review (by the developer)? I guess I expected Sphinx would do its work by a simple lexical analysis of the Python files... and not actually import/execute them. I admit: the docs (the tutorials that I've read so far) don't SAY that. But they also don't say all will be imported/executed... which should be a VERY BIG caveat/warning in the tutorials. I love how the Sphinx docs look, and I'd like to use the system for my project and my team... But I need to have a rock solid way to make sure that the documentation build doesn't start trashing my operational system by executing scripts that had no intention of running or even have anything useful to be documented in them. -- You received this message because you are subscribed to the Google Groups sphinx-users group. To unsubscribe from this group and stop receiving emails from it, send an email to sphinx-users+unsubscr...@googlegroups.com mailto:sphinx-users+unsubscr...@googlegroups.com. To post to this group, send email to sphinx-users@googlegroups.com mailto:sphinx-users@googlegroups.com. Visit this group at http://groups.google.com/group/sphinx-users. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups sphinx-users group. To unsubscribe from this group and stop receiving emails from it, send an email to sphinx-users+unsubscr...@googlegroups.com. To post to this group, send email to sphinx-users@googlegroups.com. Visit this group at http://groups.google.com/group/sphinx-users. For more options, visit https://groups.google.com/groups/opt_out.
Re: [sphinx-users] Is sphinx-build extremely dangerous? Or is it just me?
Am 04.03.2014 15:09, schrieb Dan Harasty: I'm new to Sphinx, but a seasoned Python programmer. I'm working through the Sphinx tutorials, and I may have by sheer luck barely avoided a disaster. Please: someone tell me if I'm being over dramatic, and calm this Sphinx-noob down. I'm so new, I'm not even really sure which part of Sphinx calls which, and which is exhibiting the [what I consider] extremely dangerous behavior: sphinx-build? sphinx-apidoc? make? So in my ignorance, I'll just attribute all to sphinx-build. It is in fact autodoc, but you're right, for the user that does not matter. It seems that sphinx-build imports every Python file it finds in the directory it is pointed to. But of course, it can't distinguish a true module (reusable code that performs no side-effects until functions are invoked) from a script (code invoke to do something). It can't do this because that distinction is purely in the developer's head, not in Python or Python files themselves. Therein is -- what I consider -- the extreme danger. In our system, we occasionally have maintenance scripts sitting in the directories with the modules. Script that do minor stuff like, oh, delete important system logs, kick off long-running (multi-hour) table generation routines, or alter production tables, or even drop entire databases. Imagine my panic when running sphinx-build for the first time, and I realize by the output that EVERYTHING is being imported... which means everything is being executed. Did I leave any scripts in a state where they are deleting important files or dropping databases? Are any of those configured to execute against our production system??? Apparently, by sheer luck, no script was configured so as to produce an irrecoverable side effect, and I think my system escaped unscathed. But it could easily have happened. Once my blood pressure returned to normal and my panic subsided, I went back to the tutorial docs, looking for an explanation that ALL files would actually be imported/executed. I didn't see that. I looked for a warning: if any of your modules or scripts in the tree perform side effects, put all that code in an if __name___=='__main__' block, or put ::sphinx-ignore-this-file as a comment somewhere in the file.. I didn't see that. The former is good practice, so I encourage it in any case. Did I miss that? Has anyone considered how dangerous it is to execute EVERY PYTHON FILE in a large directory without proper warning (from Sphinx) and thorough code review (by the developer)? I guess I expected Sphinx would do its work by a simple lexical analysis of the Python files... and not actually import/execute them. I admit: the docs (the tutorials that I've read so far) don't SAY that. But they also don't say all will be imported/executed... which should be a VERY BIG caveat/warning in the tutorials. I agree, this needs to be documented better. Please have a look at https://bitbucket.org/birkenfeld/sphinx/commits/00e44121ac086e0728ced51300251300f9040498 which I just pushed. I love how the Sphinx docs look, and I'd like to use the system for my project and my team... But I need to have a rock solid way to make sure that the documentation build doesn't start trashing my operational system by executing scripts that had no intention of running or even have anything useful to be documented in them. The best way to ensure that is -- and that is unrelated to Sphinx -- is the if __name__ == '__main__' guard. It's only 3 lines and a bit of indentation. cheers, Georg -- You received this message because you are subscribed to the Google Groups sphinx-users group. To unsubscribe from this group and stop receiving emails from it, send an email to sphinx-users+unsubscr...@googlegroups.com. To post to this group, send email to sphinx-users@googlegroups.com. Visit this group at http://groups.google.com/group/sphinx-users. For more options, visit https://groups.google.com/groups/opt_out.
