Re: [Python-Dev] Private header files (Was: Renaming Include/object.h)
On 1/3/07, Martin v. Löwis [EMAIL PROTECTED] wrote: Neal Norwitz schrieb: By private, I mean internal only to python and don't need to prefix their identifiers with Py and are subject to change without backwards compatibility. Include/graminit.h is one example of what I mean. Some others are: bitset.h, grammar.h, opcode.h, metagrammar.h, errcode.h Ah. This seems to be a requirement completely different from the one I'm talking about. By this definition, object.h is *not* an internal header file, yet I want it to be renamed. Agreed. I was mixing two things that aren't necessarily related because I see the same possible solution. I'm also using this one example as a good opportunity to clean up more things. Let me try to explain a bit more below. As for this issue: how about moving all such private header files out of Include entirely? The parser-related ones should go into Parser, for example (grammar.h, bitset.h, graminit.h, metagrammar.h, errcode.h). This would leave us with opcode.h only. Others are kinda questionable (they have some things that are definitely public, others I'm not so sure about): code.h, parsetok.h, pyarena.h, longintrepr.h, osdefs.h, pgen.h, node.h Thomas said that at least code.h must stay where it is. What is the reason that you want them to be renamed? Sorry, I wasn't trying to imply that these should necessarily be renamed, only that the internal portions be moved elsewhere. I guess I should explain my mental model first which might make things clearer. Then again, I'm tired, so who knows if it will explain anything. :-) I'm a Python embedder and I want to know what's available to me. I look in Include and see a ton of header files. Do I need all these? What do I *need* and what can I *use*? I only want to see the public stuff that is available to me. Thus I want anything that has internal/implementation details/etc out of my sight to reduce my learning curve. I don't ever want to learn about something I won't need nor include anything I won't need. That's one part. Another part of my mental model is that I'm a Python developer and I'm modifying a header file that is implementation specific. I need to share it among different subdiretories (say Python and Objects). So I really need to stick the header file in a common place, Include/ is it. I don't want to export anything, but I don't know if other third party developers will use the header or not. Or maybe I need to include some implementation details in another public header. I'll probably be lazy and just make a single header which has some internal and some public stuff. I want clear rules on when identifiers need to be prefixed. If it's internal (e.g., in an internal directory or prefixed with _), it can have any name and can't be included from any non-internal header. I can also change it in a point release. If anyone uses anything from here, they are on their own. If I see any identifier in a non-internal header, it must be public and therefore prefixed with Py or _Py. The Python headers are pretty good about prefixing most things. But they could be better. I think it gets harder to maintain without the rules though. Finally, by putting everything in directories and always including a directory in the header file, like: #include python/python.h #include python/internal/foo.h There can never be an include file name collision as what started this thread. It also provides a simple way of demonstrating what's public and what is not. It addresses all my complaints. There are only a few rules and they are simple. But I am addressing several points that are only loosely related which I what I think generated some confusion. Adding the directory also makes clear were the header file comes from. If you see: #include node.h you don't know if that's a python node.h, from some other part of the code or a third party library. Not to try to confuse things even more, but I will point out something Google does that is only indirectly related. Google requires importing modules. You aren't supposed to import classes. You wouldn't do: from foo.bar import Message # ... msg = Message(...) You would do: from foo import bar # ... msg = bar.Message(...) This makes it clear where Message comes from, just like adding a python prefix to all header file names makes it clear where the header file lives. Both are good for traceability, though in different ways. This technique makes it easier to manage larger code bases, particularly when there are multiple libraries used. n ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Private header files (Was: Renaming Include/object.h)
Neal Norwitz schrieb: I'm a Python embedder and I want to know what's available to me. I look in Include and see a ton of header files. Do I need all these? What do I *need* and what can I *use*? I only want to see the public stuff that is available to me. Thus I want anything that has internal/implementation details/etc out of my sight to reduce my learning curve. I don't ever want to learn about something I won't need nor include anything I won't need. Ideally, you should look at the documentation. It should have everything you can use, and only that. I know that this currently doesn't work, for two reasons: - the documentation is incomplete, i.e. it omits things that people do use on a regular basis - people read header files, even if the documentation was complete Another part of my mental model is that I'm a Python developer and I'm modifying a header file that is implementation specific. I need to share it among different subdiretories (say Python and Objects). So I really need to stick the header file in a common place, Include/ is it. There could be another standard directory for header files also, like Python. I don't want to export anything, but I don't know if other third party developers will use the header or not. I find that debatable. I see no reason not to export everything, unless a stable ABI is also an objective (which it may be). If people really want to get to the internals, they can access structures even if they aren't defined in a header file, by replicating the structure definition in their own code. There should be a mostly stable API, certainly, and this is the one that is documented. Other things in the header files aren't internal, they are just undocumented and thus may change without notice. I want clear rules on when identifiers need to be prefixed. Always, unless they are static functions. Even an internal function must be prefixed, as you may otherwise get conflicts when embedding Python. At the moment, only the AST exports symbols that aren't properly mangled. [internal symbols] I can also change it in a point release. If anyone uses anything from here, they are on their own. No. There shouldn't be any possible ABI breakage in a point release. Finally, by putting everything in directories and always including a directory in the header file, like: #include python/python.h #include python/internal/foo.h There can never be an include file name collision as what started this thread. It also provides a simple way of demonstrating what's public and what is not. It addresses all my complaints. There are only a few rules and they are simple. As specified, above, it is incompatible with the current API. I think #include Python.h should be preserved. I personally see no problem with a single header file, and would prefer that include to indicate all API that we are committed to document and keep mostly stable. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Private header files (Was: Renaming Include/object.h)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Jan 4, 2007, at 4:17 AM, Martin v. Löwis wrote: As specified, above, it is incompatible with the current API. I think #include Python.h should be preserved. I personally see no problem with a single header file, and would prefer that include to indicate all API that we are committed to document and keep mostly stable. I agree. I don't mind if Python.h is just a wrapper around #includes from python/*.h. I think we should add structmember.h and structseq.h to Python.h and perhaps move everything else into a 'python' subdirectory. - -Barry -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRZ0pfnEjvBPtnXfVAQKuyAP+J8Tm3z4am5BOfXCSJIeHsA1tEddeniuM dqSUAPUQUag9WkvkAreQYXu3iRC26e52mJ0B8eceqiuBa16WPILb0CvRFCBoW2fc /FAg4EROlMBhrE/MWVSfGSi76bL+4CwaogmHOnsvyCDEppA420C/8Zkz1VZYiGGd T+Ppo1ZVsHA= =b/Fm -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Private header files (Was: Renaming Include/object.h)
Barry Warsaw schrieb: I agree. I don't mind if Python.h is just a wrapper around #includes from python/*.h. I think we should add structmember.h and structseq.h to Python.h and perhaps move everything else into a 'python' subdirectory. For the python subdirectory, there is the issue that the framework includes in OSX magically look for python.framework when searching for python/foo.h, which they find, so that may get us the wrong version. Somebody would have to study the details here, first. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Private header files (Was: Renaming Include/object.h)
On Thursday 04 January 2007 11:33, Martin v. Löwis wrote: For the python subdirectory, there is the issue that the framework includes in OSX magically look for python.framework when searching for python/foo.h, which they find, so that may get us the wrong version. Somebody would have to study the details here, first. If everything public gets included from Python.h, perhaps python/object.h and friends could become pythonX.Y/object.h; I'm not sure this will solve the Mac OS framework magic issue, though, not being a Mac OS developer. -Fred -- Fred L. Drake, Jr. fdrake at acm.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Private header files (Was: Renaming Include/object.h)
On 4 Jan, 2007, at 17:56, Fred L. Drake, Jr. wrote: On Thursday 04 January 2007 11:33, Martin v. Löwis wrote: For the python subdirectory, there is the issue that the framework includes in OSX magically look for python.framework when searching for python/foo.h, which they find, so that may get us the wrong version. Somebody would have to study the details here, first. If everything public gets included from Python.h, perhaps python/ object.h and friends could become pythonX.Y/object.h; I'm not sure this will solve the Mac OS framework magic issue, though, not being a Mac OS developer. That would solve the problem, however I don't think there is one. An experiment seems to indicate that the include path is prefered over the magic inclusion of framework headers (see the trace below). I haven't checked yet if the behaviour shown below is intentional, but I'd be surprised if it isn't. $ mkdir include $ mkdir include/Python $ cat include/Python/object.h EOF #error my header included EOF $ cat demo.c -EOF #include Python/object.h EOF $ $ cc -c demo.c In file included from demo.c:1: /System/Library/Frameworks/Python.framework/Headers/object.h:227: error: parse error before ‘FILE’ /System/Library/Frameworks/Python.framework/Headers/object.h:353: error: parse error before ‘PyType_IsSubtype’ ... more errors removed, this clearly includes a header from the python framework $ $ cc -I include -c t.c In file included from demo.c:1: include/Python/object.h:1:2: error: #error my error Therefore moving all headers into a directory named Python would cause no problems users that use the normal way of linking with python and would even allow users to use the python framework as a normal framework. Ronald -Fred -- Fred L. Drake, Jr. fdrake at acm.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ ronaldoussoren%40mac.com smime.p7s Description: S/MIME cryptographic signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Private header files (Was: Renaming Include/object.h)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Jan 4, 2007, at 2:36 PM, Ronald Oussoren wrote: On 4 Jan, 2007, at 17:56, Fred L. Drake, Jr. wrote: On Thursday 04 January 2007 11:33, Martin v. Löwis wrote: For the python subdirectory, there is the issue that the framework includes in OSX magically look for python.framework when searching for python/foo.h, which they find, so that may get us the wrong version. Somebody would have to study the details here, first. If everything public gets included from Python.h, perhaps python/ object.h and friends could become pythonX.Y/object.h; I'm not sure this will solve the Mac OS framework magic issue, though, not being a Mac OS developer. That would solve the problem, however I don't think there is one. An experiment seems to indicate that the include path is prefered over the magic inclusion of framework headers (see the trace below). I haven't checked yet if the behaviour shown below is intentional, but I'd be surprised if it isn't. $ mkdir include $ mkdir include/Python $ cat include/Python/object.h EOF #error my header included EOF $ cat demo.c -EOF #include Python/object.h EOF $ $ cc -c demo.c In file included from demo.c:1: /System/Library/Frameworks/Python.framework/Headers/object.h:227: error: parse error before ‘FILE’ /System/Library/Frameworks/Python.framework/Headers/object.h:353: error: parse error before ‘PyType_IsSubtype’ ... more errors removed, this clearly includes a header from the python framework $ $ cc -I include -c t.c In file included from demo.c:1: include/Python/object.h:1:2: error: #error my error Therefore moving all headers into a directory named Python would cause no problems users that use the normal way of linking with python and would even allow users to use the python framework as a normal framework. I think that's basically correct: framework includes are searched after normal includes. You might be able to confuse things if you used -F/-framework options, because I believe those are interleaved with -I paths, but then if you add -F presumably you should know what you're doing. Thanks for testing this. - -Barry -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRZ1abHEjvBPtnXfVAQJQtAP/XgGgI2z7xUGJlxBGfZiggIEtxRYzJObn TVl/2r7tJ58QCwTzc+eI/m18gcfi85q+hmS1hPc9tjq0ICiqZGjSI9hpSsq0Uqva WXKFscmvnNyZLrhemy8AjHSbA7dKKBGKBmqycjEt26am4LetoCD/HCt44+AaoI3d SIzFFiSKw/4= =4FpY -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Private header files (Was: Renaming Include/object.h)
Neal Norwitz schrieb: By private, I mean internal only to python and don't need to prefix their identifiers with Py and are subject to change without backwards compatibility. Include/graminit.h is one example of what I mean. Some others are: bitset.h, grammar.h, opcode.h, metagrammar.h, errcode.h Ah. This seems to be a requirement completely different from the one I'm talking about. By this definition, object.h is *not* an internal header file, yet I want it to be renamed. As for this issue: how about moving all such private header files out of Include entirely? The parser-related ones should go into Parser, for example (grammar.h, bitset.h, graminit.h, metagrammar.h, errcode.h). This would leave us with opcode.h only. Others are kinda questionable (they have some things that are definitely public, others I'm not so sure about): code.h, parsetok.h, pyarena.h, longintrepr.h, osdefs.h, pgen.h, node.h Thomas said that at least code.h must stay where it is. What is the reason that you want them to be renamed? Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
