Re: Package organization
You read me like a book :) I just moved from C# a few months ago, and I played with C++ for a while before that. Seriously though, for some of my more complicated subsystems (graphics, for example), putting all the classes in one module seems excessive. Right now that subsystem has around 10 classes, and may end up with about twice that. I could find a logical split between a few modules, but that would defeat the purpose of reducing the level of nesting. Having all my code in a single module would make it hard to find the code I want to edit quickly. Subject: Re: Package organization From: Matt Good [EMAIL PROTECTED] Date: Sat, 15 Jul 2006 15:57:22 -0700 To: python-list@python.org One class per module? Sounds like you've been programming in Java (or C#) for too long :) But skipping the package structure for a moment you can simplify your use of the Timer class by changing your imports: from engine.timer.timer import Timer t = Timer() OR from engine.timer import timer t = timer.Timer() Ok, back to package structure. In Python putting 1 class per module basically means you're adding an extra level of nesting beyond the equivalent structure in languages like Java and C# that require you to create a new file per-class. In Java defining "engine/timer/Timer.java" you'd get an "engine.timer.Timer" class. As you saw in Python defining Timer in "engine/timer/timer.py" the class is now "engine.timer.timer.Timer". Dropping that extra level by combining the classes in engine.timer into a single module will simplify things. -- Matt Good -- http://mail.python.org/mailman/listinfo/python-list
Re: Package organization
Mike Wyatt wrote: I've been playing around with Python for a few months now, and I just recently started looking at packages to organize my growing project. So far, I've been organizing my application into one class per module. This has been working pretty well. For example, I simply import timer, then use t = timer.Timer() to allocate a new Timer object, . Unfortunately, this is yielding some pretty odd syntax when I use packages. Here is a sample of my package structure: /engine /graphics /input /world /timer timer.py # contains Timer class main.py Let's say I want to create a Timer object in main.py. I would need to do something like this: import engine.timer.timer.Timer t = engine.timer.timer.Timer() Maybe I shouldn't have a module and package with the same name, but it seems the most logical design. Unfortunately, the code is a bit ugly with timer included in the import three times. Is there a better way to do this? How do you guys organize your packages and modules? Basically you are over-organizing for your needs here. Is there any reason why the timer.py file defininf the Timer function needs to be in its own sub-package? It's bad enough having to write timer.Timer(), don't compound that folly unnecessarily! regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC/Ltd http://www.holdenweb.com Skype: holdenweb http://holdenweb.blogspot.com Recent Ramblings http://del.icio.us/steve.holden -- http://mail.python.org/mailman/listinfo/python-list
Package organization
I've been playing around with Python for a few months now, and I just recently started looking at packages to organize my growing project. So far, I've been organizing my application into one class per module. This has been working pretty well. For example, I simply import timer, then use t = timer.Timer() to allocate a new Timer object, . Unfortunately, this is yielding some pretty odd syntax when I use packages. Here is a sample of my package structure: /engine /graphics /input /world /timer timer.py # contains Timer class main.py Let's say I want to create a Timer object in main.py. I would need to do something like this: import engine.timer.timer.Timer t = engine.timer.timer.Timer() Maybe I shouldn't have a module and package with the same name, but it seems the most logical design. Unfortunately, the code is a bit ugly with timer included in the import three times. Is there a better way to do this? How do you guys organize your packages and modules? -- http://mail.python.org/mailman/listinfo/python-list
Re: Package organization
Mike Wyatt wrote: I've been playing around with Python for a few months now, and I just recently started looking at packages to organize my growing project. So far, I've been organizing my application into one class per module. This has been working pretty well. For example, I simply import timer, then use t = timer.Timer() to allocate a new Timer object, . Unfortunately, this is yielding some pretty odd syntax when I use packages. Here is a sample of my package structure: /engine /graphics /input /world /timer timer.py # contains Timer class main.py Let's say I want to create a Timer object in main.py. I would need to do something like this: import engine.timer.timer.Timer t = engine.timer.timer.Timer() Maybe I shouldn't have a module and package with the same name, but it seems the most logical design. Unfortunately, the code is a bit ugly with timer included in the import three times. Is there a better way to do this? How do you guys organize your packages and modules? One class per module? Sounds like you've been programming in Java (or C#) for too long :) But skipping the package structure for a moment you can simplify your use of the Timer class by changing your imports: from engine.timer.timer import Timer t = Timer() OR from engine.timer import timer t = timer.Timer() Ok, back to package structure. In Python putting 1 class per module basically means you're adding an extra level of nesting beyond the equivalent structure in languages like Java and C# that require you to create a new file per-class. In Java defining engine/timer/Timer.java you'd get an engine.timer.Timer class. As you saw in Python defining Timer in engine/timer/timer.py the class is now engine.timer.timer.Timer. Dropping that extra level by combining the classes in engine.timer into a single module will simplify things. -- Matt Good -- http://mail.python.org/mailman/listinfo/python-list
Re: Package organization: where to put 'common' modules?
fortepianissimo wrote: Paul Boddie wrote: fortepianissimo wrote: Hm this doesn't work. Say I have the following directory structure: A |--- util ||--- foo.py | |--- B |--- bar.py And bar.py has this line from util import foo [...] Placing code modifying sys.path in main would be a problem if the utility loading line is at the beginning of the module - error would occur before sys.path can be modified. Well, I suppose you could start your module with... import sys, os sys.path.append(some_nasty_tricks_with_paths) Although I don't really recommend this, since such code is going to get run as the modules get imported, and this could cause some additional complications if various things then conspire to import the wrong modules somewhere else in your system. It is especially true for me since I need to load these common functionality not just for testing - they are necessary to implement the functionality of the module. Besides, adding code to manipulate sys.path in *every* module that needs the common functionality module seems to be quite a hassle... I suppose you're one of the people who may benefit from the relative import mechanisms in Python 2.5. Myself, I can't see why it's so hard to write... from A.util import foo [...] I need to import this common functionality not just for testing code, so it probably doesn't matter if I separate the testing code from the module itself. I stopped running modules inside packages directly precisely because of the behaviour you've seen. I haven't experienced many problems with my test from the outside approach since. In addition, the common functionality is common only for the system I'm building - they are probably not that useful for general purposes. So placing them in an absolute path (like site packages) doesn't make a lot of sense. Right. So it's just a matter of making sure that everything can import such functionality without either triggering circular imports or mixing up internal modules with external ones (as my example showed). Paul -- http://mail.python.org/mailman/listinfo/python-list
Re: Package organization: where to put 'common' modules?
Kent Johnson wrote: Paul Boddie wrote: Yes, Python does this - it puts the directory of bar.py (B in this case) in sys.path, but not the directory in which you're sitting when you run the program from the shell (A in this case). This seems to be OS dependent. If I put 'print sys.path' at the start of site.py (which does most of the setup of sys.path), one element of the path is an empty string. This is expanded by main() - removeduppaths() - makepath() - os.path.abspath() to the current working dir. This is Python 2.4.2 on Win2K. Kent Following your example, I tried to use symlink to solve my problem. I have the following dir structure: A |--- util ||--- foo.py | |--- B ||--- util (symlink to ../util) ||--- bar.py | |--- main.py -- util/foo.py: print 'foo initialized' def baz(): print 'foo.baz() here' -- B/bar.py: from util import foo foo.baz() -- main.py: import sys from util import foo print id(sys.modules['util.foo']) from B import bar print id(sys.modules['util.foo']) -- Now when I run python main.py in dir A, I got the following result (Python 2.4.2, Mac OS X): foo initialized 3698320 foo initialized foo.baz() here 3698320 My question is why foo got initialized twice? -- http://mail.python.org/mailman/listinfo/python-list
Re: Package organization: where to put 'common' modules?
Hm this doesn't work. Say I have the following directory structure: A |--- util ||--- foo.py | |--- B |--- bar.py And bar.py has this line from util import foo I then run python B/bar.py in directory A. Still got error ImportError: No module named util A check of sys.path in bar.py when running from A revealed that sys.path has A/B, not A. Am I missing something here? -- http://mail.python.org/mailman/listinfo/python-list
Re: Package organization: where to put 'common' modules?
fortepianissimo wrote: Hm this doesn't work. Say I have the following directory structure: A |--- util ||--- foo.py | |--- B |--- bar.py And bar.py has this line from util import foo I then run python B/bar.py in directory A. Still got error ImportError: No module named util Do you have a file util/__init__.py? This is required to make python recognize util as a package. This works for me: C:\WUTemp\Adir /b/s C:\WUTemp\A\B C:\WUTemp\A\util C:\WUTemp\A\B\bar.py C:\WUTemp\A\B\__init__.py C:\WUTemp\A\util\foo.py C:\WUTemp\A\util\foo.pyc C:\WUTemp\A\util\__init__.py C:\WUTemp\A\util\__init__.pyc C:\WUTemp\Atype util\foo.py def baz(): print 'foo.baz() here' C:\WUTemp\Atype B\bar.py import sys print sys.path from util import foo foo.baz() C:\WUTemp\Apython B\bar.py ['C:\\WUTemp\\A\\B', 'C:\\Python24\\python24.zip', 'C:\\WUTemp\\A', snip lots more dirs] foo.baz() here Kent -- http://mail.python.org/mailman/listinfo/python-list
Re: Package organization: where to put 'common' modules?
fortepianissimo wrote: Hm this doesn't work. Say I have the following directory structure: A |--- util ||--- foo.py | |--- B |--- bar.py And bar.py has this line from util import foo This would only work with A in sys.path/PYTHONPATH. However... I then run python B/bar.py in directory A. Still got error ImportError: No module named util Yes, Python does this - it puts the directory of bar.py (B in this case) in sys.path, but not the directory in which you're sitting when you run the program from the shell (A in this case). I always get round this by running programs like this: PYTHONPATH=. python B/bar.py For me, this is a minor inconvenience since I know that ultimately the imported packages will be installed and available via site-packages. A check of sys.path in bar.py when running from A revealed that sys.path has A/B, not A. Am I missing something here? Well, I imagine that you want some kind of sys.path modification to take place, and you can certainly do this in bar.py, taking care only to do so in the main program section of the module - you don't usually want sys.path magic going on when just importing from the bar module. However, my advice with regard to package structure and testing would be to put common functionality either in modules below those that need such functionality in the package structure, or in a separate hierarchy in the package structure, to use absolute references to such packages (A.util.foo) rather than relative ones (util.foo or stuff soon to be permitted in Python 2.5), and to put your test programs outside the package and have them use the package code like any other piece of software. Despite the presumed benefits to the upcoming relative import support (less typing, perhaps), it's good to see that the existing import rules will be tightened up with absolute importing being enforced. Here's an example of import misbehaviour in today's Python: ---x |---y ...print x |---z ...import y.a # we want this to be absolute! ---y |---a ...print y Now try and import x.z in Python 2.4! It won't even get to printing y. Paul -- http://mail.python.org/mailman/listinfo/python-list
Re: Package organization: where to put 'common' modules?
Interesting - Python seems to act differently under Windows then. Here I'm using Python 2.4 on Mac OS X. With all of the files, directories and command exactly like yours, it's still not working. One thing I noticed is that you have this 'C:\\WUTemp\\A' when you print sys.path in B/bar.py, but mine doesn't. -- http://mail.python.org/mailman/listinfo/python-list
Re: Package organization: where to put 'common' modules?
Paul Boddie wrote: fortepianissimo wrote: Hm this doesn't work. Say I have the following directory structure: A |--- util ||--- foo.py | |--- B |--- bar.py And bar.py has this line from util import foo This would only work with A in sys.path/PYTHONPATH. However... I then run python B/bar.py in directory A. Still got error ImportError: No module named util Yes, Python does this - it puts the directory of bar.py (B in this case) in sys.path, but not the directory in which you're sitting when you run the program from the shell (A in this case). I always get round this by running programs like this: PYTHONPATH=. python B/bar.py For me, this is a minor inconvenience since I know that ultimately the imported packages will be installed and available via site-packages. A check of sys.path in bar.py when running from A revealed that sys.path has A/B, not A. Am I missing something here? Well, I imagine that you want some kind of sys.path modification to take place, and you can certainly do this in bar.py, taking care only to do so in the main program section of the module - you don't usually want sys.path magic going on when just importing from the bar module. Placing code modifying sys.path in main would be a problem if the utility loading line is at the beginning of the module - error would occur before sys.path can be modified. It is especially true for me since I need to load these common functionality not just for testing - they are necessary to implement the functionality of the module. Besides, adding code to manipulate sys.path in *every* module that needs the common functionality module seems to be quite a hassle... However, my advice with regard to package structure and testing would be to put common functionality either in modules below those that need such functionality in the package structure, or in a separate hierarchy in the package structure, to use absolute references to such packages (A.util.foo) rather than relative ones (util.foo or stuff soon to be permitted in Python 2.5), and to put your test programs outside the package and have them use the package code like any other piece of software. I need to import this common functionality not just for testing code, so it probably doesn't matter if I separate the testing code from the module itself. In addition, the common functionality is common only for the system I'm building - they are probably not that useful for general purposes. So placing them in an absolute path (like site packages) doesn't make a lot of sense. Despite the presumed benefits to the upcoming relative import support (less typing, perhaps), it's good to see that the existing import rules will be tightened up with absolute importing being enforced. Here's an example of import misbehaviour in today's Python: ---x |---y ...print x |---z ...import y.a # we want this to be absolute! ---y |---a ...print y Now try and import x.z in Python 2.4! It won't even get to printing y. Thanks for pointing this out - I can see this would be a problem if not careful. -- http://mail.python.org/mailman/listinfo/python-list
Re: Package organization: where to put 'common' modules?
Paul Boddie wrote: Yes, Python does this - it puts the directory of bar.py (B in this case) in sys.path, but not the directory in which you're sitting when you run the program from the shell (A in this case). This seems to be OS dependent. If I put 'print sys.path' at the start of site.py (which does most of the setup of sys.path), one element of the path is an empty string. This is expanded by main() - removeduppaths() - makepath() - os.path.abspath() to the current working dir. This is Python 2.4.2 on Win2K. Kent -- http://mail.python.org/mailman/listinfo/python-list
Re: Package organization: where to put 'common' modules?
fortepianissimo wrote: Say I have the following package organization in a system I'm developing: A |B |C |D I have a module, say 'foo', that both package D and B require. What is the best practice in terms of creating a 'common' package that hosts 'foo'? I want to be able to - Testing modules in D right in D's directory and testing modules in B in B's directory; - If possible, minimize the modification to PYTHONPATH or sys.path while doing the above. What I do is run always from the base directory (violates your first requirement). I make a util package to hold commonly used code. Then B and D both use from util import foo In Python 2.5 you will be able to say (in D, for example) from ..util import foo http://www.python.org/peps/pep-0328.html Kent -- http://mail.python.org/mailman/listinfo/python-list
Re: Package organization: where to put 'common' modules?
Kent Johnson [EMAIL PROTECTED] writes: What I do is run always from the base directory (violates your first requirement). I make a util package to hold commonly used code. Then B and D both use from util import foo In Python 2.5 you will be able to say (in D, for example) from ..util import foo http://www.python.org/peps/pep-0328.html I do work a bit different here. When two programs start sharing code, then I convert this code into a library, i.e., I make it a module, put it in PYTHONPATH and then import it. Probably more than two projects will use it if two of them already are :-) -- Jorge Godoy [EMAIL PROTECTED] Quidquid latine dictum sit, altum sonatur. - Qualquer coisa dita em latim soa profundo. - Anything said in Latin sounds smart. -- http://mail.python.org/mailman/listinfo/python-list
Package organization: where to put 'common' modules?
Say I have the following package organization in a system I'm developing: A |B |C |D I have a module, say 'foo', that both package D and B require. What is the best practice in terms of creating a 'common' package that hosts 'foo'? I want to be able to - Testing modules in D right in D's directory and testing modules in B in B's directory; - If possible, minimize the modification to PYTHONPATH or sys.path while doing the above. Is it possible? Thanks! -- http://mail.python.org/mailman/listinfo/python-list
Re: Package organization: where to put 'common' modules?
Ok I guess a little more hunch might be needed to elicit suggestions. If I chose to put foo in A, is there a way for code in B to import foo from A? Namely, is there a way to import stuff from a parent directory/package? If it's not possible without changing sys.path, what's the path of least effort - i.e., the way that requires least amount of added code (adding code to prepend sys.path in every module under B certain does the trick but requires a lot code). Thanks! -- http://mail.python.org/mailman/listinfo/python-list
Re: Package organization
In [EMAIL PROTECTED], Thomas Lotze wrote: Assume I have a package called PDF. Should the classes then be called simply File and Objects, as it is clear what they do as they are imported from PDF? Or should they be called PDFFile and PDFObjects, as the names would be too undescriptive otherwise? It depends on the length of the module name in my projects. There's not much difference in importing `PDFFile` from the module or using `PDF.File`. If the name of the module is long, I prefer to explicitly import the names I need from the module. In that case it's nice to have a more descriptive name than `File`. Ciao, Marc 'BlackJack' Rintsch -- http://mail.python.org/mailman/listinfo/python-list
Package organization
Hi, I've two questions concerning organizing and naming things when writing a Python package. - Naming of classes: I'm writing a library that reads PDF files. I have a data structure that represents the large-scale structure of a PDF file (header, trailer, incremental updates etc), and I'll have others, e.g. one that represents the document as a collection of logical objects (page descriptions, images etc). Assume I have a package called PDF. Should the classes then be called simply File and Objects, as it is clear what they do as they are imported from PDF? Or should they be called PDFFile and PDFObjects, as the names would be too undescriptive otherwise? - Organizing subpackages and interfaces: I'm using the zope.interface package in order to define interface classes. In a small package called foo, one might define interfaces IReadableFoo and IWritableFoo in foo.interfaces. However, in a large package foo with subpackages bar and baz, interface definitions might either sit in foo.bar.interfaces and foo.baz.interfaces, or in foo.interfaces.bar and foo.interfaces.baz. Which is preferable? Thanks for any thought on this. -- Thomas -- http://mail.python.org/mailman/listinfo/python-list
Re: Package organization
Le Wed, 22 Jun 2005 20:42:24 +0200, Thomas Lotze a écrit : Hi, I've two questions concerning organizing and naming things when writing a Python package. Assume I have a package called PDF. Should the classes then be called simply File and Objects, as it is clear what they do as they are imported from PDF? Or should they be called PDFFile and PDFObjects, as the names would be too undescriptive otherwise? As you whish :-) if in the package ie in the __init__.py (not the best idea) from PDF import File as PDFFile # always possible if File is defined in a module Objects from PDF.Objects import File # as PDFFile is always possible. Have you installed the reportlab package ? It is full of from ... import .. and it generates PDF. -- http://mail.python.org/mailman/listinfo/python-list
Re: Package organization
F. Petitjean wrote: As you whish :-) Damn freedom of choice *g if in the package ie in the __init__.py (not the best idea) from PDF import File as PDFFile # always possible Technically, this is clear - however I don't like the idea of giving the same thing different names, especially if there's a chance that other people get to look at and try to understand the code... Using short names being unique by virtue of the subpackage hierarchy internally and leaving it to the user (which might even be another subpackage of the library) to import it as something more descriptive in his context is probably the easiest, cleanest and least obtrusive thing, as I think about it. Have you installed the reportlab package ? It is full of from ... import .. and it generates PDF. I do know ReportLab. IIRC, last time I looked, it didn't simply expose an API that models and operates on a PDF document's structures, but was designed to produce PDF files with a certain kind of content. It didn't seem to be of much easy use for anything wildly different from that. -- Thomas -- http://mail.python.org/mailman/listinfo/python-list
Re: Package organization
On Wednesday 22 June 2005 01:42 pm, Thomas Lotze wrote: Assume I have a package called PDF. Should the classes then be called simply File and Objects, as it is clear what they do as they are imported from PDF? Or should they be called PDFFile and PDFObjects, as the names would be too undescriptive otherwise? If you import PDF instead of importing from PDF, you will get that anyway: import PDF then you can refer to: PDF.File and PDF.Object the only downside to this is that you are using a . operator each time, which is a (TINY) performance hit. In tightly optimized loops, some folks recommend avoiding this. But then, so what? PDE_File = PDF.File and the problem goes away. This always seems cleaner to me than: PDF.PDFFile etc, which drives me crazy to read. Useless repetition just gets annoying. -- Terry Hancock ( hancock at anansispaceworks.com ) Anansi Spaceworks http://www.anansispaceworks.com -- http://mail.python.org/mailman/listinfo/python-list
