> everything was global, ....how you guys handle a modern structured
> language

Don't worry this is one of the hardest bad habits to break.
You are not alone. The easiest way is to just pass the data
from function to function in the function parameters. Its not
at all unusual for functions to have lots of parameters, "global"
programmers tend to panic when they have more than a couple,
but its not at all bad to have 5 or 6 - more than that gets
unweildy I admit and is usually time to start thinking about
classes and objects.

> I have ended up with my application in several separate directories.

Separate modules is good. Separate directories for anything
other than big programs (say 20 or more files?) is more hassle
than its worth. The files are better kept in a single directory
IMHO. The exception being modules designed for reuse...
It just makes life simpler!

> My problem is that pretty much all the modules need to fix where
they
> are when they exit and pick up from that point later on,

There are two "classic" approaches to this kind of problem:

1) batch oriented - each step of the process produces its own
output file or data structure and this gets picked up by the
next stage. Tis usually involved processing data in chunks
- writing the first dump after every 10th set of input say.
This is a very efficient way of processing large chuinks of
data and avoids any problems of synchronisation since the
output chunks form the self contained input to the next step.
And the input stage can run ahead of the processing or the
processing aghead of the input. This is classic mainframe
strategy, ideal for big volumes. BUT it introduces delays
in the end to end process time, its not instant.

2) Real time serial processing, typically constructs a
processing chain in a single process. Has a separate thread
reading the input data and kicks off a separate processing
thread (or process) for each bit of data received. Each
thread then processes the data to completion and writes
the output. A third process or thread then assembles the
outputs into a single report.

This produces results quickly but can overload the computer
if data starts to arrive so fast that the threads start to
back up on each other. Also error handling is harder since
with the batch job data errors can be fixed at the
intermediate files but with this an error anywhere means
that whole data processing chain will be broken with no way
to fix it other than resubmitting the initial data.

> With my code now running to a few hundred lines
> (Don't laugh this is BIG for me :-D )

Its big for me in Python, I've only writtenone program with
more than a thousand lines of Python wheras I've written
many C/C++ programs in ecess of 10,000 lines and worked
on several of more than a million lines. But few if any
Python programs get to those sizes.

HTH,

Alan G
Author of the Learn to Program web tutor
http://www.freenetpages.co.uk/hp/alan.gauld

_______________________________________________
Tutor maillist  -  [EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/tutor

Reply via email to