These are really valuable info. I will try. Thanks a lot.
On Fri, Feb 6, 2009 at 7:44 PM, Kent Johnson <ken...@tds.net> wrote: > On Fri, Feb 6, 2009 at 4:11 AM, Daniel <daniel.chaow...@gmail.com> wrote: > > Hi Tutors, > > > > I want to use python to finish some routine data processing tasks > > automatically (on Windows). > > > > The main task could be split to sub small tasks. Each can be done by > > executing some small tools like "awk" or by some other python scripts. > > One example of such task is conducting a data processing job, including: > > > > use tool-A to produce some patterns. > > feed tool-B with these patterns to mine more related data > > repeat these tasks circularly until meeting some conditions. > > > > The real task includes more tools which run in parallel or sequential > > manner. > > > > I know how to do this with modules like subprocess, but the final python > > program looks somewhat messy and hard to adapt for changes. > > > > Do you have any best practices on this? > > My first thought was, use shell pipelines and bash. Then I remembered, > David Beazley shows how to use generators to implement a processing > pipeline in Python: > http://www.dabeaz.com/generators-uk/ > > It's a fascinating read, it might take a couple of times to get it but > it might fit your needs quite well. You would write a generator that > wraps a subprocess call and use that to access external tools; other > pieces and the control logic would be in Python. > > Kent >
_______________________________________________ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
