On Fri, Feb 19, 2010 at 5:45 AM, Johann Borck <johann.bo...@densedata.com> wrote: > I see. So if I get these requirements right you have 5-10 services with > 1-30 instances of each, with following properties relevant to the task > at hand: > > 1. those pollable by your existing program. > 2. those incompatible with your existing program. > 3. those that do not stream additional data. > 4. those that do stream additional data. > > since sets (1 and 2) and (3 and 4) are distinct, the combinations > (1,3),(1,4),(2,3),(2,4) are possible. > > (1,3) the easy one, obviously you just need an object that polls the > service using your program in intervals specific to the service. > (1,4) Question: will the data you're interested in be collected by your > existing program or by twisted? In the former case, it's basically the > same as (1,3), in the latter you'll have to implement a protocol. > (2,3) for these you'll need to implement a protocol class. > (2,4) here you'll have to implement one or two protocols, depending on > how the service is implemented. > > Is the above about correct? I think it would be a good idea to have an > object OB that keeps references to all objects that gather data from the > services, grouped by the type of service they're responsible for > (defaultdict(list or dict) comes to mind). And then you'll probably > either want factories that take care of handing newly created protocol > instances over to OB or some instances (one or two per service in sets > (2,3,4) ) of a multipurpose factory that can be initialized with the > respective protocol and the information how to pass the created protocol > over to OB, maybe just a simple method that is able to distinguish the > protocols by the interfaces they implement. > > One pitfall might be your polling program in case you're using > t.i.u.getProcessOutput. It (t.i.u...) provides an asynchronous > interface, so the worst that can happen is a stray process that doesn't > return, but you still might want to consider implementing a > ProcessProtocol > (http://twistedmatrix.com/documents/current/core/howto/process.html) > with a reasonable timeout, to be able to kill the spawned process in > case it doesn't terminate. Thinking about it, it's the better solution > anyway, because process protocols are just another type of protocol in > twisted, and can be integrated consistently with the rest of your app. > > Another utility you might want is t.i.task.LoopingCall, for obvious > reasons. Given your requirements something along these lines would be my > approach, although I'd probably reimplement the polling thingy in > twisted if it's not too complex :) > > hope that makes sense, > Johann
Johann, Thank you very much for your detailed response. It is greatly appreciated. I think what you describe makes sense, and I hope I understand what you mean and how I can implement it: I need to create a "suberobject" OB which references all the poller-objects. I need to create one object per detected server instance, which takes care of creating a timed spawning of the external poller-process, and passing it along to OB. For those servers that require streamlisteners, I really only need one factory per server type, with the ablity to match streamed data with polled data via the poller-object. Is this somewhat on the right track? I absolutely hate creating lots of code just to find that it was done in the "wrong" way, structurally speaking, and that it requires a lot of work to rewrite in the "correct way" to allow for further expansion of the application. Your answers are very helpful in letting me avoid spaghetti-code. Cheers, Einar _______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
