I'm having difficulty wrapping my arms around sockets and threading, not so much from a 10,000 foot/ network perspective, but from a low level perspective.
In our production environment we have three machines that generate data and forward it over tcp to a computer that stores the data, parses it and sends it on to another computer over tcp for analytics. In our test environment we have simulated this by building three vm's. VM1 has a python script that sends raw data over tcp to VM2 which parses the data and sends it over tcp to VM3 upon which we are developing our analytics apps. Note that VM1 script reads from a .log file which is the actual output from three real machines, differentiated by a unique deviceID, and each line or 'event' in the .log file is read and sent over tcp with a 0.5 second delay between each read. VM2 has a python script that receives through a socket the raw data over tcp, parses it and sends it on through a socket over tcp to VM3. My lack of confusion arises for a couple reasons. 1. The data from the three different machines each gets it's own thread in production, so that would have to happen on 'VM2' as the 'VM1' are actually just microcontrollers out in production. From a socket and threading perspective, which would be considered the client and which the server, VM1 (the sender) or VM2 (the receiver)? 2. The process has worked mediocre at best thus far. When I developed the two python scripts (tunnelSim to send over socket and parser to rx and tx over socket) I started by just reading and writing to files so I could concentrate on the parsing bit. Once that was done and worked very well I added in sockets for data flow and commented out the read from and to files bits, and everything seemed to work fine, VM1 sent a number of 'lines', VM2 received the same number of 'lines', parsed them and, seemed to send them on. Some days analytics (VM3) got them all, some days it did not. Not sure where to look, and any thoughts on troubleshooting this would be helpful, but the main point of the entire email is question 1, threading. regards, Richard _______________________________________________ Tutor maillist - [email protected] To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
