To be a bit more clear, I am not sure how to get the data off of the communications medium and into a db, that is where I am at loss. I assume that I have to create a table in the db that is laid out like each row, and then have a tool that can pipe the data from the source into the db.
The format of the connection is http://URL/ipaddy:PortNum PortNum may be 9010, 9030 or 9050, the last one spits out the data in binary format, I will deal with that later, right now I just want to get my data in the CSV format stuffed into a db. On Mon, Jun 23, 2014 at 2:44 PM, Chuck Hast <[email protected]> wrote: > Rich, > Yes, I want to store each row in a db or a flat file, but I fear the > FF may get messy, then once I have it somewhere for posterity, > I will start to process it. Generally we look at it in the following > form, > Rejects by mould number > Reject type by mould number > > Rejects last hour > Rejects last X hours > Rejects last shift > Rejects last 24 hours > Other time periods > Also we look at a breakdown of reject types on those time > periods. > > At this point I am just trying to get the data stream into a db or a > file, once there I can start playing with how to graph it, I used to > do RF coverage studies and we graphed all the different parameters > on a RF system, signal level, BER, BERT, S/N, etc both historically > and real time. It has been a while, but I have a lot of machines here > that are trying to supply all manner of status info, and that format is > more or less standardized among them all. > > At this point I am not worried about the graphing, I am trying to figure > how to get live data into a file/db so I can graph it from there. > > > > On Mon, Jun 23, 2014 at 2:30 PM, Rich Shepard <[email protected]> > wrote: > >> On Mon, 23 Jun 2014, Chuck Hast wrote: >> >> > Hmmm I guess I did not get the data sample in it. >> >> Chuck, >> >> You did include the data sample, but not what questions you had about >> it. >> >> > I know how to get csv files into spead sheets and i have sort of a idea >> of >> > getting them into a db, but how do I handle a stream of data like that >> > below. All of them are CSV formated, so once I can get one into >> something >> > I can graph and twiddle, the rest should be easy. >> >> So, you want to store each row in a database table. Then you want to >> produce some sort of plot (histogram? scatter plot? box-and-whisker plot?) >> from selected rows or continuously? What sort of twiddling do you >> envision? >> >> I would approach a solution by using a Python script (the psycopg2 >> module >> is appropriate for the interface between python and postgres, and wxPython >> is the GUI I use) to store the rows in a postgres database table as each >> row >> was sent from a machine, then use pandas to analyze those data and produce >> plots using matplotlib. There are probably a gazillion alternatives, >> including using R for the analyses, but I suspect that would be overkill >> for >> your needs. >> >> Rich >> >> _______________________________________________ >> PLUG mailing list >> [email protected] >> http://lists.pdxlinux.org/mailman/listinfo/plug >> > > > > -- > > Chuck Hast -- KP4DJT -- > Glass, five thousand years of history and getting better. > The only container material that the USDA gives blanket approval on. > > > -- Chuck Hast -- KP4DJT -- Glass, five thousand years of history and getting better. The only container material that the USDA gives blanket approval on. _______________________________________________ PLUG mailing list [email protected] http://lists.pdxlinux.org/mailman/listinfo/plug
