Thanks for the responses. I have it working ok with pandas, using 'skiprows' and only reading out the number of rows I need via 'nrows'
This seems to help as the file gets bigger, but I need to test it over the course of a few hours and see what effect this has on performance. On Friday, February 8, 2019 at 10:21:11 AM UTC, Alex Strickland wrote: > > Hi there, > > I'm pretty new to python (am an embedded C programmer by trade), but I > have a need to be able to plot data being written to a CSV file and show > the plot in pseudo-real time in pyqtgraph. > > I have built the GUI with the plots and am happy with it plotting static > data and rolling round (it reads data from a CSV file once using pandas) > but I'm having trouble getting pandas to work for real time data that is > being written to the CSV file by another process once a second. > > I am wondering if pandas are the right way to go for this? Once a second > my CSV file will be written to, it has a variable data rate of either > 1,16,32 or 128Hz and each sample (on a new line) contains a timestamp and > three vectors (it's telemetry data from a magnetometer). > > So depending on the data rate I need to read each CSV file once a second > and pull out either 1, 16,32 or 128 lines and then put the vectors into a > list and then update the plot moving the data along at the data rate speed. > > I wonder if there are better solutions than pandas for this? As in my > update function I am forced to re-read the CSV file and pull the whole file > out each time and tail the necessary lines into a copy then using the > tolist() method > > But as the file grows this is probably fighting a losing battle to be > efficient and could even slow down the drawing of the plots? > > Any advice appreciated! Thanks, Alex > > > > > -- You received this message because you are subscribed to the Google Groups "pyqtgraph" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/pyqtgraph/6d7d21c7-a132-48b9-a617-d2d34f36238c%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
