Hey Dilawar, thanks for the hint with lualatex, it's actually significantly faster:
With filtering (dataset changed a bit from the one I used before) - pdflatex: 80s - lualatex: 20s Without filtering (i.e. I split the CSV file before): - pdflatex: 2s - lualatex: 2s (generally a bit slower than pdflatex here) I don't think it's the size of the dataset, but the filtering, especially because I apply four filters. Another idea might be use use the `shell` functionality of pgfplots to have an external program filter the rows. Or just live with it and split up the csv before latex. Best, Florian Am 30.10.18 um 11:44 schrieb Dilawar Singh: > Hi Florian > > I have used pgfplots with lualatex/gnuplot with much larger dataset. pdflatex > runs out of memory pretty soon on large dataset. It is PROBABLY slower than > lualatex. You can checkout the generated images and scripts to generate them > here. > > > https://github.com/dilawar/SinghAndBhalla_CaMKII_SubunitExchange_2018/tree/master/PaperFigures > > Moreover, I use `gnuplot` with its 'every' keyword which is very helpful if > you > need not plot every datapoint. In the example below, I plot every 10th point. > If > you need to plot every point, then would be of no help. > > \begin{tikzpicture}[scale=1] > \begin{axis}[ xlabel=xlabel,ylabel=ylabel ] > \addplot [color=blue] gnuplot [ raw gnuplot ] { > use datafile separator ","; > plot "data.csv" using 2:3 with lines every 10 > }; > \end{axis} > \end{tikzpicture} > > The learning curve was bit steep for me but it paid well. > cheers, > Dliawar > > On Mon, Oct 29, 2018 at 02:26:12PM +0100, Florian Lindner wrote: >> Hey, >> >> unfortunately I got to walk away from using pgfplots like that again. It's >> just to slow. >> >> I have a document with nothing but 2 groupplots with 2 graphs and 4 or 5 >> addplots each. >> >> All are plot from the same CSV file with 3600 lines, which is filtered by 3 >> onlyif commands to 50 data points per addplot. >> >> On my machine pdflatex on that documents take 95s. (CPU is i5 @ 1.9 GHz, >> SSD). >> >> So I will split up data in chunks of CSV files before plotting. >> >> Best, >> Florian >> >> Am 25.10.18 um 10:13 schrieb Florian Lindner: >>> Hi, >>> >>> ok, I see the issue. >>> >>> One other thing. I get tons of messages like >>> >>> NOTE: coordinate (0.6,0.1) has been dropped because it is unbounded (in x). >>> (see also unbounded coords=jump). >>> >>> I added filter discard warning=false, so the log output is not cluttered so >>> much: >>> >>> \pgfplotsset{ >>> only if/.style 2 args={ >>> filter discard warning=false, >>> x filter/.append code={ >>> \edef\tempa{\thisrow{#1}} >>> \edef\tempb{#2} >>> \ifx\tempa\tempb >>> \else >>> \def\pgfmathresult{NaN} >>> \fi >>> }, >>> }, >>> } >>> >>> (so this message was less of a question, more of documentation) >>> >>> Best Regards, >>> >>> Florian >>> >>> >>> >>> Am 24.10.18 um 20:29 schrieb Stefan Pinnow: >>>> Hi Florian, >>>> >>>> I think there was already a discussion about an inclusion of such a >>>> feature. I >>>> think the conclusion was that it is not a trivial task, because the >>>> elements >>>> to filter could be of numerical or non-numerical type, and the data could >>>> come >>>> from a table with and without headers. And many, many more things need to >>>> be >>>> considered. >>>> >>>> Thus I don't have much hope that Christian -- the developer -- will find a >>>> "global" solution to this problem in the near future. But to keep track of >>>> that, consider adding a feature request to the PGFPlots Tracker >>>> <https://sourceforge.net/p/pgfplots/feature-requests/> >>>> if there isn't already one in for that feature. >>>> >>>> >>>> Best regards, >>>> Stefan >>>> >>>> >>>>> -----Original Message----- >>>>> From: Florian Lindner [mailto:mailingli...@xgm.de] >>>>> Sent: Wednesday, October 24, 2018 5:34 PM >>>>> To: Stefan Pinnow >>>>> Cc: pgfplots-features@lists.sourceforge.net >>>>> Subject: Re: [Pgfplots-features] Filter data before plotting >>>>> >>>>> Hi Stefan, >>>>> >>>>> thanks for that link, works like a charm. I renamed it to "only if" for >>>>> brevity. >>>>> >>>>> Still, I think that this is a worthy feature for inclusion into pgfplot, >>>>> don't? >>>>> >>>>> Best Thanks, >>>>> Florian >>>>> >>>>> Am 23.10.18 um 17:36 schrieb Stefan Pinnow: >>>>>> Hello Florian, >>>>>> >>>>>> this is already possible with a minor modification of the code. >>>>>> Have a look at >>>>>> <https://tex.stackexchange.com/a/342529/95441> >>>>>> I am pretty sure that this question was also already there for `discard >>>>> if' >>>>>> but after a quick search I couldn't find the answer. >>>>>> >>>>>> But I think you already know what needs to be done to combine/"append" >>>>>> filters ;) >>>>>> >>>>>> >>>>>> Best regards, >>>>>> Stefan >>>>>> >>>>>> >>>>>>> -----Original Message----- >>>>>>> From: Florian Lindner [mailto:mailingli...@xgm.de] >>>>>>> Sent: Tuesday, October 23, 2018 11:10 AM >>>>>>> To: pgfplots-features@lists.sourceforge.net >>>>>>> Subject: [Pgfplots-features] Filter data before plotting >>>>>>> >>>>>>> Hello, >>>>>>> >>>>>>> it seems to be common demand to filter data that is to be plotted. >>>>>>> >>>>>>> As an example, you have a table: >>>>>>> >>>>>>> | h | Model | Testset | Result | >>>>>>> |-----+-------+---------+--------| >>>>>>> | 1 | A | X | 0.1 | >>>>>>> | 0.5 | A | X | 0.2 | >>>>>>> | 0.2 | A | X | 0.3 | >>>>>>> | 1 | B | X | 0.11 | >>>>>>> | 0.5 | B | X | 0.12 | >>>>>>> | 0.2 | B | X | 0.13 | >>>>>>> | 1 | B | Y | 0.1 | >>>>>>> | 0.5 | B | Y | 0.12 | >>>>>>> | 0.2 | B | Y | 0.1 | >>>>>>> >>>>>>> Now you want to plot all results (x = h, y = Result) for Model=B, >>>>>>> Testset=Y. >>>>>>> >>>>>>> There are some code snippets, like: >>>>>>> >>>>>>> https://tex.stackexchange.com/questions/66640/how-can-i-filter-select- >>>>>>> data-float-text-from-a-table-and-plot-it >>>>>>> https://tex.stackexchange.com/questions/98003/filter-rows-from-a-table >>>>>>> >>>>>>> but they only allow to filter for one criteria. >>>>>>> >>>>>>> As far as I see, pgfplots already have a number of >>>>> filtering/restrictions >>>>>>> possiblilities, but they all act on coordinates only. >>>>>>> >>>>>>> Or is there something like \addplot[restrict={Model==B and Testset=Y}, >>>>>>> x=h, y=Result] {my_data.csv}; ? >>>>>>> >>>>>>> What would be the official way of doing that? >>>>>>> >>>>>>> Best Thanks, >>>>>>> >>>>>>> Florian Lindner >>>>>>> >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Pgfplots-features mailing list >>>>>>> Pgfplots-features@lists.sourceforge.net >>>>>>> https://lists.sourceforge.net/lists/listinfo/pgfplots-features >>>>>> >>> >>> >>> _______________________________________________ >>> Pgfplots-features mailing list >>> Pgfplots-features@lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/pgfplots-features >>> >> >> >> _______________________________________________ >> Pgfplots-features mailing list >> Pgfplots-features@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/pgfplots-features _______________________________________________ Pgfplots-features mailing list Pgfplots-features@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/pgfplots-features
