Hi, When manipulating huge datasets, the bottleneck is certainly not the viz tool (shiny, c#, html5 etc). This won't be data-manipulation part (you won't find faster than data.table with c# manipulation since most part of data.table is C based). The slow part of your tool is probably the data-loading phase.
Open-CPU allows to cache data when the opencpu services is starting. This means each user querying your dataset won't have to wait until the dataset is loaded into open-cpu memory. see http://stackoverflow.com/questions/12468574/are-data-objects-persistent-in-opencpus-r-session Rserver and other stuff are quite old comparing to open-cpu. And I would go with them. Keep separate your view (html5, c#), backend (opencpu/R) and database (opencpu/data.table). And one day you can moove to more flexible architecture such backend (java/c#) and database (distributed such hive) if your dataset cannot feet into RAM memory. But the opencpu way allows you to fast deploy something compatible with your 10K lines of data.table/R code Le 16 avril 2017 à 13:22, Don Brady écrivait : > Thanks you very much for the comments. Those are some good ways to get he > data > back and forth. In fact, I think we can regard those mechanical issues as > taken care of. I can see a number of ways to do that. > > However, once I get the columnar data (what was data.tables in R) into C# and > stored in memory, what representation of it should I use in C# that would be > most like data.table? And how would I manipulate it then and there in C#, > but > in data.table like ways, without involving R at that point. Does anyone know > what .Net way would be most like data.table? Would it be LINQ Objects, for > example, or some in-memory data package? I do not want to do SQL etc. which > is so painful compared to data.table. I know this is not a data.table > question > as such, but people here are the only people who can understand it because > they > know what data.table is. I want to be able to add one column to another etc. > as I can do in data.table. (Then maybe eventually send it back to R as a > data.table which is just mechanics again). > > If I am too far off topic let me know and I apologize. > > Thanks, > > Don > _______________________________________________ > datatable-help mailing list > [email protected] > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help _______________________________________________ datatable-help mailing list [email protected] https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help
