Another option might be to use OpenRefine http://openrefine.org - this should easily handle 250,000 rows. I find it good for basic data analysis, and there are extensions which offer some visualisations (e.g. the VIB BITs extension which will plot simple data using d3 https://www.bits.vib.be/index.php/software-overview/openrefine <https://www.bits.vib.be/index.php/software-overview/openrefine>)
I’ve written an introduction to OpenRefine available at http://www.meanboyfriend.com/overdue_ideas/2014/11/working-with-data-using-openrefine/ <http://www.meanboyfriend.com/overdue_ideas/2014/11/working-with-data-using-openrefine/> Owen Owen Stephens Owen Stephens Consulting Web: http://www.ostephens.com Email: o...@ostephens.com Telephone: 0121 288 6936 > On 5 Aug 2015, at 21:07, Harper, Cynthia <char...@vts.edu> wrote: > > Hi all. What are you using to process circ data for ad-hoc queries. I > usually extract csv or tab-delimited files - one row per item record, with > identifying bib record data, then total checkouts over the given time > period(s). I have been importing these into Access then grouping them by bib > record. I think that I've reached the limits of scalability for Access for > this project now, with 250,000 item records. Does anyone do this in R? My > other go-to- software for data processing is RapidMiner free version. Or do > you just use MySQL or other SQL database? I was looking into doing it in R > with RSQLite (just read about this and sqldf > http://www.r-bloggers.com/make-r-speak-sql-with-sqldf/ ) because I'm sure my > IT department will be skeptical of letting me have MySQL on my desktop. > (I've moved into a much more users-don't-do-real-computing kind of > environment). I'm rusty enough in R that if anyone will give me some > start-off data import code, that would be great. > > Cindy Harper > E-services and periodicals librarian > Virginia Theological Seminary > Bishop Payne Library > 3737 Seminary Road > Alexandria VA 22304 > char...@vts.edu<mailto:char...@vts.edu> > 703-461-1794
