Elasticsearch is a no SQL database (http://www.slideshare.net/DmitriBabaev1/elastic-search-moscow-bigdata-cassandra-sept-2013-meetup) and much easier to install and manage than Mongo or CouchDB.
Why 'boggle'? I it's a 'hello world' sketch, no exception guarding, hard coded URLs' and other embarrassing no-nos... ... ok, fine https://github.com/anisbet/hist Edmonton Public Library Andrew Nisbet ILS Administrator T: 780.496.4058 F: 780.496.8317 -----Original Message----- From: Code for Libraries [mailto:CODE4LIB@LISTSERV.ND.EDU] On Behalf Of Cary Gordon Sent: March-19-15 1:15 PM To: CODE4LIB@LISTSERV.ND.EDU Subject: Re: [CODE4LIB] Anyone analyzed SirsiDynix Symphony transaction logs? Has anyone considered using a NoSQL database to store their logs? With enough memory, Redis might be interesting, and it would be fast. The concept of "too experimental to post to Github" boggles the mind. Cary > On Mar 19, 2015, at 9:38 AM, Andrew Nisbet <anis...@epl.ca> wrote: > > Hi Bill, > > I have been doing some work with Symphony logs using Elasticsearch. It is > simple to install and use, though I recommend Elasticsearch: The Definitive > Guide (http://shop.oreilly.com/product/0636920028505.do). The main problem is > the size of the history logs, ours being on the order of 5,000,000 lines per > month. > > Originally I used a simple python script to load each record. The script > broke down each line into the command code, then all the data codes, then > loaded them using curl. This failed initially because Symphony writes > extended characters to title fields. I then ported the script to python 3.3 > which was not difficult, and everything loaded fine -- but took more than a > to finish a month's worth of data. I am now experimenting with Bulk > (http://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html) > to improve performance. > > I would certainly be willing to share what I have written if you would like. > The code is too experimental to post to Github however. > > Edmonton Public Library > Andrew Nisbet > ILS Administrator > > T: 780.496.4058 F: 780.496.8317 > > -----Original Message----- > From: Code for Libraries [mailto:CODE4LIB@LISTSERV.ND.EDU] On Behalf Of > William Denton > Sent: March-18-15 3:55 PM > To: CODE4LIB@LISTSERV.ND.EDU > Subject: [CODE4LIB] Anyone analyzed SirsiDynix Symphony transaction logs? > > I'm going to analyze a whack of transaction logs from our Symphony ILS so > that we can dig into collection usage. Any of you out there done this? > Because the system is so closed and proprietary I understand it's not easy > (perhaps > impossible?) to share code (publicly?), but if you've dug into it I'd be > curious to know, not just about how you parsed the logs but then what you did > with it, whether you loaded bits of data into a database, etc. > > Looking around, I see a few examples of people using the system's API, but > that's it. > > Bill > -- > William Denton ↔ Toronto, Canada ↔ https://www.miskatonic.org/
