Re: [CODE4LIB] clarification about file visualization
The following application may be useful for your task. I created this application at the National Archives. The team that I was on used this application for a number of file system analysis tasks. https://github.com/usnationalarchives/File-Analyzer This application will allow you to select a recipe to use when crawling a file system. The recipe that you select will determine the type of report that will be generated. Once the report is generated, you can filter and sort for information of interest. Essentially, the application converts the tree structure of the file system into a table structure. The table structure seemed to simplify decisions about a complex file hierarchy. Terry On Thu, Aug 30, 2012 at 6:02 PM, Shearer, Timothy J tshea...@email.unc.eduwrote: Hi Folks, My query may have been poorly expressed... What we have is a webserver with 64,665 files (html, css, js, jpg, you get the idea) and lots of directories with subdirectories. The goal is to be able to conveniently take all that in in a way that makes it pretty simple to see/navigate (say for a public services staff member tasked with doing a survey of the old content) so that we can get a handle on what's there (prior to say, moving from a php+html template approach to a CMS). It's about exploring the website from under the hood. In my limited imagination it might look like: the document tree represented in xml as viewed through a web browser. Expanding/contracting nodes (and being able to recursively explode the view at at any node). Maybe choose to hide things like image, css, and js files. Annotation would be lovely (say at a subdirectory be able to say: this one's old and needs to go, this one we keep as is, this one needs to be reworked entirely). And in an ideal world state could be preserved...if you'd expanded/contracted chunks as you were exploring, you could come back later and be where you were in your exploration. tree expresses the file system as (strangely enough) a tree, but the output is not interactive and it's huge and unwieldy to deal with. If you find a subdirectory that's full of thousands of files that are irrelevant to the task of getting a handle on the overall content, they're on the screen and you page and page down and eventually lose track of where they are in the directory hierarchy. I'm more interested in how other shops help users understand a huge old webserver's content than focusing on a specific tool such as the one my brain imagines. Thanks for the feedback so far! Tim -- Terry Brady Applications Programmer Analyst Lauinger Information Technology 202-687-7053
Re: [CODE4LIB] clarification about file visualization
I agree. Kibana looks promising for log analysis and visualization... http://rashidkpc.github.com/Kibana/index.html On 8/30/12 9:10 PM, Lars Aronsson wrote: On 2012-08-31 00:02, Shearer, Timothy J wrote: What we have is a webserver with 64,665 files (html, css, js, jpg, you get the idea) and lots of directories with subdirectories. The goal is to be able to conveniently take all that in in a way that makes it pretty simple to see/navigate (say for a public services staff member tasked with doing a survey of the old content) so that we can get a handle on what's there (prior to say, moving from a php+html template approach to a CMS). It's about exploring the website from under the hood. I'd recommend starting with the web server log files. Maybe a handful of those files are 95% of your traffic, and the rest is odd or peculiar long tail information. -- Shaun D. Ellis Digital Library Interface Developer Firestone Library, Princeton University voice: 609.258.1698 | sha...@princeton.edu
[CODE4LIB] Timelines (was: visualize website)
On 2012-08-30, at 1:03 PM, miles stauffer wrote: Is this what you are looking for? http://selection.datavisualization.ch/ The site points to TimelineJS at http://timeline.verite.co/ for timeline visualization. There is also the widget from the SIMILE project at MIT at http://www.simile-widgets.org/timeline/ Are there other suggestions for tools for time line visualizations? Walter
Re: [CODE4LIB] Timelines (was: visualize website)
There's also timemap (SIMILE Timeline + mapping libraries like Google Maps or OpenLayers) if you need to display geography in conjunction to chronology. http://code.google.com/p/timemap/ Ethan On Fri, Aug 31, 2012 at 9:27 AM, Walter Lewis wltrle...@gmail.com wrote: On 2012-08-30, at 1:03 PM, miles stauffer wrote: Is this what you are looking for? http://selection.datavisualization.ch/ The site points to TimelineJS at http://timeline.verite.co/ for timeline visualization. There is also the widget from the SIMILE project at MIT at http://www.simile-widgets.org/timeline/ Are there other suggestions for tools for time line visualizations? Walter
[CODE4LIB] Fwd: Hack4Europe! Dublin 2012 - announcing a competition to use open cultural data from the Europeana portal
For folks in Europe -- there are a series of Europeana hack days. Here's info on the one in Dublin -- they have some funding for developers (150E for those outside the island). -Jodi *Hack4Europe! Dublin* ** *ANNOUNCING A COMPETITION TO USE OPEN CULTURAL DATA* Hack4Europe! Dublin is part of a series of hack days organised by the Europeana Foundation and its partners throughout Europe. Hackathons provide an exciting environment to explore the potential of open cultural data from the Europeana portal to create products for social and economic growth in Europe. Europeana - www.europeana.eu http://www.europeana.eu - enables people to explore over 20 million digital objects from Europe's libraries, museums, archives and galleries. The Department of Arts, Heritage and the Gaeltact is organising the Dublin hackathon in conjunction with the Irish Manuscripts Commission, the Digital Humanities Observatory and Fáilte Ireland. Hack4Europe! Dublin will take place on 24 and 25 September 2012 in the Science Gallery, Pearse Street, Dublin 2. ** *CONTACT AND QUESTIONS *- hack4europedub...@gmail.com mailto:hack4europedub...@gmail.com ** *REGISTRATION - *Register your interest *before* Friday 31 August 2012 at: http://hack4europe2012-dublin.eventbrite.ie ** * * On behalf of the organising committee * * Sharon Barry (D/AHG) Cathy Hayes (IMC) Niall O'Leary (DHO) Ursina O'Riordan (D/AHG)
Re: [CODE4LIB] clarification about file visualization
Something like Quick View Plus? http://www.avantstar.com/metro/home/products/quickviewplusstandardedition john -Original Message- From: Shearer, Timothy J [mailto:tshea...@email.unc.edu] Sent: Thursday, August 30, 2012 6:03 PM To: CODE4LIB@LISTSERV.ND.EDU Subject: [CODE4LIB] clarification about file visualization Hi Folks, My query may have been poorly expressed... What we have is a webserver with 64,665 files (html, css, js, jpg, you get the idea) and lots of directories with subdirectories. The goal is to be able to conveniently take all that in in a way that makes it pretty simple to see/navigate (say for a public services staff member tasked with doing a survey of the old content) so that we can get a handle on what's there (prior to say, moving from a php+html template approach to a CMS). It's about exploring the website from under the hood. In my limited imagination it might look like: the document tree represented in xml as viewed through a web browser. Expanding/contracting nodes (and being able to recursively explode the view at at any node). Maybe choose to hide things like image, css, and js files. Annotation would be lovely (say at a subdirectory be able to say: this one's old and needs to go, this one we keep as is, this one needs to be reworked entirely). And in an ideal world state could be preserved...if you'd expanded/contracted chunks as you were exploring, you could come back later and be where you were in your exploration. tree expresses the file system as (strangely enough) a tree, but the output is not interactive and it's huge and unwieldy to deal with. If you find a subdirectory that's full of thousands of files that are irrelevant to the task of getting a handle on the overall content, they're on the screen and you page and page down and eventually lose track of where they are in the directory hierarchy. I'm more interested in how other shops help users understand a huge old webserver's content than focusing on a specific tool such as the one my brain imagines. Thanks for the feedback so far! Tim
Re: [CODE4LIB] Timelines (was: visualize website)
TimeGlider seems promising as well: http://timeglider.com/ -Scott On Fri, Aug 31, 2012 at 9:42 AM, Ethan Gruber ewg4x...@gmail.com wrote: There's also timemap (SIMILE Timeline + mapping libraries like Google Maps or OpenLayers) if you need to display geography in conjunction to chronology. http://code.google.com/p/timemap/ Ethan On Fri, Aug 31, 2012 at 9:27 AM, Walter Lewis wltrle...@gmail.com wrote: On 2012-08-30, at 1:03 PM, miles stauffer wrote: Is this what you are looking for? http://selection.datavisualization.ch/ The site points to TimelineJS at http://timeline.verite.co/ for timeline visualization. There is also the widget from the SIMILE project at MIT at http://www.simile-widgets.org/timeline/ Are there other suggestions for tools for time line visualizations? Walter
Re: [CODE4LIB] Timelines (was: visualize website)
A vertical timeline: http://builtbybalance.com/github-timeline/ Jason On Fri, Aug 31, 2012 at 9:27 AM, Walter Lewis wltrle...@gmail.com wrote: On 2012-08-30, at 1:03 PM, miles stauffer wrote: Is this what you are looking for? http://selection.datavisualization.ch/ The site points to TimelineJS at http://timeline.verite.co/ for timeline visualization. There is also the widget from the SIMILE project at MIT at http://www.simile-widgets.org/timeline/ Are there other suggestions for tools for time line visualizations? Walter
