Re: [CODE4LIB] Good Database Software for a Digital Project?

[Scancella, John]

I would definitely pick postgres over mysql. It has all of the same features 
and more, plus it is easier to use (in my own opinion).

But before I even pick a database I would consider these:
What are the speed requirements? 
How do you plan on doing searching? 
How much data? 
Does it need to be redundant? 
What about clustering? 
Geographically diverse for faster local retrieval? 
What languages or other technologies do you plan on interfacing with?

and then, based on those answers more questions will arise.
Best of luck!

From: Code for Libraries [CODE4LIB@LISTSERV.ND.EDU] On Behalf Of Ben Cail 
[benjamin_c...@brown.edu]
Sent: Friday, April 15, 2016 2:23 PM
To: CODE4LIB@LISTSERV.ND.EDU
Subject: Re: [CODE4LIB] Good Database Software for a Digital Project?

I would suggest looking at postgresql . It
may not be as widely used as mysql, but it is used a lot, and it's a
high-quality piece of database software. It's also free.

-Ben

On 04/15/2016 02:18 PM, Matt Sherman wrote:
> Hi all,
>
> I am looking to pick the group brain as to what might be the most useful
> database software for a digital project I am collaborating on.  We are
> working on converting an annotated bibliography to a searchable database.
> While I have the data in a few structured formats, we need to figure out
> now what to actually put it in so that it can be queried.  My default line
> of thinking is to try a MySQL since it is free and used ubiquitously
> online, but I wanted to see if there were any other database or software
> systems that we should also consider before investing a lot of time in one
> approach.  Any advice and suggestions would be appreciated.
>
> Matt Sherman

Re: [CODE4LIB] bagit java version

[Scancella, John]

Terry,

Thanks for responding! There are libraries that already deal with making 
Tar(and other archive formats) files much better than bagit-java ever could 
simply because they have more resources to maintain and add new features.
After a quick search here are two that you could use
http://commons.apache.org/proper/commons-compress/tar.html
https://github.com/kamranzafar/jtar

Would it help to create an examples folder and show how you would create a tar 
yourself when using bagit-java?

Thanks

-Original Message-
From: Code for Libraries [mailto:CODE4LIB@LISTSERV.ND.EDU] On Behalf Of Terry 
Brady
Sent: Wednesday, March 23, 2016 12:09 PM
To: CODE4LIB@LISTSERV.ND.EDU
Subject: Re: [CODE4LIB] bagit java version

John,

I am glad to hear that the bagit library will be enhanced.

At Georgetown, I have integrated the the bagit java library into our 
FileAnalyzer application.  We use this application for a number of digitization 
related tasks.

https://github.com/Georgetown-University-Libraries/File-Analyzer/wiki/Bagit-automation

Our real use case is to prepare bags for the APTrust preservation repository.

https://github.com/Georgetown-University-Libraries/File-Analyzer/wiki/Bagit-automation-for-Academic-Preservation-Trust-(APTrust)

APTrust requires bags to be packaged as TAR files.  This code creates some 
APTrust tag files and then packages the bag as a tar file.  It would be useful 
to have a tar option built into the bagit library.

Since we have a number of applications that are already in Java, we will 
continue to work with the Java version of the library.

Terry

On Wed, Mar 23, 2016 at 6:36 AM, Scancella, John <j...@loc.gov> wrote:

> Hi All,
>
> I am currently rewriting the library so that it better conforms to the 
> spec, and to make it easier to extend and use.  I was wondering if 
> people would be so kind as to provide any feedback on:
> * How they currently use the bagit-java library
> * Do you use the command line?
> * If so what is stopping you from using the python version instead?
> * Do you use the library in a java application?
> * If so what functionality do you use? What do you wish it did?
> * Trying out the latest version (you can see examples here 
> https://github.com/LibraryOfCongress/bagit-java/blob/master/README.md#
> examples
> on how to use it)
>
> Thanks
>
> John
>
> Please note, all opinions expressed in this email are my own.
>



--
Terry Brady
Applications Programmer Analyst
Georgetown University Library Information Technology 
https://www.library.georgetown.edu/lit/code
425-298-5498 (Seattle, WA)

[CODE4LIB] bagit java version

[Scancella, John]

Hi All,

I am currently rewriting the library so that it better conforms to the spec, 
and to make it easier to extend and use.  I was wondering if people would be so 
kind as to provide any feedback on:
* How they currently use the bagit-java library
* Do you use the command line? 
* If so what is stopping you from using the python version instead?
* Do you use the library in a java application?
* If so what functionality do you use? What do you wish it did?
* Trying out the latest version (you can see examples here 
https://github.com/LibraryOfCongress/bagit-java/blob/master/README.md#examples 
on how to use it) 

Thanks

John

Please note, all opinions expressed in this email are my own.

Re: [CODE4LIB] Anyone Doing Interesting Things With Digital Collection Systems?

[Scancella, John]

Hi Matt,

I work on the digital repository for the Library of Congress. We have a lot of 
our tools on our public github
https://github.com/LibraryOfCongress

Of particular interest would be the bagit-python, and bagit-java. Note that for 
bagit-java we are in the middle of a rewrite so if you plan on using it for 
more than the near term you should check out the 
https://github.com/LibraryOfCongress/bagit-java/tree/rewrite branch or BETA 
release http://search.maven.org/#artifactdetails|gov.loc|bagit|5.0.0-BETA|jar

John
Please note: all opinions expressed in this email are my own and do not reflect 
those of The Library Of Congress

-Original Message-
From: Code for Libraries [mailto:CODE4LIB@LISTSERV.ND.EDU] On Behalf Of Erin 
Tripp
Sent: Monday, February 29, 2016 7:19 AM
To: CODE4LIB@LISTSERV.ND.EDU
Subject: Re: [CODE4LIB] Anyone Doing Interesting Things With Digital Collection 
Systems?

Hi Matt, 

The Islandora Community (http://islandora.ca/about) is releasing some lovely 
open source digital repositories. Islandora is interoperable and extensible 
through the Tuque API, the Islandora OAI module, and many other tools that are 
included in the software stack. 

Here are a few repositories to explore: 
http://dcmny.org/
http://dlib.bc.edu/
http://repository.lib.cuhk.edu.hk/
http://arcabc.ca/

We have monthly webinars on Islandora if you'd like to join and learn more. 

~ Erin 

Erin Tripp, BJH MLIS 
Business Development Manager
discoverygarden inc. 
e...@discoverygarden.ca 

Re: [CODE4LIB] Accordion menus & mobile web best practices

[Scancella, John]

Kyle,

It is possible. I typically do back end development but it also doesn't seem 
unreasonable to have only one part of the menu open at the same time. Perhaps 
there is another reason why the developer is unable/unwilling to change it. 
Personally I would then ask that developer to propose another solution, or mock 
up another navigation UI. 

After doing a little googling I found 
http://webdesignledger.com/best-practices-accordions-in-web-design which seems 
like a good overview. In the first example it points to http://designmodo.com/ 
which you has the behavior I believe you want in their demo 
http://designmodo.com/demo/jqueryaccordionmenu/

Hope this helps. I find I always design better when I am forced to use the 
product I am designing. Maybe if they have to use it for a couple months they 
will change their mind.

John
Note: opinions expressed in this email are my own and not those of the Library 
of Congress

From: Code for Libraries [CODE4LIB@LISTSERV.ND.EDU] On Behalf Of Kyle Breneman 
[tomeconque...@gmail.com]
Sent: Friday, December 18, 2015 3:01 PM
To: CODE4LIB@LISTSERV.ND.EDU
Subject: [CODE4LIB] Accordion menus & mobile web best practices

Our library website is currently being redesigned to be responsive.  The
work is being done by an outside design firm and the project is being
managed by University Relations, our school's PR department.

The mobile version of our responsive site has several accordion menus
(similar to attached).  I've asked for these accordion menus to be
self-closing; in other words, there is never more than one expansion of an
accordion open at one time - if a user clicks to open another part of the
accordion, the first part simultaneously slides shut.

I've been told that self-closing accordions are contrary to best practices:

"Unfortunately, no, as this isn’t best practice. Accordions should require
a click each to open and close; in other words, nothing on your page should
move without a user action. This is true throughout our sites. See the
universal Quick Links in mobile."

Is it true that self-closing accordion menus run counter to best practices
in mobile web design?  The sort of behavior that I'm asking for seems, to
me, intuitive and expected.

Thanks for your input!

Kyle Breneman
Integrated Digital Services Librarian
University of Baltimore

Re: [CODE4LIB] New Release: Diva.js 4.0 (with IIIF support)

[Scancella, John]

Andrew,

I am getting this error when trying out the default on the website(see attached)

-Original Message-
From: Code for Libraries [mailto:CODE4LIB@LISTSERV.ND.EDU] On Behalf Of Andrew 
Hankinson
Sent: Wednesday, September 09, 2015 4:54 AM
To: CODE4LIB@LISTSERV.ND.EDU
Subject: [CODE4LIB] New Release: Diva.js 4.0 (with IIIF support)

We’re pleased to announce a new version of our open-source document image 
viewer, Diva.js. Diva is an ideal for archival book digitization initiatives 
where viewing high-resolution images is a crucial part of the user experience. 
Using Diva, libraries, archives, and museums can present high-resolution 
document page images in a user-friendly “instant-on” interface that has been 
optimized for speed and flexibility.

In version 4.0 we’re introducing support for the International Image 
Interoperability Framework (IIIF). Through IIIF, Diva becomes part of a larger 
movement to enhance archival image collections through promoting sharing of 
these resources.

With 4.0 we’re also introducing the “Book Layout” view, presenting document 
images as openings, or facing pages. This will provide our users with a 
valuable way of visualizing document openings, providing more tools for viewing 
and understanding the structure of a digitized document.

Several demos are available at http://ddmal.github.io/diva.js/try/ 


Other improvements in 4.0 include:
• Improved integration with existing web applications
• New plugins: Autoscroll (animated page scrolling), Page Alias (pages 
may have multiple identifiers), IIIF Metadata (displays document metadata from 
IIIF manifest), IIIF Highlight (displays annotations from a IIIF manifest)
• Improved build system with Gulp
• Support for switching documents without reloading the viewer
• Numerous bug fixes and optimizations

For more information, demos, and documentation visit 
http://ddmal.github.io/diva.js/.

Diva.js is developed by the Distributed Digital Music Archives and Libraries 
laboratory, part of the Music Technology Area of the Schulich School of Music 
at McGill University and is funded by the Social Sciences and Humanities 
Research Council of Canada.

Re: [CODE4LIB] Looking for Ideas on Line Breaks in OCR Text

[Scancella, John]

Matt,

A word document does funny things to the text since it is actually html (try 
opening a .doc in a plain text editor and you will see it is html). I would try 
and get the plain ASCII text instead, and then install Cygwin which contains 
Sed and a bunch of other usful Unix/Linux commands.
see http://stackoverflow.com/a/127567/2896744 for more info.

From: Code for Libraries [CODE4LIB@LISTSERV.ND.EDU] On Behalf Of Matt Sherman 
[matt.r.sher...@gmail.com]
Sent: Tuesday, August 04, 2015 9:09 AM
To: CODE4LIB@LISTSERV.ND.EDU
Subject: Re: [CODE4LIB] Looking for Ideas on Line Breaks in OCR Text

I am on Windows machines, so I don't have quite the easy access to
that useful command.  Someone had earlier put the OCR in a doc file so
I've been playing with that more than with the raw PDF OCR.

On Tue, Aug 4, 2015 at 8:19 AM, Scancella, John j...@loc.gov wrote:
 Matt,

 There are probably a dozen ways to do this, but it would be really helpful to 
 know what operating system you are on? For example, if you are using Linux, 
 you can run it through sed using
   cat OCR_FILE | sed 's/\n//'  STRIPPED_OCR_FILE
 see http://stackoverflow.com/a/800644/2896744 for more info
 
 From: Code for Libraries [CODE4LIB@LISTSERV.ND.EDU] On Behalf Of Matt Sherman 
 [matt.r.sher...@gmail.com]
 Sent: Monday, August 03, 2015 10:29 PM
 To: CODE4LIB@LISTSERV.ND.EDU
 Subject: [CODE4LIB] Looking for Ideas on Line Breaks in OCR Text

 Hi Code4Lib folks,

 I was wondering if anyone had some experience cleaning up OCR text.
 Particularly I am trying to figure out how I can deal with the random
 line breaks that come from OCR.  I am trying to parse out a
 bibliography with regex.  I think I've figured out which queries I
 need to run to break it up so I can make it into a tab delimited text
 file but I noticed that the text does the classic thing of OCR
 inserting line breaks where they physically are on the page.  This
 will obviously be a bit of an issue since it would break the
 annotation into a bunch of lines rather than leaving it one block so I
 can manipulate it into a database.  So I am wondering if anyone who
 has worked with OCR text before has a suggested way to clean up those
 line breaks without doing 300 + pages by hand?  Any thoughts would be
 welcome.

 Matt Sherman

Re: [CODE4LIB] Looking for Ideas on Line Breaks in OCR Text

[Scancella, John]

Matt,

There are probably a dozen ways to do this, but it would be really helpful to 
know what operating system you are on? For example, if you are using Linux, you 
can run it through sed using 
  cat OCR_FILE | sed 's/\n//'  STRIPPED_OCR_FILE
see http://stackoverflow.com/a/800644/2896744 for more info

From: Code for Libraries [CODE4LIB@LISTSERV.ND.EDU] On Behalf Of Matt Sherman 
[matt.r.sher...@gmail.com]
Sent: Monday, August 03, 2015 10:29 PM
To: CODE4LIB@LISTSERV.ND.EDU
Subject: [CODE4LIB] Looking for Ideas on Line Breaks in OCR Text

Hi Code4Lib folks,

I was wondering if anyone had some experience cleaning up OCR text.
Particularly I am trying to figure out how I can deal with the random
line breaks that come from OCR.  I am trying to parse out a
bibliography with regex.  I think I've figured out which queries I
need to run to break it up so I can make it into a tab delimited text
file but I noticed that the text does the classic thing of OCR
inserting line breaks where they physically are on the page.  This
will obviously be a bit of an issue since it would break the
annotation into a bunch of lines rather than leaving it one block so I
can manipulate it into a database.  So I am wondering if anyone who
has worked with OCR text before has a suggested way to clean up those
line breaks without doing 300 + pages by hand?  Any thoughts would be
welcome.

Matt Sherman

Re: [CODE4LIB] Where to look for a JavaScript Code Coach?

[Scancella, John]

Hi Eliza,

Although I am not a javascript programmer expert I have used it enough to know 
that 6 hours will not be enough. if you break it down (roughly) 4 weeks a 
month, that is 1.5 hours a week, which is not enough time to help a group. In 
my own opinion the type of expert you are seeking is one who is highly in 
demand and could make much more money doing something else. So unless they 
specifically want to help you train these people it is not worth the time and 
effort they will need to put into in for the money getting paid.

I would instead recommend finding several people willing to volunteer to help 
out when needed. You probably could find these through a local meetup 
(http://javascript.meetup.com/).

John

From: Code for Libraries [CODE4LIB@LISTSERV.ND.EDU] On Behalf Of Eliza Carrie 
Bettinger [betti...@uwm.edu]
Sent: Friday, May 08, 2015 11:14 AM
To: CODE4LIB@LISTSERV.ND.EDU
Subject: [CODE4LIB] Where to look for a JavaScript Code Coach?

Hello All,


I received a grant from my university to organize a 7-month program for 
academic staff to build their Javascript skills. The idea is all the staff who 
participate will get together 1-2x per month to share their progress on 
individual projects they're developing, and get feedback and troubleshooting 
help from both a peer group AND a paid expert I'm calling a coach. (Full 
description below.)


The problem I'm having is finding anyone who is interested in taking the 
coaching the job. I have not made a public job ad, but I have emailed about 10 
folks from campus and community, and asked them to notify their networks and 
affiliated lis-servs. For example: the leaders of Milwaukee MeetUp groups for 
tech women, for Javascript programmers, and open data; the web development 
professor at our iSchool; our campus app development lab; the leader of a local 
nonprofit that teaches coding to kids; etc. I've gotten positive responses form 
many of these folks, but I haven't received a nibble of interest from a single 
person in the job itself. I'm continuing to identify key people whose networks 
I can tap, but I'm discouraged by the apparent lack of interest I've 
encountered so far, and scratching my head over what I'm doing wrong.


So I'm turning to you for advice -- specifically, two questions:

1) Do you know someone who might know someone in the Milwaukee area who might 
be interested in a gig like this? If so, please feel free to circulate this 
posting

2) Can you spot anything in the ad itself that I could improve? Anything that's 
a turn-off? Should I include the pay rate in the ad?

(We have a total of about $1100; I figured I would negotiate the hourly rate 
and total number of hours with the person we found. Although I listed 6 
hours/month, that's only the ideal; we could bring down the total number of 
hours, and pay a higher rate.)

Any other suggestions or ideas? I'd love to get your feedback.


Thank you!!

Eliza


Javascript Workshop Leader / Coach

6 hours per month, July - December, 2015


This summer, the Digital Humanities Lab at UWM will begin a professional 
development series for academic staff from across campus to improve their 
skills in coding for front-end Web development -- specifically, Javascript. 
Workshop participants will be professionals from diverse academic backgrounds 
who each have a real-life work-related project under development. They will 
have completed some basic training in Javascript (such as the JavaScript course 
in Codecademy), but are novice-level developers.  In twice-monthly meetings 
with peers and an expert coach, up to 12 participants will meet for 
collaborative workshop sessions in which they present progress on their 
projects, get help on overcoming obstacles, receive short lessons on important 
topics, and gradually “level up” to become more proficient programming problem 
solvers.  Our goal is to support staff who have already learned some basics of 
coding, but need help to start becoming fluent enough to apply those basics in !
 order to design and build their desired products.


We seek an expert front-end programmer with Javascript expertise who will act 
as coach, mentor, and troubleshooter for this group of learners. Overall, the 
goal of the workshop sessions will be to build a community of learners to 
support one another and provide motivation, camaraderie, and practical 
assistance in overcoming roadblocks. The paid expert will act a coach for the 
group, providing guidance in matters that are opaque to novice learners, and 
sharing strategies from his/her experience. Sessions may include short lessons 
prepared and presented by the coach, but we envision workshop sessions in which 
the focus of the majority of the meeting time is on individual participants' 
projects and questions. Participants could also take turns researching and 
presenting on relevant topics, with guidance from the coach.  The exact format 
of the sessions will 

Re: [CODE4LIB] Amazon Glacier - tracking deposits

[Scancella, John]

Have you looked at google's cloud storage nearline? it is about $0.01 per 
gigabyte per month with about 3 second access time
http://googlecloudplatform.blogspot.com/2015/03/introducing-Google-Cloud-Storage-Nearline-near-online-data-at-an-offline-price.html


-Original Message-
From: Code for Libraries [mailto:CODE4LIB@LISTSERV.ND.EDU] On Behalf Of Cary 
Gordon
Sent: Wednesday, April 08, 2015 7:49 PM
To: CODE4LIB@LISTSERV.ND.EDU
Subject: Re: [CODE4LIB] Amazon Glacier - tracking deposits

We have been playing with Glacier, but so far neither us nor our clients have 
been convinced of its cost-effectiveness. A while back, we were discussing a 
project with 15 PB of archival assets, and that would certainly have made 
Glacier cost-effective, saving about $30k/mo. over S3, although requests could 
cut into that.

The Glacier location is in the format /Account ID/vaults/Vault 
Name/archives/Archive ID, so you might want to consider using the whole 
string.

Thanks,

Cary


 On Apr 8, 2015, at 3:32 PM, Sara Amato sam...@willamette.edu wrote:
 
 Has anyone leapt on board with Glacier?   We are considering using it for 
 long term storage of high res archival scans.  We have derivative copies for 
 dissemination, so don't intend touching these often, if ever.   The question 
 I have is how to best track the  Archive ID that glacier attaches to 
 deposits, as it looks like that is the only way to retrieve information if 
 needed (though you can attach a brief description also that appears on the 
 inventory along with the id.)   We're considering putting the ID in Archivist 
 Toolkit, where the location of the dissemination copies is noted, but am 
 wondering if there are other tools out there specific for this scenario that 
 people are using. 

Re: [CODE4LIB] Question about choice of frameworks for JavaScript, Perl and python

[Scancella, John]

Ray,

Thanks this helps a lot to frame the question.

Do you know any more details about what each technology stack(perl, python, 
coldfusion, etc) does in your system? It sounds like you might benefit from 
stepping back from what you have, and these new proposed technologies, and 
instead developing an architecture that makes sense for you doing. Also 
thinking about good design patterns, the answer should present itself, instead 
of you seeking out a new technology stack just to up our game. 
Some good books to take a look 
* http://www.amazon.com/Head-First-Design-Patterns-Freeman/dp/0596007124 
* 
http://www.amazon.com/Clean-Code-Handbook-Software-Craftsmanship/dp/0132350882/ref=pd_sim_b_4?ie=UTF8refRID=1R62G9ZDKDRFK3RY1MRG
* 
http://www.amazon.com/Patterns-Enterprise-Application-Architecture-Martin/dp/0321127420/ref=sr_1_1?s=booksie=UTF8qid=1428341924sr=1-1keywords=Patterns+of+Enterprise+Application+Architecture
 

For instance, if you are merely doing reporting, perhaps you could create a 
backend application that does the heavy lifting(like a java process that 
updates the reports every 5 minutes). Then create a lightweight UI 
application to display those reports(a django app that simply displays the sql 
report that the java app created). Having an overarching design will then allow 
you to pick a stack of technologies that fit that design. 

Having worked on several projects now (most of them inherited) I cannot 
overemphasize the importance of good design. It has meant the difference 
between me dreading work and new work being a pleasure. Play to your strengths, 
use what you know or develop a prototype to test something new. But please 
PLEASE, DO NOT PUT THE PROTOTYPE INTO PRODUCTION! Once you are done with it, 
rewrite it, and then when you are done with the rewrite, rewrite it again. Just 
my $0.02

John

-Original Message-
From: Code for Libraries [mailto:CODE4LIB@LISTSERV.ND.EDU] On Behalf Of 
Schwartz, Raymond
Sent: Monday, April 06, 2015 1:11 PM
To: CODE4LIB@LISTSERV.ND.EDU
Subject: Re: [CODE4LIB] Question about choice of frameworks for JavaScript, 
Perl and python

Thanks John,

One of our coders recommended that we should up our game, so to speak--get a 
framework.

What we do now is use javascript for web apps communicating with our ILS 
(Voyager).  We use perl, python, and php for data extraction and parsing 
between multiple systems--usually marc records, but also patron and 
transactional records.  Presently we use coldfusion for a great deal of 
reporting, put we would like to move to d3.js.  We are talking about web 
applications--not desktop.

/Ray

-Original Message-
From: Code for Libraries [mailto:CODE4LIB@LISTSERV.ND.EDU] On Behalf Of 
Scancella, John
Sent: Monday, April 06, 2015 12:17 PM
To: CODE4LIB@LISTSERV.ND.EDU
Subject: Re: [CODE4LIB] Question about choice of frameworks for JavaScript, 
Perl and python

Ray,

We all have to start somewhere, so do not get discouraged we are here to help!

Like I mentioned before, if we know what problem you are trying to solve we can 
more easily point you in the right direction. Let's start with why choose 
javascript, perl, and or python? What made you choose those? Is this a web 
application or a desktop application? Will it talking to other applications? 
What operating systems will it need to run on?

Once you answer those perhaps we can ask more targeted questions and be better 
able to help you.
I would also recommend the great website stackoverflow.com. They hold a vast 
amount of knowledge when it comes to all things development related.

John

-Original Message-
From: Code for Libraries [mailto:CODE4LIB@LISTSERV.ND.EDU] On Behalf Of 
Schwartz, Raymond
Sent: Monday, April 06, 2015 12:08 PM
To: CODE4LIB@LISTSERV.ND.EDU
Subject: Re: [CODE4LIB] Question about choice of frameworks for JavaScript, 
Perl and python

Well I guess that shows how little we know about frameworks.  :(

-Original Message-
From: Code for Libraries [mailto:CODE4LIB@LISTSERV.ND.EDU] On Behalf Of 
Scancella, John
Sent: Monday, April 06, 2015 12:06 PM
To: CODE4LIB@LISTSERV.ND.EDU
Subject: Re: [CODE4LIB] Question about choice of frameworks for JavaScript, 
Perl and python

Ray,

That is a very broad question. We could probably better answer which ones to 
look at if you gave us some more information, like what specifically are you 
trying to do with these frameworks/technologies?

John
Developer at The Library Of Congress

-Original Message-
From: Code for Libraries [mailto:CODE4LIB@LISTSERV.ND.EDU] On Behalf Of 
Schwartz, Raymond
Sent: Monday, April 06, 2015 11:35 AM
To: CODE4LIB@LISTSERV.ND.EDU
Subject: [CODE4LIB] Question about choice of frameworks for JavaScript, Perl 
and python

We are considering getting frameworks for each of these languages.  Since we 
are new to this technology, I would like to know which frameworks do you 
use/recommend and why?  We did get a recommendation for zend/php.

Thanks for any

Re: [CODE4LIB] Question about choice of frameworks for JavaScript, Perl and python

[Scancella, John]

Ray,

That is a very broad question. We could probably better answer which ones to 
look at if you gave us some more information, like what specifically are you 
trying to do with these frameworks/technologies?

John
Developer at The Library Of Congress

-Original Message-
From: Code for Libraries [mailto:CODE4LIB@LISTSERV.ND.EDU] On Behalf Of 
Schwartz, Raymond
Sent: Monday, April 06, 2015 11:35 AM
To: CODE4LIB@LISTSERV.ND.EDU
Subject: [CODE4LIB] Question about choice of frameworks for JavaScript, Perl 
and python

We are considering getting frameworks for each of these languages.  Since we 
are new to this technology, I would like to know which frameworks do you 
use/recommend and why?  We did get a recommendation for zend/php.

Thanks for any assistance. /Ray

Ray Schwartz
Systems Specialist Librarian  
schwart...@wpunj.edublocked::mailto:schwart...@wpunj.edu
David and Lorraine Cheng Library   Tel: +1 973 720-3192
William Paterson University  Fax: +1 973 720-2585
300 Pompton Road Mobile: +1 201 424-4491
Wayne, NJ 07470-2103 USA  
http://nova.wpunj.edu/schwartzr2/http://euphrates.wpunj.edu/faculty/schwartzr2/

Re: [CODE4LIB] Question about choice of frameworks for JavaScript, Perl and python

[Scancella, John]

Ray,

We all have to start somewhere, so do not get discouraged we are here to help!

Like I mentioned before, if we know what problem you are trying to solve we can 
more easily point you in the right direction. Let's start with why choose 
javascript, perl, and or python? What made you choose those? Is this a web 
application or a desktop application? Will it talking to other applications? 
What operating systems will it need to run on?

Once you answer those perhaps we can ask more targeted questions and be better 
able to help you.
I would also recommend the great website stackoverflow.com. They hold a vast 
amount of knowledge when it comes to all things development related.

John

-Original Message-
From: Code for Libraries [mailto:CODE4LIB@LISTSERV.ND.EDU] On Behalf Of 
Schwartz, Raymond
Sent: Monday, April 06, 2015 12:08 PM
To: CODE4LIB@LISTSERV.ND.EDU
Subject: Re: [CODE4LIB] Question about choice of frameworks for JavaScript, 
Perl and python

Well I guess that shows how little we know about frameworks.  :(

-Original Message-
From: Code for Libraries [mailto:CODE4LIB@LISTSERV.ND.EDU] On Behalf Of 
Scancella, John
Sent: Monday, April 06, 2015 12:06 PM
To: CODE4LIB@LISTSERV.ND.EDU
Subject: Re: [CODE4LIB] Question about choice of frameworks for JavaScript, 
Perl and python

Ray,

That is a very broad question. We could probably better answer which ones to 
look at if you gave us some more information, like what specifically are you 
trying to do with these frameworks/technologies?

John
Developer at The Library Of Congress

-Original Message-
From: Code for Libraries [mailto:CODE4LIB@LISTSERV.ND.EDU] On Behalf Of 
Schwartz, Raymond
Sent: Monday, April 06, 2015 11:35 AM
To: CODE4LIB@LISTSERV.ND.EDU
Subject: [CODE4LIB] Question about choice of frameworks for JavaScript, Perl 
and python

We are considering getting frameworks for each of these languages.  Since we 
are new to this technology, I would like to know which frameworks do you 
use/recommend and why?  We did get a recommendation for zend/php.

Thanks for any assistance. /Ray

Ray Schwartz
Systems Specialist Librarian  
schwart...@wpunj.edublocked::mailto:schwart...@wpunj.edu
David and Lorraine Cheng Library   Tel: +1 973 720-3192
William Paterson University  Fax: +1 973 720-2585
300 Pompton Road Mobile: +1 201 424-4491
Wayne, NJ 07470-2103 USA  
http://nova.wpunj.edu/schwartzr2/http://euphrates.wpunj.edu/faculty/schwartzr2/

Re: [CODE4LIB] JAVA deployment and msi

[Scancella, John]

Bohyun,

Let me preface this with the fact that the only desktop admin I have done 
enterprise wide was during an internship in college. But, it looks like Specops 
just sits on top of Microsoft software products and extends it functionality. 
So, I would imagine if all you need to do is install the latest version of java 
is to download the jre.exe, and use Specops to run it on all your computers 
using the /s command line switch. Not having used Specops I don't know the 
specifics but I would also imagine you have someone there that handles 
interacting with it and should be able to create a new install using the 
jre.exe and /s.

Now if on the other hand you want all the extra functionality that is promised 
by oracle's MSI, you will have to do that yourself by writing some kind of 
script to handle the logic. If it is windows vista or newer you should have 
powershell installed. Powershell would provide lots of functionality in regards 
to handling the logic of what to remove, where, etc.

Hope this helps, good luck!

John
Software developer at The Library Of Congress

-Original Message-
From: Code for Libraries [mailto:CODE4LIB@LISTSERV.ND.EDU] On Behalf Of Kim, 
Bohyun
Sent: Thursday, March 26, 2015 2:38 PM
To: CODE4LIB@LISTSERV.ND.EDU
Subject: Re: [CODE4LIB] JAVA deployment and msi

Hi John - 

This is not my area, but the link below is the page my staff directed me to. We 
are looking to do the silent install with the auto update prompt disabled for 
all staff machines, where users do not have the admin privilege. We use 
Specops.  

And it seems like now that is not possible without purchasing the new MSI 
Enterprise JRE installer. My staff tells me this is a new thing with JAVA 8.40. 
There are 3rd party services but they are not free. 

https://www.java.com/en/download/help/msi_install.xml

MSI Enterprise JRE Installer

With the release of Java SE 8u20, Oracle introduced an MSI Enterprise JRE 
Installer. This is a new MSI compatible installer that enables system 
administrators to install the JRE across the enterprise without end user 
interaction. Integrated with the MSI Installer is the Java Uninstall Tool, 
which provides the option to remove older versions of Java from the system. Now 
the common features such as rollback of unsuccessful installs, repair of broken 
installations and installing over existing broken installations are all 
accessible with the MSI Installer in place.

The MSI Enterprise JRE Installer is only available as part of Oracle Java SE 
Advanced products1 and is available to customers via My Oracle Support (MOS).

Bohyun




From: Code for Libraries [CODE4LIB@LISTSERV.ND.EDU] on behalf of Scancella, 
John [j...@loc.gov]
Sent: Wednesday, March 25, 2015 1:07 PM
To: CODE4LIB@LISTSERV.ND.EDU
Subject: Re: [CODE4LIB] JAVA deployment and msi

Hi Bohyun,

How specifically are you deploying the msi package? I would imagine any 
organization large enough to have this problem to have Microsoft System Center 
Configuration Manager or some other kind of central software management.

Msi files are nothing more than a payload with a special script that is run 
when you double click on it (or deploy it via some management software). So you 
could roll your own by creating a simple powershell script that installs it for 
you, or even a super simple batch file.

Looking at http://www.oracle.com/technetwork/java/javase/silent-136552.html you 
simply need to add the /s for a silent install.
Perhaps you could provide more details as to the problem you are running into? 
More details is always better than less.

John

-Original Message-
From: Code for Libraries [mailto:CODE4LIB@LISTSERV.ND.EDU] On Behalf Of Kim, 
Bohyun
Sent: Wednesday, March 25, 2015 11:52 AM
To: CODE4LIB@LISTSERV.ND.EDU
Subject: [CODE4LIB] JAVA deployment and msi

Hi all -

Oracle no longer offers .msi packages for JAVA which we have been using to 
deploy JAVA to library PCs on the network through Spec Ops. Anyone has a 
workaround? We have internally discussed download the msi package from Source 
Forge or pay for the 3rd party msi packaging service.

Any feedback would be appreciated. Thanks!

Bohyun
--
Bohyun Kim, MA, MSLIS
Associate Director for Library Applications and Knowledge Systems University of 
Maryland, Baltimore Health Sciences and Human Services Library

Re: [CODE4LIB] JAVA deployment and msi

[Scancella, John]

Hi Bohyun,

How specifically are you deploying the msi package? I would imagine any 
organization large enough to have this problem to have Microsoft System Center 
Configuration Manager or some other kind of central software management.

Msi files are nothing more than a payload with a special script that is run 
when you double click on it (or deploy it via some management software). So you 
could roll your own by creating a simple powershell script that installs it for 
you, or even a super simple batch file.

Looking at http://www.oracle.com/technetwork/java/javase/silent-136552.html you 
simply need to add the /s for a silent install.
Perhaps you could provide more details as to the problem you are running into? 
More details is always better than less.

John

-Original Message-
From: Code for Libraries [mailto:CODE4LIB@LISTSERV.ND.EDU] On Behalf Of Kim, 
Bohyun
Sent: Wednesday, March 25, 2015 11:52 AM
To: CODE4LIB@LISTSERV.ND.EDU
Subject: [CODE4LIB] JAVA deployment and msi

Hi all -

Oracle no longer offers .msi packages for JAVA which we have been using to 
deploy JAVA to library PCs on the network through Spec Ops. Anyone has a 
workaround? We have internally discussed download the msi package from Source 
Forge or pay for the 3rd party msi packaging service.

Any feedback would be appreciated. Thanks!

Bohyun
--
Bohyun Kim, MA, MSLIS
Associate Director for Library Applications and Knowledge Systems University of 
Maryland, Baltimore Health Sciences and Human Services Library

Re: [CODE4LIB] Data Lifecycle Tracking Documentation Tools

[Scancella, John]

At my previous job we often had to ingest and support several versions of 
ingest files from multiple outside data providers. One approach we took was to 
use the supportable pattern. Basically we would create a singleton component 
per file/version, with two methods required.
1) Boolean supports(file) - singleton did whatever necessary to see if it could 
support that file
2) void process(file) - singleton did its magic transforming. We ended up 
transforming them into objects that we then stored in our database, your 
requirements might be different.
What was really nice about this is was that it made it super easy to test (unit 
test), as well as have spring framework automatically create a list of these 
singletons during startup and simply loop over each singleton per file. When 
performance became an issue it was also easy to put this in a thread since the 
singletons didn't keep state.

I know for xml it can conform to a specific schema. Do you have any formal 
schema for the files you are ingesting? If so, is there any way to enforce they 
follow their schema?

As for needing a human to quality review the ingest, if after looping through 
all singletons and none applying you know it needs human review. The code could 
put that file in a list and present it via some web interface (restful or GUI 
based).

As for the wiki, we found that a good front page that linked to the most used 
pages was the best. From what I recall mediawiki was ok at search but not the 
best. We had the understanding that if ANYONE found something wrong or 
duplicate, it was up to them to fix it (including new members of the team). But 
even given the worst case of it being disorganized, I would still take it over 
just a select few people knowing it. Communication is key, so if you are 
finding it becoming a mess perhaps that is a better indication that there is 
some communication gap. To combat this we made it mandatory to be in the dev 
chat room on our IM client. People would often post links to the wiki in the 
chat room, which allowed anyone to mention if it was a duplicate.

Anyways, these are just my ramblings as a developer. Take it with a grain of 
salt, your mileage may vary.

John 

-Original Message-
From: Code for Libraries [mailto:CODE4LIB@LISTSERV.ND.EDU] On Behalf Of 
davesgonechina
Sent: Tuesday, March 10, 2015 10:35 PM
To: CODE4LIB@LISTSERV.ND.EDU
Subject: Re: [CODE4LIB] Data Lifecycle Tracking  Documentation Tools

Hi John,

Good question - we're taking in XLS, CSV, JSON, XML, and on a bad day PDF of 
varying file sizes, each requiring different transformation and audit 
strategies, on both regular and irregular schedules. New batches often feature 
schema changes requiring modification to ingest procedures, which we're trying 
to automate as much as possible but obviously require a human chaperone.

Mediawiki is our default choice at the moment, but then I would still be 
looking for a good workflow management model for the structure of the wiki, 
especially since in my experience wikis are often a graveyard for the best 
intentions.

Dave




On Tue, Mar 10, 2015 at 8:10 PM, Scancella, John j...@loc.gov wrote:

 Dave,

 How are you getting the metadata streams? Are they actual stream 
 objects, or files, or database dumps, etc?

 As for the tools, I have used a number of the ones you listed below. I 
 personally prefer JIRA (and it is free for non-profit). If you are ok 
 if editing in wiki syntax I would recommend mediaWiki (it is what 
 powers Wikipedia). You could also take a look at continuous deployment 
 technologies like Virtual Machines (virtualbox), linux containers 
 (docker), and rapid deployment tools (ansible, salt). Of course if you 
 are doing lots of code changes you will want to test all of this continually 
 (Jenkins).

 John Scancella
 Library of Congress, OSI

 -Original Message-
 From: Code for Libraries [mailto:CODE4LIB@LISTSERV.ND.EDU] On Behalf 
 Of davesgonechina
 Sent: Tuesday, March 10, 2015 6:05 AM
 To: CODE4LIB@LISTSERV.ND.EDU
 Subject: [CODE4LIB] Data Lifecycle Tracking  Documentation Tools

 Hi all,

 One of my projects involves harvesting, cleaning and transforming 
 steady streams of metadata from numerous publishers. It's an infinite 
 loop but every cycle can be a little bit or significantly different. 
 Many issue tracking tools are designed for a linear progression that 
 ends in deployment, not a circular workflow, and I've not hit upon a 
 tool or use strategy that really fits.

 The best illustration I've found so far of the type of workflow I'm 
 talking about is the DCC Curation Lifecycle Model  
 http://www.dcc.ac.uk/sites/default/files/documents/publications/DCCLif
 ecycle.pdf
 
 .

 Here are some things I've tried or thought about trying:

- Git comments
- Github Issues
- MySQL comments
- Bash script logs
- JIRA
- Trac
- Trello
- Wiki
- Unfuddle
- Redmine
- Zendesk
- Request Tracker

Re: [CODE4LIB] Data Lifecycle Tracking Documentation Tools

[Scancella, John]

Dave,

How are you getting the metadata streams? Are they actual stream objects, or 
files, or database dumps, etc?

As for the tools, I have used a number of the ones you listed below. I 
personally prefer JIRA (and it is free for non-profit). If you are ok if 
editing in wiki syntax I would recommend mediaWiki (it is what powers 
Wikipedia). You could also take a look at continuous deployment technologies 
like Virtual Machines (virtualbox), linux containers (docker), and rapid 
deployment tools (ansible, salt). Of course if you are doing lots of code 
changes you will want to test all of this continually (Jenkins).

John Scancella
Library of Congress, OSI  

-Original Message-
From: Code for Libraries [mailto:CODE4LIB@LISTSERV.ND.EDU] On Behalf Of 
davesgonechina
Sent: Tuesday, March 10, 2015 6:05 AM
To: CODE4LIB@LISTSERV.ND.EDU
Subject: [CODE4LIB] Data Lifecycle Tracking  Documentation Tools

Hi all,

One of my projects involves harvesting, cleaning and transforming steady 
streams of metadata from numerous publishers. It's an infinite loop but every 
cycle can be a little bit or significantly different. Many issue tracking tools 
are designed for a linear progression that ends in deployment, not a circular 
workflow, and I've not hit upon a tool or use strategy that really fits.

The best illustration I've found so far of the type of workflow I'm talking 
about is the DCC Curation Lifecycle Model 
http://www.dcc.ac.uk/sites/default/files/documents/publications/DCCLifecycle.pdf
.

Here are some things I've tried or thought about trying:

   - Git comments
   - Github Issues
   - MySQL comments
   - Bash script logs
   - JIRA
   - Trac
   - Trello
   - Wiki
   - Unfuddle
   - Redmine
   - Zendesk
   - Request Tracker
   - Basecamp
   - Asana

Thoughts?

Dave

Re: [CODE4LIB] Checksums for objects and not embedded metadata

[Scancella, John]

The library of congress has several tools for making and working with bagit 
bags.

Java command line tool and library
https://github.com/LibraryOfCongress/bagit-java

a python command line tool and library
https://github.com/LibraryOfCongress/bagit-python

or a standalone java desktop application (GUI based)
https://github.com/LibraryOfCongress/bagger 

-Original Message-
From: Code for Libraries [mailto:CODE4LIB@LISTSERV.ND.EDU] On Behalf Of Joe 
Hourcle
Sent: Saturday, January 24, 2015 10:07 PM
To: CODE4LIB@LISTSERV.ND.EDU
Subject: Re: [CODE4LIB] Checksums for objects and not embedded metadata

On Jan 23, 2015, at 5:35 PM, Kyle Banerjee wrote:

 Howdy all,
 
 I've been toying with the idea of embedding DOI's in all our digital 
 assets and possibly inserting/updating other metadata as well. 
 However, doing this would alter checksums created using normal methods.
 
 Is there a practical/easy way to checksum only the objects themselves 
 without the metadata? If the metadata in a tiff or other kind of file 
 is modified, it does nothing to the actual object. Since providing 
 more complete metadata within objects makes them more 
 usable/identifiable and might simplify migrations down the road, it 
 seems like this wouldn't be a bad way to go.


The only file format that I'm aware of that has a provision for this is FITS 
(Flexible Image Transport System), which was a concept of a 'CHECKSUM' and a 
'DATASUM'.  (the 'DATASUM' is the checksum for only the payload portion, the 
'CHECKSUM' includes the metadata)[1].  It's possible that there are others, but 
I suspect that most consumer file formats won't have specific provisions for 
this.

The problems with 'metadata' in a lot of file formats is that they're just 
arbitrary segments -- you'd have to have a program that knew which segments 
were considered 'headers' vs. not.  It might be easier to have it be able to 
compute a separate checksum for each segment, so that should the modifications 
change their order, they'd still be considered valid.

Of course, I personally don't like changing files if I can help it.
If it were me, I'd keep the metadata outside the file;  if you're using BagIt, 
you could easily add additional metadata outside of the data directory.[2]

If you're just doing this internally, and don't need the DOI to be attached to 
the file when it's served, you could also look into file systems that support 
arbitrary metadata.  Older Macs used to use this, where there was a 'data fork' 
and a 'resource fork', but you had to have a service that knew to only send the 
data fork.
Other OSes support forks, but some also have 'extended file attributes', which 
allows you to attach a few key/value pairs to the file.  (exact limits are 
dependent upon the OS).

-Joe


[1] http://fits.gsfc.nasa.gov/registry/checksum.html
[2] https://tools.ietf.org/html/draft-kunze-bagit ; 
http://en.wikipedia.org/wiki/BagIt

Re: [CODE4LIB] Automatically updating documentation with screenshots

[Scancella, John]

Owen,

A number of years ago I saw this video 
https://www.youtube.com/watch?v=UMnZiTL0tUc about doing pixel diffs (or pdiff 
as they call it) for UI elements on a web application. It sounds like it might 
be part of the answer you are looking for. Something like, run pdiff on all 
pages that are documented, if different run script to capture images again, and 
have user/developer check changes.

I don't know of an automated way to generate user documentation with 
screenshots, but it sounds like it might be useful.

-Original Message-
From: Code for Libraries [mailto:CODE4LIB@LISTSERV.ND.EDU] On Behalf Of Owen 
Stephens
Sent: Monday, January 26, 2015 6:38 AM
To: CODE4LIB@LISTSERV.ND.EDU
Subject: [CODE4LIB] Automatically updating documentation with screenshots

I work on a web application and when we release a new version there are often 
updates to make to existing user documentation - especially screenshots where 
unrelated changes (e.g. the addition of a new top level menu item) can make 
whole sets of screenshots desirable across all the documentation.

I'm looking at whether we could automate the generation of screenshots somehow 
which has taken me into documentation tools such as Sphinx 
[http://sphinx-doc.org] and Dexy [http://dexy.it]. However, ideally I want 
something simple enough for the application support staff to be able to use.

Anyone done/tried anything like this?

Cheers

Owen

Owen Stephens
Owen Stephens Consulting
Web: http://www.ostephens.com
Email: o...@ostephens.com
Telephone: 0121 288 6936

Re: [CODE4LIB] Library community web standards (was: LibGuides v2 - Templates and Nav)

[Scancella, John]

Hey guys I am new to this list so I beg your pardon if I am responding to the 
wrong people.

I have been trying to follow the conversation below and agree with Michael, I 
am still not clear what the end goal is.
 
Having been developer for a number of years now(and looking at this from that 
perspective), I worry that any suggestions/best practices now will be wrong in 
the near future (change is constant). I know it stinks, but I don't see any 
other way but wade through lots of technical documents to understand WHY 
they(document writer) suggest something. What is applicable now to someone is 
not the case for someone else/ or in the future.

Case in point, which is better to use for hosting a web application Tomcat or 
Jetty? The answer is it really depends. Until we have computers that can 
write/manage code for you, I don't see this changing.

John Scancella

-Original Message-
From: Code for Libraries [mailto:CODE4LIB@LISTSERV.ND.EDU] On Behalf Of Brad 
Coffield
Sent: Tuesday, September 30, 2014 10:23 AM
To: CODE4LIB@LISTSERV.ND.EDU
Subject: Re: [CODE4LIB] Library community web standards (was: LibGuides v2 - 
Templates and Nav)

I agree that it would be a bad idea to endeavor to create our own special 
standards that deviate from accepted web best practices and standards. My own 
thought was more towards a guide for librarians, curated by librarians, that 
provides a summary of best practices. On the one hand, something to help those 
without a deep tech background to quickly get up to speed with best practices 
instead of needing to conduct a lot of research and reading.
But beyond that, it would also be a resource that went deeper for those who 
wanted to explore the literature.

So, bullet points and short lists of information accompanied by links to 
additional resources etc. (So, right now, it sounds like a libguide lol)

Though I do think there would potentially be additional information that did 
apply mostly/only to libraries and our particular sites etc. Off the top of my 
head: a thorough treatment and recommendations regarding libguides v2 and 
accessibility, customizing common library-used products (like Serial Solutions 
360 link, Worldcat Local and all their competitors) so that they are most 
usable and accessible.

At it's core, though, what I'm picturing is something where librarians get 
together and cut through the noise, pull out best web practices, and display 
them in a quickly digested format. Everything else would be the proverbial 
gravy.

On Tue, Sep 30, 2014 at 10:01 AM, Michael Schofield mschofi...@nova.edu
wrote:

 I am interested but I am a little hazy about what kind of standards 
 you all are suggesting. I would warn against creating standards that 
 conflict with any actual web standards, because I--and, I think, many 
 others--would honestly recommend that the #libweb should aspire to and 
 adhere more firmly to larger web standards and best practices that 
 conflict with something that's more, ah, librarylike. Although that 
 might not be what you folks have in mind at all : ).

 Michael S.

 -Original Message-
 From: Code for Libraries [mailto:CODE4LIB@LISTSERV.ND.EDU] On Behalf 
 Of Brad Coffield
 Sent: Tuesday, September 30, 2014 9:30 AM
 To: CODE4LIB@LISTSERV.ND.EDU
 Subject: Re: [CODE4LIB] Library community web standards (was: 
 LibGuides v2
 - Templates and Nav)

 Josh, thanks for separating this topic out and starting this new 
 thread. I don't know of any such library standards that exist on the 
 web. I agree that this sounds like a great idea. As for this group or not... 
 why not!
 It's 2014 and they don't exist yet and they would be incredibly useful 
 for many libraries, if not all. Now all we need is a cool 'working 
 group' title for ourselves and we're halfway done! Right???

 But seriously, I'd love to help.

 Brad




 --
 Brad Coffield, MLIS
 Assistant Information and Web Services Librarian Saint Francis 
 University
 814-472-3315
 bcoffi...@francis.edu




--
Brad Coffield, MLIS
Assistant Information and Web Services Librarian Saint Francis University
814-472-3315
bcoffi...@francis.edu