Re: [OPEN-ILS-DEV] Leaving Calvin

2020-06-30 Thread Dan Scott
We'll miss you, Remington. Thanks for everything!

On Tue, 30 Jun 2020 at 11:10, Remington Steed  wrote:

> Hi Evergreen Devs,
>
> Today is my last day at Calvin University, and I wanted to say goodbye and
> thanks.  It's been great working with you all, and I've learned a lot from
> you.  Thanks for taking the time to answer my questions and to review my
> code.  Keep up the good work, and God bless.
>
> Remington
>
> --
> Remington Steed
> Electronic Resources Specialist
> Hekman Library, Calvin University
> http://library.calvin.edu/
>


Re: [OPEN-ILS-DEV] IDL is not backward compatible?

2019-02-18 Thread Dan Scott
Hi Ken:

In addition to the headers you're getting back, it might be helpful to
include the exact GET request you're making. If you're referring to
/IDL2js, then that's ~400kb of data that's being shipped--a not
inconsiderable amount to send over the wire.

Back in 2017, Evergreen's web environment adopted the generally accepted
performance approach of setting the cache time as long as possible and
relying on cache-busting GET params to only cause browsers to request new
versions of files when things actually change. When things change (for
example, when a server is upgraded and autogen.sh is run), the GET param
changes from ?abc123 to ?def456 (or some other such hash), breaking the
cache and forcing a fresh retrieval of the file in question.

Prior to commit 1cb0d8c63c44, the default caching time was 18 hours. So
crashes due to IDL mismatches could still occur if the cache-busting GET
param wasn't updated; the window was just shorter.

It sounds like in the the iOS app, if you want to force a refetch of the
cached file, perhaps you could generate a similar cache-busting token and
just append it to your GET URL?

Best,
Dan

Best,
Dan

Dan

On Mon, Feb 18, 2019 at 6:12 PM Ken Cox  wrote:

> I think I know what is going on, but I would like somebody to confirm
> my understanding.
>
> After a server upgrade from 3.0.1 to 3.2.2, I saw a significant spike
> the iOS app crashes.  It looks like a cached IDL specifies fewer
> fields than is returned by the /osrf-gateway-v1 endpoint.
>
> I fixed the app so that does not cause a crash, reasoning that since
> the IDL fields must be newly added fields, and because the app didn't
> require them, I could safely ignore them.  On deeper inspection, it
> looks as though new fields have been added to the middle of IDL
> classes.  For instance, the class 'au' had the following new fields
> added, all in the middle:
> * Privacy Waiver Entries
> * Preferred Prefix
> * Preferred First Name
> ...
>
> Short term, I think a partial fix is to disable caching of the IDL,
> even though the server indicates it can be cached:
>
> HTTP/1.1 200 OK
> Date: Mon, 18 Feb 2019 09:27:03 GMT
> Cache-Control: max-age=31536000
> Expires: Tue, 18 Feb 2020 09:27:03 GMT
>
> Long term, I think a better approach would be to add new fields only
> at the end.  This is a common technique for maintaining binary
> compatibility in compiled code.  If the app is suspended during a
> server upgrade, and if fields are added in the middle, then the IDL is
> out of sync, right?  Or maybe, all sessions are lost during the down
> time and the next request with an auth token will fail, and the IDL
> should be loaded on every login?  (Sorry I have been trying to test
> this hypothesis but I am having trouble upgrading my own EG server).
>
> Thanks,
> Ken
>


Re: [OPEN-ILS-DEV] Python binding for OpenSRF and Evergreen

2019-02-11 Thread Dan Scott
Laurentian is still using the Python bindings to sync users from our LDAP
directory to our library system.

I believe it is also required for any library using Syrup Reserves?

Due to other commitments, I cannot commit to becoming a maintainer of the
Python code at this point in time, unfortunately.

On Mon, Feb 11, 2019 at 3:37 PM Galen Charlton 
wrote:

> Hi,
>
> Similar to the question Jason Stephenson asked about the Java bindings
> [1], I would like to know if anybody is actively using OpenSRF's and
> Evergreen's Python binding.
>
> I do know that the Python binding is a bit less dusty than the Java
> binding, as srfsh.py does still work. I also know that Syrup [2] uses the
> binding and has at least a couple active installations, although to my
> knowledge nobody is actively maintaining that project.
>
> I would also like to know whether anybody has any interest in becoming a
> maintainer for the Python binding. If so, projects include adding support
> for Python 3.x (and deprecating Python 2 support?) and updating the binding
> to support new OpenSRF features such as bundling and chunking.
>
> Regards,
>
> Galen
>
> [1]
> http://libmail.georgialibraries.org/pipermail/open-ils-dev/2019-February/010659.html
> [2] http://git.evergreen-ils.org/?p=Syrup.git;a=tree
> --
> Galen Charlton
> Implementation and Services Manager
> Equinox Open Library Initiative
> phone:  1-877-OPEN-ILS (673-6457)
> email:  g...@equinoxinitiative.org
> web:  https://equinoxInitiative.org
> direct: +1 770-709-5581
> cell:   +1 404-984-4366
>


Re: [OPEN-ILS-DEV] jQuery plus jQuery-ui in OPAC

2017-09-11 Thread Dan Scott
For the past couple of years, thanks to the maturing of the web platform,
standardization of EcmaScript across browsers, the adoption of transpiling,
and a focus on performance (not just network transfer but memory
consumption), there has been a movement away from all-in-one libraries like
jQuery and Dojo that sought to pave over differences between browsers at
execution time (talk about boilerplate!), and towards the use of small
packages that get bundled for delivery at the point of need (see bower ->
npm -> yarn and webpack which is used to split JS and CSS up into the
bundles needed for any given page).

I understand the community felt a need for jQuery in the catalogue. But
given jquery-ui's relative staleness (the last stable release 1.12.1 was a
year ago, and the roadmap was last updated a year ago), I'm worried that we
would be tying ourselves to the equivalent of another Dojo at this point,
when much of the rest of the web development world has already moved on.

On Mon, Sep 11, 2017 at 9:44 AM, Cesar Velez <
cesar.ve...@equinoxinitiative.org> wrote:

> With the coming release of 3.0, the OPAC will now make use of the jQuery
> javascript library.
> This I think will help greatly to reduce boilerplate JS code, and make it
> easier to add more dynamic features and content there.
>
> As such, I was wondering whether it would also be beneficial if we went
> ahead and added the related UI library, jQuery-ui? This UI library has
> several useful widgets (http://jqueryui.com/demos/) and would complement
> the jquery library nicely. Mainly, it would help greatly in speeding up
> future UI work.
>
> I'm curious what the community's general experience is with this UI
> library, and whether there's general consensus for it to be included along
> with the main jQuery.
>
> thanks!
>
> Cesar Velez
> EOLI
>


Re: [OPEN-ILS-DEV] Customizing Evegreen OPAC display questions

2014-08-26 Thread Dan Scott
Hi Jesse:

You'll find the font sizes defined in
Open-ILS/src/templates/opac/parts/css/fonts.tt2

Everything else, it's going to be hard to debug without seeing what
customizations you've made to the stock catalogue HTML and CSS (vs. just
looking at screen shots).

Note however that many of the design changes are intended to support both
accessibility and mobile devices. In some cases we had to trade off some
visual polish for better all-round functionality. That said, your My
Account menu shouldn't be floating off the top of the screen :)

Dan


On Tue, Aug 26, 2014 at 12:07 PM, Jesse McCarty jes...@burlingtonwa.gov
wrote:

  Hi Everyone,



 I am not sure it this is better suited for the Evergreen-Admin list (which
 seems to have very little activity) but figured I would have some luck
 here. As I am still in the process of upgrading Evergreen 2.4.4 to  2.6.2
 (I’m working on a good write up on what all was involved for others that
 may be interested). I’ve had to revert to the stock style.css.tt2 and three
 other configuration files (searchbar.tt2 advanced/search.tt2 and
 misc_utils.tt2) as using these files from our Production 2.4.4 system
 produces a 500 server error when searching the OPAC online and doesn’t
 display very nicely on top of that.

 I have nearly all of my look/feel visuals back in place (the style.css.tt2
 is very close to the original used in our 2.4.4 production system, but with
 the addition of the new configurations for Evergreen 2.6.2 and some lines I
 did not change as they distorted the display) sans a few outstanding
 configurations as noted below.


 For the following 4 issues, I have attached FontSizes.JPG for visual
 reference that shows a side by side image of our current production
 environment (right side) vs. the testing environment being upgraded.

 1) The font size - the in progress upgrade (on the left in the attached
 screen shots) has larger fonts, I haven’t been able to find where to get
 this changed to display in the smaller font style like we currently have in
 our 2.4.4 install of Evergreen.

 2) Quantity alignment - the numbers in () after the author name are
 aligned nicely to the right in the current production system, and when I
 edit the style.css.tt2 that aligns them to the right like in the production
 system, they garble/run together with some of the links to the left.

 3) Place Hold/ Add to my list are fluid in the production system (when the
 browser windows resizes they adjust appropriately), but are static and move
 below the text when the window is too small in the development system, what
 can I edit/change in the configuration to get the fluidity back?

 4) Language material book icon – this doesn’t show in the new system,
 where can I get it back?

 When logging into my Account, the My Account and Logout buttons at the top
 are pushed too far up as to not be visible or readable. Also the text color
 is yellow for the fines value, where we want it red like our production
 environment (MyAccount.JPG). I am not sure where to edit the style.css.tt2
 file to get this to be more like the production system.



 Any help is greatly appreciated, thanks in advance!

  Jesse McCarty

 City of Burlington

 IT Technical Assistant





Re: [OPEN-ILS-DEV] [OPEN-ILS-GENERAL] browser client update 2014-05-09 / UI lists for sprint 1; packaging, etc.

2014-05-09 Thread Dan Scott
On Fri, May 9, 2014 at 9:35 AM, Bill Erickson ber...@esilibrary.com wrote:
 Hi All,

 I don't have any specific feedback requests this time, but feedback is of
 course always welcome / appreciated.

 http://wiki.evergreen-ils.org/doku.php?id=dev:browser_staff:dev_notes

Feedback: this looks awesome, Bill. Your choices for the JavaScript
tools match everything that I've seen for best practices, and seeing
automated unit tests for the staff client coming together fills me
with great hope. You're putting together a nice, solid, user _and_
developer friendly foundation for the future.

Thanks!


Re: [OPEN-ILS-DEV] [OPEN-ILS-GENERAL] browser staff feedback request / integration

2014-04-04 Thread Dan Scott

On Fri, Apr 04, 2014 at 10:25:56AM -0400, McCanna, Terran wrote:

My preference would be to leave the existing client as is (with the exception 
of bug fixes, etc.), but to offer the browser-based modules as options once 
they become available. My reasons for this are:

1) It will allow for indepth use of the new module in a production environment 
to uncover any bugs or performance issues while still having a known entity as 
a backup. No amount of testing is going to uncover all of the odd little 
problems like working in a live environment will.

2) When moving back and forth between the two environments, staff will be more 
aware of the improved user interface and performance, and I believe they will 
be more eager to transition completely to the new environment.

3) The browser-based circulation module may offer some immediate relief to 
those frontline circ staff who are struggling with antiquated machines and slow 
networks.

4) The new modules can begin to be thoroughly tested on mobile devices that 
cannot run the current client.


+1 to what Terran says. Let's forgo the complications of trying to force
new-style development into an aging security / performance /
functionality-impaired container and instead focus on fleshing out the
browser-based modules on a workflow-by-workflow basis as much as
possible, so that some staff (as Terran says) can benefit ASAP and
help us find where we can optimize workflows or where we've missed yet
another piece of existing functionality that most sites don't use but
that is critical to another site's workflow.


Re: [OPEN-ILS-DEV] Developer meeting at the Evergreen Conference

2014-03-20 Thread Dan Scott
+1

Thanks Galen!

On Thu, Mar 20, 2014 at 10:11 PM, Ben Shum bs...@biblio.org wrote:
 +1 from me to the lunch time meeting.  That would be fine for me.

 -- Ben

 On Thu, Mar 20, 2014 at 10:01 PM, Galen Charlton g...@esilibrary.com wrote:
 Hi,

 On Thu, Mar 20, 2014 at 8:33 PM, Galen Charlton g...@esilibrary.com wrote:
 I suggest that we meet in the hotel lobby at 11:45 a.m. and find a
 nearby restaurant to invade. I'll bring my laptop along to relay
 messages from anybody who intends to join via #evergreen.

 After some discussion, unless there's a general outcry, let's run with
 meeting at lunch.  The general idea would be to grab food nearby
 (e.g., Chipotle) and take it back to the Discovery Room at the hotel.
 The room is booked for us from 12-1 (thanks, Kathy!).  Please note
 that there is another group meeting in that room before then.

 Regards,

 Galen
 --
 Galen Charlton
 Manager of Implementation
 Equinox Software, Inc. / The Open Source Experts
 email:  g...@esilibrary.com
 direct: +1 770-709-5581
 cell:   +1 404-984-4366
 skype:  gmcharlt
 web:http://www.esilibrary.com/
 Supporting Koha and Evergreen: http://koha-community.org 
 http://evergreen-ils.org



 --
 Benjamin Shum
 Evergreen Systems Manager
 Bibliomation, Inc.
 24 Wooster Ave.
 Waterbury, CT 06708
 203-577-4070, ext. 113


Re: [OPEN-ILS-DEV] a request regarding development communication in the bug tracker

2014-03-05 Thread Dan Scott
On Wed, Mar 5, 2014 at 8:58 AM, Sharp, Chris csh...@georgialibraries.orgwrote:

 Good Morning All,

 I've had my head down with local projects since early January, so I'm just
 now able to tune in a bit on 2.6 development.  I was just chatting with Ben
 Shum about some 2.6 new features and I came across the discussion happening
 on https://bugs.launchpad.net/evergreen/+bug/1198465.  The discussion
 goes pretty deeply into the rationale for the features being proposed
 and/or developed, and I would like to request that those sorts of
 discussions happen here on this list rather than within the dark depths of
 Launchpad.

 I happen to subscribe to Launchpad bug mail, so I get every communication
 of new bugs, status changes, and comments added, but I don't always have
 the time/bandwidth to tune in to every email that comes through (especially
 during bug retargeting times like now).  I do, however, read every email
 that comes to this and the Open-ILS-General lists.  When I first started
 learning about the inner workings of Free and Open Source Software
 projects, I purchased Producing Open Source Software by Karl Fogel (
 http://producingoss.com/), which I think is a great guide for projects
 like ours written by a veteran of the Subversion project (one of our SFC
 cousins, incidentally).  In his chapter on communication, he has a section
 entitled No Conversations in the Bug Tracker, that I think is worth a
 read: http://producingoss.com/en/bug-tracker-usage.html.

 Given the availability and higher visibility of this list, the difficulty
 of navigating/searching Launchpad, and perhaps most importantly, the fact
 that we have recurring discussions about ditching Launchpad in favor of
 some other bug tracker, I'd like to request that we have discussions like
 the one on bug 1198465 on the public lists instead.  I'm welcome to
 alternative opinions on this, so feel free to respond frankly!


+1

I read through this bug a few days back and had similar thoughts. I have
been guilty of participating in (at least one, probably more) conversation
in the bug tracker in the past where the discussion veered heavily back
into design rather than straight-up implementation. That said, it's tough
to distinguish between design and implementation sometimes as you iterate
over code commits, where the implemented behaviour ends up revealing an
unanticipated design decision or fundamental miscommunication.

In any case, trying to keep the bulk of the design discussion in the lists
also helps documentation writers and QA-oriented people, who might in this
case envision the possible combinations of the suggested six settings that
would be added to control the behaviour of crediting/voiding and throw up
their hands at any realistic way of sanely documenting and ensuring the
reliability of anything beyond the default settings.

Yes, there is some sensitivity in the community of the over-settingization
of Evergreen. For good reason: pain teaches rational actors to avoid
repeating the steps that led to that pain. Anyone who has been bitten by
taking advantage of non-default features or settings that worked great when
they were initially created but then were forgotten about and broken in
subsequent upgrades can attest to this.


Re: [OPEN-ILS-DEV] a request regarding development communication in the bug tracker

2014-03-05 Thread Dan Scott
On Wed, Mar 5, 2014 at 8:58 AM, Sharp, Chris csh...@georgialibraries.orgwrote:

  When I first started learning about the inner workings of Free and Open
 Source Software projects, I purchased Producing Open Source Software by
 Karl Fogel (http://producingoss.com/), which I think is a great guide for
 projects like ours written by a veteran of the Subversion project (one of
 our SFC cousins, incidentally).  In his chapter on communication, he has a
 section entitled No Conversations in the Bug Tracker, that I think is
 worth a read: http://producingoss.com/en/bug-tracker-usage.html.


One side note from an update on Fogel's effort to produce a 2.0 version of
that book (and why it has been delayed beyond the original Kickstarter
delivery date):

I started hearing from a ton of people who had thoughtful, constructive
suggestions for areas to update. One example: the original edition had a
section about not having long conversations in the bug tracker (because the
tracker is a poor tool for that, and mailing lists are better).  Well,
multiple people have independently written in to say that that might not be
so true anymore: bug trackers have evolved, they now integrate better with
other communications mechanisms, and for these and other reasons actual
practice in many projects has changed -- certain kinds of development
conversations do happen in the bug tracker now, and it works.

https://www.kickstarter.com/projects/kfogel/updating-producing-open-source-software-for-2nd-ed/posts/578279


[OPEN-ILS-DEV] Features and defaults and navigation (was: [Bug 1261939] Re: Add per-library TPAC pages with schema.org structured data support)

2014-01-22 Thread Dan Scott
I've forwarded comments from Dan Wells on bug # 1261939 around merging a
new feature; and instead of responding in the bug (which is now closed),
I've put my thoughts below that. There's a mix of discussion around whether
new features should be on by default, as well as design how navigation in
the TPAC should work. I was going to respond in the bug, but thought that
it might make more sense to respond on open-ils-dev as the issues apply
pretty generally and a broader discussion is probably warranted.

-- Forwarded message --
From: Dan Wells d...@calvin.edu
Date: Tue, Jan 21, 2014 at 3:43 PM
Subject: [Bug 1261939] Re: Add per-library TPAC pages with
schema.orgstructured data support
To: d...@coffeecode.net


I have rebased this branch and pushed it into master.  Thanks, Dan!  I
opted not to squash any commits, as I find seeing the process to be of
value historically, and it all seemed understandably linear to me (with
no commits I'd consider trivial).

Praises:
- I think exposing this library data in a public way is a very valuable
addition.
- Making it structured data gives it some added value beyond simple
exposure.

Concerns:
- The library about links added in 2.5 had the same behavior in both the
results list (show more details) and the details page.  This new feature
only affects the details page, and can therefore lead to some confusing
interface behavior.  It should be easy to add the same logic to the results
page, but it remains a TODO.
- This feature adds a large number of (frequently redundant) links to the
page.  In addition, I don't find it intuitive to predict what these links
will do before I click them for the first time.  I think a number of
installations (mine included) will either not need these links, or will
decide they detract more from the focus of the catalog than they add.
 Unless a feature is a slam dunk, I am in favor of making it optional
(despite the drawbacks).

Questions:
- Should we consider making some display changes to help clarify the links?
 For instance, perhaps something as simple as a 'title' attribute which
says something like 'See library hours and contact information'?  Or what
about using small icons (with alt text) to identify either the external
links, the 'info' links, or both?  I think if we are consistent with
applying such conventions, we could get a lot of usability with minimal
disruption to the 'normal' flow of the catalog.

I pushed this branch since I felt that all of this could be addressed
(or not, as needed) in an ongoing way.  Thanks again, Dan!

=

Response:

Thanks Dan:

(Results page consistency is TODONE at 1271600)

I cannot prove whether this feature is going to be a slam dunk.
(Hypothetically, Google / Yandex  / Bing could read all of the structured
data this make available from the agent-promise-object model linking copies
to records to libraries and surface richer results to users directly at
their level, given that they know where a user is physically located and
where the libraries  copies are... that would indeed be a slam dunk, and
this feature makes that actually possible, but I'm not going to place bets
on it.)

As far as I know slam dunk criteria has never been formally applied to
any previous changesets. Perhaps we should formalize how we make such
decisions and apply it consistently for _all_ new features, otherwise
conflict, hurt feelings, and wasted time may result.

Realistically, though, like most features, this is likely to be warmly
welcomed by some libraries and turned off by others. The power of the
default dictates whether libraries are likely to even be aware of the
feature; if the default is that it is off, then most libraries are unlikely
to ever become aware of the feature; even with excellent release notes, the
difference between Oh, sounds neat and Oh, sounds neat; I'm going to go
into Library Settings and turn that on is huge.

In addition, if it's made into YAOUS, then they get to deal with
configuration combinations of two different OUS dealing with one relatively
simple set of behaviour around library name links, and we get to figure out
how to test the various combinations thoroughly and figure out how we can
still supply structured data that provides a useful seller property for
the offers of copies independently of the choice of configurations... all
of which will complicate the template markup even further.

On redundancy in the copy tables: the existing copy tables already feature
plenty of potentially redundant information--the names of the libraries,
shelving locations, and copy availability. Any redesign seeking to reduce
redundancy should look at the broader context rather than just this
feature-specific narrow part of it.

While hover help via a title attribute or the like might help for desktop
browsers, the mobile browser situation is more complicated. I'll admit that
I shudder a bit at the thought of 90's-era external link / more info
icons and such for 

Re: [OPEN-ILS-DEV] Vote Needed: Release Manager 2.6 Approval

2013-12-19 Thread Dan Scott
+1

Thanks Dan!

On Thu, Dec 19, 2013 at 10:43 AM, Dan Wells d...@calvin.edu wrote:
 Hello all,

 Back at the dev meeting on Nov. 12, I made an offer to continue as RM for an
 abbreviated 2.6 release.  In the meeting, the discussion concluded with
 Galen suggesting that “voting the same day as the proposal seems rushed -- I
 propose that we give a week, then assent via email if no other proposal is
 made.”

 A week has turned into a month, but in the spirit of no time like the
 present, I think it is time we make this official in one way or the other.
 Obviously no other proposal has been made, and rather than continue to wait
 passively for assent, I’d like to call for a simple up/down vote on whether
 we accept my offer of continuation.

 Thanks all,
 Dan


 Daniel Wells
 Library Programmer/Analyst
 Hekman Library, Calvin College
 616.526.7133





Re: [OPEN-ILS-DEV] Putting the community's QA money where our dev mouth is

2013-11-05 Thread Dan Scott
On Tue, Nov 5, 2013 at 1:41 PM, Mike Rylander mrylan...@gmail.com wrote:
 CORRECTION: the commit hash is
 3e7f6f1503938e3e5a1cfe3bd90eaa8f021a0d48 ... I failed to fetch before
 attempting to pick. Sorry!

Hey Mike:

It looks like that test assumes that the concerto sample records have
been loaded, which means we can't run it on a production system.

What I've done in my tests, instead, is insert the desired record with
a known-and-entirely-improbable record ID, then tested against that.
As we're using an explicit ID, the sequence doesn't get touched, and
as the test occurs within a transaction, the record and all derived
rows get rolled back at the end.

To make my approach more airtight, I suppose I should test for the
existence of a record with the entirely-improbable ID first, and skip
the test if that record ID for some reason exists...


Re: [OPEN-ILS-DEV] Putting the community's QA money where our dev mouth is

2013-11-05 Thread Dan Scott
On Tue, Nov 5, 2013 at 2:38 PM, Mike Rylander mrylan...@gmail.com wrote:
 Dan,

 I understand the process of building mock environments, but thanks.

You're welcome. I'm sure that many other people reading this thread,
now and in the future, might not, so hopefully my response will be
helpful to them in understanding what we're talking about.

 It's my understanding that concerto is intended to be a valid test
 environment, so I decided to go with that.  Is that not the case?

As I mentioned in IRC, I have experience with a previous project where
a sample database meant to demonstrate the usage of a product had
regression tests depend on its structure over time, so much so that
changing the sample database to use better examples and to show off
new features became out of the question. I would prefer to avoid a
similar fate with this project.

I think the Concerto sample database is useful for demonstrating
Evergreen and for users to report problems. I'm not, however, a big
fan of building dependencies on it for testing purposes.

Building a test that relies on Concerto also means that once you log
into the system and make any changes that affect the database, you can
no longer depend on the results of tests run against that database.
You need a full stop, database rebuild, start of the system to get a
valid set of test results again. And anything that slows down a
developer is less likely to actually happen, right?

 As I mentioned in IRC, changing my most recent test to create a mock
 env would be fine, of course, and might even be a good learning tool
 for someone else interested in working with pgTAP tests in EG.  But,
 if nobody else gets around to it, I'll try to find time to remove the
 dependency on concerto data.  It'll be lower priority than other tests
 or new development, though, since the test exists now ... and that's
 100% more existing than it was doing before. :)

It's hard to argue with that.


[OPEN-ILS-DEV] Putting the community's QA money where our dev mouth is

2013-11-01 Thread Dan Scott

In the QA report for which several members of the community generously
paid $30,000, the Moving Forward section at
http://nox.esilibrary.com/~jason/qareport/qa.html#_moving_forward
states:

We recommend that the development community start including integration
tests with their changes to the backend, and pgTAP tests with their
database changes (there was discussion and general interest in this
during a developers meeting).

The referenced developer meeting is minuted at
http://evergreen-ils.org/meetings/evergreen/2013/evergreen.2013-08-27-14.04.html
with the agreement general interest in easing into qa practices
demonstrated by phasefx, phasefx to hold hands with everyone, especially
through the mailing list

I'm concerned that since then we have seen a number of database changes
without corresponding pgTAP tests (the most recent pgTAP test was
committed Sept 3rd, while there were a ton of changes committed through
October). I did bring this up in the IRC channel with respect to one of
the recent bug fixes that was committed for a fairly fundamental
function, but my gentle prod in that case appears to have been
overlooked.

I recognize there is pressure to get the 2.5 release out, but if we
continue to follow our past approach of not including unit tests where
the path has been blazed for us when we commit changes to the database,
our quality assurance is going to assuredly be similar to the quality we
have produced in the past.

In short, we have heard strongly from the community that we are not
producing software of the quality that they expect; and the community
has followed up those words with a significant investment in the QA
project; and we as a developer team are _not_ following through with the
process improvements we had agreed to adopt.

FWIW, should anyone want to follow some commits to teach themselves how
to add pgTap tests, I did go through the learning process to support
https://bugs.launchpad.net/evergreen/+bug/1242999 - the tests I created
and the instructions for running them are basic, but they're a start.
Thanks to Galen for giving me a pointer in the right direction.


Re: [OPEN-ILS-DEV] LDAP OSRF Synchronization Questions

2013-07-31 Thread Dan Scott
Hi Elliot:

On Wed, Jul 31, 2013 at 06:57:25PM +, Elliot Voris wrote:
 Hi, all!
 
 I'm very interested in using Dan Scott's script
 (http://git.evergreen-ils.org/?p=contrib/Conifer.git;a=blob;f=tools/patron-load/ldap_osrf_sync;h=cf3c9b8edbffdb4d458baa2d88dcb61a2a7c0ac1;hb=9d1f461152bfc1079bd1647f25ebd872f5dc55f5)
 to load our patron (student/faculty) records from our Active Directory
 domain into Evergreen. I've tried to find as much info as I can, but I
 wasn't able to find much detail about how the script is intended to be
 used. I'm hoping someone here may be able to answer some questions for
 me.

I'm probably the only person who can answer authoritatively, so I'll
give it a go...

   1.  This probably goes without saying; I need to provide an AD
   account with read/writ privileges, right?

Yes, if you want to push the generated Evergreen barcode into your LDAP
server, then your AD administrator should be able to limit your
privileges appropriately (so that you can't totally blow things up!).

If you're just using the existing employeeNumber from the LDAP server
and never need to write to the LDAP server, then read-only would be
fine. Our work flow is a bit torturous:

a. General user account gets created in LDAP
b. ldap_osrf_sync checks for recently created LDAP users, if found:
  i. for each new user, create a corresponding Evergreen account
  ii. for each new user, create an Evergreen barcode
  iii. for each new user, update the LDAP server with the Evergreen
   barcode

If you don't need to generate the barcodes, then you can avoid a few
other branches that I created to support barcode-generation in
Evergreen. But you will probably need one branch that adds support for
searching across all user accounts (overriding patron scopes).

   2.  The script looks like it keeps track of the barcode in LDAP as
   lulLibraryBarcode, am I reading that right? I've been using the AD
   attribute employeeNumber to store student ID numbers (barcodes).
   Will the script be able to check if that attribute is defined in AD
   and then use it for the barcode in EG?

Yes, you should be able to swap out your local mappings in place of the
ones that were specific to our institutional needs. I could have done a
better job of centralizing the settings so that they were easier to fix
up in one place, but at a certain point getting a working script took
priority over generalization :/

   3.  Is this script meant to be run as-needed, or is it intended to
   run at a regular interval (as a cronjob)?

I run it in a cron job every half hour, using the query-date option
along the lines of:

ldap_osrf_sync --query-date `date --date='1 hour ago' +'%Y%m%d%H%M%SZZ'`

(escaped for cron, naturally)

   4.  For the OSRF_USER information in credentials.py, am I supposed
   to use the egadmin account? What does OSRF_WORK_OU mean?

You're connecting via OpenSRF to create the Evergreen accounts, so these
are really your Evergreen credentials. The OSRF_WORK_OU is a registered
Evergreen workstation.

   5.  Is there a syntax to run this (i.e. ./ldap_osrf_sync.py
   credentials.py), or do I just run it after creating
   credentials.py where the script can find it?

./ldap_osrf_sync --help gives you syntax help; there are lots of ways
of looking up users. I noticed that if credentials.py doesn't exist,
then it errors out with an ugly exception, so I just pushed a revision
that makes it a bit more graceful (by printing the script's doc block
and then exiting).
 
 I'm sure there are more questions that I've forgotten since starting
 this email, but I wanted to at least get started with it.

pydoc ldap_osrf_sync will provide some documentation about the script
as a whole, as well as class / method descriptions.


Re: [OPEN-ILS-DEV] LDAP OSRF Synchronization Questions

2013-07-31 Thread Dan Scott
On Wed, Jul 31, 2013 at 09:51:17PM +, Elliot Voris wrote:
 I've gone through, and adjusted the script to better suit our environment. I 
 keep getting these errors:
 
 opensrf@evergreen-test:~/tools/patron-load$ ./ldap_osrf_sync --dump-ldap
 [WARN:6134:idl.py:166:MainThread:] No field 'owner' in IDL class 'vie'
 [WARN:6134:idl.py:166:MainThread:] No field 'item_attr_def' in IDL class 'vie'
 [WARN:6134:idl.py:166:MainThread:] No field 'match_set' in IDL class 'vie'
 [WARN:6134:idl.py:166:MainThread:] No field 'renewals' in IDL class 'circ'
 [WARN:6134:idl.py:166:MainThread:] No field 'renewals' in IDL class 'acirc'
 [WARN:6134:idl.py:166:MainThread:] No field 'distribution_formula_entries' in 
 IDL class 'acpl'
 [WARN:6134:idl.py:166:MainThread:] No field 'default_fund' in IDL class 
 'acqpo'
 [WARN:6134:idl.py:166:MainThread:] No field 'filter_sets' in IDL class 'cfdi'
 opensrf@evergreen-test:~/tools/patron-load$
 
 Does any of that mean anything to you? Here's that section of 
 /usr/local/lib/python2.7/dist-packages/oils/utils/idl.py (evergreen 2.4.1, 
 btw). Any clues there?

Those are actually just warnings; basically places in the fieldmapper
IDL that don't match up the way that the IDL is supposed to.

Also, the --dump-ldap option won't dump anything to STDOUT if it doesn't
find any matches, so you will need to pair that up with a
--query-something option :)


Re: [OPEN-ILS-DEV] Anyone currently working on an opac redesign? Especially mobile-freindly?

2013-06-28 Thread Dan Scott
On Fri, Jun 28, 2013 at 04:16:56PM +, Boyer, Jason A wrote:
 I'm trying to make sure I'm not duplicating a bunch of effort. I'm
 working on making the opac more accessible on small screens. I saw a
 branch that Dan Scott made available in April, but I've not seen
 anything on the list about any similar work since. 
 
 Here's what I'm hoping to do, in case it piques anyone else's interest:
 Make the font-size entries relative, so that changing the body font-size 
 adjusts them all.

Hmm. grep -r font Open-ILS/src/templates/opac/ suggests that's already
true for the most part; at least for anything in TPAC under the /opac/
directory. What version of Evergreen are you working with?

 Use pt or mm for the body font-size rather than px.

Right now, that's a matter of changing one line (again, assuming a
current version of Evergreen) in Open-ILS/src/templates/opac/parts/css/fonts.tt2

 Rearrange things so it's easier to use the My Account and holds features.

Big thumbs up to that; in my experimental branch, the holds / my lists
bits were the most troublesome, IIRC.

I would like to keep working on this as well; my time is opening up over
the summer, I just need a little motivation now and then :)


Re: [OPEN-ILS-DEV] What are authorities sorted in the cataloging manage authorities view?

2013-06-24 Thread Dan Scott
On Mon, Jun 24, 2013 at 05:16:43PM -0400, Yamil Suarez wrote:
 Hello,
 
 My catalogers are confused by how authorities are sorted in the
 cataloging manage authorities view. I wanted to understand the
 approach being taken by the current code, versus what the catalogers
 are expecting.

The code in authority_tag_sf_browse() in OpenILS/Application/SuperCat.pm
is sorting purely by the value of the subfield that was being searched;
other subfields currently play no actual role in the sorting. (That is,
if the code hasn't changed significantly over the past couple of years).

In addition, the examples you gave are great, but they're missing the
4xx / 5xx fields which can also play a role in the sorting, as you might
get a match for a 4xx see from or 5xx see also field, per
http://www.loc.gov/marc/authority/ad400.html.

For example, if you searched for Singh, Bhagat, you might be surprised
to see Bhagata Si appearing apparently out of order. However, the idea
was that if you displayed the pertinent fields for the hit, you would
see that it was a see from reference and you would select the
authorized result. Obviously there's lots of improvement that could go
into the UI for this...


Re: [OPEN-ILS-DEV] slow(!) import

2013-06-06 Thread Dan Scott
On Thu, Jun 06, 2013 at 10:48:32AM -0400, Joe Thornton wrote:
 Thanks very much to everyone who replied to my plea for help.
 
 Elliot  -- Thanks for pointing me to the right list.
 
 Jason: I'm just kicking the tires. I was sure we'd need a much bigger
 server if we decide to use Evergreen in production, but I thought  16GB of
 RAM and 200GB of disk would be okay for testing. And at this point it's
 just 160k bib records. I haven't even gotten to items or patrons yet.
 Thanks for sharing your git project. I'm not a git user but I'll give it a
 try if I don't find a more familiar solution.
 
 Dan: Thanks -- I'll probably give your suggestion a try, but 550,000
 records 1,000 at a time will certainly take a while.

I think it will take a lot less time than importing them all in one
shot. Just generate an SQL file that imports each block of 1,000 records
in a sequence of statements.

The problem is that the database can get pretty bloated with a huge
transaction. Smaller transactions might give autovacuum a chance to run
in parallel with the import. Also, have you tweaked the database
configuration at all for things like maintenance_work_mem and work_mem?
 
 Three things:
 
 --  The process I started three days ago to import 160,000 records using
 the method on the Evergreen site is still running.

Yeah. That's ridiculous and not representative of a normal Evergreen
experience. We imported our ~2 million records more than 4 years ago in
less than a weekend. Things have changed since then: ingest is happening
entirely in-database now instead of outside of the database, which might
introduce some more overhead; but PostgreSQL has also gotten faster and
certainly there's lots better hardware available now...

 --  Maybe an unfair comparison, but we use VuFind as an alternative
 interface to Horizon, and a full import of all 550k records takes about 45
 minutes.

To some extent that's Solr vs PostgreSQL, and in some ways it's not so
much PostgreSQL's fault as it is that we do an awful lot to shred each
record out into potentially hundreds of rows across many tables, all of
which are indexed. For example, you can search by specific
field/subfield in Evergreen, which adds 25 - 125 rows per record
(average of about 40 rows per record in our sample data). VuFind doesn't
have to meet the same requirements as Evergreen does for searching
records; but yeah, Solr is also just plain fast.

 --  It's surprising to me that there isn't a faster method. We're looking
 seriously at Evergreen as a replacement for Horizon, but this would be a
 problem. I'll try Dan's and then Jason's methods (again, thank you very
 much) and hope that they're significantly faster. If I had the time and
 ability (unfortunately I have neither) I'd take a shot at it myself.

Parallelizing the import, as Jason suggested, certainly can help. I
haven't tried this for quite a while, but a really simplistic way of
parallelizing the import would be to take that script with many 1,000
record chunks and splitting it into n-1 separate scripts (where n is
the number of cores on your database server--again, credit due to
Jason), then run each script from a separate psql session. You might run
into deadlocks this way, but it's worth a shot.


[OPEN-ILS-DEV] We need a translation coordinator!

2013-04-13 Thread Dan Scott
Per the developer update yesterday at the Evergreen conference, we would
really like to have a translation manager as part of the developer team.
If you have internationalization expertise, and have an interest in
improving the quality of Evergreen's support for internationalization
and coordinating the efforts of the translators, we would like to talk
to you!

Here's a little background:

Translation has largely been on a best-effort basis since the Evergreen
1.0 release. It has largely focused on the public interface since 1.0,
with significant chunks of staff interface left behind over time.

We chose Launchpad as the lowest barrier to entry for casual
translators, but this poses a barrier for more professional translators.
In short: tools, testing, coordination... all have been on a best-effort
basis.

And we have a _good_ problem: there are a number of languages with
members of the community that complain when things break: Canadian
French, Armenian, Czech, Georgian, Finnish... and we want to keep this
list growing!

To improve the quality of Evergreen's support for internationalization,
we believe that we need a focal point for translations on the
development team. While the role (or roles) are not tightly defined,
some of the potential roles and responsibilities that person (or
persons) could take on include:

* Someone willing to work as part of the dev team;
* Partially to beat developers over the head until we learn;
* Partially to improve or replace translation tools and processes;
* Partially to coordinate translators translation teams;
* And it will not be a thankless job! Thank you / miigwetch / merci will
  roll in (and doubtless many other languages)

If you're interested, respond! We will help you help us!


[OPEN-ILS-DEV] EG2013 Hackfest wrap-up thanks and thoughts

2013-04-11 Thread Dan Scott
Hey all:

Yesterday's hackfest was a lot of fun!

Huge thanks to the EG2013 local organizing team for putting together a
great space for the hackfest. Lots of power + lots of wifi == a good
environment for learning and doing!

I also want to extend special thanks to Mark Jordan for ensuring that
everything went as smoothly as possible and for guiding us to all of the
resources that we really needed to get things done (wifi, coffee, power,
and food!)

I hope that everyone who came out keeps on hacking and sending in
patches, branches, bugs, and more. One thing that would be great for
future conferences would be honest feedback on what you think went well,
and what you think we could benefit from when organizing future
hackfests.

For example, it might have been useful to have virtual images or hosted
images available for those who arrive without already having access to a
test instance of Evergreen... or perhaps we could have explicitly told
would-be attendees well in advance that they're expected to have a
working dev instance of Evergreen available when they arrive at the
hackfest (possibly pointing at a virtual image).

As a reminder, the core hackfest coordination document that we used is
at http://goo.gl/CQMvu if you want to relive the glory that was
yesterday, or record the blazing insights that you might have had at
3:00AM while you were still riding the remnants of your hackfest high.

Happy hacking,
Dan


Re: [OPEN-ILS-DEV] Need basic information on Journal Title search for 2.4 docs

2013-03-11 Thread Dan Scott
On Mon, Mar 11, 2013 at 12:53:45PM -0400, Yamil Suarez wrote:
 
 Thanks Kathy, I was looking at the wrong release notes file for 2.3.
 Fo right now I only need a one sentence summary to add to the docs
 where all the qtypes are listed[1].
 
 The title entry is listed as Title—finds the terms you enter in
 the title of an item.
 
 I guess I could write something like Journal Title—finds the terms
 you enter in the title of a journal bib record, but I want to make
 sure this is both accurate and not too verbose.
 
 BTW, I am still not 100% clear of how it works in the back end,
 because I am not familiar with the bib_level selector and want to
 learn more about this search for when we upgrade.

Hi Yamil:

bib_level comes from MARC leader[07] bibliographic level attribute:
http://www.loc.gov/marc/bibliographic/bdleader.html

For the case of Journal Title, it simply appends bib_level=s to the
backend search to limit results to records that have that leader[07]
value set to s for serial.

Dan


Re: [OPEN-ILS-DEV] Need basic information on Journal Title search for 2.4 docs

2013-03-11 Thread Dan Scott
On Mon, Mar 11, 2013 at 01:11:01PM -0400, gislaine.hame...@statcan.gc.ca wrote:
 Question:
 
 Are these keyword type searches? (i.e. not exact?)

Yes, they're the same as regular title searches, but restricted to bib
records with bibliographic level s.

You could enclose your search with quotes blah blah to make it a
phrase search, just like you can with the regular title search.

Dan


Re: [OPEN-ILS-DEV] Evergreen Staff Client 2.3 and XULRunner 17.x

2013-02-21 Thread Dan Scott
On Thu, Feb 21, 2013 at 11:50 AM, Robert Heller hel...@deepsoft.com wrote:
 Does anyone know if the 2.3 version of the Evergreen Staff Client will
 work with XULRunner 17?  CentOS 5's latest set of updates includes
 XULRunner 17 and I was wondering if the 2.3 version of the  Evergreen
 Staff Client (the version currently in use by C/W Mars) will actually
 work with this version of XULRunner.  The application.ini file *says*
 the max version is 14.*, but I would like to know if that is a
 conservitive setting or a real hard limit.

I regularly set my max version to say 20.* and run with xulrunner
18.x for development / testing purposes without any consequences.

I think the 14.* is just an artifact of what we tested with going into
2.3 and what the make staff_client Makefile targets currently
download. Thomas has an updated branch that grabs the latest version
of XULRunner  updates the application.ini automatically that has not
yet been merged.


Re: [OPEN-ILS-DEV] Feb. maintenance releases - Thursday

2013-02-20 Thread Dan Scott
On Wed, Feb 20, 2013 at 09:05:54AM -0500, Bill Erickson wrote:
 Hi all,
 
 If there are no objections, I'd like to cut the monthly maintenance
 releases tomorrow (21st) instead of today.

I have no objections to seeing the release cut tomorrow rather than
today...

 Merge ye bugs while ye may.

...but I would rather not see a rush of bug fixes getting merged
today. We have to get out of the last-minute-rush mindset. And we have
another maintenance release in one more month.

Wouldn't it be better to just cut an RC based on what we have right now,
and only modify it if testing reveals that a regression slipped in?


[OPEN-ILS-DEV] Cutting releases

2013-01-16 Thread Dan Scott
So, on the bright side, we got three releases out today, and turned
around a fix for a critical security problem in minimal time in a
collaborative fashion, and had blog posts and everything.

I have a few concerns, though, which at the risk of sounding overly
negative I'll list out here...

I saw no mention of testing the release tarballs on various distros
before they were released. I had asked for sanity testing of my 2.1.5
tarball but didn't get any response. Security releases suck but still
need QA. How much sanity testing did these most recent releases get?
I thought the outcome of one of the last meetings was that we were going
to adopt a more open security process, one that would enable community
members to contribute towards testing?

Perhaps due to the lack of QA, our 2.2 and 2.3 tarballs once again
contain the full xulrunner binary packages for both win32 and
linux-i686, bloating their sizes by tens of megabytes. Bloat aside,
for legal reasons it's generally not advisable to ship binaries
from other projects if you can avoid it.

I think our release processes need some work.

Exhibit A: Even for a security release, we should be able to prep a
git branch over in the security repo, cut a tarball from it, and
then when the release is announced, just git push origin 
security/rel_2_1_5:rel_2_1 
so that the upstream release branch is a perfect replica of what went
into the tarball.

Instead, at least for the rel_2_1 branch, two times today
commits went into the rel_2_1 branch despite my having painstakingly
prepped a complete branch over in the security repo last night (which
also included commits from rel_2_1_4 that were missing from rel_2_1). I
found that pretty frustrating, although I certainly overreacted today
(probably due to staying up until 2:00 AM to prep the materials for the
release and not getting enough sleep to act like a decent human being).

Exhibit B: We seem to keep stumbling over how to keep upgrade scripts in
sync. Back when I proposed the version-upgrade directory as a means
of stamping out the problems we had historically had with fixes for
point-to-point scripts not making their way into all the right
releases, I also proposed that all point-to-point version upgrade
scripts should be committed first to master, then backported to each
applicable release. Can we commit to taking this approach? Or is there a
better alternative?

Exhibit C: Open-ILS/src/perlmods/lib/OpenILS.pm keeps saying 'our
$VERSION = 2.00;' - in rel_2_1 land I've been trying to remember
to rev that (although I think the 2.1.5 release actually says 2.14 -
sigh).  The release script is all well and good--I believe in better
living through automation--but the script needs regular injections of
human intelligence too. See also QA above! (Aside--what happens when we
get to 2.2.10? Maybe we should be using 2.015, 2.025, 2.033 instead of
2.15, 2.25, 2.33...)

So... maybe we can document our release process a bit more, with a
checklist of steps to follow to create a tarball and of things to QA?

Exhibit D: General confusion around translation processes. They're a
black box for most people, they don't result in quality output for
groups serious about their translations, there's an impedance mismatch
between the tools some translation groups want to use and between casual
translators... we could really use a dedicated translation coordinator
to step up and sort out some improvements on this front. Any volunteers?

On tagging releases:

I've brought this up in IRC a few times, but as far as I can tell the
only reason that we're not using actual git tag to create annotated
GPG-signed tags for releases is because one of the git admins doesn't
like git tags. I have to say thanks to Galen for having documented
the procedure for OpenSRF, which has worked well for me over there to
date, and I wish we would use actual tags in Evergreen instead of
branches that are named tag/blah.

Bah, this probably would have benefited from being broken up into
several different emails, each issue for its own thread. Oh well.


Re: [OPEN-ILS-DEV] Question about error when upgrading to 2.2 database schema

2013-01-08 Thread Dan Scott
On Tue, Jan 08, 2013 at 01:41:43PM -0500, Lebbeous Fogle-Weekley wrote:
 On Tue, Jan 8, 2013 at 1:18 PM, Lazar, Alexey Vladimirovich 
 alexey.la...@mnsu.edu wrote:
 
  Hello, developers.
 
  I am running the 2.1-2.2-upgrade-db.sql script from the 2.3.2 release
  tarball.
 
  [...]
  How important is it for the Authority Record ID to be 28 specifically?
 
  Can I safely change the script to make the ID be 31 to get past this
  problem? Or do I need a different solution?
 
 
 Hi Alexey.  Yes, you can change that to 31 for both appearances of 28 in
 the line of SQL you cite.  There should be no problem doing so.

This is arguably a bug similar to others where we've left space in the
sequence values for stock system values, so I've opened
https://bugs.launchpad.net/evergreen/+bug/1097399 accordingly - maybe it
will help new libraries launching Evergreen avoid this upgrade issue in
the future.


Re: [OPEN-ILS-DEV] JSPAC removal blockers, what and which are they?

2012-12-19 Thread Dan Scott
On Wed, Dec 19, 2012 at 12:21:36PM -0500, Mike Rylander wrote:
 Well, do don't really know, exactly.  But, thanks to Kathy Lussier and Ben
 Shum, we learned at the dev IRC meeting this week that we have a good
 starting point.  It lives at
 http://evergreen-ils.org/dokuwiki/doku.php?id=dev:opac:template-toolkit:plan#jspac_features_missing_from_tpacand
 each feature now has a Launch Pad bug.

snip 

 What we need from everyone that can spare a few minutes are data and
 opinions, and the best way to offer both is on the Launch Pad bugs linked
 from the above wiki page.  Please take a moment to scan that list of JSPAC
 features not yet in the TPAC, find the one that looks important or
 interesting to you and your institution, follow the link to the LP bug and
 leave a comment after having read the description and any previous
 comments.  Even if your comment is, we don't care about this feature and
 are planning to move to TPAC with or without it, that is important
 information to have.

snip 

 Questions, comments or concerns?

If adding comments to bugs in Launchpad is a barrier for participation
(having to sign up for an account, etc), then posting to the mailing
list would be helpful too, IMO.

Also, in the unlikely event that Kathy and Ben missed something that
matters to you in the current round-up of missing features, feel free to
raise that!

Dan

P.S. crazy flu-influenced idea - we could push one JSPAC removal bug to the
mailing list each day to solicit feedback, so that everyone is looking
at the same bug at approximately the same time...


Re: [OPEN-ILS-DEV] [OPEN-ILS-GENERAL] Library Communications Framework

2012-11-19 Thread Dan Scott
On Mon, Nov 19, 2012 at 10:13:01AM -0800, Lori Bowen Ayre wrote:
 I have just returned from the UK where I met with BIC and CILIP (Chartered
 Institute of Library and Information Professionals) representatives as well
 as RFID vendors.
 
 BIC has commissioned of a new set of library interoperability standards,
 which define a framework for the communication of data between self-service
 devices and other library terminal applications to and from library
 management systems. This framework replicates and extends the range of
 activities commonly conducted using 3M's open SIP.2 protocol and
 additionally provides web services functionality for the exchange of
 information.
 
 It is anticipated that further functionality will be added over time, as
 new requirements are identified by libraries and by their management system
 and terminal application developers and suppliers. Support for additional
 functionality in the areas such as stock management is under consideration.

I suspect saying here's yet another set of emerging standards, you guys
should spend your limited development hours chasing them in the promise
that one day it will be useful and adopted by the world isn't going to
go very far. Libraries or equipment vendors generally want to get an
immediate return on their investment rather than a promise that one day
what they're paying for might be useful.

On the other hand, BIC and CILIP would certainly be welcome to use
Evergreen as a reference platform for implementing their emerging draft
protocols. Reference implementations are pretty common in standards
processes, and having it available as part of a reasonably large open
source library system, with test suites that help enforce adherence to
the documented standards, might sway equipment vendors to actually adopt
the standards themselves.


Re: [OPEN-ILS-DEV] [OPEN-ILS-GENERAL] Library Communications Framework

2012-11-19 Thread Dan Scott
On Mon, Nov 19, 2012 at 02:12:29PM -0800, Lori Bowen Ayre wrote:
 I think instead of inviting BIC to use Evergreen, it would be more fruitful
 for we, US Libraries, build on the work they've done to look beyond those
 many protocols (from among the yet another you mention) that a) are
 implemented completely differently across systems) and b) are limited in
 the functionality they support and c) are built around one-at-a-time
 barcode technology.

I think that you completely misunderstood what I meant, which was the
BIC / CILIP could use Evergreen as a reference implementation for their
LFC work. As in, they could go ahead and develop modules that add the
LFC standard protocols to Evergreen, thereby making at least one ILS
capable of speaking to equipment that would also implement the LFC
standard protocols.


Re: [OPEN-ILS-DEV] Pull Request for MARC Edit Window Tiling.

2012-11-12 Thread Dan Scott
On Mon, Nov 12, 2012 at 11:33:29AM -0800, David Boyle wrote:
 I have submitted a pull request for the new feature MARC Edit Window Tiling.
 
 The pull request is at https://github.com/evergreen-library-system/Evergreen
 
 The wiki page is dev:proposal:marc_edit_tile
 The blueprint is at
 https://blueprints.launchpad.net/evergreen/+spec/marc-edit-window-tiling

Hi David:

It's nice to see you pulling together some code for Evergreen! However,
I noticed that
https://github.com/CatalystIT/Evergreen/commit/f723881ddc771365cac4930f773e24ddc68a9f64
has no commit message, no sign-off indicating your developer's
certificate of origin(DCO) and license compatible with Evergreen - all
of which is covered in (too much, and occasionally conflicting) detail
under
http://www.evergreen-ils.org/dokuwiki/doku.php?id=contributing#code_contribution

In short, we at the very least need to have the DCO sign-off to indicate
that you have the right to submit the code that you're submitting, and
that you're making it available under the GPL version 2 with the or
later option, to cover our basic legal behinds.


Re: [OPEN-ILS-DEV] Evergreen upgrade instructions for 2.3

2012-11-08 Thread Dan Scott
On Wed, Oct 03, 2012 at 02:26:36PM -0400, Soulliere, Robert wrote:
 Hi All,
 
 I updated the upgrade instructions to reflect the 2.3.0 version of Evergreen.
 http://docs.evergreen-ils.org/2.3/_upgrade_the_evergreen_code.html
 
 I wonder if I could get someone to double check the documentation?
 Then, add a link from the downloads page, if it passes the test?
 
Hi Robert:

I was going to make a few tweaks to those upgrade docs for 2.3.1 but it
looks like the version of docs/installation/server_upgrade.txt in master
is for 2.3-beta1; I guess the commit for the 2.3.0 instructions only got
into rel_2_3.  I'll push your commit into master and hopefully have some
time to work on it tomorrow.


Re: [OPEN-ILS-DEV] TPAC localization

2012-10-23 Thread Dan Scott
On Tue, Oct 23, 2012 at 11:00:47AM +, Kivilahti Olli-Antti wrote:
 Thanks for your reply Dan!
 
 What I mean with can't be localized:
 We have been doing our localization according to the instructions here
 http://evergreen-ils.org/dokuwiki/doku.php?id=evergreen-admin:customizations:i18n
 We have translated all of the generated .po -files and none of them 
 affect our TPAC, as in Evergreen v2.3 from this September.

I don't see any translated strings at all in
https://translations.launchpad.net/evergreen/master/+lang/fi - this is
where we draw our translations from for releases, and is the easiest way
to get a base set of translated files into a release.

 If there is a separate instructable which describes how to translate the 
 text entries for TPAC, please instruct?

Translate build/i18n/po/tpac/tpac.pot to create a Finnish fi-FI.po file,
then follow the Apache directives section in the official docs at
http://docs.evergreen-ils.org/2.3/_creating_a_new_skin_the_bare_minimum.html
for defining OILSWebLocale fi_FI and pointing at the location of the
fi-FI.po file. One Apache reload and your strings should be there.


Re: [OPEN-ILS-DEV] TPAC localization

2012-10-22 Thread Dan Scott
On Mon, Oct 22, 2012 at 11:10 AM, Kivilahti Olli-Antti
olli-antti.kivila...@jns.fi wrote:
 Howdy hou Evergreensters! (sorry bshum)
 It looks like the new TPAC can't really be localized and we are planning
 to start making it localizable using the existing gettext tools. Anyone
 working on that yet?

Can you please explain exactly what you mean by can't really be
localized? I think some of my neurons popped when they read that
sentence.

 We are planning to start work on the matter on 11.11. So far we have
 encountered unlocalizable text fields in Create New Patron -view, and
 various staff_client views, like acquisitions etc.

In the staff client, sure, there are some interfaces that need to have
hardcoded strings replaced.

 The main concern atm
 is TPAC, as we haven't found a way to localize it with traditional
 tools. Any insight into the matter will be appreciated, anyone working
 on that?

As noted above, I find this a surprising statement. Examples would
help a lot. The TPAC is almost entirely set up to be translated, and
does a far better job of supporting complex translations (including
standard gettext singular/plural support) than most of the other
sections of Evergreen. Or by localized do you mean
date/time/currency formatting, which probably does need some work?


Re: [OPEN-ILS-DEV] Proposal for search modifications for Evergreen 2.4

2012-10-15 Thread Dan Scott
On Mon, Oct 15, 2012 at 02:03:09PM -0400, Mike Rylander wrote:
 On Mon, Oct 15, 2012 at 12:52 PM, Jeffrey Bond
 jbond.evergr...@gmail.com wrote:
  Greetings Evergreen Developers,
 
  I’d like to propose a few changes to search functionality for the Evergreen
  application.  Here is some relevant information on the changes. I had read
  in an e-mail that blueprints may no longer be needed? I added one anyways
  but I can remove it if needed. Note that all changes proposed here can be
  turned on or off as defined in the wiki; if and when these changes are added
  they will not change the functionality of Evergreen unless an organization
  specific option is selected to do so. Thank you all for considering these
  changes.
 
 
 Without details or some prototype code the community can't really help
 evaluate the design.  That's something that you'll want to happen to
 improve the chances of having your work included in main-line
 Evergreen.
 
 Also, as Thomas mentioned, there is ongoing search improvement being
 both discussed and developed collaboratively, mostly in the IRC
 channel.  Some of Thomas' might impact yours, and there is higher
 level QueryParser work happening that might also either impact or
 inform your design.
 
 Jumping in IRC and discussing (and pointing to any code you might be
 experimenting with) might make things easier.

Jeffrey did point to the wiki, where at least pg_trgm was mentioned and
five new tables were mentioned; but beyond that, yeah, more details
would certainly help.

I think it's encouraging that both Thomas and Jeffrey have started by
writing up some of their design thoughts in the wiki; Thomas has done a
good job of also adding in use cases with some specific examples. I
haven't totally bought into the way he wants to try to address those use
cases via the tsvector weights-as-classes, personally, but the use cases
themselves can be kept around for any potential solution. And it's way
easier to get a sense of the overall design  direction of a change of
this size from a document / email conversation than from slowly doling
it out on IRC (and hoping that the pertinent folks are on-channel and
able to pay attention).

While I'm babbling, I would like to take the opportunity to point to
https://bugs.launchpad.net/evergreen/+bug/1054755 primarily for the
stop adding TSearch2 contrib/extension in new installs.  TSearch2
doesn't offer any functionality that we actually use these days, but it
does break ts_rewrite(), which offers fairly simple on-the-fly thesaurus
functionality that might be useful. Yes, there's still the problem of
getting rid of the TSearch2 extension from existing sites, but let's not
add to that pile of existing sites in the mean time...


Re: [OPEN-ILS-DEV] Request for installation assistance

2012-10-11 Thread Dan Scott
Hi Larry:

On Thu, Oct 11, 2012 at 08:39:34AM -0400, Larry W Arnold wrote:
 Hello,
 
 I hope I am posting to the correct listserv list, if not please advise.

Thanks for posting to the Evergreen dev list, it's a reasonable spot to
ask for technical assistance, and you've given us a lot of useful debug
info; I'll see if I can offer any help inline below:

 I have installed Evergreen 2.2 on a Ubuntu 10.04.4 server, and while the
 installation was smooth, I have encountered problems probably related to
 configuration (Network or Hostname ??).
 
 Any guidance or assistance is much appreciated
 
 When I run autogen.sh to boot-strap Opensrf I get the following:
 opensrf@eg-test:~$ autogen.sh
 Updating Evergreen organization tree and IDL
 
 Updating fieldmapper
 Exception: OpenSRF::EX::Session 2012-10-11T07:49:17
 OpenILS::Utils::Cronscript /
 usr/local/share/perl/5.10.1/OpenILS/Utils/Cronscript.pm:276 Session Error:
 route
 r@private.localhost/opensrf.settings IS NOT CONNECTED TO THE NETWORK!!!

snip

Okay, we see that fairly frequently, and it's often an authentication
problem with one or more of the ejabberd users.

 I then ran settings-tester.pl with the following result:
 
 opensrf@eg-test:~$
 /home/larry/Evergreen-ILS-2.2.2/Open-ILS/src/support-scripts/settings-tester
 .pl

Cool. This script is getting long in the tooth but occasionally provides
useful feedback, such as...

 1..7
 ok 1 - use OpenSRF::Utils::Config;

snip
 
 Checking Jabber connection for user opensrf, domain public.localhost
 Use of uninitialized value $@ in concatenation (.) or string at
 /usr/local/share/perl/5.10.1/OpenSRF/Transport/SlimJabber/Client.pm line
 157, DATA line 35.
 * Error connecting to jabber:
 SCALAR(0x2528538)

Aha - I'm pretty sure this indicates (clumsily!) that the opensrf user
was unable to connect to to the public.localhost ejabberd domain with
the credentials you supplied. Double-check opensrf_core.xml for a typo
in the password, or remove and reregister the opensrf@public.localhost
user with the same password.

(Also note that ejabberd sadly doesn't deal with special shell
characters in passwords, so avoid $ and @ and the like...)

Good luck!


Re: [OPEN-ILS-DEV] Code Documentation Proposal

2012-09-27 Thread Dan Scott
On Tue, Sep 25, 2012 at 01:49:31PM -0700, Jeffrey Bond wrote:
 Greetings Evergreen Devs,
 
 Catalyst is currently seeking to provide in line documentation for perl
 modules on the Evergreen project. The goal of this effort is to provide new
 developers and contributors a better glimpse into the code. It will also
 clear up misunderstandings about code that may, or may not be deprecated
 from use. Documentation for the code will include sub routine comments
 formatted in ASCIIdocs format, and completing all OpenSRF API calls with
 proper information.

Hmm. I'm curious about the choice of AsciiDoc format for the inline
documentation; wouldn't POD markup make much more sense (so that
perldoc OpenILS::Actor::Friends can return something)? Or is there a
perldoc variant that understands AsciiDoc?

 Details for the proposal can be found
 herehttp://evergreen-ils.org/dokuwiki/doku.php?id=dev:proposal:perl_module_documentation
 
 We wrote up this proposal assuming the perl modules we've committed to
 match the desires of the community. Please give us your input on what type
 of documentation should be added or taken away. Also we would like your
 input on what modules should be documented. We picked what we felt would be
 the best choice, but would rather comment on what would help the community
 the most.

It's pretty tough for me to have an opinion about which modules are most
valuable in terms of inline documentation, because when I need to I tend
to dig into the code itself. I guess it depends on which modules areas
the community is most interested in extending or making use of.


Re: [OPEN-ILS-DEV] Managing RC bug merges

2012-09-10 Thread Dan Scott
On Sep 5, 2012 11:48 AM, Galen Charlton g...@esilibrary.com wrote:

 Hi,


 On 09/05/2012 11:41 AM, Bill Erickson wrote:

 Surely there's a psychological difference between 2.2.3 and 2.3.0,
 but I don't see one in practice.  If anything, 2.2.3 has *more* cause
 to be handled with kid gloves, since 2.2 is much more widely used in
 production systems.


 .0 releases often feel bigger, but I agree with your overall point that
we should avoid brown-bagging *any* release, be it a functionality release
or a minor bugfix release.

 I actually support triple-signoffs as a matter of course -- author,
tester, QAer, for example -- but I don't think we've got enough numbers yet
to do that without unduly slowing down the flow of code.

For the record, I don't see what an RC branch buys us; we should be as
careful in rel_2_3 as we would be in the theoretical RC branch.

I'd be in favour of triple sign off for anything in a branch past RC.
Including after a .0 has come out.

Maybe in the short term an automated test (whether unit or system) could
serve as a replacement for one of those sign-offs, if we feel that there
aren't enough skilled individuals in the community... but it's definitely
been encouraging recently to have sign-offs from Kathy, Ben, Jason,
Michael, and others adding immensely to the QA effort!


Re: [OPEN-ILS-DEV] The Joensuu Regional Library is hiring!

2012-09-06 Thread Dan Scott
I hope you don't mind, but I reposted the job posting to the Code4lib
jobs board (at http://jobs.code4lib.org/job/2723/ ) to try and get
some broader visibility for you. Good luck!

On Thu, Sep 6, 2012 at 11:50 AM, Kivilahti Olli-Antti
olli-antti.kivila...@jns.fi wrote:
 The Joensuu Regional Library is looking for an Open Source
 revolutionary, passionate about programming, to it's Systems Migration
 Project.


Re: [OPEN-ILS-DEV] Authorities - alternate formats

2012-09-05 Thread Dan Scott
On Wed, Sep 05, 2012 at 02:32:14PM +, Duimovich, George wrote:
 
 Hello all,
 
 Regarding unAPI services, we've only used them in the context of 
 bibliographic records, but I assume that some of the same plumbing could be 
 adapted for authority records - or is it there already for use in 2.2.2 / 
 2.3.x (or planned for a future release)?

Some plumbing is indeed already there:

http://laurentian.concat.ca/opac/extras/supercat/retrieve/marcxml/authority/100

 
 For examples of authority record transforms, see Alternate formats section 
 of this entry here:
 
 http://id.loc.gov/authorities/subjects/sh2008118536.html
 
 I also wonder if any of those LoC crosswalks have been spotted anywhere under 
 any public licence?

Time to start hunting for XSLT for the likes of
http://www.loc.gov/standards/mads/mads-mapping.html I guess.

Maybe open a wishlist bug and start collecting the required bits?


Re: [OPEN-ILS-DEV] Authorities - alternate formats

2012-09-05 Thread Dan Scott
On Wed, Sep 05, 2012 at 10:42:07AM -0400, Dan Scott wrote:
 On Wed, Sep 05, 2012 at 02:32:14PM +, Duimovich, George wrote:
  
  Hello all,
  
  Regarding unAPI services, we've only used them in the context of 
  bibliographic records, but I assume that some of the same plumbing could be 
  adapted for authority records - or is it there already for use in 2.2.2 / 
  2.3.x (or planned for a future release)?
 
 Some plumbing is indeed already there:
 
 http://laurentian.concat.ca/opac/extras/supercat/retrieve/marcxml/authority/100
 
  
  For examples of authority record transforms, see Alternate formats 
  section of this entry here:
  
  http://id.loc.gov/authorities/subjects/sh2008118536.html
  
  I also wonder if any of those LoC crosswalks have been spotted anywhere 
  under any public licence?
 
 Time to start hunting for XSLT for the likes of
 http://www.loc.gov/standards/mads/mads-mapping.html I guess.
 
 Maybe open a wishlist bug and start collecting the required bits?

On that note,
http://www.loc.gov/standards/marcxml/xslt/MARC21slim2MADS.xsl should be
freely available to US citizens, I think.


Re: [OPEN-ILS-DEV] MFHD Display

2012-08-24 Thread Dan Scott
Heh, okay, it looks like there are 6 MFHD records attached to the bib
record for AIC (locg 152). Let's narrow it down with:

SELECT id, record, owning_lib
  FROM serial.record_entry
  WHERE record = 140858
AND deleted IS FALSE
AND marc LIKE '%Elms College%'
  ORDER BY owning_lib;



On Fri, Aug 24, 2012 at 09:49:25AM -0400, Tim Spindler wrote:
 Dan,
 
 
 Tim
 
 On Fri, Aug 24, 2012 at 9:35 AM, Dan Scott d...@coffeecode.net wrote:
 
  On Fri, Aug 24, 2012 at 09:23:49AM -0400, Tim Spindler wrote:
   I may have been confused about how this is designed to work but I was
   thinking the MFHD display would show only for owning library when the
   catalog is set to search that library only.  However, we are finding that
   all libraries holdings are displaying regardless of limit.  For instance,
   see
  
  
  https://bark.cwmars.org/eg/opac/record/140858?query=title%3AAmerican%20Sociological%20Review;qtype=title;locg=152;_adv=1;page=0;expand=issues#issues
  
   If you look at issues held, it shows not only AIC's holdings but Elms
   College.
  
   I am also only referring the MFHD holdings statement and not the Serials
   holdings statemnt generate by batch receiving or receiving in the serials
   control view.
 
  You're right, it should only show the records for the indicated library
  (or its children in the hierarchy).
 
  Can you show us the results of the SQL statement:
 
  SELECT id, record, owning_lib FROM serial.record_entry WHERE record =
  140858;
 
  Thanks,
  Dan
 
 
 
 
 -- 
 
 Tim Spindler
 
 Manager of Library Applications
 
 tspind...@cwmars.org
 
 508-755-3323 x20
 
 
 C/W MARS, Inc.
 
 67 Millbrook St, Suite 201
 
 Worcester, MA 01606
 http://www.cwmars.org
 
 
 
 *P**   Go Green - **Save a tree! Please don't print this e-mail unless it's
 really necessary.*


Re: [OPEN-ILS-DEV] MFHD Display

2012-08-24 Thread Dan Scott
On Fri, Aug 24, 2012 at 10:38:04AM -0400, Tim Spindler wrote:
 Dan,
 
 This is the result.
 
 idrecordowning_lib
 1375  140858  190
 
 I'll have to say I'm not sure what you are looking for with these queries
 other than to possibly see if there is an Org_unit id mismatch.

That's exactly what I was looking for. It seems, though, that that's not
what's happening here, as 152 was the OU id for AIC and 190 is the OU id
for Elms College.

Is that a consistent pattern? Do Elms College's holdings always show up
in AIC's context?


Re: [OPEN-ILS-DEV] Automatic stemming in Evergreen

2012-08-14 Thread Dan Scott
On Tue, Aug 14, 2012 at 06:22:15AM -0400, Kathy Lussier wrote:
 Hi all,
 
 We've had difficulty finding records in our catalog due to the
 automatic stemming that occurs when records are indexed in
 Evergreen. As an example, a title on one of our summer readings
 lists was The Assist by Neil Swidey. However, when users were
 searching for the assist as a title search with the phrase
 enclosed in quotations, they still had to page through several pages
 of results before finding the title they needed. Many of the records
 that ranked higher contained words like assistance, assistive,
 assisted, etc. because they were automatically stemmed at
 indexing, and the stemmed version of the word (assist) was what was
 stored in the index vector column. We've had many other examples
 where this stemming has made it difficult to conduct searches.

This particular example is quite a concern! I haven't noticed anything
similar yet, since we moved to Evergreen 2.3ish last week, and nobody
has brought a similar problem to my attention, but it might just be
early days for us.

 In digging through IRC logs and other list messages regarding
 stemming, people have mentioned that this stemming can be turned off
 so that the full words are indexed rather than the stemmed versions
 of a word. Can anybody tell me how this is done? I understand that
 the records would need to be reingested, but is there a flag that
 needs to be disabled to turn off the stemming or does it require
 something else? 

The simplest way to do this in a new Evergreen instance is to change the
configuration of the text search dictionary in
Open-ILS/src/sql/Pg/000.english.pg91.fts-config.sql - for example,
instead of using the snowball stemming algorithm as a basis for the full
text search, just use the simple dictionary which returns the
lowercase version of the incoming text:

CREATE TEXT SEARCH DICTIONARY english_nostop (TEMPLATE=pg_catalog.simple);

Note, however, that this is likely to cause other problems for
searchers; in the default concerto sample set of records, for example,
people will have to search for concertos to get matches for
concertos; concerto won't result in a match (and vice versa).

 Also, is there a way to use another dictionary for
 the stemmer so that the stemming is somewhat less aggressive than is
 used by the snowball stemmer? Overall, we like the concept of
 stemming, particularly when it retrieves results for both singular
 and plural versions of a word, but we've had many examples where
 stemming seems to be throwing users off course.

ispell support was added in the last few versions of PostgreSQL, which
might be worth exploring. I plan to dig into the current state of PostgreSQL
full-text search over the next few weeks, so the timing of your question
is quite good!


Re: [OPEN-ILS-DEV] 2.3 Documentation progress.

2012-08-09 Thread Dan Scott
Hi Robert:

On Thu, Aug 9, 2012 at 9:31 AM, Soulliere, Robert
robert.soulli...@mohawkcollege.ca wrote:
 Hi All,

 I set up Documentation processing for Evergreen  2.3. This is in “Alpha mode” 
 and available for review.

 It is available in the “Under Development” section of our documentation 
 launch page:

 http://docs.evergreen-ils.org/


 I also added an outline page for folks working on content to update progress 
 on chapters and sections:

 http://www.open-ils.org/dokuwiki/doku.php?id=evergreen-docs_2.3:outline

 You might notice a number of “Can this be pulled from 2.2 content?” notes in 
 red.

 The content for those chapters are already in 2.2 and are easy to pull into 
 2.3. All I need is for content authors or developers who are familiar with 
 the content or development of the features in these chapter to indicate “yes” 
 using the outline or the DIG list or launchpad.  Then, I can pull it in.

I'm sorry, but I don't think this is a sustainable model. I think we
have to assume that all of the 2.2 content should be pulled into 2.3,
and then deal with the exceptions as they arise. Beta-testers can open
bugs against the documentation if they find discrepancies between what
is documented and what they see in the beta release, but they can't do
that if there's no content to look at.

Perhaps as part of the Launchpad pullrequest and the review process,
we could start flagging areas of the documentation that need to be
changed as new features are added, or as old features are deprecated?


Re: [OPEN-ILS-DEV] Fwd: errors while running autoget.sh -c

2012-08-08 Thread Dan Scott
On Aug 8, 2012 2:45 AM, Net Wolf netwolf...@yahoo.com wrote:

 Hi Dan,

 there is no IDL section in either /opensrf/etc/opensrf.xml or
/openils/conf/opensrf.xml (or the respective opensrf_core.xml files)
 Incidentally, I was asked to install opensrf in the documentation on the
site before installing evergreen.

 http://evergreen-ils.org/downloads.php

It's not incidental; it's fundamental. OpenSRF is the foundation on which
Evergreen builds.

 It mentions OpenSRF at the top and the link to download and install it.
 I have also followed each step to the T for installation.

You need to follow them all the way to Z though!

 Should I reinstall from scratch?

Nope!

 What would be the steps then?

Evergreen bundles its own opensrf*.xml.example files and installs them into
the conf directory. You need to follow the Evergreen steps that tell you to
copy and configure them; then restart the OpenSRF services. Even though
those steps seem similar to the OpenSRF steps and people often skip that
step in the Evergreen steps because they think I already did that... silly
instructions, they're actually quite different.
 On Aug 8, 2012 2:45 AM, Net Wolf netwolf...@yahoo.com wrote:

 Hi Dan,

 there is no IDL section in either /opensrf/etc/opensrf.xml or
 /openils/conf/opensrf.xml (or the respective opensrf_core.xml files)
 Incidentally, I was asked to install opensrf in the documentation on the
 site before installing evergreen.

 http://evergreen-ils.org/downloads.php

 It mentions OpenSRF at the top and the link to download and install it.
 I have also followed each step to the T for installation. Should I
 reinstall from scratch? What would be the steps then?

 regards
 Siddharth

   --
 *From:* Dan Scott d...@coffeecode.net
 *To:* Evergreen Development Discussion List 
 open-ils-dev@list.georgialibraries.org
 *Cc:* Net Wolf netwolf...@yahoo.com
 *Sent:* Friday, August 3, 2012 10:09 PM
 *Subject:* Re: [OPEN-ILS-DEV] Fwd: errors while running autoget.sh -c

 On Fri, Aug 03, 2012 at 10:20:11AM -0400, Ben Shum wrote:
  Forwarded from where it was wrongly send to the catalogers list,
  this is a Fedora installation issue.
 
 
   Original Message 
  Subject: [Evergreen-catalogers] errors while running autoget.sh -c
  Date: Fri, 3 Aug 2012 02:46:34 -0700 (PDT)
  From: Net Wolf netwolf...@yahoo.com
  Reply-To: Net Wolf netwolf...@yahoo.com, Evergreen Community
  Catalogers evergreen-catalog...@list.evergreen-ils.org
  To: evergreen-catalog...@list.evergreen-ils.org
  evergreen-catalog...@list.evergreen-ils.org
 
 
 
  Hi,
 
  I am getting the following errors while running autogen.sh -c after
  a fresh installation of opensrf and evergreen.
 
  [opensrf@siddharth opensrf-2.1.0]$ autogen.sh -u
 
  Updating Evergreen organization tree and IDL
 
  Updating fieldmapper
  Empty filename at
  /usr/local/share/perl5/OpenILS/Utils/Fieldmapper.pm line 208.

 This suggests that you might be using the opensrf.xml file from OpenSRF
 rather than from Evergreen - and if that's the case, then nothing else
 will work. Check for an IDL section in the file and a pointer to the
 right location of fm_IDL.xml.

 snip

  Also, I had to install UUID::Tiny, MARC::Record::XML, SRU::Request
  and maybe a few more that I don't remember using perl -MCPAN. I also
  installed postgresql-plperl. All the above are not a part of the
  installation dependencies mentioned in the documentation.

 A lot of these problems were fixed after the 2.2.0 release, so see the
 upcoming 2.2.1 release, or just grab the latest Makefile.install
 directly in git at

 http://git.evergreen-ils.org/?p=Evergreen.git;a=blob;f=Open-ILS/src/extras/Makefile.install

 Long story short: I've packaged some of the dependencies for Fedora 17,
 in an effort to ease the install process on Fedora. MARC::File::XML is
 unfortunately still waiting approval (mostly my lack of prompting the
 reviewer) so that needs to be pulled from CPAN still, but I've added
 that to the FEDORA_CPAN list in the prereq installer. Note that these
 packages have _not_ been backported to Fedora 16, so we're effectively
 supporting only the latest stable version of Fedora at this point.

 Many thanks for giving Evergreen a go on Fedora! I'm a little lonely out
 here, but I've been running Fedora as my primary environment for years
 now; it's really helpful having some feedback to catch what I've
 overlooked.

 One other thing I should note is that when you get to the point of
 running Apache, you will need to disable SELinux via setenforce 0 due
 to our weird run as opensrf with a webroot in a weird location issue.
 Some day we'll fix that...





Re: [OPEN-ILS-DEV] Fwd: errors while running autoget.sh -c

2012-08-03 Thread Dan Scott
On Fri, Aug 03, 2012 at 10:20:11AM -0400, Ben Shum wrote:
 Forwarded from where it was wrongly send to the catalogers list,
 this is a Fedora installation issue.
 
 
  Original Message 
 Subject:  [Evergreen-catalogers] errors while running autoget.sh -c
 Date: Fri, 3 Aug 2012 02:46:34 -0700 (PDT)
 From: Net Wolf netwolf...@yahoo.com
 Reply-To: Net Wolf netwolf...@yahoo.com, Evergreen Community
 Catalogers evergreen-catalog...@list.evergreen-ils.org
 To:   evergreen-catalog...@list.evergreen-ils.org
 evergreen-catalog...@list.evergreen-ils.org
 
 
 
 Hi,
 
 I am getting the following errors while running autogen.sh -c after
 a fresh installation of opensrf and evergreen.
 
 [opensrf@siddharth opensrf-2.1.0]$ autogen.sh -u
 
 Updating Evergreen organization tree and IDL
 
 Updating fieldmapper
 Empty filename at
 /usr/local/share/perl5/OpenILS/Utils/Fieldmapper.pm line 208.

This suggests that you might be using the opensrf.xml file from OpenSRF
rather than from Evergreen - and if that's the case, then nothing else
will work. Check for an IDL section in the file and a pointer to the
right location of fm_IDL.xml.

snip

 Also, I had to install UUID::Tiny, MARC::Record::XML, SRU::Request
 and maybe a few more that I don't remember using perl -MCPAN. I also
 installed postgresql-plperl. All the above are not a part of the
 installation dependencies mentioned in the documentation.

A lot of these problems were fixed after the 2.2.0 release, so see the
upcoming 2.2.1 release, or just grab the latest Makefile.install
directly in git at
http://git.evergreen-ils.org/?p=Evergreen.git;a=blob;f=Open-ILS/src/extras/Makefile.install

Long story short: I've packaged some of the dependencies for Fedora 17,
in an effort to ease the install process on Fedora. MARC::File::XML is
unfortunately still waiting approval (mostly my lack of prompting the
reviewer) so that needs to be pulled from CPAN still, but I've added
that to the FEDORA_CPAN list in the prereq installer. Note that these
packages have _not_ been backported to Fedora 16, so we're effectively
supporting only the latest stable version of Fedora at this point.

Many thanks for giving Evergreen a go on Fedora! I'm a little lonely out
here, but I've been running Fedora as my primary environment for years
now; it's really helpful having some feedback to catch what I've
overlooked.

One other thing I should note is that when you get to the point of
running Apache, you will need to disable SELinux via setenforce 0 due
to our weird run as opensrf with a webroot in a weird location issue.
Some day we'll fix that...


Re: [OPEN-ILS-DEV] 'My Account' vs. 'Your Account'

2012-07-31 Thread Dan Scott
On Tue, Jul 31, 2012 at 12:48:49PM -0400, Dan Wells wrote:
 Hello all,
 
 Apologies if I missed any prior discussion, but as I am working on 
 implementing TPAC for my library, I noticed that the 'My Account' link from 
 JSPAC is now labeled as 'Your Account Log in'.  I have no problem with either 
 wording, and it isn't a huge deal, but I want to make sure we are being 
 consistent and deliberate with this label.  In particular, the current alpha 
 Android app uses 'My Account', so I am trying to determine for certain which 
 direction we wish to go as a group before I go ahead and change it to match 
 the TPAC.
 
 Since it interested me, I just spent the last 30 minutes or so looking at 20 
 top Internet retailers across a variety of product types to see which sort of 
 labeling is more common.  Here are the results:
 
 My Account:
 barnesandnoble.com
 bestbuy.com
 buy.com
 costco.com
 dell.com
 homedepot.com
 jcp.com
 kohls.com
 officemax.com
 petco.com (in 'My Petco' menu)
 petsmart.com (only visible after 'Sign in')
 sears.com (listed as 'My Profile')
 staples.com
 target.com
 toysrus.com
 zappos.com
 
 Your Account:
 amazon.com
 lowes.com
 nordstrom.com
 
 Other:
 newegg.com ('Your Account' within 'My NewEgg')
 
 
 Rather than spend any more time providing an analysis of this data, I would 
 like to step back and hear if others have any strongly held opinions about 
 this.
 
 Thanks,
 Dan

After you log in, the button splits into two: My Account and Logout.

I would be fine with changing Your Account Log In to My Account in
the default TPAC, if that's the preference, as the string is already
translated.


Re: [OPEN-ILS-DEV] 2.2.1 coming next week

2012-07-20 Thread Dan Scott
On Thu, Jul 19, 2012 at 01:26:44PM -0400, Lebbeous Fogle-Weekley wrote:
 Hi everyone.  The community aims to release Evergreen 2.2.1 early next week.
 
 If there are bug fixes that are important to you, and if they have
 not been tested and included in the master branch yet, now is a good
 time to draw attention to them on the mailing lists (preferably
 open-ils-dev). Please reference them by Launchpad bug number.
 
 There are always unsolved problems and new development to talk
 about, but to be clear, patches for the 2.2 series can only contain
 bug fixes and cleanup now.  New features are for the 2.3 series.

Hi Lebbeous:

I have been testing  merging  creating bug fixes for the past week or
so, including a number for 2.2; here are a few 2.2 candidates that
haven't been merged yet:

https://bugs.launchpad.net/evergreen/+bug/1015108 is a fix for MARC
Batch Import of authority records for sites that have migrated from 2.1
to 2.2. I have not tested this yet, however.

https://bugs.launchpad.net/evergreen/+bug/1003738 improves the situation
on TPAC for most browsers where the focus gets placed in the search box
in all cases, rather than more obvious locations in some cases (for
example, focus should be set to the User name input field when a user
clicks Log in to my account). I didn't address the JSPAC in my branch,
and due to a lack of support for the autofocus HTML5 attribute the
situation does not improve for IE 9 and less, but on TPAC and for most
other browsers the situation improves greatly, so I hope we won't let
the perfect be the enemy of the good in moving forward on this...

Finally, I think it's a bug that there's no user-driven means of
switching locales in the TPAC in 2.2, thus would put
https://bugs.launchpad.net/evergreen/+bug/1025921 forward for
consideration. If, however, you consider the addition of a locale picker
a feature rather than a bug and decide not to backport it, you should
definitely backport the second commit from the linked branch, as it
fixes a serious bug that affects any TPAC translation that uses Unicode.

Dan


[OPEN-ILS-DEV] AsciiDoc docs merged to rel_2_2

2012-07-13 Thread Dan Scott
Hi folks:

Wielding some git-fu, I've untangled the fairly interesting mess of
commits and merges from the DocBook repository  master and merged the
AsciiDoc docs into rel_2_2 now.

It should be pretty straight-forward to commit docs to master and then
backport them to rel_2_2 from here on out. Cheers!

Dan


Re: [OPEN-ILS-DEV] [GSOC] System Log

2012-07-09 Thread Dan Scott
Hi Swenyu:

On Tue, Jul 03, 2012 at 10:04:15PM +0800, sy duan wrote:
 Hi everyone,
 
  I’m Swenyu, this year’s GSOC student. I’m optimizing the system
 performance in search. I have studied the log of OpenSRF in the file
 osrfsys.log. Which shows in a single search operation, DB calls takes as
 much time as the message processing in Tranport.pm. And the accumulated
 message processing time even surpass the DB search time. So I want to

Can you share a little more information - for example, what sort of
hardware and operating system configuration, how much data is loaded in
the database, what query you're running, etc? If we want to reproduce
your tests in the future, to ensure that we don't introduce performance
regressions, we'll need some more complete descriptions and
instructions.

 optimize the message processing method in Tranport.pm as well as the DB’s
 stored procedures. But the conclusion is drawn from my own virtual machine
 logs which has little test data in database. In most time the search only
 return 0 result. So I turned to your help. Could anyone send me a snippet

Which data set are you using currently? If it's just the 100 records
from concerto.sql, we could look at using something like the Open
Content Alliance records at http://books.scholarsportal.info/marc.html
as the basis for a high-volume test data set (like concerto.sql, we
would want to create call numbers and copies, including deleted
records/call numbers/copies and monographic parts and the like - just
with 270,000 records instead of 100!)

 of your running system log which contains some query with a large result
 set. The system log is located in /openils/var/log in your running system.

I've attached the activity.log entries pertaining to a search on our
production database, but I suspect you're looking for more than just the
standard split-up-for-syslog log entries. Maybe you could post what
particular parts of the log entries you want, as an example, and we
could pull the pertinent pieces together?

 Or could anyone point a place for me to find some real data to fill in my
 database. That will be a great help to support the optimization.

As noted, I think the Open Content Alliance MARC records would serve as
a reasonable starting point for a huge set of of MARC, if that's the
kind of data you're looking for. (I'm not sure if you want patron data
and org unit data and transaction histories and everything else, too,
but as you're focused on search I'm guessing MARC + call numbers +
copies is your primary interest for now.)
2012-07-09 11:46:44 protostar osrf_json_gw: 
[ACT:3038:./osrf_json_gateway.c:302:134184874830388] [142.51.8.34] [] [en-CA] 
open-ils.search open-ils.search.biblio.multiclass.query 
{org_unit:105,depth:1,limit:10,offset:0,visibility_limit:3000,default_class:keyword},
 apache, 1
2012-07-09 11:46:44 protostar osrf_json_gw: 
[ACT:3038:./osrf_json_gateway.c:302:134184874830389] [142.51.8.34] [] [en-CA] 
open-ils.search open-ils.search.biblio.record.mods_slim.retrieve 889694
2012-07-09 11:46:44 protostar osrf_json_gw: 
[ACT:3852:./osrf_json_gateway.c:302:134184878538522] [142.51.8.34] [] [en-CA] 
open-ils.search open-ils.search.biblio.record.mods_slim.retrieve 889695
2012-07-09 11:46:44 protostar osrf_json_gw: 
[ACT:3044:./osrf_json_gateway.c:302:134184874930443] [142.51.8.34] [] [en-CA] 
open-ils.search open-ils.search.biblio.record.mods_slim.retrieve 889636
2012-07-09 11:46:44 protostar osrf_json_gw: 
[ACT:3826:./osrf_json_gateway.c:302:1341848778382612] [142.51.8.34] [] [en-CA] 
open-ils.search open-ils.search.biblio.record.mods_slim.retrieve 354247
2012-07-09 11:46:44 protostar osrf_json_gw: 
[ACT:2986:./osrf_json_gateway.c:302:134184871729868] [142.51.8.34] [] [en-CA] 
open-ils.search open-ils.search.biblio.record.mods_slim.retrieve 885982
2012-07-09 11:46:44 protostar osrf_json_gw: 
[ACT:3035:./osrf_json_gateway.c:302:134184874730353] [142.51.8.34] [] [en-CA] 
open-ils.search open-ils.search.biblio.record.mods_slim.retrieve 885968
2012-07-09 11:46:44 protostar osrf_json_gw: 
[ACT:3038:./osrf_json_gateway.c:302:1341848748303810] [142.51.8.34] [] [en-CA] 
open-ils.search open-ils.search.biblio.record.mods_slim.retrieve 293264
2012-07-09 11:46:44 protostar osrf_json_gw: 
[ACT:3826:./osrf_json_gateway.c:302:1341848778382613] [142.51.8.34] [] [en-CA] 
open-ils.search open-ils.search.biblio.record.mods_slim.retrieve 832165
2012-07-09 11:46:44 protostar osrf_json_gw: 
[ACT:2986:./osrf_json_gateway.c:302:134184871729869] [142.51.8.34] [] [en-CA] 
open-ils.search open-ils.search.biblio.record.mods_slim.retrieve 38167
2012-07-09 11:46:44 protostar osrf_json_gw: 
[ACT:3852:./osrf_json_gateway.c:302:134184878538523] [142.51.8.34] [] [en-CA] 
open-ils.search open-ils.search.biblio.record.mods_slim.retrieve 355300
2012-07-09 11:46:44 protostar osrf_json_gw: 
[ACT:3038:./osrf_json_gateway.c:302:1341848748303811] [142.51.8.34] [] [en-CA] 
open-ils.search open-ils.search.biblio.record.copy_count 105, 889694, 

Re: [OPEN-ILS-DEV] Documentation commits and sign-off procedure

2012-07-07 Thread Dan Scott
On Jul 6, 2012 9:20 AM, Soulliere, Robert 
robert.soulli...@mohawkcollege.ca wrote:

 Hi,

 I was hoping to do a lot of documentation committing in the next month I
just wanted to check in about  commit sign-off procedure as it relates to
documentation under 3 scenarios:

 1. An author sends me documentation in a format needing to be converted
to asciidoc. I convert it and commit it to the repository -- Would it be OK
for me to be the single sign-off on this commit using the commit option to
indicate that there is an original author and my sign off.
 e.g. git commit --author=Original Author em...@address.com

Sounds good to me; this way the author gets credited in gitstats and the
like.

 2.  I make a few typo fixes or update the upgrade instructions to update
version  references -- Can these quick fixes be direct commits or do I need
an additional sign-offs?

I think the goal is to have commit processes that offer some value (like
who wrote or changed what and why, so that we have a meaningful commit
history and change log), not to force painful overhead. Also, the chance of
a typo fix breaking the doc build seems low, so I don't see a great
advantage to requiring a second sign-off in this case.

 3. I author a new section or chapter or do a massive overhaul of a
section -- In this case, I would create working branch – make changes --
 cherry pick commit – add pull request in launchpad – wait for
review/sign-off – commit to main repo (or will someone else do the final
commit?).
 How many sign-offs should be required for documentation commits? Also, is
there a time frame for waiting for sign-offs before I can just commit it
without additional sign-off?

I don't know if you need a second sign-off in this case, either. Given how
few DIG committers there are, we wouldn't want to impede the progress of
the docs just to keep a perfectly parallel commit process with the code.

 Also if someone adds documentation from a different working branch and
asks me to pull into the main repo, could I just do the cherry pick and
commit to the main repo as a single sign-off, or do I follow a longer
review process and go through launchpad?

I would cherry-pick and commit.

 My main goals is for documentation to be updated in a timely manner but
to also follow the proper procedure for commits. I am concerned that we
will have fewer sign-off persons to test and sign off on documentation
changes compared to code commits.

Agreed!


Re: [OPEN-ILS-DEV] Using marc_export to extract only a single library's holdings

2012-06-15 Thread Dan Scott
On Fri, Jun 15, 2012 at 12:28:44AM -0400, Ben Shum wrote:
 I'm not sure, but it seems like it may not be possible to use
 marc_export script to extract only a single library's holdings as
 associated to bib records.
 
 Using an option for --library SHORTNAME definitely gives me the bib
 records with that library attached, but using the option for --items
 seems to drag in all holdings associated to that bib record and not
 just the one for the library specified.
 
 Is it possible (or has anyone already modified marc_export) to
 include the capability of extracting only a single library's bibs
 and associated holdings if specified?

marc_export doesn't do it out of the box, but it is definitely possible
to modify to make it work that way.

Inside add_bib_holdings() in the loop where it issues the
$r-insert_grouped_field() call to add each copy, you can add
preliminary tests, e.g.:

next unless $orgs{$cn-owning_lib}-shortname eq 'BR1';
next unless $orgs{$cp-circ_lib}-shortname eq 'BR1';
next if $cp-opac_visible eq 'f';

... and the like, that would skip inserting copies unless they match the
desired criteria.

We could standardize those into flags for marc_export if there's
significant interest.


Re: [OPEN-ILS-DEV] [OPEN-ILS-GENERAL] Evergreen 2.3 alpha around the corner

2012-06-15 Thread Dan Scott
On Thu, Jun 14, 2012 at 10:21:59PM -0400, Kathy Lussier wrote:
 Hi Bill,
 
 
 Do you have opinions on these items or think some items are
 missing?  Let your voices be heard.  It's safe to say this will be
 the shortest feature development cycle of any major Evergreen
 release.  Within the next month or so, we'll have a pretty good
 sense of everything that's making the cut.  In six weeks, we hit
 the feature freeze.  Now is the time to effect change.
 
 Thanks for asking the question! Since jspac is scheduled to be
 deprecated as of 2.3, I think it might be useful to identify which
 of those unsponsored pending features from 
 http://open-ils.org/dokuwiki/doku.php?id=dev:opac:template-toolkit:planrev=1339016264
 should be addressed before jspac is phased out. There are two that
 jump out at me:
 
 * Display of public copy notes - it might seem to be minor, but
 public notes aren't really public if the public can't see them.

Copy notes could easily be displayed in search results - we're already
grabbing them, just not displaying them - but as search results and
record details currently use different methods to retrieve essentially
the same bib record / call number / copy information (unapi SQL function
for search results, json_query Perl method for record details), we would
need to add the 'acpn' (copy note) field to the json_query in
OpenILS::Application::AppUtils::basic_opac_copy_query(). Shouldn't be
very hard to go this route.

At some point, though, we should settle on one really complex way of
grabbing this info, rather than having two really complex ways. I had
been working towards that with unapi, but ran out of time to accomplish
that for the 2.2 deadline.


Re: [OPEN-ILS-DEV] commiting documentation changes from the branches of others into Evergreen repo

2012-06-14 Thread Dan Scott
On Wed, Jun 13, 2012 at 07:57:43AM -0400, Soulliere, Robert wrote:
 Hi Dan,
 
 I did try to follow the instructions and using git cherry-pick -s
 commit-hash. However, when I got to the point of pushing the
 changes, I received a git error regarding non fast-forward commits and
 requiring a merge.. Thus, I merged and could push. Did I miss a step
 or is there a good way to get around this error without merging? I did
 pull for the latest changes just before pushing.

I think you want to git pull first, to get the latest changes, then
git cherry-pick -s commit to pull in the commit of interest, then
push.

If someone else pushes to the upstream branch while you were working,
causing that non fast-forward commit message, then, you could go ahead
and merge those changes, and then use git rebase to pop the
cherry-picked commit to the top and get rid of the merge commit noise.

In any case, next time if you run into a problem, drop into #evergreen
and I'm sure someone will walk through it with you step by step. (That
goes for anybody!)
 
 I was wondering if I could delete the old 1.6 folder under docs? This
 is an old directory used in the early days of DocBook testing. Seems
 like it could be confusing and is non useful at this point in time.
 http://git.evergreen-ils.org/?p=Evergreen.git;a=tree;f=docs/1.6;h=ee59c7ad272d9262a5034d854511290a4d4d4cd8;hb=master

I would second deleting the docs/1.6 folder for the reason you suggest.


Re: [OPEN-ILS-DEV] commiting documentation changes from the branches of others into Evergreen repo

2012-06-12 Thread Dan Scott
On Mon, Jun 11, 2012 at 01:49:04PM -0400, Soulliere, Robert wrote:
 Hi Developers,
 
 I have tested committing a change for the docs folder in the code
 repository. However I am a bit fuzzy on committing changes from the
 branches of other into the main repo. For example, bshum made did some
 asciidoc cleanup here:
 
 http://git.evergreen-ils.org/?p=working/Evergreen.git;a=commit;h=af19fa6419e5328058338cada40b42a44b0522de
 
 and I want to merge the changes into the repo. I don't think the
 changes have been merged yet?

Actually, I think they were pushed to master, based on the shortlog at
http://git.evergreen-ils.org/?p=Evergreen.git;a=summary (or running git
log against a current copy of master).

It looks like you used git merge to merge Ben's branch into your copy
of the master repository, then committed the merge of that branch with
your own commit message of Merge remote branch 'origin' into
documentation_test, then pushed your documentation_test branch to
master.

 What appropriate steps are used to make
 that happen? Do I need to set up my own working branch, merge bshum's
 changes into my branch and then merge my branch into the master repo,
 or can I merge bshum's branch directly into the master branch?

Generally, the process that we've been following for the code is to use
git cherry-pick to sign-off on each commit that we want to bring into
the master branch, rather than git merge. As documented at
http://evergreen-ils.org/dokuwiki/doku.php?id=dev:git#guidelines_for_contributors
under the Signing off on one or more commits section, the process is
basically:

1. git checkout -b new_branch origin/master
2. git cherry-pick -s commit-hash
3. test the changes to ensure docs still build
4. git push origin new_branch:master

The main difference to note is the push to origin instead of
working; origin is where the canonical set of code lives. If you
want to push your branch out to the working repository so that others
can check your efforts first for things like proper sign-offs / comments
/ everything works, then you push to the working repo in your own user
directory -- e.g.

git push working new_branch:user/rsoulliere/new_branch

Also note that new_branch should be something meaningful, like
dos_to_unix_linefeeds, so that a few months later if you're wondering
what the branch was for you don't have to wade through a bunch of
quick_test fix_bug generic branch names (speaking from experience
here!)

Also also note that when the docs get merged into the rel_2_2 branch, to
apply changes there you would checkout the new branch against
origin/rel_2_2 instead of origin/master in step 1.

If you're ever feeling unsure about any of these steps, feel free to
throw questions at #evergreen - there's plenty of us who will be happy
to help you out!
 
 Also, I am concerned about keeping track of these branched changes to
 ensure that they do not slip through the cracks as we get more
 contributors. Is there a recommended strategy for keeping up with
 contribution branches.

I think this was mentioned elsewhere, but the best practice (again,
keeping in sync with what we're doing for code) is probably to file a
bug at http://bugs.launchpad.net/evergreen and tag it documentation so
that you can generate lists of doc bugs that have not yet made it to
Fix committed or Fix released status.


Re: [OPEN-ILS-DEV] Evergreen 2.1.2-rc1 release available for testing

2012-06-07 Thread Dan Scott
On Thu, Jun 07, 2012 at 01:04:14AM -0400, Dan Scott wrote:
 Hello all:
 
 I've posted the Evergreen 2.1.2-rc1 release at
 http://evergreen-ils.org/downloads.php - this is a testing release, with
 hopes that minimal changes will be required before the final 2.1.2
 release in the next few days.
 
 Given that the last 2.1 release, 2.1.1, came out in November 2011, there
 are a very large number of bug fixes rolled into the 2.1.2 release. For
 the gory details, see the Changelog at 
 http://evergreen-ils.org/downloads/previews/ChangeLog-2.1.1-2.1.2-rc1
 
 I would appreciate feedback from testers who can give the upgrade
 SQL a try (it's a brief number of lines, but affects indexing so might
 take a long time to run) and would also delight in feedback from anyone
 who can exercise the various catalogue and staff client interfaces; we
 want to ensure that we have introduced no regressions in this pure
 bug-fix release.

Note: thanks to George Duimovich for noticing this; the tarball contains
an unwanted SQL upgrade script named
Open-ILS/src/sql/Pg/version-upgrade/2.1.1-2.1.2-rc1-upgrade-db.sql; I
had purged this from the release branch (user/dbs/rel_2_1_2_rc1) but it
managed to sneak into the tarbal anyway.

Do not use this script to upgrade! It will fail.

Instead, please use
Open-ILS/src/sql/Pg/version-upgrade/2.1.1-2.1.2-upgrade-db.sql and
report any problems based on your experience with that script.

Thanks again to George, and my apologies for this inconvenience.


[OPEN-ILS-DEV] Evergreen 2.1.2-rc1 release available for testing

2012-06-06 Thread Dan Scott
Hello all:

I've posted the Evergreen 2.1.2-rc1 release at
http://evergreen-ils.org/downloads.php - this is a testing release, with
hopes that minimal changes will be required before the final 2.1.2
release in the next few days.

Given that the last 2.1 release, 2.1.1, came out in November 2011, there
are a very large number of bug fixes rolled into the 2.1.2 release. For
the gory details, see the Changelog at 
http://evergreen-ils.org/downloads/previews/ChangeLog-2.1.1-2.1.2-rc1

I would appreciate feedback from testers who can give the upgrade
SQL a try (it's a brief number of lines, but affects indexing so might
take a long time to run) and would also delight in feedback from anyone
who can exercise the various catalogue and staff client interfaces; we
want to ensure that we have introduced no regressions in this pure
bug-fix release.

Thanks,
Dan


[OPEN-ILS-DEV] OpenSRF 2.1.0 - now generally available

2012-06-04 Thread Dan Scott
I am pleased to announce that the 2.1.0 release of OpenSRF is now
officially released and available at http://evergreen-ils.org/opensrf.php

Significant changes are outlined in the release notes at
http://evergreen-ils.org/documentation/release/OpenSRF/RELEASE_NOTES_2_1_0.html

Thanks to everyone who contributed bug reports, code, testing, and
documentation to OpenSRF for this release cycle!

Of course, we never rest; now begins the next feature release cycle for
OpenSRF, which already has some interesting features (cross-origin
JavaScript) in the pipeline.


Re: [OPEN-ILS-DEV] Internationalization support in Evergreen

2012-06-01 Thread Dan Scott
On Fri, Jun 1, 2012 at 2:40 PM, Job D R Borges j...@nupelia.uem.br wrote:
 Hello All,


 I am reviewing and updating the Brazilian translations at launchpad.net.
 Nowadays the restrictions status for Evergreen translations in Launchpad is
 Open,
 which means that anyone can translate without revision.
 I am working to get a very good quality for Brazilian Translations,
 revising all entries.

 Then I am looking, for someone that know how to change the current Status,
 to Restricted,
 to change to a more quality control process for translations.

Hi Job:

Thanks so much for you contributions to date!

To date we haven't really had anyone stepping up like you to take more
direct responsibility for a given language, so we haven't had to
figure out that process. This is a good problem to have! I'll take a
look at Launchpad to figure out what the process is for assigning a
given language to Restricted and granting you reviewer status so
that you can sign off on any contributed translations to Brazilian
languages.

It's probably a good time to ask if anyone wants to take on a similar
responsibility for other languages. And maybe someday we'll have a
real overall translation coordinator too!

Dan


Re: [OPEN-ILS-DEV] Internationalization support in Evergreen

2012-06-01 Thread Dan Scott
On Fri, Jun 1, 2012 at 4:27 PM, Dan Scott d...@coffeecode.net wrote:
 On Fri, Jun 1, 2012 at 2:40 PM, Job D R Borges j...@nupelia.uem.br wrote:
 Hello All,


 I am reviewing and updating the Brazilian translations at launchpad.net.
 Nowadays the restrictions status for Evergreen translations in Launchpad is
 Open,
 which means that anyone can translate without revision.
 I am working to get a very good quality for Brazilian Translations,
 revising all entries.

 Then I am looking, for someone that know how to change the current Status,
 to Restricted,
 to change to a more quality control process for translations.

 Hi Job:

 Thanks so much for you contributions to date!

 To date we haven't really had anyone stepping up like you to take more
 direct responsibility for a given language, so we haven't had to
 figure out that process. This is a good problem to have! I'll take a
 look at Launchpad to figure out what the process is for assigning a
 given language to Restricted and granting you reviewer status so
 that you can sign off on any contributed translations to Brazilian
 languages.

 It's probably a good time to ask if anyone wants to take on a similar
 responsibility for other languages. And maybe someday we'll have a
 real overall translation coordinator too!

Okay, I've opted to set things at Structured rather than
Restricted for now. Per
https://help.launchpad.net/Translations/YourProject/PermissionPolicies,
Structured enables people to contribute to languages that have no
assigned team or person, whereas Restricted prevents such
submissions. I think we still want to allow drive by contributions
to languages for which no-one has stepped up to take formal
responsibility.

As Job has asked to be responsible for the Brazilian languages, I
think it would make sense to create an evergreen-i18n-br group that
contains Job, rather than just assigning Job directly; that way more
people can be added to the group if necessary to share the burden with
Job (a problem that we can hope to happen!).

And that would give us a similar approach to follow for other
languages, so if people step up, we could create evergreen-i18n-cs
and evergreen-i18n-ru, etc. Sound good?

Dan


Re: [OPEN-ILS-DEV] Internationalization support in Evergreen

2012-06-01 Thread Dan Scott
On Fri, Jun 1, 2012 at 4:37 PM, Dan Scott d...@coffeecode.net wrote:
 On Fri, Jun 1, 2012 at 4:27 PM, Dan Scott d...@coffeecode.net wrote:
 On Fri, Jun 1, 2012 at 2:40 PM, Job D R Borges j...@nupelia.uem.br wrote:
 Hello All,


 I am reviewing and updating the Brazilian translations at launchpad.net.
 Nowadays the restrictions status for Evergreen translations in Launchpad is
 Open,
 which means that anyone can translate without revision.
 I am working to get a very good quality for Brazilian Translations,
 revising all entries.

 Then I am looking, for someone that know how to change the current Status,
 to Restricted,
 to change to a more quality control process for translations.

 Hi Job:

 Thanks so much for you contributions to date!

 To date we haven't really had anyone stepping up like you to take more
 direct responsibility for a given language, so we haven't had to
 figure out that process. This is a good problem to have! I'll take a
 look at Launchpad to figure out what the process is for assigning a
 given language to Restricted and granting you reviewer status so
 that you can sign off on any contributed translations to Brazilian
 languages.

 It's probably a good time to ask if anyone wants to take on a similar
 responsibility for other languages. And maybe someday we'll have a
 real overall translation coordinator too!

 Okay, I've opted to set things at Structured rather than
 Restricted for now. Per
 https://help.launchpad.net/Translations/YourProject/PermissionPolicies,
 Structured enables people to contribute to languages that have no
 assigned team or person, whereas Restricted prevents such
 submissions. I think we still want to allow drive by contributions
 to languages for which no-one has stepped up to take formal
 responsibility.

 As Job has asked to be responsible for the Brazilian languages, I
 think it would make sense to create an evergreen-i18n-br group that
 contains Job, rather than just assigning Job directly; that way more
 people can be added to the group if necessary to share the burden with
 Job (a problem that we can hope to happen!).

 And that would give us a similar approach to follow for other
 languages, so if people step up, we could create evergreen-i18n-cs
 and evergreen-i18n-ru, etc. Sound good?

To continue replying to myself... Sounds good, but that's not how it
works. You can't assign arbitrary teams to languages in Launchpad; you
have to pick from predefined teams in Launchpad, which then have
sub-teams for particular languages. And they highly recommend the
Launchpad Translators, so I've done that. (To be clear, the
Launchpad admins do leave open the possibility of creating a new team
for a specific project, but they clearly don't want to.)

So if you look at
https://translations.launchpad.net/evergreen/master/+lang/pt_BR you'll
see that it's managed by Launchpad Brazilian Translators. And
English (Canada) has no team, so anyone can contribute.

This is one of those areas where we're adapting to what Launchpad
gives us, I guess.


[OPEN-ILS-DEV] OpenSRF 2.1.0 release: final(?) release candidate

2012-05-30 Thread Dan Scott
Hello OpenSRF users and testers and developers:

I have tagged (as osrf_rel_2_1_0) and uploaded what I hope to be the
final release candidate for the OpenSRF 2.1.0 release. Let's give this
until Friday, June 1st to shake out any horrible bugs that would warrant
another kick at the 2.1.0 release, otherwise on Friday I will declare
this release generally available.

Your links:

* Release notes: 
http://evergreen-ils.org/documentation/release/OpenSRF/RELEASE_NOTES_2_1_0.html

* Install docs: http://evergreen-ils.org/documentation/install/README_2_1_0.html

* ChangeLog: http://evergreen-ils.org/downloads/OpenSRF-ChangeLog-2.1.0

* Tar ball: http://evergreen-ils.org/downloads/opensrf-2.1.0.tar.gz

* md5sum of tar ball: 
http://evergreen-ils.org/downloads/opensrf-2.1.0.tar.gz.md5

Please go forth and test, and report any errors to
http://bugs.launchpad.net/opensrf and/or on this mailing list and/or to
the #evergreen IRC channel on the Freenode network.


Re: [OPEN-ILS-DEV] Customized Search Functionality

2012-05-29 Thread Dan Scott
On Tue, May 29, 2012 at 04:30:08PM +, Michael Kerrigan wrote:
 Hello Developers,

Welcome!
 
 We have been hired to customize some of the search functionality for
 Evergreen for a client and was hoping someone could point us in the
 right direction regarding a couple of issues.

It would be interesting to know what you're working on customizing so
that we don't end up duplicating efforts, if you can possibly share...

 1. Should we submit our changes back to the community as a bulk
 patch to 2.2 (with all of our changes) or would it be more applicable
 to submit each change we are making independently?

There's a code contribution section under Contributing at
http://evergreen-ils.org/dokuwiki/doku.php?id=contributing#code_contribution
that is rather wordy but pretty much complete.

Long story short:

* Please use git (we offer access to the working Evergreen repository
  to anyone who sends in an SSH key)
* Create feature branches based against master for each feature you
  want to contribute. Keep'em separated, if you can.
* Open a bug at bugs.launchpad.net/evergreen and point at your branch
  when you're ready to have it reviewed

More info on how we use git at
http://evergreen-ils.org/dokuwiki/doku.php?id=dev:git
 
 2. What IDE do you all use for working with and debugging
 Evergreen PERL code?  Eclipse, Komodo?

Most Evergreen devs (at least the core committers) use Vim.
 
 3. Who is the resident expert on Evergreen search functionality?

If you follow the commit logs / bugs / mailing lists, Mike Rylander is
probably the closest thing to a resident expert on search functionality,
although many of us have recently had our fingers in parts of that pie.
Why don't you ask questions on the list and see what shakes out?

Dan


Re: [OPEN-ILS-DEV] Internationalization support in Evergreen

2012-05-24 Thread Dan Scott
On Thu, May 24, 2012 at 02:53:28PM +, Duimovich, George wrote:
 
 Hello Dan / Lebbeous,
 
 Regarding the GSOC proposal on Overhauling internationalization
 support in Evergreen here:
 
 http://evergreen-ils.org/dokuwiki/doku.php?id=dev:summer_of_coding_ideas
 
 Are there any examples in the Template::Toolkit code for 2.2+ that
 demonstrate any of the i18n advantages you folks had in mind over the
 current approach (for either Staff Client or TPAC)?
 
 How much 'heavy lifting' would be required to revisit and re-code a
 sample approach for best practices going forward? Or is there already
 a pretty clear approach that would have to be taken for any TT-based
 interface etc.?

Hi George:

Actually, I think the TPAC currently demonstrates best practices when it
comes to i18n support in Evergreen - at least as far as text goes. I
think it was Bill Erickson who did the bulk of the work to support
things like pluralization forms.

The TPAC presentation we did at EGConf 2013 (http://goo.gl/qyEqV) has
some examples of using the l() wrapper for marking translated text.

There is also a note in the older TPAC tutorial I wrote that got pulled
into the docs at
http://docs.evergreen-ils.org/2.2/_changing_some_text_in_the_tpac.html
about localizing text.

There is also some info about providing your own customized or
translated strings at
http://evergreen-ils.org/dokuwiki/doku.php?id=dev:opac:template-toolkit

Someday we'll pull all of that back into the docs in one authoritative
place :)

There's definitely more work that needs to be done to support globalization,
in terms of currency display, date / time formats, etc, but as far as
text goes in the TPAC, we're in good shape. (Well... I hadn't thought
about right-to-left text yet; we probably need a domain expert to tackle
that support if we want to go there. The good news is that TPAC should
be infinitely easier to work with to provide that support...)

So that's TPAC.

The same i18n support is available for anything built on
Template::Toolkit, so acquisitions (which currently has lots of
hardcoded strings), bookings (which currently uses Dojo's i18n approach
to completely decouple text from the template), serials (hardcoded
strings), etc could all benefit from an i18n overhaul.

Dan


Re: [OPEN-ILS-DEV] Should GSOC schedule affect 2.3 (etc) release schedule?

2012-05-24 Thread Dan Scott
On May 24, 2012 12:19 PM, Lazar, Alexey Vladimirovich 
alexey.la...@mnsu.edu wrote:

 Generally, I think all contributed code should go through the equal
review procedures and not specially prioritized, unless we are talking
about critical fixes, but can be decided on a case-by-case basis or
whatever the current process. I think keeping with release schedule is most
important. I would rather see GSCO or any other code or feature come in a
later release, but properly reviewed, tested and documented.

+1

  1. Don't let it affect the schedule.  Any GSOC code that is completed
early enough to make the cut, get reviewed, and merged is included in 2.3.
 Otherwise, it waits in the pool for the 2013-03 release along with the
rest of the un-merged community code.

 Yes.

I would prefer to keep the schedule as is.

Also, mentors, try to find some early contributions to core from your
students. This will build their confidence and pride and familiarity with
our code submission process... small wins early for big gains!


Re: [OPEN-ILS-DEV] Scheduling the next developer's meeting

2012-05-22 Thread Dan Scott
On Mon, May 21, 2012 at 6:19 PM, Dan Wells d...@calvin.edu wrote:
 Hello,

 I guess I was too late in filling out the survey, as that is the only slot I 
 cannot make.  Since there aren't any better slots in terms of overall 
 availability, I don't think there is anything to be done, so this is just an 
 FYI concerning my absence.

Alas.

There is something to be done, though! I've thrown some items into the
Draft agenda for the next meeting standing placeholder at
http://evergreen-ils.org/dokuwiki/doku.php?id=dev:meetings (for
example: merging the documentation repository with the code
repository; dealing with github pull requests; release progress). The
idea of having a next meeting standing placeholder page is that it
can start to accumulate items we need to talk about that come up in
passing in IRC or on the list or wherever...

Anyone who can't make a given meeting is always welcome to add agenda
items or comments in the draft agenda, or of course to discuss on the
lightly-used -dev list.

Dan


Re: [OPEN-ILS-DEV] ***SPAM*** OpenSRF Gateway/Translator Cross Origin Support

2012-05-12 Thread Dan Scott
On Sat, May 12, 2012 at 10:38:32PM -0400, nivardus wrote:
 Summary:
 I've written a small function that appends the correct headers and used it
 to enable CORS on the OSRF-HTTP-Translator. This could be useful if a
 library wants to put web applications and functionality that interacts with
 Evergreen on a separate domain. I'd appreciate any suggestions/feedback!

This sounds good. I'd love to see a simple but realistic scenario
outlined in the commit message, if at all possible; it will help people
understand why they want this even more.
 
 Cross-Site Background:
 Browsers' same-origin policy currently restricts requests to the current
 website's domain to prevent various nefarious scenarios. However, because
 APIs and other web resources need to remain open to cross-site use Cross
 Origin Resource Sharing (CORS) was created to allow services to formally
 authorize cross-origin requests.
 
 Questions:
 * Configuration: Is the current allowed origin location appropriate?
 (opensrf_core.xml, xpath: /config/gateway/cross_origin)

Location seems okay, but I wonder whether the example
opensrf_core.xml.example should contain '*' as a default. Maybe include
that as a commented-out option with a comment?

 * Gateway support: CORS is only implemented on the http-translator, given
 that the translator is The Future is there any reason to implement it on
 the gateway?

It seems that the gateway will continue to have a long life, so perhaps.

 Other Notes:
 * OpenSRF X-Headers are available as defines in osrf_http_gateway.c, in
 order to avoid a circular import I had to define a long string of all the
 headers in apachetools.h for use when setting Access-Control-Allow-Headers
 * Access-Control-Max-Age is currently set to 6 hours, but I see no reason
 why it couldn't be much longer.
 * Were it desirable the allowed-origin matching (currently just equality)
 could be fleshed out into something with pattern matching.
 * OpenSRF's Javascript client library currently defaults to the root of the
 current web host.
 * Using the wildcard * in the config whitelists all origins.
 
 Github diff:
 https://github.com/nivardus/OpenSRF/commit/3df3807e9cf69c60e5282717e97cadf429a39555
 CORS Spec: http://www.w3.org/TR/cors/

Two or three things here, but only #1 is necessary:

1) Per the code section of Contributing
   (http://evergreen-ils.org/dokuwiki/doku.php?id=contributing), we
   absolutely want to have a sign-off from you indicating that you have
   read and are asserting a Developer's Certificate of Origin.

2) For a pull request, something like Please pull from
   git://github.com/nivardus/OpenSRF.git master (to give the
   git repo location and branch) would make our lives a little
   easier, although not necessary. (Aside: you might not want to
   commit your own changes to the master branch of your repo;
   generally you want to keep your changes in separate branches.
   I've learned this the hard way!)

3) We would welcome you to submit your SSH pubkey to
   gitad...@evergreen-ils.org so that you can commit directly
   to the Evergreen working repo, which most of the current
   developers keep under our fingertips (slowly, we will indoctrinate
   you in our ways and pull you into our circle!). See
   http://evergreen-ils.org/dokuwiki/doku.php?id=dev:git
   for more info on our workflows, etc.

Thanks so much for this!


Re: [OPEN-ILS-DEV] Evergreen API

2012-05-11 Thread Dan Scott
On Fri, May 11, 2012 at 07:18:09PM -0700, Lori Bowen Ayre wrote:
 Hi Devs and Community,
 
 Has there ever been serious consideration given to developing an Evergreen
 API?  And if not, why not?
 
 I'm just wondering if that might be something that we could make happen as
 a community, perhaps with funds we raise through some clever activity or by
 selling some groovy merchandise.  We've been talking about doing something
 like that on the Oversight Board although no particular project has been
 suggested.
 
 But, I may suggest developing an Evergreen API unless there's some reason
 not do.  Please advise!

Maybe you need to define what you mean by API. If you mean a set of
methods that you can call to return data or produce results, then I
think Evergreen has had that since day one. It would be really hard for
us to develop Evergreen without an API. A Google search for Evergreen
API returns this as hit #2 for me: http://evergreen-ils.org/blog/?p=46

What we have could certainly use a lot more documentation in many
places, but we currently have native bindings for Perl, Python, Java, C,
and a Google Summer of Code project to pull together an official PHP
binding... all talking to the same API.


Re: [OPEN-ILS-DEV] Having issues with getting emails from d...@evergreen-ils.org

2012-05-07 Thread Dan Scott
On Mon, May 07, 2012 at 12:33:59PM -0400, Anoop Atre wrote:
 Yamil
 I can confirm you are still setup to receive emails from that address.
 
 On 05/06/2012 07:02 PM, Yamil Suarez wrote:
 Hello,
 
 About a month or so ago I was set up to receive emails from
 d...@evergreen-ils.org, and I have sent two emails to that address in
 the last 48 hours and I have not received copies of the emails. This
 year my institution added a new layer of spam filetering, which could
 be the cause of my current issue, but I would like if someone could
 check if I am still set up for receiving emails from
 d...@evergreen-ils.org.

Whether or not you receive copies of your own posts from a mailing list
depends on mailing list settings, which often include per-user
preferences, as well as your mail client / server (Google, for example,
tends to prevent you from receiving copies of your own posts)... so
whenever I wonder whether my outgoing mail is actually getting
distributed by a mailing list, I check the archives - in this case, at:
http://libmail.georgialibraries.org/mailman/listinfo/open-ils-documentation



Re: [OPEN-ILS-DEV] Having issues with getting emails from d...@evergreen-ils.org

2012-05-07 Thread Dan Scott
On Mon, May 07, 2012 at 12:45:59PM -0400, Yamil Suarez wrote:
 
 On May 7, 2012, at 12:40 PM, Dan Scott wrote:
 
 On Mon, May 07, 2012 at 12:33:59PM -0400, Anoop Atre wrote:
 
 Whether or not you receive copies of your own posts from a mailing
 list
 depends on mailing list settings, which often include per-user
 preferences, as well as your mail client / server (Google, for
 example,
 tends to prevent you from receiving copies of your own posts)... so
 whenever I wonder whether my outgoing mail is actually getting
 distributed by a mailing list, I check the archives - in this
 case, at:
 http://libmail.georgialibraries.org/mailman/listinfo/open-ils-documentation
 
 
 Is the d...@evergreen-ils.org address a mailing list? If so, what
 Dan mentioned makes perfect sense, though I I do not know what the
 list home page is to make changes.

Oh, apologies - I'm an idiot, I translated d...@evergreen-ils.org into
open-ils-documentat...@list.georgialibraries.org in my head.

I think d...@evergreen-ils.org is probably just an email forwarding
alias, no public visibility or settings to be applied there :/

Dan


Re: [OPEN-ILS-DEV] Need access to web server space to host largeconference presentations

2012-05-05 Thread Dan Scott
Hi Yamil:

On Fri, May 4, 2012 at 3:08 PM, Yamil Suarez ysua...@berklee.edu wrote:

 LOCKSS! I will used that acronym from now on.

 I will email Michael and store the files on both evergreen2012.org site and
 Google drive.

 Btw, should I store all presentation files on Google drive or just the large
 files?

I would just throw all of the files on the Google Drive - that makes
it easy for others to grab a complete set, or link to them from the
wiki, etc, rather than having to hunt around for the smaller files.


Re: [OPEN-ILS-DEV] Developer meeting summary

2012-04-27 Thread Dan Scott
On Fri, Apr 27, 2012 at 11:21:45PM -0400, Galen Charlton wrote:
 Hi,
 
 Here's my summary of the high points from the in-person developer's meeting 
 at the conference this morning.  This is from memory, so if you've got 
 corrections, please let me know or reply to the thread.  For the sake of this 
 email, I'm using 2.3 to refer to the release of Evergreen to follow 2.2.
 
 Time-based releases
 ---
 We agreed to continue working towards time-based releases, with two major 
 releases a year.  To improve the changes of meeting this goal, after side 
 discussion in recent days with various people, I had formally propose 
 establishing a position of release manager.
 
 Release Manager/Release Maintainers
 ---
 We discussed the proposal to define a release manager position, with the 
 following results.  First, the position of release manager comes with the 
 following responsibilities:
 
 - shepherding the release to ensure it happens on schedule and with 
 acceptable quality
 - acting as a neutral technical arbiter regarding what is ready to include in 
 the release by the scheduled release date
 - acting as final tie-breaker in case of otherwise irresolvable technical 
 disputes
 - as needed to ensure proper vetting, signing off on major commits, 
 particular ones that involve architectural changes
 - providing predictable, periodic updates on the state of the release to the 
 rest of the community
 
 The position of the release maintainer has similar responsibilities as the 
 release manager, but applied to point releases within a maintenance release 
 series.  RMaints are expected to release point releases more frequently, 
 e.g., every month.
 
 RMs and RMaints are chosen among the committers (in the case of RMs) or 
 committers and active developers (in the case of RMaints) by consensus (or if 
 necessary, election) among the developers and other contributors who attend 
 the development meetings.  At today's meeting, the following people were 
 chosen:
 
 RM for Evergreen 2.3: Bill Erickson
 RMaint for 2.2: Lebbeous Fogle-Weekley
 RMaint for 2.1: Dan Scott
 
 Release of 2.2.0
 
 We agreed to the following actions to get 2.2.0 out the door:
 
 - Jason Stephenson will spearhead a triage of open Launchpad bugs to find out 
 what can be replicated and to target bugs to 2.2 when appropriate.
 - To encourage focusing on the release, for the next while, the master branch 
 is frozen for all merges except commits related to getting 2.2.0 released.
 - We will cut a release candidate within 3-4 weeks, with the specific date to 
 be set next week after the bug triage is complete.
 - We will release 2.2.0 by June 8th, possibly earlier (again, depending on 
 the results of the bug triage).
 - Dan Scott will update the translations in the next few days; further string 
 changes are expected to be modest.

Thanks Galen! The only things I have to add from my own notes are -
under assistance with bug wrangling, I had:

** Ben Shum for everything but authority
** Dan Scott for authority
** GSoC students to be asked to help Jason Stephenson

We identified a few candidate show-stoppers but the triage will tell
all:
** Advanced search options bug - needs to be reverted, reopen bug?
** Slow search results

Thanks again for this summary!


Re: [OPEN-ILS-DEV] Developer's meeting at EG conf 2012 - Friday 4/27 at 8:00 a.m.

2012-04-26 Thread Dan Scott
On Thu, Apr 26, 2012 at 07:44:10AM -0400, Galen Charlton wrote:
 Hi,
 
 Face-to-face meetings?  We can't have that!  But seriously, I'm calling a 
 meeting of developers at the Evergreen conference for tomorrow (Friday) at 8 
 a.m.  Let's gather near the conference registration desk and claim space 
 nearby.  Food will be available from the conference continental breakfast 
 spread.
 
 There are a few items to discuss:
 
 - putting together the developers' update for Saturday
 - discussing the RM proposal
 - seeing if we can nail down a release schedule for 2.2.0
 - any other topics?

Great idea, Galen. 8:00 AM hurts a bit but we've got to do it some time.

To nail down a release schedule for 2.2.0, do we need to have a
reasonable definition of critical and showstopper bugs so that we
can produce a release candidate  golden master per
http://evergreen-ils.org/dokuwiki/doku.php?id=versioning ? Or is that
secondary to just needing actual testing for a variety of scenarios
(fresh install vs. upgrade, exercising a wide variety of functionality)
and given bug reports we'll know critical and showstopper when we
see them... at least for 2.2.0?

I wouldn't mind sorting out what we're doing Saturday / Sunday either;
right now I'm booked for Saturday night at an airport hotel, but would
cancel if I could get a reasonable rate around here (the conf rate
wasn't available for Saturday). My flight on Sunday doesn't take off
until 4:00 pm so I've potentially got lots of time. Alternatively, we
could all congregrate in a cheaper airport hotel - heh.


[OPEN-ILS-DEV] Dev hackfest: prep and thoughts on remote participation

2012-04-20 Thread Dan Scott
Hello:

Two things on my mind today, not fully-formed but in progress...

1. There's a list of proposed hackfest projects / time slots /
   interested people on the wiki at
   http://evergreen-ils.org/dokuwiki/doku.php?id=dev:hackfest:eg2012 - feel
   free to add other ideas to the table, or update existing rows to link
   to related bugs or wiki pages describing the idea / goals / work that has
   already taken place (for example, Jeff... I think you had a branch at
   one time for the use bib ID instead of ISBN for added content idea
   that might enable us to fast-forward on to other projects?)

2. Given that there are a number of developers, including Google Summer
   of Code students, who will be unable to attend this year's conference,
   I'm thinking that maybe we could host a Google Hangout per project to
   enable remote attendees to participate. I've poked around looking for
   a good open source alternative to Google Hangouts but nothing seems
   to fit the bill. This would depend on bandwidth, naturally, and might
   end up being utterly unworkable, but at least for the projects that
   I end up working on I hope to give it a shot.

   The best that I can come up with for connecting remote people to
   projects is to ask the person who wants to be the primary contact
   for a given Hackfest project to link to their Google+ profile (in
   your Google+ page, click Profile and copy the link, then run it
   through a link shortener like goo.gl - a la http://goo.gl/g5jSV

Dan


Re: [OPEN-ILS-DEV] #help Evergreen 2.2 Release

2012-04-09 Thread Dan Scott
On Mon, Apr 9, 2012 at 10:48 AM, Soulliere, Robert
robert.soulli...@mohawkcollege.ca wrote:
 Hi Dan,

 I agree with your proposals for more single sourcing and providing the docs 
 in the tarball. Our switch to AsciiDoc makes this more viable given the 
 usability of AsciiDoc in its raw form compared to DocBook XML. I am not 
 sure of other processed formats (HTML, PDF, etc...). I guess we need to 
 consider a balance of providing mulitple formats vs the size of the tarball?

 Note that I think Developers can currently commit to the DIG repository.
  The actual live git repository which can be committed to is located on the 
 same server as Evergreen Code: 
 http://git.evergreen-ils.org/?p=Evergreen-DocBook.git;a=summary. I gained 
 access by sending SSH keys to Galen. I am assuming that developers an also 
 gain access the same way if they don't already have commit access given their 
 current access to the Evergreen code repo.
 The github repository (https://github.com/rsoulliere/Evergreen-DocBook/) I 
 usually refer to is the a mirror of the main evergreen documentation repo.

Oh great! I was under the impression that the mirroring was still the
other way around :)

Thomas just gave my account write permission on the
git.evergreen-ils.org/Evergreen-DocBook.git repo now (funny how that
name is getting out of date rapidly) and I just pushed some more quick
fixes to the upgrade docs. Many thanks!


Re: [OPEN-ILS-DEV] RELEASE TEAM

2012-04-09 Thread Dan Scott
On Mon, Apr 09, 2012 at 04:44:37PM -0400, Thomas Berezansky wrote:
 Uploaded 2.2-beta2 downloads (source, staff client) and Changelog.
 
 Testing and Wiki/Web updates would be appreciated.

Using the README and docs/RELEASE_NOTES_2_2.txt in to source directory,
I've updated:

http://evergreen-ils.org/documentation/install/README_2_2.html

and

http://evergreen-ils.org/documentation/release/RELEASE_NOTES_2_2.html

(although the latter is, in particular, still pretty skeletal - like, no
mention at all of TPAC, heh).

Thanks Thomas!


[OPEN-ILS-DEV] Buildbot continuous integration environment is alive! Aliiiiive!

2012-04-08 Thread Dan Scott
Hi all:

Galen at Equinox got the VM for http://testing.evergreen-ils.org up
and running and connected to the 'net again last week, and I've got
the buildbot updated to 0.86pl1 and running and testing builds on
every commit again for OpenSRF master / rel_2_1 / rel_2_0 (on Ubuntu
10.04) and Evergreen master / rel_2_2 / rel_2_1 / rel_2_0 (on Debian
Squeeze, thanks to UPEI who kept their VM running faithfully and
patiently for the past four months).

GPLS did have Ubuntu 8.04 and 10.04 build slaves. If they're still in
the game, I would suggest we scrap the 8.04 build slave - perhaps in
favour of a 12.04 build slave?

Also, if anyone else has a VM they want to add to the mix (say, if
you're keen on FreeBSD or the like), let's talk!

Dan


Re: [OPEN-ILS-DEV] Including preferred library in search for located URIs

2012-04-06 Thread Dan Scott
On Fri, Mar 23, 2012 at 8:28 AM, Kathy Lussier kluss...@masslnc.org wrote:
 Thanks Dan!

 I also wanted to add that the use case where we see this feature as being of
 most benefit is when a user has set their search library to the entire
 consortium. Many of our at-home users start off by searching the consortium,
 and giving them the ability to retrieve electronic resources owned by their
 home libraries will be a big plus.

 Kathy

-Original Message-
From: open-ils-dev-boun...@list.georgialibraries.org [mailto:open-ils-
dev-boun...@list.georgialibraries.org] On Behalf Of Dan Scott
Sent: Friday, March 23, 2012 3:39 AM
To: open-ils-dev@list.georgialibraries.org
Subject: [OPEN-ILS-DEV] Including preferred library in search for
located URIs

I've been asked to teach search to include the user's preferred search
library (as set in search preferences, falling back to home OU) in
searches, specifically for the purposes of ensuring that located URIs
at
the user's preferred library would trigger hits where physical copies
would be out of scope.

Practical example:

1. User sets preferred search library to BR1.

2. They jump onto the catalogue, log in (which changes their search org
   to their preferred search library of BR1), but then for some reason
   change their search org in the org selector to BR3.

3. They issue a search for Harry Potter and the Philosopher's Stone.
   BR3 doesn't hold any copies or have any located URIs, but SYS1
(BR1's
   parent) has a PotterMore licence and has added their 856 $9 SYS1 to
a
   bib record.

As it currently stands, User is out of luck; they won't see any hits in
search results as there are no copies or located URIs in the BR3 scope.

The proposed enhancement would, however, make the search results
contain
a hit for Harry Potter and the Philosopher's Stone at the user's
preferred search library. With the enhancements to TPAC search results
 record details copy / located URI display in
https://bugs.launchpad.net/evergreen/+bug/907056 the located URI at the
preferred library would then be displayed.

So, two questions:

1) Any strong resistance to the addition of this feature?

2) My rough implementation plan would be:

   a) Add another parameter to search.query_parser_fts() :
param_pref_ou INT
      - that would simply concatenate the pref_ou ancestors onto
      luri_org_list (deduping, naturally)

   b) Then make the existing 10-parameter search.query_parser_fts()
function
      a wrapper that calls the new 11-parameter function.

   c) Then make O:A:Storage:Publisher::query_parser_fts() capable of
calling
      the 11-param function if param_pref_ou has a value.

   d) Then teach callers of open-ils.storage.query_parser_search to
pass in
      a pref_ou argument (when appropriate).

Any thoughts / concerns before I sally forth?


Okay, hearing no objection, I've pushed a first-cut working branch to
user/dbs/preferred_lib_located_uris that adds this capability.

Thus far it has only lightly been tested in TPAC, but in theory should
work in JSPAC. The major difference from the proposed plan was that I
simply added the 11th parameter to search.query_parser_fts() and made
it default to NULL; then dropped the 10-parameter version.


Re: [OPEN-ILS-DEV] Testing 2.1-2.2-beta1 upgrade script

2012-04-05 Thread Dan Scott
On Thu, Apr 5, 2012 at 7:16 AM, Ben Shum bs...@biblio.org wrote:
 It seemed to be going well till I hit this and the rest of the script died
 out:

 psql:Evergreen/Open-ILS/src/sql/Pg/version-upgrade/2.1-2.2-upgrade-db.sql:15328:
 ERROR:  cannot drop language plperl because other objects depend on it
 DETAIL:  function migration_tools.add_codabar_checkdigit(text) depends on
 language plperl
 function migration_tools.expand_barcode(text,text,integer,text,text) depends
 on language plperl
 HINT:  Use DROP ... CASCADE to drop the dependent objects too.

 So I suppose that I have to figure out which other custom functions (I think
 that one is an ESI migration_tools one?) exist that utilize plperl and
 change those before I can drop plperl from our Evergreen database.

I hit that overnight as well, and would recommend that we move the
DROP LANGUAGE statement outside of the transaction so that it doesn't
roll back all of the other work.

It seems likely that many sites will have added custom plperl
functions over the past few years and will hit this roadblock after
what can be a very long upgrade script (somewhere in the vicinity of
6-8 hours for us on our testing server), which in turn slows down the
testing  adoption process.


Re: [OPEN-ILS-DEV] Fwd: GSOC [2012] for PL/SQL C based stored procedures

2012-04-05 Thread Dan Scott
Hi Preeti:

On Thu, Apr 5, 2012 at 10:51 AM, preeti . ppreet...@gmail.com wrote:
 Hello everyone,
 I am eagerly Waiting for response from EverGreen. Please give answer to
 queries as mentioned in last mail ASAP.
 I want to submit Propsal in Mono Project as the deadline is tomorrow for
 GSOC 2012.

You can submit a proposal to the Mono project as well as the Evergreen
project if you like.


 I would be interested in this project:
 Translate certain search-related SQL query components to C-based stored
 procedures

 I am working on understanding the basics as of now. I shall be thankful for
 any guidance from the team members about
 how to go about.

The basics would be:

1) Install Evergreen
2) Build one or more realistic workloads that exercise the database
(constrictor might help here)
3) Measure the performance and determine where there are bottlenecks
in the database performance
4) Address those bottlenecks - with one approach being to translate
non-C stored procedures / database functions to C stored procedures
5) Measure the performance with the translated database stored
procedures / functions to determine where improvements have occurred

 Please provide me more details regarding above project, so that i can decide
 timelines of project, followed by mentioning them in proposal that i want to
 submit in EverGreen for GSOC 2012.

You are the person responsible for working out the details and
creating the proposal. If you ask specific questions, we can help with
those. If you're having problems installing Evergreen or running it,
we can help with that. But if we write your proposal for you, we have
no way of knowing if you actually have the ability to do what you / we
are proposing for the summer.


Re: [OPEN-ILS-DEV] Fwd: GSOC [2012] Question regarding PL/SQL C based stored procedure

2012-04-05 Thread Dan Scott
Hi Sumit:

(with some apologies for the form-based reply)

On Thu, Apr 5, 2012 at 10:54 AM, sumit verma sumitverma2...@gmail.com wrote:
 Hello everyone,

 I am eagerly Waiting for response from EverGreen. Please give answer to
 queries as mentioned in last mail ASAP.

 I want to submit Proposal  as the deadline is tomorrow for GSOC 2012.

snip

 -- Forwarded message --
 From: sumit verma sumitverma2...@gmail.com
 Date: Wed, Apr 4, 2012 at 3:04 PM
 Subject: GSOC [2012] Question regarding PL/SQL  C based stored procedure
 To: open-ils-dev@list.georgialibraries.org


 Hello everyone,

 As I would be interested in this project:

 Optimize Evergreen: Convert PL/Perl-based PostgreSQL stored procedures to
 PL/SQL or PL/C



 So please provide me more details regarding above project, so that i can
 decide timelines of projects, followed by mentioning them in proposal that i
 want to submit in EverGreen for GSOC 2012.

The basics would be:

1) Install Evergreen
2) Build one or more realistic workloads that exercise the database
(constrictor might help here)
3) Measure the performance and determine where there are bottlenecks
in the database performance
4) Address those bottlenecks - with one approach being to translate
non-C stored procedures / database functions to C stored procedures
5) Measure the performance with the translated database stored
procedures / functions to determine where improvements have occurred

You are the person responsible for working out the details and
creating the proposal. If you ask specific questions, we can help with
those. If you're having problems installing Evergreen or running it,
we can help with that. But if we write your proposal for you, we have
no way of knowing if you actually have the ability to do what you / we
are proposing for the summer.


Re: [OPEN-ILS-DEV] #help Evergreen 2.2 Release

2012-04-05 Thread Dan Scott
Hi Robert:

I apologize, I was supposed to respond to this weeks ago.

On Tue, Feb 28, 2012 at 2:01 PM, Soulliere, Robert
robert.soulli...@mohawkcollege.ca wrote:
 Hi Anoop,

 Does the release-notes-writer role basically take what is in the raw release 
 notes from the master file as added by developers here:
 http://git.evergreen-ils.org/?p=Evergreen.git;a=blob_plain;f=docs/RELEASE_NOTES_2_2.txt;hb=master

 and get them into the official docs such as here?:
 http://docs.evergreen-ils.org/2.2/_release_notes.html

That's a good short term model, I think. We just started trying to
keep release notes up to date late in the 2.2 cycle, so they're pretty
limited this time around, but hopefully we'll get better in future
releases.

 If so, perhaps I can help with that and take on that role?

That would be great, in my opinion.

 Can we just use one source for that in the official docs or do we need to add 
 a wiki page as well?

Yes, ideally we would only need one source for that. I would
personally prefer to avoid adding a wiki page to the mix.

 Also note that we have upgrade instruction in the official docs here:
 http://docs.evergreen-ils.org/2.2/_upgrading_the_evergreen_server.html
 -- this is based on the alpha release, but if you give me a few days heads up 
 before the official release will be released, I could get those updated to 
 reflect the official tarball version. It might also be good to have a few 
 folks review the upgrade instructions chapter as well.

Yes, single-sourcing the upgrade instructions would be a really good
idea too. Just taking a quick peek at the current version, there are a
number of things that I think we would want to change. Holding an
official review would be a good idea, I think.

 In general I think DIG has more documentation ready to go for 2.2 than we did 
 for past releases.
 See:
 http://docs.evergreen-ils.org/2.2/

Yes, it's exciting to see this progress! DIG++

Here's a crazy longer-term idea: what if, as part of the release
process, we pulled the documentation repository into the Evergreen
tarball - effectively replacing what we have in the /doc/ directory
today with the DIG documentation? We could do that manually, or
automate it as part of Thomas' release-cutting script, or we could use
git submodules (http://book.git-scm.com/5_submodules.html) to do it
more formally.

The advantage would be that users would get a copy of the Evergreen
documentation with the tarball - and it would be truly single sourced.
(We could also talk about building the HTML  PDF  ePub versions of
the documentation in the distributed tarball as well; depends on how
far we want to go with that).

Developers would need to be able to commit to the DIG repository if we
wanted to continue to maintain the install intructions and release
notes as we do today, and add upgrade instructions to our bailiwick,
but that seems to be a small price to pay for a pretty big win
overall.


Re: [OPEN-ILS-DEV] #help Evergreen 2.2 Release

2012-04-05 Thread Dan Scott
Hi DIG!

On Thu, Apr 5, 2012 at 2:56 PM, Dan Scott d...@coffeecode.net wrote:
 On Tue, Feb 28, 2012 at 2:01 PM, Soulliere, Robert
 robert.soulli...@mohawkcollege.ca wrote:

snip

 Also note that we have upgrade instruction in the official docs here:
 http://docs.evergreen-ils.org/2.2/_upgrading_the_evergreen_server.html
 -- this is based on the alpha release, but if you give me a few days heads 
 up before the official release will be released, I could get those updated 
 to reflect the official tarball version. It might also be good to have a few 
 folks review the upgrade instructions chapter as well.

 Yes, single-sourcing the upgrade instructions would be a really good
 idea too. Just taking a quick peek at the current version, there are a
 number of things that I think we would want to change. Holding an
 official review would be a good idea, I think.

Towards that end, here's a bug with a quick patch I whipped up to
address some issues in the 2.2 upgrade docs:
https://bugs.launchpad.net/evergreen/+bug/974600


[OPEN-ILS-DEV] RELEASE TEAM: OpenSRF 2.1.0 RC1 uploaded

2012-04-03 Thread Dan Scott
Not that I think there is an OpenSRF release team, but if there was you
might be interested in knowing that I've uploaded OpenSRF 2.1.0 RC1 to

http://evergreen-ils.org/downloads/previews/opensrf-2.1.0-alpha1.tar.gz
http://evergreen-ils.org/downloads/previews/opensrf-2.1.0-alpha1.tar.gz.md5

and now with install instructions at:

http://evergreen-ils.org/documentation/install/OpenSRF/README_2_1_0.html

OpenSRF 2.1.0 is a prerequisite for version 2.2.0 of Evergreen, so you
might be interested in trying it out.

Web folks: I've pushed a branch to the Evergreen website repository with
the required changes (I think) to point to 2.1.0-rc1 at
dbs/opensrf_2.1.0-rc1 - commit f4bed652e957d if you just want to
cherry-pick the relevant commit.


[OPEN-ILS-DEV] Planning development hackfest for EG2012

2012-04-03 Thread Dan Scott
Hi all:

I wanted to draw your attention to the
http://www.open-ils.org/dokuwiki/doku.php?id=dev:hackfest:eg2012 wiki
page where we're pulling together some ideas about what we want to work
on, who wants to work on what, and when, for the dev hackfest at the
Evergreen conference this year.

If you're interested in attending, feel free to add to the table in the
wiki:

  * add your name to an existing idea;
  * propose a new idea;
  * associate a bug with an idea and start sketching out your design;
  * link to branches from the bug if there is existing code to build on
  * extend it however it seems appropriate

Of course, you should also feel free to just show up! Some of us are
just hoping to achieve more concrete results from our efforts with this
year's kick at the can :)

Dan


[OPEN-ILS-DEV] Including preferred library in search for located URIs

2012-03-23 Thread Dan Scott
I've been asked to teach search to include the user's preferred search
library (as set in search preferences, falling back to home OU) in
searches, specifically for the purposes of ensuring that located URIs at
the user's preferred library would trigger hits where physical copies
would be out of scope.

Practical example:

1. User sets preferred search library to BR1.

2. They jump onto the catalogue, log in (which changes their search org
   to their preferred search library of BR1), but then for some reason
   change their search org in the org selector to BR3.

3. They issue a search for Harry Potter and the Philosopher's Stone.
   BR3 doesn't hold any copies or have any located URIs, but SYS1 (BR1's
   parent) has a PotterMore licence and has added their 856 $9 SYS1 to a
   bib record.

As it currently stands, User is out of luck; they won't see any hits in
search results as there are no copies or located URIs in the BR3 scope.

The proposed enhancement would, however, make the search results contain
a hit for Harry Potter and the Philosopher's Stone at the user's
preferred search library. With the enhancements to TPAC search results
 record details copy / located URI display in
https://bugs.launchpad.net/evergreen/+bug/907056 the located URI at the
preferred library would then be displayed.

So, two questions:

1) Any strong resistance to the addition of this feature?

2) My rough implementation plan would be:

   a) Add another parameter to search.query_parser_fts() : param_pref_ou INT
  - that would simply concatenate the pref_ou ancestors onto
  luri_org_list (deduping, naturally)

   b) Then make the existing 10-parameter search.query_parser_fts() function
  a wrapper that calls the new 11-parameter function.

   c) Then make O:A:Storage:Publisher::query_parser_fts() capable of calling
  the 11-param function if param_pref_ou has a value.

   d) Then teach callers of open-ils.storage.query_parser_search to pass in
  a pref_ou argument (when appropriate).

Any thoughts / concerns before I sally forth?


Re: [OPEN-ILS-DEV] Modifying the Did you mean *****? vocabulary.

2012-03-23 Thread Dan Scott
Welcome!

I think
http://open-ils.org/dokuwiki/doku.php?id=evergreen-admin:customizations:i18ncovers
this, although that was from a number of years ago.
On Mar 23, 2012 7:24 AM, Olli-Antti Kivilahti olli-antti.kivila...@jns.fi
wrote:

 Is it possible to localize the vocabulary used in search suggestions?
 How is it done?

 --
 Olli-Antti Kivilahti
 Open Library 2013
 Library of Joensuu



[OPEN-ILS-DEV] OpenSRF 2.1.0 alpha1 release in progress

2012-03-11 Thread Dan Scott
Hi all:

If you have an interest in OpenSRF 2.1.0, I've posted alpha1 files at
the following locations:

 * http://evergreen-ils.org/downloads/OpenSRF-ChangeLog-2.1.0-alpha1
 * http://evergreen-ils.org/downloads/opensrf-2.1.0-alpha1.tar.gz
 * http://evergreen-ils.org/downloads/opensrf-2.1.0-alpha1.tar.gz.md5

If you have an interest in Evergreen 2.2, then by definition you have an
interest in OpenSRF 2.1.0, because OpenSRF 2.1.0 will be a prerequisite
for Evergreen 2.2.

So, please step on up and test out OpenSRF 2.1.0; report any bugs to
http://bugs.launchpad.net/opensrf and consider writing up some release
notes (in the wiki or a working branch, whereever you please!).

If you experience success with the release, please post here and let us
know what distribution / version / bitness you were testing on.

Thanks,
Dan


Re: [OPEN-ILS-DEV] system admin training

2012-02-28 Thread Dan Scott
On Tue, Feb 28, 2012 at 04:42:28PM -0500, gislaine.hame...@statcan.gc.ca wrote:
 Hello List,
 
 We hope to be running Evergreen soon. Does anyone know if the training
 for system administration includes Linux and SQL portions? No one in
 our shop has this base knowledge and the training offered here is very
 expensive.

Hi Gislaine:

I just gave a class introducing StatsCan resources to a Research
Methodology students today, so it's only fair I share back...

There's the intro to SQL I wrote a few years ago at
http://bzr.coffeecode.net/intro_to_sql/introduction_to_sql.txt with
a PDF version and accompanying presentation slides at
http://coffeecode.net/archives/212-Introduction-to-SQL-for-Evergreen-administrators.html
which is a fairly gentle introduction (if I do say so myself). Thanks
again to Bibliomation for agreeing to allow me to release this under the
CC-BY-SA license!


[OPEN-ILS-DEV] Google Summer of Code 2012

2012-02-26 Thread Dan Scott
Hi folks:

Per http://www.google-melange.com/gsoc/events/google/gsoc2012 the
period during which organizational applications are accepted for the
Google Summer of Code 2012 is February 27th through March 9th. We're
getting close to the starting gate, and I'd much rather have an
application in early than cut it close to the wire _if_ we want to
participate again this year.

Thus, a quick set of thoughts of what we need to do:

1) Review and refresh our
http://evergreen-ils.org/dokuwiki/doku.php?id=dev:summer_of_coding_ideas
wiki page to ensure that the ideas are still desired, capture a good,
interesting list off the projects that we think are attainable by a
computer science student through the course of a summer, and ensure that
the page itself gives appropriate guidance to prospective students.

Steps that I've taken thus far:

  * Revised the Contacting us section to break it out into
org administrators (Galen and I) and mentors (Thomas and I, so far)

  * Removed the Overhaul config madness entry that Joe Lewis addressed
during last year's GSoC project

  * Revised the i18n project to reflect the predominance of the TPAC and
the potential to standardize further on TPAC for UIs and
internationalization

I've also noted that others have adjusted the page to offer information
on getting started with a new developer's virtual image. Can I suggest
that we break most of that information out into a separate page? I thin
it currently makes the page pretty top-heavy with technical info rather
than introducing potential applicants to our community - and the virtual
image is a valuable resource in and of itself!

I think we should also build on some of the lessons learned from last
year's GSoC experience:

a) State clearly that our goal is to enable students to experience
   success in contributing working code to our project. For example, as
   noted in my reflections on GSoC 2011, I think we made a mistake last
   year in creating separae student repos rather than treating students
   like any other contributor and expecting them to use the working
   repos.

   Based on my discussions with other projects at the GSoC mentors'
   summit, I also think it is more important to get code from the students
   committed early (with accompanying follow-up commits from mentors that
   address small mistakes) than to force them to attain perfection prior
   to submission. Last year we allowed code to drift in branches
   for long periods of time, as we attempted to teach them how to create
   perfect submissions, but that can have a negative effect on morale
   and slow down the willingness to put forward code for integration.

   In short, we want the experience to be positive for the student,
   mentor, and project.

b) State clearly that we expect students to communicate publicly with
   the project, either via blog posts or posts to the mailing list, on a
   regular basis (I would suggest weekly as a minimum bar). And more
   communication should also be occurring between the student and the
   development team on a daily basis through the normal modes of IRC,
   bug tracker, and mailing list.

c) Introduce and enforce a requirement that any student applications
   _must_ submit a patch or point to a branch that addresses some
   problem or adds some small enhancement. Bite-size bugs are good
   candidates. If people want to tag some more bugs as 'bitesize' that
   would be a good use of time. (If you think that this is a barrier to
   applicants, then bingo: that's definitely the goal. If you saw how
   many applications we had to wade through last year, you would
   understand why we want to have more of a barrier to entry this time
   around).

2) We need to firm up the list of willing mentors for this summer.
   Mentors must have time available each week (some estimates suggest
   10 hours / week) to provide assistance to a new developer in the
   project; you must be willing to help nurture a new developer; and
   you must be technically capable of Evergreen development (including
   all of our norms around git usage, etc).

   Prospective mentors are advised to consult the Google Summer of Code
   Mentors Guide at http://en.flossmanuals.net/GSoCMentoring/ (heck, it
   contains a lot of accumulated wisdom about GSoC, it's worth browsing
   if you're just interested in the program).

3) I guess we should confirm that we actually do want to participate
   again this year, although my gut tells me yes we haven't really had
   any developer meetings for quite some time. For what it's worth,
   Galen did ask the Oversight Board and the initial reaction, at least,
   is that they're cool with us participating again.

That's a quick brain dump on what I think we need over the course of the
next week or so, based on my experiences  lessons learned from last
year. Other thoughts would be greatly appreciated!

Dan


Re: [OPEN-ILS-DEV] upgrade sql script

2012-02-09 Thread Dan Scott
On Thu, Feb 9, 2012 at 11:35 AM, Peters, Michael
mrpet...@library.in.gov wrote:
 I could be wrong, but isn't this a job for update_db.sh?

 Something like:

 /home/opensrf/Evergreen/build/tools/update_db.sh localhost evergreen 
 evergreen  (while in Open-ILS/src/sql/Pg)

*RED ALERT* That's not a good way to try and upgrade from version to version.

update_db.sh grabs the largest value from config.upgrade_log and
applies all of the numbered updates in order after that point.

The problem is that if you're currently on rel_2_1, you might have had
upgrades # 705 and #805 backported from master to fix problems that
also exist in rel_2_1. However, if you want to upgrade to rel_2_2,
update_db.sh will say Ahh, you already have everything up to #805, so
I'll start with # 806 - even though you might really need all of #
706 through #804 to be applied for new features that only exist in
rel_2_2 and master. After which - hello brokenness.

update_db.sh is useful if you're a developer syncing up a test
database to the latest version of the database schema, or a production
site running master. Otherwise, I'd stay away.


[OPEN-ILS-DEV] Firefox 10, BibTemplate, and the catalogue - heads up

2012-02-03 Thread Dan Scott
Hi folks:

We discovered today that Firefox 10 (coming to desktops near you over
the past few days) reacts strongly to some things that it always used to
accept without complaint - which can end up in a degraded or completely
broken catalogue, depending on how you've customized it with
BibTemplate.

Long story short, we've been able to figure out that Firefox 10 is
reacting to particular CSS3 selector syntax combinations. One thing that
is known to make it angry is attribute values without quotes; for
example:


* query='datafield[tag=245]' /* breaks in Firefox 10 */
* query='datafield[tag=245]' /* works in Firefox 10 */

We have also determined that CSS3 selector syntax that uses hierarchies
is also broken, at least in some cases. For example:

* query='datafield[tag=245] subfield[code=a]' /* breaks in Firefox 10 */

The recommended workaround is to wrap another dojo.query() function and
select the second element of the CSS2 selector syntax that way; for
example:

td type='opac/slot-data' query='datafield[tag=245]'
script type='opac/slot-format'![CDATA[
dojo.query('subfield[code=a]', item).forEach(function(item) {
/* code you would have previously dumped directly in the script tag */
});
]]/script

This isn't perfect, by any means, but it should get you on the road to
getting Firefox 10 working better than it might be right now. If you
want to see what we're doing at Conifer, you can follow along with our
skins branch at
http://git.evergreen-ils.org/?p=contrib/Conifer.git;a=shortlog;h=refs/heads/feature/skins-bulk_2_1
(I'm trying not to put too much profanity in the commit messages).

Many thanks to Dan Wells for figuring out the CSS3 selector quoting;
this Firefox 10 incompatibility absolutely blindsided us all and it's
always nice to have a calm voice of reason sorting things out :)

Dan Scott


Re: [OPEN-ILS-DEV] Evergreen 2.2 Release

2012-01-23 Thread Dan Scott
On Mon, Jan 23, 2012 at 09:31:56AM -0500, Tim Spindler wrote:
 Thanks Jason,  I appreciate your comments.  I remember the statement about
 the March release but since there hasn't been much talk about the timetable
 and there was still talk about an Alpha 2 release before beta. I'm
 wondering if a March release date is feasible?

We haven't talked about releases much recently, but we haven't been
talking much about anything recently. (See also Ben Shum's question
about dev meetings.)

Assuming we maintain high standards of entry for commits to master, we
should be able to cut a 2.2 release at pretty much any point in time.
Cutting a second alpha within the next week and a beta a few weeks after
that leaves some time for a release candidate or two to catch any
glaring problems before a March release. Oh yeah, and for release notes
and documentation and translations to get caught up too.

In the absence of broadly automated testing, it would be wise to have
manual tests for common scenarios so the community can avoid the kind of
problems Conifer has been experiencing in 2.1. We upgraded to 2.1
over the holiday break and I've been putting in a lot of time
subsequently dealing with upgrade issues [1] that would not be apparent
with data that has been loaded into a fresh database. Hopefully a lesson
learned?

1. Issues such as https://bugs.launchpad.net/evergreen/+bug/920134 and
   https://bugs.launchpad.net/evergreen/+bug/918020


Re: [OPEN-ILS-DEV] Evergreen 2.2 TPac language/locale questions

2012-01-18 Thread Dan Scott
On Wed, Jan 18, 2012 at 2:07 PM, Bill Erickson erick...@esilibrary.com wrote:
 On Wed, Jan 18, 2012 at 01:52:25PM -0500, Soulliere, Robert wrote:
 [...]

 2. When I ran:   sudo -u opensrf xgettext.pl --plugin=tt2 
 --directory=Open-ILS/src/templates/opac/ 
 --output-dir=/openils/var/data/locale --output=tpac.en_CA.po

 to roll my own en_CA.po file for Canadian English I noticed that the label 
 Search the Catalog was not added to the PO file while other labels in the 
 same searchbar.tt2 file were added.

 I could add this label manually to the file, but was wondering  why some 
 seem to get added but others were not even though they seem to use the same 
 template identifiers?

 Hmm, I just gave that a try and it's generating the label for me.

Robert - are you on master, or on 2.2-alpha1?


Re: [OPEN-ILS-DEV] writing tests for OpenSRF methods?

2011-12-28 Thread Dan Scott
On Wed, Dec 28, 2011 at 12:53:17PM -0500, Mike Rylander wrote:
 On Dec 27, 2011 6:35 PM, Dan Scott d...@coffeecode.net wrote:
 
  On Tue, Dec 27, 2011 at 05:12:26PM -0600, Scott Prater wrote:
   What I'd really like to do is write tests for the OpenSRF methods I
   created that simulate as closely as possible the requests made by the
   javascript to the OpenSRF backend, so that I can make sure I cover all
   the possible use cases, get expected responses, and be able to rerun
   the tests whenever any changes are made.
  
   My tests would do all the things normal things tests do:  seed the
   database with test data, execute the methods with some mock objects,
   and compare the responses to other mock objects, then delete the test
   data from the database.
  
   Where would be the best place to put such tests in the source tree?
 
  For functional verification tests like this that would require a
  complete running system, I think a subdirectory under Open-ILS/tests
  would be perfectly appropriate. If you need seed data for bib records,
  copies, call numbers, located URIs, monograph parts, and conjoined items
  you might find Open-ILS/tests/datasets/concerto.sql useful. Sounds like
  that's not the focus of your current efforts, but perhaps a similar
  approach would be useful for seeding the data you need - particularly if
  you need to create historical data such as past circulation history,
  etc, that might not be as easy to create using strict OpenSRF API calls.
 
 
 Outside the (unfortunately, yes) minimal in-EG tests, there's also the
 Constrictor project.  Bill Erickson built this specifically for API testing
 and benchmarking.  It's driven by relatively simple configuration files,
 provides full-stack testing with expected result comparison, and measures
 various timing components of each test.  It has the added benefit of being
 able to control a cluster of test-running clients to simulate load for
 those parts of the code that are load-sensitive, such as optional database
 replication, process-local caching and transaction control.
 
 I'll have to defer to Bill on the current whereabouts of a Constrictor
 repo, though, as even the readonly svn repo from before the age of git
 seems to be missing.

http://svn.open-ils.org/trac/ILS-Contrib/wiki/Constrictor - the
Subversion repo checkout instructions there still work for me.

Drawbacks that come to mind may be that you're dealing with Python, and
nothing else really uses Python in the current Evergreen / OpenSRF
stack, and the constrictor framework isn't integrated in any way with
the core repo, so as APIs and data structures change the constrictor
framework can get out of sync. IIRC from the times I've run constrictor,
the data set that constrictor uses is also up to the user and the user
is responsible for setup and teardown? Don't get me wrong, constrictor
does some great things, but I think those are more on the stress-testing
side than on regression testing.

Being able to run standardized tests after ./autogen.sh  configure 
make  make install  setting up a clean database schema 
osrf_ctl.sh  autogen.sh (assuming preexisting opensrf_core.xml /
opensrf.xml / ejabberd setup - geez, we really do that to people?) via a
simple prove command would be awesome. I'm not sure constrictor is
setup to fit into a TAP harness but maybe it could work too.


Re: [OPEN-ILS-DEV] writing tests for OpenSRF methods?

2011-12-27 Thread Dan Scott
On Tue, Dec 27, 2011 at 05:12:26PM -0600, Scott Prater wrote:
 I've made the changes needed in the admin client (statistical category
 editor) to implement the enhancements to patron statistical categories
 ( see the proposal at
 http://evergreen-ils.org/dokuwiki/doku.php?id=dev:proposal:patron_statistical_categories
 ).  I did some pretty thorough and mind-crushingly dull manual testing
 in the staff admin client to make sure that everything behaved as
 expected in all the possible combinations I could think of different
 org units, categories, entries, etc.

Wow! At first glance that looks like an impressively well-scoped and
thought-out chunk of work!
 
 What I'd really like to do is write tests for the OpenSRF methods I
 created that simulate as closely as possible the requests made by the
 javascript to the OpenSRF backend, so that I can make sure I cover all
 the possible use cases, get expected responses, and be able to rerun
 the tests whenever any changes are made.

 My tests would do all the things normal things tests do:  seed the
 database with test data, execute the methods with some mock objects,
 and compare the responses to other mock objects, then delete the test
 data from the database.
 
 Where would be the best place to put such tests in the source tree?

For functional verification tests like this that would require a
complete running system, I think a subdirectory under Open-ILS/tests
would be perfectly appropriate. If you need seed data for bib records,
copies, call numbers, located URIs, monograph parts, and conjoined items
you might find Open-ILS/tests/datasets/concerto.sql useful. Sounds like
that's not the focus of your current efforts, but perhaps a similar
approach would be useful for seeding the data you need - particularly if
you need to create historical data such as past circulation history,
etc, that might not be as easy to create using strict OpenSRF API calls.

 What would be the preferred way to invoke them?  Are there any
 examples in the current source I can use as a model for writing and
 executing my own tests?  Ideally, this would become part of a larger
 framework of tests that would eventually completely cover the entire
 Evergreen source code.

I can't say they're a perfect model for you in that the existing tests
only exercise static code (that is, when they do anything more than just
ensuring that the code is actually valid syntax), but if you're
comfortable with Perl (as you appear to be), there are basic unit tests
using the Test::More framework in the Open-ILS/src/perlmods/t directory.

(Also, can I say that getting a solid chunk of system tests into the
code base would be the best end-of-year present for Evergreen that I can
imagine?)

Dan


Re: [OPEN-ILS-DEV] MARC Record Import from txt

2011-12-22 Thread Dan Scott
On Thu, Dec 22, 2011 at 05:05:25PM +, Elliot Voris wrote:
 I've made a bit of progress with help from the PostgreSQL mailing list. 
 
 I installed XML::LibXML, and that did indeed clear up the 'ParserDetails.ini' 
 trouble, but now I get another error. This time, it references three specific 
 lines in three separate .pm files.

That suggests that the make -f Open-ILS/src/extras/Makefile.install
distro command, or the corresponding command for OpenSRF, either
failed or was skipped, and/or that the corresponding target for your distro
is broken.
  
 evergreen=# INSERT INTO biblio.record_entry (marc, last_xact_id) SELECT marc, 
 'IMPORT' FROM marcxml_import;
 ERROR:  Tag  is not a valid tag. at /usr/share/perl5/MARC/File/SAX.pm line 
 92 at /usr/local/lib/perl/5.10.1/XML/LibXML/SAX.pm line 99
 at /usr/share/perl5/MARC/File/XML.pm line 445
 CONTEXT:  PL/Perl function maintain_control_numbers
 evergreen=#
  
 I've tried looking at the actual files, Googling it, but it's all Greek to 
 me. I'm attaching the files, hoping that someone could take a look at them. 
 Does this make sense to anyone? Which file/line is the actual problem?

Well, one thing is that you're using an older version of MARC::File::XML
(0.92, whereas 0.93 is what introduces some measure of support for
namespaced elements). Try sudo cpan MARC::File::XML, restart OpenSRF /
Evergreen, and see if that makes a difference.


[OPEN-ILS-DEV] Dev meeting: 2011-12-20?

2011-12-19 Thread Dan Scott
So, according to http://evergreen-ils.org/calendar we're supposed to
have a dev meeting tomorrow (Tuesday, December 20th).

There were a number of items discussed, particularly around release
timing and release teams during the last meeting
(http://evergreen-ils.org/irc_logs/evergreen/2011-12/%23evergreen.06-Tue-2011.log).

Rough action items from last meeting:
  * Send out the revised proposed time-based release schedule and support cycle
for feedback (Anoop Atre)
  * Move OpenSRF 1.6 to code museum from downloads page (?)
  * Ask a DIG representative to join the Release Team (and review
related processes) (Ben Shum)

Any new business to add? Do we still want to have a meeting tomorrow?


  1   2   3   4   5   6   7   8   >