Re: [Dspace-devel] [DSpace-changelog] r5076 - sandbox/aip-external-1_6-prototype/dspace-api/src/main/java/org/dspace/content/packager

2010-06-11 Thread Mark H. Wood
On Thu, Jun 10, 2010 at 11:45:49AM -0700, Mark Diggory wrote:
 Did you look over the Quartz Job Scheduler Addon, this type of asynchronous 
 thread spawning activity is what I intended it to alleviate us from doing.

I had not thought of it.  I was thinking to get the export done
immediately, not schedule it for later.  But later could be now +
1µs.  Pleased as I was with my spawning and monitoring code, I'd be
more pleased to ditch it and let someone else manage all that hair.
Thanks for the nudge.

But I was never happy with spawning subprocesses at all.  I felt
compelled to run the packager from the commandline because the only
otherwise suitable method for directly calling it is 'protected' and
thus not visible from my code.  But it was pointed out to me that I
can just do what it does: ask the PluginManager for a reference to the
AIP plugin and call *that* directly, since the plugin's methods are
'public'.  The code has been updated to do that.  This also removed a
dodgy path assumption.

The subprocess approach was also unsatisfactory because of the
potential to fork-bomb the system when batch loading large quantities
of items, or making other bulk changes.  Using a job scheduler could
have tamed that.

-- 
Mark H. Wood, Lead System Programmer   mw...@iupui.edu
Balance your desire for bells and whistles with the reality that only a 
little more than 2 percent of world population has broadband.
-- Ledford and Tyler, _Google Analytics 2.0_


pgpUiZTaiGt1U.pgp
Description: PGP signature
--
ThinkGeek and WIRED's GeekDad team up for the Ultimate 
GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the 
lucky parental unit.  See the prize list and enter to win: 
http://p.sf.net/sfu/thinkgeek-promo___
Dspace-devel mailing list
Dspace-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-devel


Re: [Dspace-devel] [DSpace-changelog] r5076 - sandbox/aip-external-1_6-prototype/dspace-api/src/main/java/org/dspace/content/packager

2010-06-10 Thread Mark Diggory
MarkW,

Did you look over the Quartz Job Scheduler Addon, this type of asynchronous 
thread spawning activity is what I intended it to alleviate us from doing.

MarkD

On Jun 9, 2010, at 9:49 AM, nore...@scm.dspace.org wrote:

 Author: mwoodiupui
 Date: Wed Jun  9 16:49:36 2010
 New Revision: 5076
 
 Log:
 Add an event consumer to drive the packager, producing AIPs for affected 
 objects
 
 Added:
   
 sandbox/aip-external-1_6-prototype/dspace-api/src/main/java/org/dspace/content/packager/PackageConsumer.java
 
 Added: 
 sandbox/aip-external-1_6-prototype/dspace-api/src/main/java/org/dspace/content/packager/PackageConsumer.java
 ==
 --- (empty file)
 +++ 
 sandbox/aip-external-1_6-prototype/dspace-api/src/main/java/org/dspace/content/packager/PackageConsumer.java
   Wed Jun  9 16:49:36 2010
 @@ -0,0 +1,219 @@
 +/**
 + * PackageConsumer.java
 + *
 + * Version: $Revision$
 + *
 + * Date: $Date$
 + *
 + * Copyright (c) 2010, The DSpace Foundation.  All rights reserved.
 + *
 + * Redistribution and use in source and binary forms, with or without
 + * modification, are permitted provided that the following conditions are
 + * met:
 + *
 + * - Redistributions of source code must retain the above copyright
 + * notice, this list of conditions and the following disclaimer.
 + *
 + * - Redistributions in binary form must reproduce the above copyright
 + * notice, this list of conditions and the following disclaimer in the
 + * documentation and/or other materials provided with the distribution.
 + *
 + * - Neither the name of the DSpace Foundation nor the names of its
 + * contributors may be used to endorse or promote products derived from
 + * this software without specific prior written permission.
 + *
 + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
 + * ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
 + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
 + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
 + * HOLDERS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
 + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
 + * BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS
 + * OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
 + * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR
 + * TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE
 + * USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH
 + * DAMAGE.
 + */
 +
 +package org.dspace.content.packager;
 +
 +import java.io.BufferedReader;
 +import java.io.File;
 +import java.io.IOException;
 +import java.io.InputStreamReader;
 +import java.util.ArrayList;
 +import java.util.HashMap;
 +import java.util.List;
 +import java.util.Map;
 +import java.util.Map.Entry;
 +
 +import org.dspace.core.ConfigurationManager;
 +import org.dspace.core.Constants;
 +import org.dspace.core.Context;
 +import org.dspace.event.Consumer;
 +import org.dspace.event.Event;
 +import org.slf4j.Logger;
 +import org.slf4j.LoggerFactory;
 +
 +/**
 + * An Event Consumer which creates AIPs as the DSpace model changes. This is
 + * done by invoking the packager utility in subprocesses, one per affected
 + * object. Any console output from the packager is logged.
 + * p
 + * If a single operation (from the user's point of view) causes multiple
 + * update()s of a given object, then some objects may be exported more than 
 once
 + * per user operation, but no object should be exported more than once per
 + * update(). See the object implementations to understand how this happens. 
 In
 + * other words, over time the number of exports is not minimal, but I think 
 it
 + * is as close as we can get.
 + * p
 + * It is mandatory to configure {...@code packageConsumer.workingDirectory} 
 when
 + * employing this class; otherwise {...@code initialize} will throw a
 + * NullPointerException.
 + * 
 + * @author Mark Wood
 + */
 +public class PackageConsumer implements Consumer
 +{
 +/** Log file access */
 +private static final Logger log = LoggerFactory
 +.getLogger(PackageConsumer.class);
 +
 +/** Configuration property: subprocess' working directory. REQUIRED. */
 +private static final String WORKING_DIRECTORY = 
 packageConsumer.workingDirectory;
 +
 +/**
 + * Creates pre-configured subprocesses to execute external packaging
 + * commands.
 + */
 +private ProcessBuilder processFactory;
 +
 +/** Accumulator for unique objects to be exported */
 +private MapString, Integer objects;
 +
 +/** Path to the external packager utility */
 +private String verb;
 +
 +public void initialize() throws Exception
 +{
 +verb = ConfigurationManager.getProperty(dspace.home) + 
 /bin/dspace;
 +objects = new HashMapString, Integer();
 +
 +