> On 2011-11-22 23:26:39, Mohammad Islam wrote:
> > Overall, it's an important problem.
> >
> > In every product, we could use some sub-dirs for future (such as lib/
> > conf/). In other words, oozie should look into $SHARE_LIB/pig/lib/ for
> > classes/jars.
> >
> > Alternatively, we can coordinate this with multi-versions. There could be
> > multiple directories like $SHARELIB/pig/default/{lib, conf},
> > $SHARELIB/pig/0.7/{lib, conf}, $SHARELIB/pig/0.8/{lib, conf} etc. I'm
> > leaning towards this way to make it consistent with multi-version support.
> >
> >
>
> Alejandro Abdelnur wrote:
> Yes, could have lib/ & conf/. However this is not a requirement at the
> moment, and the change would be transparent when we need it. Like moving from
> a single lib/ to a per-action one, users apps won't be affected.
>
> Adding multi version support is a good idea. It would also require
> changes in the action XML to specify the version.
>
> Both could be future enhancements, but we should think more the use cases
> and requirements. The current patch addresses an immediate need that could be
> augmented later.
>
> Alejandro Abdelnur wrote:
> I'm not against that, actually I like it. But I do that as a follow up
> feature. Note that, similar from the current single lib/ to multiple dirs
> (one per action) we are not breaking applications, it is transparent, the
> case will be when we ad lib/ and conf/ and versions. BTW, handling version
> requires some thought on how the user will specify the version he/she wants
> to use (in the schema URI, a config property, a new element; this has to be
> thought). Because of this I'd like to do things incrementally, the current
> patch now and a follow up JIRA to discuss your proposal.
>
> Thanks.
>
> Mohammad Islam wrote:
> I think the "use_version" support is considered here.
> Long time back, Virag had a pending patch :
> https://github.com/virag/oozie/commit/0f2c7616a4c8d0f669a3df9ca22edb071744a474#diff-2
> Please review this patch.
>
> We had a long discussion on this design. Main proposal is captured here
> (some of your comments lost in the old JIRA system)
> https://issues.apache.org/jira/browse/OOZIE-555
>
>
>
>
> Alejandro Abdelnur wrote:
> Mmmh (regarding my lost comments).
>
> OK, the idea is similar, the main difference with what I'm proposing is
> that in Virag's patch it requires a new WF schema and that the library paths
> are configured in oozie-site.xml. (BTW, the commit in GH has spurious changes
> that make difficult to see what is a change and what is not)
>
> To achieve similar functionality on top of OOZIE-610 we could:
>
> * Change the sharelib path to contain action library versions, ie:
>
> share/lib/streaming/0.20.204.0/
> share/lib/pig/0.8.0/
> share/lib/pig/0.9.0/
>
> * Have an oozie-site configuration, per action that defines the stable
> version of the library, ie:
>
> oozie.action.stable.library.version.streaming=0.20.204.0
> oozie.action.stable.library.version.pig=0.8.0
>
> * The ActionExecutor getShareLibPostFix() method would have the following
> logic:
>
> getShareLibPostFix(...) {
>
> String version = action.config.get("oozie.action.library.version");
> if (version == null) {
> version = oozie.config.get("oozie.action.stable.library.version");
> }
> if (version != null) {
> //construct the library path using the specified version
> }
> }
>
> This approach does not require changes in the WF schema, instead uses a
> configuration value in case you want a different version than the stable, if
> not it just works for existing WF apps.
>
>
>
> If we are OK with this approach I'll file a JIRA with it.
>
In addition to your proposal, we want to give the user the flexibility to
choose the version *optionally*. User can specify that in wf.xml using
use-version. Basically the effective version will be user-specified version ,
if it is defined in wf.xml. Otherwise, the default one that you mentioned will
be the effective version.
there is already a JIRA on this:
https://issues.apache.org/jira/browse/OOZIE-555
You can add your comments there.
- Mohammad
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2909/#review3446
-----------------------------------------------------------
On 2011-11-22 21:03:19, Alejandro Abdelnur wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/2909/
> -----------------------------------------------------------
>
> (Updated 2011-11-22 21:03:19)
>
>
> Review request for oozie.
>
>
> Summary
> -------
>
> breaks the Oozie sharelib into one subdir per action type
>
>
> This addresses bug OOZIE-610.
> https://issues.apache.org/jira/browse/OOZIE-610
>
>
> Diffs
> -----
>
> /trunk/core/pom.xml 1205165
> /trunk/core/src/main/java/org/apache/oozie/action/ActionExecutor.java
> 1205165
>
> /trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
> 1205165
>
> /trunk/core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java
> 1205165
>
> /trunk/core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java
> 1205165
> /trunk/core/src/main/java/org/apache/oozie/service/WorkflowAppService.java
> 1205165
> /trunk/docs/src/site/twiki/DG_Examples.twiki 1205165
> /trunk/docs/src/site/twiki/DG_QuickStart.twiki 1205165
> /trunk/examples/pom.xml 1205165
> /trunk/examples/src/main/apps/custom-main/job.properties 1205165
> /trunk/examples/src/main/apps/demo/job.properties 1205165
> /trunk/examples/src/main/apps/hadoop-el/job.properties 1205165
> /trunk/examples/src/main/apps/java-main/job.properties 1205165
> /trunk/examples/src/main/apps/pig/job.properties 1205165
> /trunk/examples/src/main/apps/streaming/job.properties 1205165
> /trunk/pom.xml 1205165
> /trunk/sharelib/pig/pom.xml PRE-CREATION
> /trunk/sharelib/pom.xml 1205165
> /trunk/sharelib/streaming/pom.xml PRE-CREATION
> /trunk/src/main/assemblies/distro.xml 1205165
> /trunk/src/main/assemblies/examples.xml 1205165
> /trunk/src/main/assemblies/partial-sharelib.xml PRE-CREATION
> /trunk/src/main/assemblies/sharelib.xml 1205165
>
> Diff: https://reviews.apache.org/r/2909/diff
>
>
> Testing
> -------
>
> built, installed and run pig and streaming examples
>
>
> Thanks,
>
> Alejandro
>
>
