> On 2012-01-06 23:24:17, Alejandro Abdelnur wrote:
> > The following is now being done in several commands:
> >
> > protected String getEntityKey() {
> > if (coordAction != null) {
> > return coordAction.getJobId();
> > }
> > return null;
> > }
> >
> > The entity cannot be NULL or lazy loaded. When the command is created the
> > entity key MUST be provided. this guarding and returning NULL if not
> > present is not correct as the lock is acquired before doing a loadState.
> >
> > So, all these checks should be removed and the entity key should be
> > provided in all XCommand constructors.
> >
> > -------
> >
> > There are still many false changes adding 'this.'
> >
> > -------
> >
> > I'm still questioning the need for adding the lockKey, the entityKey should
> > be enough.
> >
> > If not, please explain why
> >
> > -------
> >
> > Thxs.
> >
> > Alejandro
>
> Mohamed Battisha wrote:
> As far as providing the entity key providers [like the coordAction in
> this case] in the constructor, I changed some of the XCommands to assure this
> is taking place. I am not sure if i did cover all the commands this is why i
> added this check to avoid a null pointer exception. I will eliminate these
> changes and will open another Jira to track this is in a deterministic way,
> meanwhile checking the lock while executing the XCommand instead of at the
> Callable Queue service will assure the entity key is initialized as all of
> them will be initialized prior of calling execute.
>
> As far as the lock key/entity key. The entity key is part of XCommand
> and can not be accessed through the XCallable interface, as such I created
> and interface method in the XCallable interface [getLockKey()] to return the
> the entity key ] which will be used while creating and maintaining the
> interrupt map. This was the simplest way of doing it, other options will
> require a bigger footprint which I prefere to avoid at this point
>
>
> Alejandro Abdelnur wrote:
> on first paragraph, sounds good.
>
> on second paragraph, my concern with 2 keys here is that the lock is
> acquired for the entity-key and the interruption is is keyed with the
> lock-key, and they may be different. thus creating weird issues hard to
> track. bottom line (IMO), the key used for locking MUST be the same as for
> interruption and this could not be changed.
Agreed and this is why we are using the same key
@line 480 of XCommand.Java
* Return the entity Key
* @return the callable Lock key to be used by the interrupt driven map
*/
public String getLockKey() {
return getEntityKey();
}
- Mohamed
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3030/#review4229
-----------------------------------------------------------
On 2012-01-09 19:19:08, Mohamed Battisha wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/3030/
> -----------------------------------------------------------
>
> (Updated 2012-01-09 19:19:08)
>
>
> Review request for oozie.
>
>
> Summary
> -------
>
> Oozie continues to materialize new actions after end date modification. The
> main issue is related how Oozie materialized the actions based on a FIFO
> priority queue. Changing the status of a bundle/coordinator job should take
> higher priority than executing this job.
>
> The main idea is to enable the queue to handle interruptions. Mainly, once
> you finished what you are working on currently, you should focus on executing
> this next action.
>
> The issue can be illustrated as follow:
>
> 1. Configure a pipeline to run for 1 hour
> 2. Start the pipeline
> 3. After it starts materializing new actions, change the end time (in my
> example - to 10 minutes after the pipeline
> starts)
> 4. Monitor the coordinator apps - they will continue to materialize new
> actions past the end time.
>
>
> This addresses bug OOZIE-591.
> https://issues.apache.org/jira/browse/OOZIE-591
>
>
> Diffs
> -----
>
>
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/command/Command.java
> 1228133
>
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/command/XCommand.java
> 1228133
>
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/command/coord/CoordActionInputCheckXCommand.java
> 1228133
>
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/command/coord/CoordActionReadyXCommand.java
> 1228133
>
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/command/coord/CoordActionStartXCommand.java
> 1228133
>
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/command/coord/CoordMaterializeTransitionXCommand.java
> 1228133
>
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/command/coord/CoordRerunXCommand.java
> 1228133
>
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/service/CallableQueueService.java
> 1228133
>
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/service/RecoveryService.java
> 1228133
>
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/util/XCallable.java
> 1228133
>
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/resources/oozie-default.xml
> 1228133
>
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/command/TestCommand.java
> 1228133
>
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/command/coord/TestCoordActionInputCheckXCommand.java
> 1228133
>
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/command/coord/TestCoordActionStartXCommand.java
> 1228133
>
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/service/TestCallableQueueService.java
> 1228133
>
> Diff: https://reviews.apache.org/r/3030/diff
>
>
> Testing
> -------
>
> Regression for all the Commands in addition for specific testing for the
> Interrupt Driven Map
>
>
> Thanks,
>
> Mohamed
>
>
