Re: Review Request 49074: OOZIE-2582: Populating external child Ids for action failures

2016-09-23 Thread Abhishek Bafna 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49074/
---

(Updated Sept. 24, 2016, 6:42 a.m.)


Review request for oozie.


Bugs: OOZIE-2582
https://issues.apache.org/jira/browse/OOZIE-2582


Repository: oozie-git


Description
---

Currently Oozie external child ids are populated into workflow bean, when the 
job/action completes successfully. It should populate external child ids in 
case of job failures as well.


Diffs (updated)
-

  core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 
96726da 
  core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 
b5b1bf9 
  core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 
c74e9e6 
  core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
ad07b57 
  core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 
15a641b 
  core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 
6813a37 
  core/src/test/java/org/apache/oozie/action/hadoop/TestDistcpMain.java 098995e 
  examples/src/main/apps/hive/script.q 3abc757 
  sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 
6ac5ad6 
  sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 
8de3766 
  
sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java
 b966d4b 
  sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 
d72e298 
  sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java 
e122608 
  
sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java
 5963e42 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 
054f8ea 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java 
5e80d00 
  sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 
44debbd 
  
sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java
 a5291d9 
  sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java 
38e8e8c 
  sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 
623fd2e 
  
sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java
 6474092 

Diff: https://reviews.apache.org/r/49074/diff/


Testing
---


Thanks,

Abhishek Bafna

Re: Review Request 49074: OOZIE-2582: Populating external child Ids for action failures

2016-09-23 Thread Abhishek Bafna 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49074/
---

(Updated Sept. 23, 2016, 4:16 p.m.)


Review request for oozie.


Bugs: OOZIE-2582
https://issues.apache.org/jira/browse/OOZIE-2582


Repository: oozie-git


Description
---

Currently Oozie external child ids are populated into workflow bean, when the 
job/action completes successfully. It should populate external child ids in 
case of job failures as well.


Diffs (updated)
-

  core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 
96726da 
  core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 
b5b1bf9 
  core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 
c74e9e6 
  core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
ad07b57 
  core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 
15a641b 
  core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 
6813a37 
  core/src/test/java/org/apache/oozie/action/hadoop/TestDistcpMain.java 098995e 
  examples/src/main/apps/hive/script.q 3abc757 
  sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 
6ac5ad6 
  sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 
8de3766 
  
sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java
 b966d4b 
  sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 
d72e298 
  
sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java
 5963e42 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 
054f8ea 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java 
5e80d00 
  sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 
44debbd 
  
sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java
 a5291d9 
  sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java 
38e8e8c 
  sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 
623fd2e 
  
sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java
 6474092 

Diff: https://reviews.apache.org/r/49074/diff/


Testing
---


Thanks,

Abhishek Bafna

Re: Review Request 49074: OOZIE-2582: Populating external child Ids for action failures

2016-09-20 Thread Abhishek Bafna 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49074/
---

(Updated Sept. 20, 2016, 7:21 p.m.)


Review request for oozie.


Bugs: OOZIE-2582
https://issues.apache.org/jira/browse/OOZIE-2582


Repository: oozie-git


Description
---

Currently Oozie external child ids are populated into workflow bean, when the 
job/action completes successfully. It should populate external child ids in 
case of job failures as well.


Diffs (updated)
-

  core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 
96726da 
  core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 
b5b1bf9 
  core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 
c74e9e6 
  core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
ad07b57 
  core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 
15a641b 
  core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 
6813a37 
  examples/src/main/apps/hive/script.q 3abc757 
  sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 
6ac5ad6 
  sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 
8de3766 
  
sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java
 b966d4b 
  sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 
d72e298 
  
sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java
 5963e42 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 
054f8ea 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java 
5e80d00 
  sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 
44debbd 
  
sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java
 a5291d9 
  sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java 
38e8e8c 
  sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 
623fd2e 
  
sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java
 6474092 

Diff: https://reviews.apache.org/r/49074/diff/


Testing
---


Thanks,

Abhishek Bafna

Re: Review Request 49074: OOZIE-2582: Populating external child Ids for action failures

2016-07-27 Thread Abhishek Bafna 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49074/
---

(Updated July 28, 2016, 5:09 a.m.)


Review request for oozie.


Bugs: OOZIE-2582
https://issues.apache.org/jira/browse/OOZIE-2582


Repository: oozie-git


Description
---

Currently Oozie external child ids are populated into workflow bean, when the 
job/action completes successfully. It should populate external child ids in 
case of job failures as well.


Diffs (updated)
-

  examples/src/main/apps/hive/script.q 3abc757 
  sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 
6ac5ad6 
  sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 
d8ce232 
  sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 
d72e298 
  
sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java
 5963e42 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 
fb190d6 
  sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 
b0c7635 

Diff: https://reviews.apache.org/r/49074/diff/


Testing
---


Thanks,

Abhishek Bafna

Re: Review Request 49074: OOZIE-2582: Populating external child Ids for action failures

2016-07-10 Thread Abhishek Bafna 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49074/
---

(Updated July 10, 2016, 3:10 p.m.)


Review request for oozie.


Bugs: OOZIE-2582
https://issues.apache.org/jira/browse/OOZIE-2582


Repository: oozie-git


Description
---

Currently Oozie external child ids are populated into workflow bean, when the 
job/action completes successfully. It should populate external child ids in 
case of job failures as well.


Diffs (updated)
-

  examples/src/main/apps/hive/script.q 3abc757 
  sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 
6ac5ad6 
  sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 
cb0172d 
  sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 
d72e298 
  
sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java
 5963e42 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 
fb190d6 
  sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 
360cf1e 

Diff: https://reviews.apache.org/r/49074/diff/


Testing
---


Thanks,

Abhishek Bafna

Re: Review Request 49074: OOZIE-2582: Populating external child Ids for action failures

2016-07-08 Thread Abhishek Bafna 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49074/
---

(Updated July 8, 2016, 9:45 a.m.)


Review request for oozie.


Changes
---

Uploading patch after incorporating suggestions by Satish.


Bugs: OOZIE-2582
https://issues.apache.org/jira/browse/OOZIE-2582


Repository: oozie-git


Description
---

Currently Oozie external child ids are populated into workflow bean, when the 
job/action completes successfully. It should populate external child ids in 
case of job failures as well.


Diffs (updated)
-

  examples/src/main/apps/hive/script.q 3abc757 
  sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 
6ac5ad6 
  sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 
cb0172d 
  sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 
d72e298 
  
sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java
 5963e42 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 
fb190d6 
  sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 
360cf1e 

Diff: https://reviews.apache.org/r/49074/diff/


Testing
---


Thanks,

Abhishek Bafna

Re: Review Request 49074: OOZIE-2582: Populating external child Ids for action failures

2016-07-08 Thread Abhishek Bafna 


> On July 7, 2016, 8:57 p.m., Satish Saley wrote:
> > sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java,
> >  line 41
> > 
> >
> > We don't need "Job complete" pattern. It will end up with fetching 
> > child job id twice and in UI will contain duplicate urls. Only "Submitted 
> > application" is sufficient.

I agree with you. To prevent the duplicates, I added a conditional check in the 
LauncherMain, which checks the ids first and if not present alreadt, then 
appends. I have removed the above patterns. I will marking other two also as 
Fixed. Thanks.


> On July 7, 2016, 8:57 p.m., Satish Saley wrote:
> > sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java,
> >  line 96
> > 
> >
> > It became clear after discussing with Rohini, we will write only to 
> > ACTION_DATA_EXTERNAL_CHILD_IDS.
> > 
> > And for backward compatibility, we need to make changes in the 
> > JavaActionExecutor to read from ACTION_DATA_OUTPUT_PROPS as well. Also, we 
> > need to remove getCaptureOutput() and getActionData() methods from 
> > executors. 
> > I will fix those things as part of OOZIE-2561

As per the changes here, I think it is not braeking any backward compatibility 
things. I do observe that, currently with the successful job, external child 
ids are set twice, once using the ACTION_DATA_EXTERNAL_CHILD_IDS and later in 
the getActionData();. I guess, that is what you wanted to say for, 
getActionData(). I did not get the getCaptureOutput() output part in your 
comment. Thanks.


- Abhishek


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49074/#review141204
---


On July 7, 2016, 5:41 a.m., Abhishek Bafna wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49074/
> ---
> 
> (Updated July 7, 2016, 5:41 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2582
> https://issues.apache.org/jira/browse/OOZIE-2582
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Currently Oozie external child ids are populated into workflow bean, when the 
> job/action completes successfully. It should populate external child ids in 
> case of job failures as well.
> 
> 
> Diffs
> -
> 
>   examples/src/main/apps/hive/script.q 3abc757 
>   
> sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 
> 6ac5ad6 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 
> cb0172d 
>   
> sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 
> d72e298 
>   
> sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java
>  5963e42 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 
> fb190d6 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 
> 360cf1e 
> 
> Diff: https://reviews.apache.org/r/49074/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Abhishek Bafna
> 
>

Re: Review Request 49074: OOZIE-2582: Populating external child Ids for action failures

2016-07-07 Thread Satish Saley 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49074/#review141204
---




sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 
(line 41)


We don't need "Job complete" pattern. It will end up with fetching child 
job id twice and in UI will contain duplicate urls. Only "Submitted 
application" is sufficient.



sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java (line 
46)


same as above. Only "Submitted application" should be sufficient.



sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 
(line 96)


It became clear after discussing with Rohini, we will write only to 
ACTION_DATA_EXTERNAL_CHILD_IDS.

And for backward compatibility, we need to make changes in the 
JavaActionExecutor to read from ACTION_DATA_OUTPUT_PROPS as well. Also, we need 
to remove getCaptureOutput() and getActionData() methods from executors. 
I will fix those things as part of OOZIE-2561



sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 
(line 41)


Only "Submitted application" is sufficient, else will have duplicated links


- Satish Saley


On July 6, 2016, 10:41 p.m., Abhishek Bafna wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49074/
> ---
> 
> (Updated July 6, 2016, 10:41 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2582
> https://issues.apache.org/jira/browse/OOZIE-2582
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Currently Oozie external child ids are populated into workflow bean, when the 
> job/action completes successfully. It should populate external child ids in 
> case of job failures as well.
> 
> 
> Diffs
> -
> 
>   examples/src/main/apps/hive/script.q 3abc757 
>   
> sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 
> 6ac5ad6 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 
> cb0172d 
>   
> sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 
> d72e298 
>   
> sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java
>  5963e42 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 
> fb190d6 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 
> 360cf1e 
> 
> Diff: https://reviews.apache.org/r/49074/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Abhishek Bafna
> 
>

Re: Review Request 49074: OOZIE-2582: Populating external child Ids for action failures

2016-07-06 Thread Abhishek Bafna 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49074/
---

(Updated July 7, 2016, 5:41 a.m.)


Review request for oozie.


Bugs: OOZIE-2582
https://issues.apache.org/jira/browse/OOZIE-2582


Repository: oozie-git


Description
---

Currently Oozie external child ids are populated into workflow bean, when the 
job/action completes successfully. It should populate external child ids in 
case of job failures as well.


Diffs (updated)
-

  examples/src/main/apps/hive/script.q 3abc757 
  sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 
6ac5ad6 
  sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 
cb0172d 
  sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 
d72e298 
  
sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java
 5963e42 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 
fb190d6 
  sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 
360cf1e 

Diff: https://reviews.apache.org/r/49074/diff/


Testing
---


Thanks,

Abhishek Bafna