date:20180520

Re: Review Request 64029: OOZIE-2339 Provide an API for writing jobs based on the XSD schemas

2018-05-20 Thread Robert Kanter via Review Board



> On May 20, 2018, 10:14 p.m., Robert Kanter wrote:
> > A bunch on minor things.

Looks good though!


- Robert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64029/#review203465
---


On May 14, 2018, 8:36 p.m., András Piros wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64029/
> ---
> 
> (Updated May 14, 2018, 8:36 p.m.)
> 
> 
> Review request for oozie, Attila Sasvari, Peter Cseh, and Robert Kanter.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> OOZIE-2339 Provide an API for writing jobs based on the XSD schemas
> 
> 
> Diffs
> -
> 
>   client/pom.xml 795a92d4af1a6d96603c99506cd70e8cf8e2c347 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 
> 4abc7502c0c9d8b59ded2baaed30c407ad073008 
>   client/src/main/java/org/apache/oozie/client/ApiJarFactory.java 
> PRE-CREATION 
>   client/src/main/java/org/apache/oozie/client/ApiJarLoader.java PRE-CREATION 
>   client/src/main/java/org/apache/oozie/client/OozieClient.java 
> 4c8175628ba5ee346d97ad984532662032b88acc 
>   core/pom.xml 45880a619e70a1643f4eb5ceeb61abb2090551f1 
>   core/src/main/java/org/apache/oozie/BaseLocalOozieClient.java 
> 38fb006980c0acb582b4f91175be1ced8a1afa79 
>   core/src/main/java/org/apache/oozie/local/LocalOozie.java 
> 9ab646cd53223d903e387eb92aa103a7b6577d0f 
>   core/src/main/java/org/apache/oozie/servlet/BaseJobServlet.java 
> e1bd3cf61aabfa4311881b2944fc9f10ffe9145d 
>   core/src/main/java/org/apache/oozie/servlet/BaseJobsServlet.java 
> d4b08711223356c8d4f58a3391b6b87efa0a2b73 
>   core/src/main/java/org/apache/oozie/servlet/ServletUtilities.java 
> 3eb32d50d07687c136fcc0e467a055bdc1e7ec60 
>   core/src/main/java/org/apache/oozie/servlet/V0JobsServlet.java 
> 1d800949cbff34a95f2ae111364eb8cbc6a5f5fb 
>   core/src/main/java/org/apache/oozie/servlet/V1JobsServlet.java 
> c1ca65fd2f8bf0f3e5c618dddcba3bf52eaf5420 
>   core/src/main/java/org/apache/oozie/servlet/V2ValidateServlet.java 
> b86fa6ace43aead115302aad3ae15368b15f1923 
>   core/src/main/resources/oozie-default.xml 
> 6f8925828090cee29a818de30a7c31db0617d816 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java 
> 3395cc25d27805444ba0b6463357c4ecff977ac1 
>   core/src/test/java/org/apache/oozie/servlet/TestV1JobsServlet.java 
> fb481e367653187a94199069199f1d1b06d1cecf 
>   core/src/test/java/org/apache/oozie/test/MiniOozieTestCase.java 
> 0a030d660f9c545b520afcce458a27ba2a367658 
>   docs/src/site/twiki/DG_CommandLineTool.twiki 
> 9a174596dd78fb15989daf4e88936a9a2b09ef13 
>   docs/src/site/twiki/DG_FluentJobAPI.twiki PRE-CREATION 
>   examples/pom.xml 2bfc7c8b3fc3de4876c62d70f25bed35b886ece8 
>   examples/src/main/java/org/apache/oozie/example/fluentjob/Credentials.java 
> PRE-CREATION 
>   examples/src/main/java/org/apache/oozie/example/fluentjob/Global.java 
> PRE-CREATION 
>   examples/src/main/java/org/apache/oozie/example/fluentjob/JavaMain.java 
> PRE-CREATION 
>   examples/src/main/java/org/apache/oozie/example/fluentjob/Parameters.java 
> PRE-CREATION 
>   examples/src/main/java/org/apache/oozie/example/fluentjob/Shell.java 
> PRE-CREATION 
>   examples/src/main/java/org/apache/oozie/example/fluentjob/Spark.java 
> PRE-CREATION 
>   fluent-job/fluent-job-api/pom.xml PRE-CREATION 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/Condition.java
>  PRE-CREATION 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/GraphVisualization.java
>  PRE-CREATION 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/ModifyOnce.java
>  PRE-CREATION 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/action/ActionAttributes.java
>  PRE-CREATION 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/action/ActionAttributesBuilder.java
>  PRE-CREATION 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/action/Builder.java
>  PRE-CREATION 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/action/ChFSBase.java
>  PRE-CREATION 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/action/ChFSBaseBuilder.java
>  PRE-CREATION 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/action/Chgrp.java
>  PRE-CREATION 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/action/ChgrpBuilder.java
>  PRE-CREATION 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/action/Chmod.java
>  PRE-CREATION 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/action/ChmodBuilder.java
>  PRE-CREATION 
>   
>

Re: Review Request 64029: OOZIE-2339 Provide an API for writing jobs based on the XSD schemas

2018-05-20 Thread Robert Kanter via Review Board


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64029/#review203465
---



A bunch on minor things.


client/src/main/java/org/apache/oozie/cli/OozieCLI.java
Line 29 (original), 30 (patched)


* import



client/src/main/java/org/apache/oozie/cli/OozieCLI.java
Lines 1382 (patched)


add a space after :



client/src/main/java/org/apache/oozie/cli/OozieCLI.java
Lines 1391 (patched)


"API jar was written to "



client/src/main/java/org/apache/oozie/client/ApiJarLoader.java
Lines 43 (patched)


```isFile() also checks that the file exists, so this line isn't 
necessary.



client/src/main/java/org/apache/oozie/client/ApiJarLoader.java
Lines 63 (patched)


Is the 0th item always guaranteed to be the main method?  

We could do this the same way that Oozie Actions do it:

Method mainMethod = mainClass.getMethod("main", String[].class);




client/src/main/java/org/apache/oozie/client/ApiJarLoader.java
Lines 65 (patched)


I feel like there should be a way to check that the underlying class is a 
Workflow class, but I'm not sure off the top of my head.  If not, then it's 
fine as is.



core/pom.xml
Lines 410-414 (patched)


Should this also have a test scope?  As-is, it makes the core depends on a 
test jar for non-test stuff.



core/src/main/resources/oozie-default.xml
Lines 3326 (patched)


When using the Fluent API, how does Oozie choose which HDFS to use?



core/src/test/java/org/apache/oozie/client/TestOozieCLI.java
Line 21 (original), 21 (patched)


* import



docs/src/site/twiki/DG_CommandLineTool.twiki
Lines 109 (patched)


The begining of each of these can be simplified to "based on the supplied 
Fluent Job API jar, a workflow..."



docs/src/site/twiki/DG_CommandLineTool.twiki
Lines 1112 (patched)


remove the sudo



docs/src/site/twiki/DG_CommandLineTool.twiki
Lines 1119 (patched)


remove the sudo



docs/src/site/twiki/DG_CommandLineTool.twiki
Lines 1157 (patched)


remove the sudo



docs/src/site/twiki/DG_CommandLineTool.twiki
Lines 1164 (patched)


remove the sudo



docs/src/site/twiki/DG_FluentJobAPI.twiki
Lines 27 (patched)


"XML is not an ideal way to express..."



docs/src/site/twiki/DG_FluentJobAPI.twiki
Lines 46 (patched)


"It defines dependencies..."



docs/src/site/twiki/DG_FluentJobAPI.twiki
Lines 47 (patched)


"... scientists think."



docs/src/site/twiki/DG_FluentJobAPI.twiki
Lines 60 (patched)


"The following points are not targeted for the initial release..."



docs/src/site/twiki/DG_FluentJobAPI.twiki
Lines 61 (patched)


"... other than Java."



docs/src/site/twiki/DG_FluentJobAPI.twiki
Lines 70 (patched)


"Older XSD versions, as ..."



docs/src/site/twiki/DG_FluentJobAPI.twiki
Lines 77 (patched)


"... the following points..."



docs/src/site/twiki/DG_FluentJobAPI.twiki
Lines 88 (patched)


"A Simple Example"



examples/src/main/java/org/apache/oozie/example/fluentjob/Credentials.java
Lines 1 (patched)


All of the examples need the Apache License header



fluent-job/fluent-job-api/pom.xml
Lines 122-131 (patched)


Shouldn't these be the newest 1.x XSDs?



fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/action/ActionAttributes.java
Lines 105 (patched)


Missing Javadoc



fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/action/ActionAttributes.java
Lines 229-243 (patched)


Missing Javadocs

[jira] Subscription: Oozie Patch Available

2018-05-20 Thread jira

Issue Subscription
Filter: Oozie Patch Available (104 issues)

Subscriber: ooziedaily

Key Summary
OOZIE-3249  [tools] Instrumentation log parser
https://issues.apache.org/jira/browse/OOZIE-3249
OOZIE-3237  Flaky test TestZKLocksService#testWriteReadLockThreads
https://issues.apache.org/jira/browse/OOZIE-3237
OOZIE-3221  Rename DEFAULT_LAUNCHER_MAX_ATTEMPS
https://issues.apache.org/jira/browse/OOZIE-3221
OOZIE-3218  Oozie Sqoop action with command splits the select clause into 
multiple parts due to delimiter being space
https://issues.apache.org/jira/browse/OOZIE-3218
OOZIE-3199  Let system property restriction configurable
https://issues.apache.org/jira/browse/OOZIE-3199
OOZIE-3196  Authorization: restrict world readability by user
https://issues.apache.org/jira/browse/OOZIE-3196
OOZIE-3194  Oozie should set proper permissions to sharelib after upload
https://issues.apache.org/jira/browse/OOZIE-3194
OOZIE-3193  Applications are not killed when submitted via subworkflow
https://issues.apache.org/jira/browse/OOZIE-3193
OOZIE-3186  Oozie is unable to use configuration linked using jceks://file/...
https://issues.apache.org/jira/browse/OOZIE-3186
OOZIE-3185  Conflicting JARs org.apache.derby exist in Oozie
https://issues.apache.org/jira/browse/OOZIE-3185
OOZIE-3179  Adding a configurable config-default.xml location to a workflow
https://issues.apache.org/jira/browse/OOZIE-3179
OOZIE-3170  Oozie Diagnostic Bundle tool fails with NPE due to missing service 
class
https://issues.apache.org/jira/browse/OOZIE-3170
OOZIE-3135  Configure log4j2 in SqoopMain
https://issues.apache.org/jira/browse/OOZIE-3135
OOZIE-3109  Escape log-streaming's HTML-specific characters
https://issues.apache.org/jira/browse/OOZIE-3109
OOZIE-3094  fix for grammar mistake
https://issues.apache.org/jira/browse/OOZIE-3094
OOZIE-3091  Oozie Sqoop Avro Import fails with "java.lang.NoClassDefFoundError: 
org/apache/avro/mapred/AvroWrapper"
https://issues.apache.org/jira/browse/OOZIE-3091
OOZIE-3071  Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 
than Spark 2.2.0
https://issues.apache.org/jira/browse/OOZIE-3071
OOZIE-3063  Sanitizing variables that are part of openjpa.ConnectionProperties
https://issues.apache.org/jira/browse/OOZIE-3063
OOZIE-3062  Set HADOOP_CONF_DIR for spark action
https://issues.apache.org/jira/browse/OOZIE-3062
OOZIE-3061  Kill only those child jobs which are not already killed
https://issues.apache.org/jira/browse/OOZIE-3061
OOZIE-3002  address findbugs errors in client lib
https://issues.apache.org/jira/browse/OOZIE-3002
OOZIE-2975  code clean up in pig sharelib, replace Exception with more 
explicit, add try with resources, StringBuilder instead of StringBuffer
https://issues.apache.org/jira/browse/OOZIE-2975
OOZIE-2956  Fix Findbugs warnings related to reliance on default encoding in 
oozie-core
https://issues.apache.org/jira/browse/OOZIE-2956
OOZIE-2955  Fix Findbugs warnings related to reliance on default encoding in 
oozie-client
https://issues.apache.org/jira/browse/OOZIE-2955
OOZIE-2954  Fix Checkstyle issues in oozie-client
https://issues.apache.org/jira/browse/OOZIE-2954
OOZIE-2953  Fix Checkstyle issues in oozie-tools
https://issues.apache.org/jira/browse/OOZIE-2953
OOZIE-2952  Fix Findbugs warnings in oozie-sharelib-oozie
https://issues.apache.org/jira/browse/OOZIE-2952
OOZIE-2949  Escape quotes whitespaces in Sqoop  field
https://issues.apache.org/jira/browse/OOZIE-2949
OOZIE-2942  [examples] Fix Findbugs warnings
https://issues.apache.org/jira/browse/OOZIE-2942
OOZIE-2927  Append new line character for Hive2 query using query tag
https://issues.apache.org/jira/browse/OOZIE-2927
OOZIE-2883  OOZIE throw the error "Missing 
[oozie.service.ProxyUserService.proxyuser.oozie.service.ProxyUserService.proxyuser.mr.groups]
 property"
https://issues.apache.org/jira/browse/OOZIE-2883
OOZIE-2877  Oozie Git Action
https://issues.apache.org/jira/browse/OOZIE-2877
OOZIE-2867  Timezone handling for Coordinators: emphasize "Continent/City" 
format
https://issues.apache.org/jira/browse/OOZIE-2867
OOZIE-2834  ParameterVerifier logging non-useful warning for workflow definition
https://issues.apache.org/jira/browse/OOZIE-2834
OOZIE-2833  when using uber mode the regex pattern used in the 
extractHeapSizeMB method does not allow heap sizes specified in bytes.
https://issues.apache.org/jira/browse/OOZIE-2833
OOZIE-2829  Improve sharelib upload to accept multiple source folders
https://issues.apache.org/jira/browse/OOZIE-2829
OOZIE-2826  Falcon feed fails to aws s3; Oozie joda

Re: Review Request 64029: OOZIE-2339 Provide an API for writing jobs based on the XSD schemas

Re: Review Request 64029: OOZIE-2339 Provide an API for writing jobs based on the XSD schemas

[jira] Subscription: Oozie Patch Available

3 matches

Site Navigation

Mail list logo

Footer information