date:20161020

[jira] [Created] (OOZIE-2716) Oozie Truncates Workflow Parameter Values which contain '}}'

2016-10-20 Thread Ruben Aguilar (JIRA)

Ruben Aguilar created OOZIE-2716:


 Summary: Oozie Truncates Workflow Parameter Values which contain 
'}}'
 Key: OOZIE-2716
 URL: https://issues.apache.org/jira/browse/OOZIE-2716
 Project: Oozie
  Issue Type: Bug
Affects Versions: 4.1.0
 Environment: Using Oozie 4.1.0-cdh5.8.2, Java 1.8.0_60-b27
Reporter: Ruben Aguilar


When submitting a workflow job to oozie through the web api, if the value of a 
substituted workflow parameter contains a "}}" then the rest of the value is 
truncated. This happens quite often which json data. I tried delimiting by 
}} => 
}} => \}\}
}} => '}''}'
but none of these worked.

Example:

Expected value received in jar (ending double braces):
{"hdfsArtifacts":null,"azureSqlArtifacts":null,"osdbArtifacts":{"username":"","password":"","server":"","database":"","schema":"","scriptsPath":"","outputType":""}}

Actual value received in jar (lacking ending double braces):
{"hdfsArtifacts":null,"azureSqlArtifacts":null,"osdbArtifacts":{"username":"","password":"","server":"","database":"","schema":"","scriptsPath":"","outputType":""

workflow.xml

  
  

  ${jobTracker}
  ${nameNode}
  ${master}
  cluster
  ${jobName}
  ${classToExecute}
  ${nameNode}${executableJarPath}
  
  ${artifactSpecification}



  
  
Spark Action failed, error 
message[${wf:errorMessage(wf:lastErrorNode())}]
  
  


Configuration sent to web api:

  
jobName
end-2-end-test
  
  
oozie.use.system.libpath
True
  
  
executableJarPath

/oozie-jars/e2e-job-test/version-14/E2EJobTest-assembly-1.20.jar
  
  
user.name
oozie
  
  
mapreduce.job.user.name
oozie
  
  
master
yarn-cluster
  
  
oozie.wf.application.path

hdfs://some-hdfs:8020/oozie-workflow/e2e-job-test/version-14/workflow.xml
  
  
artifactSpecification

{"hdfsArtifacts":null,"azureSqlArtifacts":null,"osdbArtifacts":{"username":"","password":"","server":"","database":"","schema":"","scriptsPath":"","outputType":""}}
  
  
jobTracker
some-namenode:8032
  
  
nameNode
hdfs://some-namenode:8020
  
  
classToExecute
E2EJob
  





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 52227: OOZIE-2225 Add wild card filter for gathering jobs

2016-10-20 Thread Sai Chirravuri



> On Oct. 18, 2016, 8:57 p.m., Sai Chirravuri wrote:
> > Any comments please ?

Just checking again !


- Sai


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52227/#review153170
---


On Oct. 14, 2016, 6:45 p.m., Sai Chirravuri wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52227/
> ---
> 
> (Updated Oct. 14, 2016, 6:45 p.m.)
> 
> 
> Review request for oozie and Robert Kanter.
> 
> 
> Bugs: OOZIE-2225
> https://issues.apache.org/jira/browse/OOZIE-2225
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> commit 05c108f61f4c3da437d826b73251e92662a4518b
> Author: krish 
> Date:   Fri Sep 23 13:50:24 2016 -0700
> 
> OOZIE-2225 Add wild card filter for gathering jobs
> 
> :100644 100644 f1d0f2b... f10de36... M  
> client/src/main/java/org/apache/oozie/cli/OozieCLI.java
> :100644 100644 a882cab... 53a6ecf... M  
> client/src/main/java/org/apache/oozie/client/OozieClient.java
> :100644 100644 01aeddf... 7597142... M  
> core/src/main/java/org/apache/oozie/DagEngine.java
> :100644 100644 3c03267... 275aabf... M  
> core/src/main/java/org/apache/oozie/executor/jpa/WorkflowsJobGetJPAExecutor.java
> :100644 100644 fb1db29... 80f0756... M  
> core/src/main/java/org/apache/oozie/store/StoreStatusFilter.java
> :00 100644 000... dd1e943... A  
> core/src/main/java/org/apache/oozie/util/db/JPAUtils.java
> :100644 100644 0b1f644... 87fccd6... M  
> core/src/test/java/org/apache/oozie/executor/jpa/TestBundleJobInfoGetJPAExecutor.java
> :100644 100644 7d468f1... 4378286... M  
> core/src/test/java/org/apache/oozie/executor/jpa/TestCoordJobInfoGetJPAExecutor.java
> :100644 100644 ddb5506... 12cb16b... M  
> core/src/test/java/org/apache/oozie/executor/jpa/TestWorkflowsJobGetJPAExecutor.java
> :100644 100644 ea778bd... 2fed1f8... M  
> core/src/test/java/org/apache/oozie/test/XDataTestCase.java
> :100644 100644 567e260... 3daf38a... M  
> docs/src/site/twiki/DG_CommandLineTool.twiki
> :100644 100644 13e1691... a66e564... M  
> docs/src/site/twiki/WebServicesAPI.twiki
> 
> 
> Diffs
> -
> 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 
> f1d0f2b7442e04e01bea93622cf378fc92934497 
>   client/src/main/java/org/apache/oozie/client/OozieClient.java 
> a882cab95c4b75c3fc641fb8e5511357d482eeff 
>   core/src/main/java/org/apache/oozie/CoordinatorEngine.java 
> cec6347905233d9a5fe0eb0e2fa27336a6a967a9 
>   core/src/main/java/org/apache/oozie/DagEngine.java 
> 01aeddfc282261d9ad396565a9d3d30efec2dc59 
>   
> core/src/main/java/org/apache/oozie/executor/jpa/WorkflowsJobGetJPAExecutor.java
>  3c032678d06641ae7bb2f2a9c0631948190ea662 
>   core/src/main/java/org/apache/oozie/store/StoreStatusFilter.java 
> fb1db29cdfe1dbe37c8e2176079f39f841232ee3 
>   core/src/main/java/org/apache/oozie/util/JobsFilterUtils.java 
> 18df836713918a2864566b56646f711dd84a5f97 
>   
> core/src/test/java/org/apache/oozie/executor/jpa/TestBundleJobInfoGetJPAExecutor.java
>  0b1f644c1ff74ce24b78eadf0ed7c0cbc87e93a3 
>   
> core/src/test/java/org/apache/oozie/executor/jpa/TestCoordJobInfoGetJPAExecutor.java
>  7d468f16981710d96473155df25b8de0d5ef322b 
>   
> core/src/test/java/org/apache/oozie/executor/jpa/TestWorkflowsJobGetJPAExecutor.java
>  ddb5506de90ae32d760657f8a867ab46599f1a86 
>   docs/src/site/twiki/DG_CommandLineTool.twiki 
> 567e260dd3e05c7619d957520ed49882b0cf37f2 
>   docs/src/site/twiki/WebServicesAPI.twiki 
> 13e1691c3497bed94957ef959e16e788d1196c67 
> 
> Diff: https://reviews.apache.org/r/52227/diff/
> 
> 
> Testing
> ---
> 
> CLI manual testing looks good and unit tests passed
> 
> 
> Thanks,
> 
> Sai Chirravuri
> 
>

[jira] [Commented] (OOZIE-2616) Add Tez profile for Hive and Pig sharelibs

2016-10-20 Thread Thomas Poepping (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15592469#comment-15592469
 ] 

Thomas Poepping commented on OOZIE-2616:


Hello, ping on this issue?

> Add Tez profile for Hive and Pig sharelibs
> --
>
> Key: OOZIE-2616
> URL: https://issues.apache.org/jira/browse/OOZIE-2616
> Project: Oozie
>  Issue Type: Improvement
>Affects Versions: trunk
>Reporter: Thomas Poepping
>Assignee: Thomas Poepping
> Attachments: OOZIE-2616-1.patch, OOZIE-2616-2.patch, 
> OOZIE-2616-3.patch
>
>
> More recent versions of Hive and Pig depend on Tez as their default execution 
> engine. As the sharelib for those applications stands today, those 
> applications are unable to run Tez jobs. This patch adds an optional profile 
> to the root pom that will include the proper Tez dependencies into those 
> sharelibs, while excluding problematic jars to keep the sharelibs from having 
> classpath duplication issues. *Note*: this patch will only be required for 
> those clients who specify their own, newer versions of Hive or Pig. This 
> change is not required for the current supported versions of those 
> applications (hence why the profile is off by default).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2713) Provide an option to log out contents of the localized jar files

2016-10-20 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-2713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15592381#comment-15592381
 ] 

Hadoop QA commented on OOZIE-2713:
--

Testing JIRA OOZIE-2713

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:red}-1{color} FindBugs diff JAR has a weird MD5 sum, rejecting.
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 1815
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}


The full output of the test-patch run is available at

.   https://builds.apache.org/job/oozie-trunk-precommit-build/3401/

> Provide an option to log out contents of the localized jar files
> 
>
> Key: OOZIE-2713
> URL: https://issues.apache.org/jira/browse/OOZIE-2713
> Project: Oozie
>  Issue Type: Improvement
>Reporter: Peter Cseh
>Assignee: Peter Cseh
> Attachments: OOZIE-2713.01.patch
>
>
> We had several issues with user jars involved in workflows containing 
> conflicting third party libraries.
> It looks like creating uberjars for UDFs in hive is a bad practice, but it's 
> sometimes hard to get the needed information to handle these issues. 
> We should introduce a new property for the actions e.g 
> oozie.action.list.jar.contents which is false by default, but if set to true, 
> list the contents of the jar files after the file list in the launcher logs. 
> This way we could see which classes are present on the classpath multiple 
> times.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Build failed in Jenkins: oozie-trunk-precommit-build #3401

2016-10-20 Thread Apache Jenkins Server

See 

Changes:

[rkanter] OOZIE-2621 Use hive-exec--core instead of hive-exec in

[rkanter] OOZIE-2705 Oozie Spark action ignores spark.executor.extraJavaOptions

--
[...truncated 6201 lines...]
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-tools ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-tools ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ oozie-tools ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:2.0-beta-6:attach-descriptor (attach-descriptor) @ 
oozie-tools ---
[INFO] 
[INFO] --- maven-assembly-plugin:2.2.1:single (default-cli) @ oozie-tools ---
[INFO] Reading assembly descriptor: ../src/main/assemblies/tools.xml
[WARNING] The following patterns were never triggered in this artifact 
exclusion filter:
o  '*:*:pom:*'

[INFO] Copying files to 

[WARNING] Assembly file: 

 is not a regular file (it may be a directory). It cannot be attached to the 
project build for installation or deployment.
[INFO] 
[INFO] 
[INFO] Building Apache Oozie MiniOozie 4.4.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ oozie-mini ---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-mini ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-mini 
---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-mini ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 3 resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-mini ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-mini ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ oozie-mini ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:2.0-beta-6:attach-descriptor (attach-descriptor) @ 
oozie-mini ---
[INFO] 
[INFO] --- maven-assembly-plugin:2.2.1:single (default-cli) @ oozie-mini ---
[INFO] Reading assembly descriptor: src/main/assemblies/empty.xml
[INFO] 
[INFO] 
[INFO] Building Apache Oozie Distro 4.4.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ oozie-distro 
---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-distro ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-distro 
---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-distro ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-distro ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-distro ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ oozie-distro ---
[INFO] Building jar:

[jira] [Updated] (OOZIE-2713) Provide an option to log out contents of the localized jar files

2016-10-20 Thread Peter Cseh (JIRA)


 [ 
https://issues.apache.org/jira/browse/OOZIE-2713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Peter Cseh updated OOZIE-2713:
--
Attachment: OOZIE-2713.01.patch

Attaching the first version.

Please help me come up with a better name for this flag.

> Provide an option to log out contents of the localized jar files
> 
>
> Key: OOZIE-2713
> URL: https://issues.apache.org/jira/browse/OOZIE-2713
> Project: Oozie
>  Issue Type: Improvement
>Reporter: Peter Cseh
>Assignee: Peter Cseh
> Attachments: OOZIE-2713.01.patch
>
>
> We had several issues with user jars involved in workflows containing 
> conflicting third party libraries.
> It looks like creating uberjars for UDFs in hive is a bad practice, but it's 
> sometimes hard to get the needed information to handle these issues. 
> We should introduce a new property for the actions e.g 
> oozie.action.list.jar.contents which is false by default, but if set to true, 
> list the contents of the jar files after the file list in the launcher logs. 
> This way we could see which classes are present on the classpath multiple 
> times.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (OOZIE-2713) Provide an option to log out contents of the localized jar files

2016-10-20 Thread Peter Cseh (JIRA)


 [ 
https://issues.apache.org/jira/browse/OOZIE-2713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Peter Cseh reassigned OOZIE-2713:
-

Assignee: Peter Cseh

> Provide an option to log out contents of the localized jar files
> 
>
> Key: OOZIE-2713
> URL: https://issues.apache.org/jira/browse/OOZIE-2713
> Project: Oozie
>  Issue Type: Improvement
>Reporter: Peter Cseh
>Assignee: Peter Cseh
>
> We had several issues with user jars involved in workflows containing 
> conflicting third party libraries.
> It looks like creating uberjars for UDFs in hive is a bad practice, but it's 
> sometimes hard to get the needed information to handle these issues. 
> We should introduce a new property for the actions e.g 
> oozie.action.list.jar.contents which is false by default, but if set to true, 
> list the contents of the jar files after the file list in the launcher logs. 
> This way we could see which classes are present on the classpath multiple 
> times.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Jenkins build is back to normal : oozie-trunk-find-patches-available #136965

2016-10-20 Thread Apache Jenkins Server

See

Build failed in Jenkins: oozie-trunk-find-patches-available #136964

2016-10-20 Thread Apache Jenkins Server

See 

--
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on H1 (yahoo-not-h2 Hadoop Mapreduce Hdfs Pig) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url 
 > https://git-wip-us.apache.org/repos/asf/oozie.git # timeout=10
Fetching upstream changes from https://git-wip-us.apache.org/repos/asf/oozie.git
 > git --version # timeout=10
 > git -c core.askpass=true fetch --tags --progress 
 > https://git-wip-us.apache.org/repos/asf/oozie.git 
 > +refs/heads/*:refs/remotes/origin/*
 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision 19014354dfd373ad5d05221d824bb3105e9271ac 
(refs/remotes/origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 19014354dfd373ad5d05221d824bb3105e9271ac
 > git rev-list 19014354dfd373ad5d05221d824bb3105e9271ac # timeout=10
[oozie-trunk-find-patches-available] $ /bin/bash 
/tmp/hudson5408442669811502852.sh
mkdir: cannot create directory 
':
 File exists
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed

  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
100  113k0  113k0 0   125k  0 --:--:-- --:--:-- --:--:--  125k
100  245k0  245k0 0   171k  0 --:--:--  0:00:01 --:--:--  172k
curl: (18) transfer closed with outstanding read data remaining
Could not retrieve available patches from JIRA
Build step 'Execute shell' marked build as failure

[jira] [Commented] (OOZIE-2715) Provide possibility to set default values for properties of a particular action type

2016-10-20 Thread Andras Piros (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-2715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15591429#comment-15591429
 ] 

Andras Piros commented on OOZIE-2715:
-

[~Bearricade] do you want a generic solution that handles any property key / 
value pairs on a per action type basis?

If you only need to suppress the retry count based on the action types, you 
could use inside {{workflow.xml}} the attribute {{retry-max}} of the 
{{application}} element like that:

{code:xml}

...

...

{code}

> Provide possibility to set default values for properties of a particular 
> action type
> 
>
> Key: OOZIE-2715
> URL: https://issues.apache.org/jira/browse/OOZIE-2715
> Project: Oozie
>  Issue Type: Wish
>  Components: action
>Affects Versions: 4.2.0
>Reporter: Istvan Vajnorak
>
> When it comes to retrying an action, one can set the default values for a 
> retry on the global level, and the override can happen then on the individual 
> step's level only.
> The proposal to be able to set retry and other properties on an action type 
> basis, could allow defaulting things and help not to do repetitive work on 
> each action instance in the workflow.
> Example:
> - One has 60 shell actions and 30 java actions in a workflow.
> - One sets " oozie.launcher.yarn.resourcemanager.am.max-attempts=1" globally
> - For shell steps they don't want this to get applied, so they have to 
> suppress the retry on those steps each time
> The proposal would allow an intermediate level of override similarly like the 
> hadoop conf allows with the *-default.xml, *-site.xml and code level 
> properties override.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2712) Merge Spark opts with two possible options

2016-10-20 Thread Peter Cseh (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-2712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15590968#comment-15590968
 ] 

Peter Cseh commented on OOZIE-2712:
---

The user would probably define these in one way or the other and won't use both.
The problem emerges when Oozie uses some of these things to set log4j settings 
or executor classpath and users can overwrite those internal things without 
even knowing it. Even after the fix in OOZIE-2705 if the user passes in 
--driver-java-options in  the the log4j settings will be 
overwritten by Spark and child IDs will be lost.

Maybe we should separate the user-defined option list parsing from adding our 
own stuff in there and add a post-process step that add Oozie-specific things.

I'll probably won't have the time to mess around with this for a couple weeks, 
if anybody has some ideas, feel free to grab this!

> Merge Spark opts with two possible options 
> ---
>
> Key: OOZIE-2712
> URL: https://issues.apache.org/jira/browse/OOZIE-2712
> Project: Oozie
>  Issue Type: Bug
>Reporter: Peter Cseh
>Assignee: Peter Cseh
> Attachments: OOZIE-2712.001.patch
>
>
> Issues like OOZIE-2658 can happen with more properties.
> || --conf ||  overwritten by ||
> |spark.driver.memory | --driver-memory |
> |spark.driver.extraJavaOptions | --driver-java-options|
> |spark.driver.extraLibraryPath|--driver-library-path |
> |spark.executor.instances|--num-executors|
> |spark.driver.extraClassPath| --driver-class-path|
> We should make sure that one is converted to the other (the current pattern 
> is to use --conf versions of these) so we don't run into issues where the 
> log4j properties are overwritten. (OOZIE-2705 can be affected by this)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2712) Merge Spark opts with two possible options

2016-10-20 Thread Peter Cseh (JIRA)


 [ 
https://issues.apache.org/jira/browse/OOZIE-2712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Peter Cseh updated OOZIE-2712:
--
Assignee: (was: Peter Cseh)

> Merge Spark opts with two possible options 
> ---
>
> Key: OOZIE-2712
> URL: https://issues.apache.org/jira/browse/OOZIE-2712
> Project: Oozie
>  Issue Type: Bug
>Reporter: Peter Cseh
> Attachments: OOZIE-2712.001.patch
>
>
> Issues like OOZIE-2658 can happen with more properties.
> || --conf ||  overwritten by ||
> |spark.driver.memory | --driver-memory |
> |spark.driver.extraJavaOptions | --driver-java-options|
> |spark.driver.extraLibraryPath|--driver-library-path |
> |spark.executor.instances|--num-executors|
> |spark.driver.extraClassPath| --driver-class-path|
> We should make sure that one is converted to the other (the current pattern 
> is to use --conf versions of these) so we don't run into issues where the 
> log4j properties are overwritten. (OOZIE-2705 can be affected by this)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] Subscription: Oozie Patch Available

2016-10-20 Thread jira

Issue Subscription
Filter: Oozie Patch Available (86 issues)

Subscriber: ooziedaily

Key Summary
OOZIE-2619  Make  Hive action defaults to match hive defaults when running from 
command line
https://issues-test.apache.org/jira/browse/OOZIE-2619
OOZIE-2618  Include hive-common in hive-beeline as compile component for 
hive-server2 (hive2 action)
https://issues-test.apache.org/jira/browse/OOZIE-2618
OOZIE-2617  Read properties files in action configurations
https://issues-test.apache.org/jira/browse/OOZIE-2617
OOZIE-2616  Add Tez profile for Hive and Pig sharelibs
https://issues-test.apache.org/jira/browse/OOZIE-2616
OOZIE-2606  Set spark.yarn.jars to fix Spark 2.0 with Oozie
https://issues-test.apache.org/jira/browse/OOZIE-2606
OOZIE-2589  CompletedActionXCommand priority param is of no effect
https://issues-test.apache.org/jira/browse/OOZIE-2589
OOZIE-2588  Support getting credentials for same cluster hcat when credentials 
config is empty
https://issues-test.apache.org/jira/browse/OOZIE-2588
OOZIE-2585  Disable or remove test case 
TestPartitionDependencyManagerEhcache.testMemoryUsageAndSpeedOverflowToDisk and 
testMemoryUsageAndSpeed
https://issues-test.apache.org/jira/browse/OOZIE-2585
OOZIE-2584  Eliminate Thread.sleep() calls in TestMemoryLocks
https://issues-test.apache.org/jira/browse/OOZIE-2584
OOZIE-2583  oozie throws EL Exception when reference variable name containing 
dot
https://issues-test.apache.org/jira/browse/OOZIE-2583
OOZIE-2582  Populating external child Ids for action failures
https://issues-test.apache.org/jira/browse/OOZIE-2582
OOZIE-2573  dataset url  which contains spaces can not be handled rightly
https://issues-test.apache.org/jira/browse/OOZIE-2573
OOZIE-2572  SLA DURATION miss not shown when job is running for longer than 
expected time
https://issues-test.apache.org/jira/browse/OOZIE-2572
OOZIE-2571  Add scala.binary.version Maven property so that Scala 2.11 can be 
used
https://issues-test.apache.org/jira/browse/OOZIE-2571
OOZIE-2569  Adding yarn-site, core-site, hdfs-site and mapred-site into spark 
launcher 
https://issues-test.apache.org/jira/browse/OOZIE-2569
OOZIE-2568  SSH  action pretends to retry automaticly when it failed
https://issues-test.apache.org/jira/browse/OOZIE-2568
OOZIE-2564  Create new log4j config for unit tests so that logged messages 
contain threads
https://issues-test.apache.org/jira/browse/OOZIE-2564
OOZIE-2552  Update ActiveMQ version for security and other fixes
https://issues-test.apache.org/jira/browse/OOZIE-2552
OOZIE-2538  Update HttpClient versions to close security vulnerabilities
https://issues-test.apache.org/jira/browse/OOZIE-2538
OOZIE-2534  Versioned action libs (similar to how SharedLibs works)
https://issues-test.apache.org/jira/browse/OOZIE-2534
OOZIE-2530  Remove jline from parent and sharelib pig pom
https://issues-test.apache.org/jira/browse/OOZIE-2530
OOZIE-2528  Print out environment variables in LauncherMapper
https://issues-test.apache.org/jira/browse/OOZIE-2528
OOZIE-2521  Filter options are case sensitive for jobtye=bundle
https://issues-test.apache.org/jira/browse/OOZIE-2521
OOZIE-2518  Oozie is creating many files and directories in /tmp per day
https://issues-test.apache.org/jira/browse/OOZIE-2518
OOZIE-2517  Add support for startCreatedTime and endCreatedTime filters for 
coord and bundles
https://issues-test.apache.org/jira/browse/OOZIE-2517
OOZIE-2513  log.scan.duration should not be used for error and audit logs
https://issues-test.apache.org/jira/browse/OOZIE-2513
OOZIE-2512  ShareLibservice returns incorrect path for jar
https://issues-test.apache.org/jira/browse/OOZIE-2512
OOZIE-2508  Documentation change for Coord action rerun [OOZIE-1735]
https://issues-test.apache.org/jira/browse/OOZIE-2508
OOZIE-2501  ZK reentrant lock doesn't work for few cases
https://issues-test.apache.org/jira/browse/OOZIE-2501
OOZIE-2500  -DtestJarSimple option mentioned in minioozie doc does not work
https://issues-test.apache.org/jira/browse/OOZIE-2500
OOZIE-2499  map-reduce launcher does not need distributed files, archives 
except  jar of input/outputformat class
https://issues-test.apache.org/jira/browse/OOZIE-2499
OOZIE-2498  Oozie CallerId configuration for downstream components 
https://issues-test.apache.org/jira/browse/OOZIE-2498
OOZIE-2495  change action status from  ErrorType.NON_TRANSIENT to TRANSIENT 
when SSH action occurs AUTH_FAILED occasionally
https://issues-test.apache.org/jira/browse/OOZIE-2495
OOZIE-2491  oozie acl cannot specify group,it does`t work
https://issues-test.apache.org/jira/browse/OOZIE-2491

[jira] Subscription: Oozie Patch Available

2016-10-20 Thread jira

Issue Subscription
Filter: Oozie Patch Available (78 issues)

Subscriber: ooziedaily

Key Summary
OOZIE-2712  Merge Spark opts with two possible options 
https://issues.apache.org/jira/browse/OOZIE-2712
OOZIE-2710  Oozie HCatalog example workflow fails
https://issues.apache.org/jira/browse/OOZIE-2710
OOZIE-2709  Log the substituted pig script by expanding macros
https://issues.apache.org/jira/browse/OOZIE-2709
OOZIE-2694  Add logging for FsActionExecutor 
https://issues.apache.org/jira/browse/OOZIE-2694
OOZIE-2693  SimpleHCatDependencyCache.removeMissingDependency can throw NPE
https://issues.apache.org/jira/browse/OOZIE-2693
OOZIE-2692  Oozie job submit doesn't report error message to user if there is 
any issue with job conf
https://issues.apache.org/jira/browse/OOZIE-2692
OOZIE-2691  Show workflow action retry count to UI
https://issues.apache.org/jira/browse/OOZIE-2691
OOZIE-2690  OOZIE NPE while executing kill()
https://issues.apache.org/jira/browse/OOZIE-2690
OOZIE-2684  Bad database schema error for WF_ACTIONS table
https://issues.apache.org/jira/browse/OOZIE-2684
OOZIE-2674  Improve oozie commads documentation
https://issues.apache.org/jira/browse/OOZIE-2674
OOZIE-2670  Upgrade Hbase to 1.2
https://issues.apache.org/jira/browse/OOZIE-2670
OOZIE-2668  Status update and recovery problems when coord action and its 
children not in sync
https://issues.apache.org/jira/browse/OOZIE-2668
OOZIE-2666  Support embedding Jetty into Oozie
https://issues.apache.org/jira/browse/OOZIE-2666
OOZIE-2662  DB migration fails if DB is too big
https://issues.apache.org/jira/browse/OOZIE-2662
OOZIE-2654  Zookeeper dependent services should not depend on Connectionstate 
to be valid before cleaning up
https://issues.apache.org/jira/browse/OOZIE-2654
OOZIE-2652  Skip trash while deleting hive table partition
https://issues.apache.org/jira/browse/OOZIE-2652
OOZIE-2650  Retry coord start on database exceptions
https://issues.apache.org/jira/browse/OOZIE-2650
OOZIE-2644  Skip queuing Notification Commands when there's nothing to notify
https://issues.apache.org/jira/browse/OOZIE-2644
OOZIE-2641  rerunning an oozie coordinator job stucks on waiting
https://issues.apache.org/jira/browse/OOZIE-2641
OOZIE-2630  Oozie Coordinator EL Functions to get first day of the week/month
https://issues.apache.org/jira/browse/OOZIE-2630
OOZIE-2619  Make  Hive action defaults to match hive defaults when running from 
command line
https://issues.apache.org/jira/browse/OOZIE-2619
OOZIE-2618  Include hive-common in hive-beeline as compile component for 
hive-server2 (hive2 action)
https://issues.apache.org/jira/browse/OOZIE-2618
OOZIE-2616  Add Tez profile for Hive and Pig sharelibs
https://issues.apache.org/jira/browse/OOZIE-2616
OOZIE-2585  Disable or remove test case 
TestPartitionDependencyManagerEhcache.testMemoryUsageAndSpeedOverflowToDisk and 
testMemoryUsageAndSpeed
https://issues.apache.org/jira/browse/OOZIE-2585
OOZIE-2573  dataset url  which contains spaces can not be handled rightly
https://issues.apache.org/jira/browse/OOZIE-2573
OOZIE-2568  SSH  action pretends to retry automaticly when it failed
https://issues.apache.org/jira/browse/OOZIE-2568
OOZIE-2564  Create new log4j config for unit tests so that logged messages 
contain threads
https://issues.apache.org/jira/browse/OOZIE-2564
OOZIE-2534  Versioned action libs (similar to how SharedLibs works)
https://issues.apache.org/jira/browse/OOZIE-2534
OOZIE-2528  Print out environment variables in LauncherMapper
https://issues.apache.org/jira/browse/OOZIE-2528
OOZIE-2518  Oozie is creating many files and directories in /tmp per day
https://issues.apache.org/jira/browse/OOZIE-2518
OOZIE-2513  log.scan.duration should not be used for error and audit logs
https://issues.apache.org/jira/browse/OOZIE-2513
OOZIE-2499  map-reduce launcher does not need distributed files, archives 
except  jar of input/outputformat class
https://issues.apache.org/jira/browse/OOZIE-2499
OOZIE-2495  change action status from  ErrorType.NON_TRANSIENT to TRANSIENT 
when SSH action occurs AUTH_FAILED occasionally
https://issues.apache.org/jira/browse/OOZIE-2495
OOZIE-2473  Connection pool for SMTP connection
https://issues.apache.org/jira/browse/OOZIE-2473
OOZIE-2461  Workflow, Coordinator and Bundle job querying should have last 
modified filter
https://issues.apache.org/jira/browse/OOZIE-2461
OOZIE-2457  Oozie log parsing regex consume more than 90% cpu
https://issues.apache.org/jira/browse/OOZIE-2457
OOZIE-2427  With Kerberos enabled, the authentication failure for the 
javascript

[jira] [Created] (OOZIE-2716) Oozie Truncates Workflow Parameter Values which contain '}}'

Re: Review Request 52227: OOZIE-2225 Add wild card filter for gathering jobs

[jira] [Commented] (OOZIE-2616) Add Tez profile for Hive and Pig sharelibs

[jira] [Commented] (OOZIE-2713) Provide an option to log out contents of the localized jar files

Build failed in Jenkins: oozie-trunk-precommit-build #3401

[jira] [Updated] (OOZIE-2713) Provide an option to log out contents of the localized jar files

[jira] [Assigned] (OOZIE-2713) Provide an option to log out contents of the localized jar files

Jenkins build is back to normal : oozie-trunk-find-patches-available #136965

Build failed in Jenkins: oozie-trunk-find-patches-available #136964

[jira] [Commented] (OOZIE-2715) Provide possibility to set default values for properties of a particular action type

[jira] [Commented] (OOZIE-2712) Merge Spark opts with two possible options

[jira] [Updated] (OOZIE-2712) Merge Spark opts with two possible options

[jira] Subscription: Oozie Patch Available

[jira] Subscription: Oozie Patch Available

14 matches

Site Navigation

Mail list logo

Footer information