[jira] Subscription: Oozie Patch Available

[jira]

Issue Subscription
Filter: Oozie Patch Available (105 issues)

Subscriber: ooziedaily

Key Summary
OOZIE-3147  Misleading documentation of oozie.service.PurgeService.purge.limit 
configuration property
https://issues.apache.org/jira/browse/OOZIE-3147
OOZIE-3135  Configure log4j2 in SqoopMain
https://issues.apache.org/jira/browse/OOZIE-3135
OOZIE-3121  bump all maven plugins to latest versions
https://issues.apache.org/jira/browse/OOZIE-3121
OOZIE-3113  Retry for ZK lock release
https://issues.apache.org/jira/browse/OOZIE-3113
OOZIE-3105  testJMXInstrumentation from the 
org.apache.oozie.util.TestMetricsInstrumentation class is flaky
https://issues.apache.org/jira/browse/OOZIE-3105
OOZIE-3094  fix for grammar mistake
https://issues.apache.org/jira/browse/OOZIE-3094
OOZIE-3091  Oozie Sqoop Avro Import fails with "java.lang.NoClassDefFoundError: 
org/apache/avro/mapred/AvroWrapper"
https://issues.apache.org/jira/browse/OOZIE-3091
OOZIE-3085  Improve logging in ActionExecutors
https://issues.apache.org/jira/browse/OOZIE-3085
OOZIE-3083  Make improved version Info backward compatible
https://issues.apache.org/jira/browse/OOZIE-3083
OOZIE-3071  Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 
than Spark 2.2.0
https://issues.apache.org/jira/browse/OOZIE-3071
OOZIE-3063  Sanitizing variables that are part of openjpa.ConnectionProperties
https://issues.apache.org/jira/browse/OOZIE-3063
OOZIE-3062  Set HADOOP_CONF_DIR for spark action
https://issues.apache.org/jira/browse/OOZIE-3062
OOZIE-3002  address findbugs errors in client lib
https://issues.apache.org/jira/browse/OOZIE-3002
OOZIE-2975  code clean up in pig sharelib, replace Exception with more 
explicit, add try with resources, StringBuilder instead of StringBuffer
https://issues.apache.org/jira/browse/OOZIE-2975
OOZIE-2973  Make sure Oozie works with Hadoop 3 
https://issues.apache.org/jira/browse/OOZIE-2973
OOZIE-2957  Documentation states that starting a coordinator is possible
https://issues.apache.org/jira/browse/OOZIE-2957
OOZIE-2956  Fix Findbugs warnings related to reliance on default encoding in 
oozie-core
https://issues.apache.org/jira/browse/OOZIE-2956
OOZIE-2955  Fix Findbugs warnings related to reliance on default encoding in 
oozie-client
https://issues.apache.org/jira/browse/OOZIE-2955
OOZIE-2954  Fix Checkstyle issues in oozie-client
https://issues.apache.org/jira/browse/OOZIE-2954
OOZIE-2953  Fix Checkstyle issues in oozie-tools
https://issues.apache.org/jira/browse/OOZIE-2953
OOZIE-2952  Fix Findbugs warnings in oozie-sharelib-oozie
https://issues.apache.org/jira/browse/OOZIE-2952
OOZIE-2949  Escape quotes whitespaces in Sqoop  field
https://issues.apache.org/jira/browse/OOZIE-2949
OOZIE-2942  Fix Findbugs warnings in oozie-examples
https://issues.apache.org/jira/browse/OOZIE-2942
OOZIE-2937  Remove redundant groupId from the child pom's
https://issues.apache.org/jira/browse/OOZIE-2937
OOZIE-2934  Fix "Exceptional return value of java.io.File.mkdirs() ignored" 
Findbugs error in oozie-sharelib-spark
https://issues.apache.org/jira/browse/OOZIE-2934
OOZIE-2927  Append new line character for Hive2 query using query tag
https://issues.apache.org/jira/browse/OOZIE-2927
OOZIE-2914  Consolidate Trim 
https://issues.apache.org/jira/browse/OOZIE-2914
OOZIE-2883  OOZIE throw the error "Missing 
[oozie.service.ProxyUserService.proxyuser.oozie.service.ProxyUserService.proxyuser.mr.groups]
 property"
https://issues.apache.org/jira/browse/OOZIE-2883
OOZIE-2877  Oozie Git Action
https://issues.apache.org/jira/browse/OOZIE-2877
OOZIE-2867  Timezone handling for Coordinators: emphasize "Continent/City" 
format
https://issues.apache.org/jira/browse/OOZIE-2867
OOZIE-2834  ParameterVerifier logging non-useful warning for workflow definition
https://issues.apache.org/jira/browse/OOZIE-2834
OOZIE-2833  when using uber mode the regex pattern used in the 
extractHeapSizeMB method does not allow heap sizes specified in bytes.
https://issues.apache.org/jira/browse/OOZIE-2833
OOZIE-2829  Improve sharelib upload to accept multiple source folders
https://issues.apache.org/jira/browse/OOZIE-2829
OOZIE-2826  Falcon feed fails to aws s3; Oozie joda time version does not meet 
required jar version 2.2 or later
https://issues.apache.org/jira/browse/OOZIE-2826
OOZIE-2812  SparkConfigurationService should support loading configurations 
from multiple Spark versions
https://issues.apache.org/jira/browse/OOZIE-2812
OOZIE-2795  Create lib directory or symlink for Oozie CLI during packaging

[jira] Subscription: Oozie Patch Available

[jira]

Issue Subscription
Filter: Oozie Patch Available (111 issues)

Subscriber: ooziedaily

Key Summary
OOZIE-3129  Fix test TestConfigurationService.testOozieConfig
https://issues-test.apache.org/jira/browse/OOZIE-3129
OOZIE-3127  Remove redundant check for user
https://issues-test.apache.org/jira/browse/OOZIE-3127
OOZIE-3126  Add option to allow list of users to access system config
https://issues-test.apache.org/jira/browse/OOZIE-3126
OOZIE-3121  bump all maven plugins to latest versions
https://issues-test.apache.org/jira/browse/OOZIE-3121
OOZIE-3118  fix for error: self-closing element not allowed
https://issues-test.apache.org/jira/browse/OOZIE-3118
OOZIE-3117  fix for warning: no @throws for
https://issues-test.apache.org/jira/browse/OOZIE-3117
OOZIE-3116  fix for warning: no description for @throws
https://issues-test.apache.org/jira/browse/OOZIE-3116
OOZIE-3115  fix for javadoc warning: empty  tag
https://issues-test.apache.org/jira/browse/OOZIE-3115
OOZIE-3113  Retry for ZK lock release
https://issues-test.apache.org/jira/browse/OOZIE-3113
OOZIE-3112  SparkConfigrationService overwrites properties provided via 
--properties-file option in SparkAction
https://issues-test.apache.org/jira/browse/OOZIE-3112
OOZIE-3105  testJMXInstrumentation from the 
org.apache.oozie.util.TestMetricsInstrumentation class is flaky
https://issues-test.apache.org/jira/browse/OOZIE-3105
OOZIE-3094  fix for grammar mistake
https://issues-test.apache.org/jira/browse/OOZIE-3094
OOZIE-3091  Oozie Sqoop Avro Import fails with "java.lang.NoClassDefFoundError: 
org/apache/avro/mapred/AvroWrapper"
https://issues-test.apache.org/jira/browse/OOZIE-3091
OOZIE-3083  Make improved version Info backward compatible
https://issues-test.apache.org/jira/browse/OOZIE-3083
OOZIE-3071  Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 
than Spark 2.2.0
https://issues-test.apache.org/jira/browse/OOZIE-3071
OOZIE-3063  Sanitizing variables that are part of openjpa.ConnectionProperties
https://issues-test.apache.org/jira/browse/OOZIE-3063
OOZIE-3062  Set HADOOP_CONF_DIR for spark action
https://issues-test.apache.org/jira/browse/OOZIE-3062
OOZIE-3002  address findbugs errors in client lib
https://issues-test.apache.org/jira/browse/OOZIE-3002
OOZIE-2975  code clean up in pig sharelib, replace Exception with more 
explicit, add try with resources, StringBuilder instead of StringBuffer
https://issues-test.apache.org/jira/browse/OOZIE-2975
OOZIE-2973  Make sure Oozie works with Hadoop 3 
https://issues-test.apache.org/jira/browse/OOZIE-2973
OOZIE-2969  Drop support for Java 1.7
https://issues-test.apache.org/jira/browse/OOZIE-2969
OOZIE-2957  Documentation states that starting a coordinator is possible
https://issues-test.apache.org/jira/browse/OOZIE-2957
OOZIE-2956  Fix Findbugs warnings related to reliance on default encoding in 
oozie-core
https://issues-test.apache.org/jira/browse/OOZIE-2956
OOZIE-2955  Fix Findbugs warnings related to reliance on default encoding in 
oozie-client
https://issues-test.apache.org/jira/browse/OOZIE-2955
OOZIE-2954  Fix Checkstyle issues in oozie-client
https://issues-test.apache.org/jira/browse/OOZIE-2954
OOZIE-2953  Fix Checkstyle issues in oozie-tools
https://issues-test.apache.org/jira/browse/OOZIE-2953
OOZIE-2952  Fix Findbugs warnings in oozie-sharelib-oozie
https://issues-test.apache.org/jira/browse/OOZIE-2952
OOZIE-2949  Escape quotes whitespaces in Sqoop  field
https://issues-test.apache.org/jira/browse/OOZIE-2949
OOZIE-2942  Fix Findbugs warnings in oozie-examples
https://issues-test.apache.org/jira/browse/OOZIE-2942
OOZIE-2937  Remove redundant groupId from the child pom's
https://issues-test.apache.org/jira/browse/OOZIE-2937
OOZIE-2934  Fix "Exceptional return value of java.io.File.mkdirs() ignored" 
Findbugs error in oozie-sharelib-spark
https://issues-test.apache.org/jira/browse/OOZIE-2934
OOZIE-2927  Append new line character for Hive2 query using query tag
https://issues-test.apache.org/jira/browse/OOZIE-2927
OOZIE-2914  Consolidate Trim 
https://issues-test.apache.org/jira/browse/OOZIE-2914
OOZIE-2883  OOZIE throw the error "Missing 
[oozie.service.ProxyUserService.proxyuser.oozie.service.ProxyUserService.proxyuser.mr.groups]
 property"
https://issues-test.apache.org/jira/browse/OOZIE-2883
OOZIE-2877  Oozie Git Action
https://issues-test.apache.org/jira/browse/OOZIE-2877
OOZIE-2867  Timezone handling for Coordinators: emphasize "Continent/City" 
format
https://issues-test.apache.org/jira/browse/OOZIE-2867
OOZIE-2834  ParameterVerifier logging 

Re: [VOTE] Release Oozie 5.0.0-beta1 (candidate 0)

[Peter Cseh]

Hi Attila!

It looks like I've messed up the extraction of the tar.gz and I was looking
at empty folders :)
Will validate the release tomorrow.

Sorry for causing confusion!
gp

On Thu, Dec 28, 2017 at 2:08 PM, Attila Sasvari 
wrote:

> @Andras: many thanks. Please let me know if you notice anything strange
> with hashes or run into other problems.
>
> @gp: thanks a lot. This is actually what the bin/create-release-artifact
> script has generated. Comparing it with previous release candidate (4.3.0
> rc1), file size of oozie-5.0.0-beta1.tar.gz looks correct to me
>
>   $ wget
> https://dist.apache.org/repos/dist/dev/oozie/oozie-4.3.0-
> rc1/oozie-4.3.0.tar.gz
>   $ wget
> http://people.apache.org/\~asasvari/oozie-5.0.0-beta1-
> rc0/oozie-5.0.0-beta1.tar.gz
>
>   $ ls -lrt oozie-*tar.gz
>   -rw-r--r--  1 asasvari  staff  2392205 Oct 25  2016 oozie-4.3.0.tar.gz
>   -rw-r--r--  1 asasvari  staff  2539718 Dec 28 00:12
> oozie-5.0.0-beta1.tar.gz
>
> oozie-5.0.0-beta1.tar.gz contains much more files (it contains all the
> sources):
>
>   $  tar tvf oozie-5.0.0-beta1.tar.gz | grep "\.java" | head -5
>   -rw-r--r--  0 asasvari wheel   10328 Dec 28 00:02
> oozie-5.0.0-beta1/zookeeper-security-tests/src/test/java/
> org/apache/oozie/util/TestZKUtilsWithSecurity.java
>   -rw-r--r--  0 asasvari wheel7357 Dec 28 00:02
> oozie-5.0.0-beta1/zookeeper-security-tests/src/test/java/
> org/apache/oozie/test/ZKXTestCaseWithSecurity.java
>   -rw-r--r--  0 asasvari wheel   32589 Dec 28 00:02
> oozie-5.0.0-beta1/tools/src/test/java/org/apache/oozie/
> tools/FakeConnection.java
>   -rw-r--r--  0 asasvari wheel1921 Dec 28 00:02
> oozie-5.0.0-beta1/tools/src/test/java/org/apache/oozie/
> tools/FakeDriver.java
>   -rw-r--r--  0 asasvari wheel2230 Dec 28 00:02
> oozie-5.0.0-beta1/tools/src/test/java/org/apache/oozie/
> tools/LauncherSecurityManager.java
>
> If you download the tarball and run bin/mkdistro.sh, it will probably fail
> due to known flaky tests. Running tests with bin/mkdistro.sh
> -Dsurefire.rerunFailingTestsCount=2, they shall pass (printing things
> like [WARNING] Tests run: 1841, Failures: 0, Errors: 0, Skipped: 2, Flakes:
> 2 ).
>
> - Attila
>
> On Thu, Dec 28, 2017 at 10:57 AM, Peter Cseh 
> wrote:
>
> > Thanks Attila!
> > The tar.gz does not contain the release, it only contains the pom files.
> > The whole thing is 2.4 Mb, an Oozie was bigger the last time I've checked
> > :)
> >
> > gp
> >
> > On Thu, Dec 28, 2017 at 12:53 AM, Attila Sasvári 
> > wrote:
> >
> > > Hi there,
> > >
> > > I have created a build for Oozie 5.0.0-beta1, release candidate 0.
> > >
> > > This release introduces major changes:
> > > - moving launcher from MapReduce AM to Oozie AM,
> > > - switching from Tomcat to embedded Jetty,
> > > - completely rewritten graph generator,
> > > - local sharelib,
> > > - JDK 8 support.
> > >
> > > Keys to verify the signature of the release artifact are available at
> > >
> > >  http://www.apache.org/dist/oozie/KEYS
> > >
> > > Please download, test, and try it out:
> > >
> > >   http://people.apache.org/~asasvari/oozie-5.0.0-beta1-rc0/
> > >
> > > The release, md5 signature, gpg signature, and rat report can all be
> > found
> > > at the above address.
> > >
> > > The vote is open until Thursday, January 04, 2018 at 12:00 CEST.
> > >
> > > Thanks,
> > > Attila Sasvari
> > >
> >
> >
> >
> > --
> > Peter Cseh
> > Software Engineer
> > 
> >
>
>
>
> --
> --
> Attila Sasvari
> Software Engineer
> 
>



-- 
Peter Cseh
Software Engineer

Re: [VOTE] Release Oozie 5.0.0-beta1 (candidate 0)

[Attila Sasvari]

@Andras: many thanks. Please let me know if you notice anything strange
with hashes or run into other problems.

@gp: thanks a lot. This is actually what the bin/create-release-artifact
script has generated. Comparing it with previous release candidate (4.3.0
rc1), file size of oozie-5.0.0-beta1.tar.gz looks correct to me

  $ wget
https://dist.apache.org/repos/dist/dev/oozie/oozie-4.3.0-rc1/oozie-4.3.0.tar.gz
  $ wget
http://people.apache.org/\~asasvari/oozie-5.0.0-beta1-rc0/oozie-5.0.0-beta1.tar.gz

  $ ls -lrt oozie-*tar.gz
  -rw-r--r--  1 asasvari  staff  2392205 Oct 25  2016 oozie-4.3.0.tar.gz
  -rw-r--r--  1 asasvari  staff  2539718 Dec 28 00:12
oozie-5.0.0-beta1.tar.gz

oozie-5.0.0-beta1.tar.gz contains much more files (it contains all the
sources):

  $  tar tvf oozie-5.0.0-beta1.tar.gz | grep "\.java" | head -5
  -rw-r--r--  0 asasvari wheel   10328 Dec 28 00:02
oozie-5.0.0-beta1/zookeeper-security-tests/src/test/java/org/apache/oozie/util/TestZKUtilsWithSecurity.java
  -rw-r--r--  0 asasvari wheel7357 Dec 28 00:02
oozie-5.0.0-beta1/zookeeper-security-tests/src/test/java/org/apache/oozie/test/ZKXTestCaseWithSecurity.java
  -rw-r--r--  0 asasvari wheel   32589 Dec 28 00:02
oozie-5.0.0-beta1/tools/src/test/java/org/apache/oozie/tools/FakeConnection.java
  -rw-r--r--  0 asasvari wheel1921 Dec 28 00:02
oozie-5.0.0-beta1/tools/src/test/java/org/apache/oozie/tools/FakeDriver.java
  -rw-r--r--  0 asasvari wheel2230 Dec 28 00:02
oozie-5.0.0-beta1/tools/src/test/java/org/apache/oozie/tools/LauncherSecurityManager.java

If you download the tarball and run bin/mkdistro.sh, it will probably fail
due to known flaky tests. Running tests with bin/mkdistro.sh
-Dsurefire.rerunFailingTestsCount=2, they shall pass (printing things
like [WARNING] Tests run: 1841, Failures: 0, Errors: 0, Skipped: 2, Flakes:
2 ).

- Attila

On Thu, Dec 28, 2017 at 10:57 AM, Peter Cseh  wrote:

> Thanks Attila!
> The tar.gz does not contain the release, it only contains the pom files.
> The whole thing is 2.4 Mb, an Oozie was bigger the last time I've checked
> :)
>
> gp
>
> On Thu, Dec 28, 2017 at 12:53 AM, Attila Sasvári 
> wrote:
>
> > Hi there,
> >
> > I have created a build for Oozie 5.0.0-beta1, release candidate 0.
> >
> > This release introduces major changes:
> > - moving launcher from MapReduce AM to Oozie AM,
> > - switching from Tomcat to embedded Jetty,
> > - completely rewritten graph generator,
> > - local sharelib,
> > - JDK 8 support.
> >
> > Keys to verify the signature of the release artifact are available at
> >
> >  http://www.apache.org/dist/oozie/KEYS
> >
> > Please download, test, and try it out:
> >
> >   http://people.apache.org/~asasvari/oozie-5.0.0-beta1-rc0/
> >
> > The release, md5 signature, gpg signature, and rat report can all be
> found
> > at the above address.
> >
> > The vote is open until Thursday, January 04, 2018 at 12:00 CEST.
> >
> > Thanks,
> > Attila Sasvari
> >
>
>
>
> --
> Peter Cseh
> Software Engineer
> 
>



-- 
-- 
Attila Sasvari
Software Engineer

[jira] [Commented] (OOZIE-3085) Improve logging in ActionExecutors

[Attila Sasvari (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16305364#comment-16305364
 ] 

Attila Sasvari commented on OOZIE-3085:
---

[~kmarton] thanks for the new patch, I left some more comments on ReviewBoard. 
Can you try it out on a real cluster? 

[~gezapeti], [~andras.piros]: this JIRA's summary and description feels quite 
general / not focused. The patch uploaded here is mainly focusing on logging 
action _starting_ and _ending_ (not killing). I would either rename it to 
express we are only dealing with logging in start() and end() methods or create 
subtasks. For example, in the SubworkflowActionExecutor there are more things 
that could be interesting to log (e.g. killing a subworkflow action). What do 
you think? At the same time, can you also have a look at the patch? You might 
see something I missed.

> Improve logging in ActionExecutors
> --
>
> Key: OOZIE-3085
> URL: https://issues.apache.org/jira/browse/OOZIE-3085
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Affects Versions: 4.3.0
>Reporter: Peter Cseh
>Assignee: Julia Kinga Marton
>  Labels: newbie
> Fix For: 5.0.0
>
> Attachments: OOZIE-3085-001.patch, OOZIE-3085-002.patch, 
> OOZIE-3085-003.patch
>
>
> Decision, ForkJoin, Email and FS ActionExecutors are producing no or very 
> little logs on info level. This makes it hard to gather usage information and 
> help with troubleshooting in the case of any issues. Please improve logging 
> in these classes.
> Several subtasks can be separated out from this one



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Review Request 64808: OOZIE-3085 - Improve logging in ActionExecutors

[Attila Sasvari via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64808/#review194570
---




core/src/main/java/org/apache/oozie/action/control/ControlNodeActionExecutor.java
Lines 52 (patched)


What does ``getId()`` return for a control node action?



core/src/main/java/org/apache/oozie/action/control/ControlNodeActionExecutor.java
Lines 59 (patched)


What does ``getExternalStatus()`` return for a control node action? It 
would might sense to log ``getStatus()`` too.



core/src/main/java/org/apache/oozie/action/decision/DecisionActionExecutor.java
Line 45 (original), 47 (patched)


What does ``getId()`` return for a control node action? I've just checked 
TestDecisionActionExecutor (executed) and saw it was ``null`` that does not 
seem very useful. 

Can you create a workflow with all changed control nodes and try it out on 
a cluster (pseudo hadoop should be fine) and double check logs about starting 
and ending actions?



core/src/main/java/org/apache/oozie/action/decision/DecisionActionExecutor.java
Lines 83 (patched)


There is a difference between status ``action.getStatus()`` (that can be 
"OK", "KILLED", "FAILED", etc.) and ``getExternalStatus()`` (in case of a 
decision action it returns where the execution will flow, transition to the 
next workflow action). 

For example:
```

 false
 false
 false
 
https://reviews.apache.org/r/64808/#comment273409>

What does ``getId()`` return?



core/src/main/java/org/apache/oozie/action/email/EmailActionExecutor.java
Lines 313 (patched)


What does ``getExternalStatus()`` return? It would might sense to log 
``getStatus()`` too.



core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java
Lines 630 (patched)


What does ``getId()`` return?



core/src/main/java/org/apache/oozie/action/oozie/SubWorkflowActionExecutor.java
Lines 167 (patched)


What does ``getId()`` return?



core/src/main/java/org/apache/oozie/action/oozie/SubWorkflowActionExecutor.java
Lines 259 (patched)


What does ``getExternalStatus()`` return? It would might sense to log 
``getStatus()`` too.



core/src/main/java/org/apache/oozie/action/ssh/SshActionExecutor.java
Lines 467 (patched)


What does ``getExternalStatus()`` return? It would might sense to log 
``getStatus()`` too.


- Attila Sasvari


On Dec. 27, 2017, 9:12 a.m., Kinga Marton wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64808/
> ---
> 
> (Updated Dec. 27, 2017, 9:12 a.m.)
> 
> 
> Review request for oozie and Attila Sasvari.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Decision, ForkJoin, Email and FS ActionExecutors are producing no or very 
> little logs on info level. This makes it hard to gather usage information and 
> help with troubleshooting in the case of any issues. Please improve logging 
> in these classes.
> Several subtasks can be separated out from this one
> 
> 
> Diffs
> -
> 
>   
> core/src/main/java/org/apache/oozie/action/control/ControlNodeActionExecutor.java
>  cc1b6108 
>   
> core/src/main/java/org/apache/oozie/action/decision/DecisionActionExecutor.java
>  8c235bda 
>   core/src/main/java/org/apache/oozie/action/email/EmailActionExecutor.java 
> d59f1d76 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 
> 63f6104d 
>   
> core/src/main/java/org/apache/oozie/action/oozie/SubWorkflowActionExecutor.java
>  d62cf686 
>   core/src/main/java/org/apache/oozie/action/ssh/SshActionExecutor.java 
> 5890b8c1 
> 
> 
> Diff: https://reviews.apache.org/r/64808/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Kinga Marton
> 
>

[jira] [Commented] (OOZIE-3145) TestDistcpMain shall remove created files after test execution

[Attila Sasvari (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16305298#comment-16305298
 ] 

Attila Sasvari commented on OOZIE-3145:
---

[~kmarton] thanks for pointing this out and linking with related JIRA.  It 
should be possible to configure where these config and log files are generated 
though, but it needs some investigation. 

You are right, we can either change the summary and description of this ticket 
(if it is not too much effort to resolve activemq and sharelib related things 
at once) or file separate JIRAs.

> TestDistcpMain shall remove created files after test execution
> --
>
> Key: OOZIE-3145
> URL: https://issues.apache.org/jira/browse/OOZIE-3145
> Project: Oozie
>  Issue Type: Bug
>  Components: tests
>Affects Versions: 5.0.0b1
>Reporter: Attila Sasvari
> Attachments: Oozie_test_garbage.png
>
>
> After running TestDistcpMain, local files created are not removed. Subsequent 
> test runs will reuse those files and fail:
> {code}
> $ ls -lrt /home/asasvari/workspace/oozie/core/distcp*
> -rw-rw-r-- 1 asasvari asasvari 1025 Dec 21 18:25 
> /home/asasvari/workspace/oozie/core/distcp-log4j.properties
> -rw-rw-r-- 1 asasvari asasvari 5756 Dec 21 18:26 
> /home/asasvari/workspace/oozie/core/distcp-oozie-1513876931701.log
> {code}
> Those files should be removed after test execution. It would be better to use 
> the oozie/core/target directory so that mvn could clean it.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: [VOTE] Release Oozie 5.0.0-beta1 (candidate 0)

[Peter Cseh]

Thanks Attila!
The tar.gz does not contain the release, it only contains the pom files.
The whole thing is 2.4 Mb, an Oozie was bigger the last time I've checked :)

gp

On Thu, Dec 28, 2017 at 12:53 AM, Attila Sasvári 
wrote:

> Hi there,
>
> I have created a build for Oozie 5.0.0-beta1, release candidate 0.
>
> This release introduces major changes:
> - moving launcher from MapReduce AM to Oozie AM,
> - switching from Tomcat to embedded Jetty,
> - completely rewritten graph generator,
> - local sharelib,
> - JDK 8 support.
>
> Keys to verify the signature of the release artifact are available at
>
>  http://www.apache.org/dist/oozie/KEYS
>
> Please download, test, and try it out:
>
>   http://people.apache.org/~asasvari/oozie-5.0.0-beta1-rc0/
>
> The release, md5 signature, gpg signature, and rat report can all be found
> at the above address.
>
> The vote is open until Thursday, January 04, 2018 at 12:00 CEST.
>
> Thanks,
> Attila Sasvari
>



-- 
Peter Cseh
Software Engineer

[jira] [Comment Edited] (OOZIE-3025) move distcp-log4j.properties out of the root of core into more appropriate location

[Julia Kinga Marton (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16305190#comment-16305190
 ] 

Julia Kinga Marton edited comment on OOZIE-3025 at 12/28/17 9:03 AM:
-

Hi [~dbist],

I found that the log4j properties and other created files are placed as well in 
the root of the working directory in case of PigMain, HiveMain and SparkMain as 
well. (See: 
https://issues.apache.org/jira/secure/attachment/12903871/12903871_Oozie_test_garbage.png)
 I think that we should handle all this cases in the same way and find an 
appropriate location for this files as well. 


was (Author: kmarton):
Hi [~dbist],

I found that the log4j properties and other created files are placed as well in 
the root of the working directory in case of PigMain, HiveMain and SparkMain as 
well. (See: 
https://issues.apache.org/jira/secure/attachment/12903871/12903871_Oozie_test_garbage.png)
 I think that we should handle all this cases in the same way. 

> move distcp-log4j.properties out of the root of core into more appropriate 
> location
> ---
>
> Key: OOZIE-3025
> URL: https://issues.apache.org/jira/browse/OOZIE-3025
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Affects Versions: 4.3.0
>Reporter: Artem Ervits
>Assignee: Artem Ervits
>
> distcp-log4j.properties file gets created in the root of core library. I 
> think it should be moved to a more appropriate location.
> https://github.com/apache/oozie/blob/master/sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java#L145



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (OOZIE-3025) move distcp-log4j.properties out of the root of core into more appropriate location

[Julia Kinga Marton (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16305190#comment-16305190
 ] 

Julia Kinga Marton commented on OOZIE-3025:
---

Hi [~dbist],

I found that the log4j properties and other created files are placed as well in 
the root of the working directory in case of PigMain, HiveMain and SparkMain as 
well. (See: 
https://issues.apache.org/jira/secure/attachment/12903871/12903871_Oozie_test_garbage.png)
 I think that we should handle all this cases in the same way. 

> move distcp-log4j.properties out of the root of core into more appropriate 
> location
> ---
>
> Key: OOZIE-3025
> URL: https://issues.apache.org/jira/browse/OOZIE-3025
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Affects Versions: 4.3.0
>Reporter: Artem Ervits
>Assignee: Artem Ervits
>
> distcp-log4j.properties file gets created in the root of core library. I 
> think it should be moved to a more appropriate location.
> https://github.com/apache/oozie/blob/master/sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java#L145



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Comment Edited] (OOZIE-3145) TestDistcpMain shall remove created files after test execution

[Julia Kinga Marton (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16305175#comment-16305175
 ] 

Julia Kinga Marton edited comment on OOZIE-3145 at 12/28/17 8:45 AM:
-

Not only TestDistcpMain produces garbage, I run the tests from core and from 
sharelib and they produced a lot of garbage. See the attached 
[^Oozie_test_garbage.png]. 

In most of the cases the files are created by the tested methods, not by the 
test case itself. I also found an open issue related the creation place of the 
files: [OOZIE-3025].

I think that we should change the place where the files are created in the 
[OOZIE-3025] issue, and then to make a general cleanup (for the other cases as 
well, not only for TestDistcpMain).

What do you think [~asasvari] ?  


was (Author: kmarton):
Not only TestDistcpMain produces garbage, I run the tests from core and from 
sharelib and they produced a lot of garbage. See the attached 
Oozie_test_garbage.png. 

In most of the cases the files are created by the tested methods, not by the 
test case itself. I also found an open issue related the creation place of the 
files: [OOZIE-3025].

I think that we should change the place where the files are created in the 
[OOZIE-3025] issue, and then to make a general cleanup (for the other cases as 
well, not only for TestDistcpMain).

What do you think [~asasvari] ?  

> TestDistcpMain shall remove created files after test execution
> --
>
> Key: OOZIE-3145
> URL: https://issues.apache.org/jira/browse/OOZIE-3145
> Project: Oozie
>  Issue Type: Bug
>  Components: tests
>Affects Versions: 5.0.0b1
>Reporter: Attila Sasvari
> Attachments: Oozie_test_garbage.png
>
>
> After running TestDistcpMain, local files created are not removed. Subsequent 
> test runs will reuse those files and fail:
> {code}
> $ ls -lrt /home/asasvari/workspace/oozie/core/distcp*
> -rw-rw-r-- 1 asasvari asasvari 1025 Dec 21 18:25 
> /home/asasvari/workspace/oozie/core/distcp-log4j.properties
> -rw-rw-r-- 1 asasvari asasvari 5756 Dec 21 18:26 
> /home/asasvari/workspace/oozie/core/distcp-oozie-1513876931701.log
> {code}
> Those files should be removed after test execution. It would be better to use 
> the oozie/core/target directory so that mvn could clean it.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (OOZIE-3145) TestDistcpMain shall remove created files after test execution

[Julia Kinga Marton (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16305175#comment-16305175
 ] 

Julia Kinga Marton commented on OOZIE-3145:
---

Not only TestDistcpMain produces garbage, I run the tests from core and from 
sharelib and they produced a lot of garbage. See the attached 
Oozie_test_garbage.png. 

In most of the cases the files are created by the tested methods, not by the 
test case itself. I also found an open issue related the creation place of the 
files: [OOZIE-3025].

I think that we should change the place where the files are created in the 
[OOZIE-3025] issue, and then to make a general cleanup (for the other cases as 
well, not only for TestDistcpMain).

What do you think [~asasvari] ?  

> TestDistcpMain shall remove created files after test execution
> --
>
> Key: OOZIE-3145
> URL: https://issues.apache.org/jira/browse/OOZIE-3145
> Project: Oozie
>  Issue Type: Bug
>  Components: tests
>Affects Versions: 5.0.0b1
>Reporter: Attila Sasvari
> Attachments: Oozie_test_garbage.png
>
>
> After running TestDistcpMain, local files created are not removed. Subsequent 
> test runs will reuse those files and fail:
> {code}
> $ ls -lrt /home/asasvari/workspace/oozie/core/distcp*
> -rw-rw-r-- 1 asasvari asasvari 1025 Dec 21 18:25 
> /home/asasvari/workspace/oozie/core/distcp-log4j.properties
> -rw-rw-r-- 1 asasvari asasvari 5756 Dec 21 18:26 
> /home/asasvari/workspace/oozie/core/distcp-oozie-1513876931701.log
> {code}
> Those files should be removed after test execution. It would be better to use 
> the oozie/core/target directory so that mvn could clean it.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (OOZIE-3145) TestDistcpMain shall remove created files after test execution

[Julia Kinga Marton (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-3145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Julia Kinga Marton updated OOZIE-3145:
--
Attachment: Oozie_test_garbage.png

> TestDistcpMain shall remove created files after test execution
> --
>
> Key: OOZIE-3145
> URL: https://issues.apache.org/jira/browse/OOZIE-3145
> Project: Oozie
>  Issue Type: Bug
>  Components: tests
>Affects Versions: 5.0.0b1
>Reporter: Attila Sasvari
> Attachments: Oozie_test_garbage.png
>
>
> After running TestDistcpMain, local files created are not removed. Subsequent 
> test runs will reuse those files and fail:
> {code}
> $ ls -lrt /home/asasvari/workspace/oozie/core/distcp*
> -rw-rw-r-- 1 asasvari asasvari 1025 Dec 21 18:25 
> /home/asasvari/workspace/oozie/core/distcp-log4j.properties
> -rw-rw-r-- 1 asasvari asasvari 5756 Dec 21 18:26 
> /home/asasvari/workspace/oozie/core/distcp-oozie-1513876931701.log
> {code}
> Those files should be removed after test execution. It would be better to use 
> the oozie/core/target directory so that mvn could clean it.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (OOZIE-3085) Improve logging in ActionExecutors

[Julia Kinga Marton (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16305163#comment-16305163
 ] 

Julia Kinga Marton commented on OOZIE-3085:
---

Thank you [~asasvari] for the review. I made the necessary changes, and updated 
the Review Board as well. Can you take a look when you will have some time?

> Improve logging in ActionExecutors
> --
>
> Key: OOZIE-3085
> URL: https://issues.apache.org/jira/browse/OOZIE-3085
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Affects Versions: 4.3.0
>Reporter: Peter Cseh
>Assignee: Julia Kinga Marton
>  Labels: newbie
> Fix For: 5.0.0
>
> Attachments: OOZIE-3085-001.patch, OOZIE-3085-002.patch, 
> OOZIE-3085-003.patch
>
>
> Decision, ForkJoin, Email and FS ActionExecutors are producing no or very 
> little logs on info level. This makes it hard to gather usage information and 
> help with troubleshooting in the case of any issues. Please improve logging 
> in these classes.
> Several subtasks can be separated out from this one



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)