[jira] Subscription: Oozie Patch Available
Issue Subscription Filter: Oozie Patch Available (101 issues) Subscriber: ooziedaily Key Summary OOZIE-3135 Configure log4j2 in SqoopMain https://issues.apache.org/jira/browse/OOZIE-3135 OOZIE-3121 bump all maven plugins to latest versions https://issues.apache.org/jira/browse/OOZIE-3121 OOZIE-3105 testJMXInstrumentation from the org.apache.oozie.util.TestMetricsInstrumentation class is flaky https://issues.apache.org/jira/browse/OOZIE-3105 OOZIE-3094 fix for grammar mistake https://issues.apache.org/jira/browse/OOZIE-3094 OOZIE-3091 Oozie Sqoop Avro Import fails with "java.lang.NoClassDefFoundError: org/apache/avro/mapred/AvroWrapper" https://issues.apache.org/jira/browse/OOZIE-3091 OOZIE-3071 Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0 https://issues.apache.org/jira/browse/OOZIE-3071 OOZIE-3063 Sanitizing variables that are part of openjpa.ConnectionProperties https://issues.apache.org/jira/browse/OOZIE-3063 OOZIE-3062 Set HADOOP_CONF_DIR for spark action https://issues.apache.org/jira/browse/OOZIE-3062 OOZIE-3002 address findbugs errors in client lib https://issues.apache.org/jira/browse/OOZIE-3002 OOZIE-2975 code clean up in pig sharelib, replace Exception with more explicit, add try with resources, StringBuilder instead of StringBuffer https://issues.apache.org/jira/browse/OOZIE-2975 OOZIE-2957 Documentation states that starting a coordinator is possible https://issues.apache.org/jira/browse/OOZIE-2957 OOZIE-2956 Fix Findbugs warnings related to reliance on default encoding in oozie-core https://issues.apache.org/jira/browse/OOZIE-2956 OOZIE-2955 Fix Findbugs warnings related to reliance on default encoding in oozie-client https://issues.apache.org/jira/browse/OOZIE-2955 OOZIE-2954 Fix Checkstyle issues in oozie-client https://issues.apache.org/jira/browse/OOZIE-2954 OOZIE-2953 Fix Checkstyle issues in oozie-tools https://issues.apache.org/jira/browse/OOZIE-2953 OOZIE-2952 Fix Findbugs warnings in oozie-sharelib-oozie https://issues.apache.org/jira/browse/OOZIE-2952 OOZIE-2949 Escape quotes whitespaces in Sqoop field https://issues.apache.org/jira/browse/OOZIE-2949 OOZIE-2942 Fix Findbugs warnings in oozie-examples https://issues.apache.org/jira/browse/OOZIE-2942 OOZIE-2937 Remove redundant groupId from the child pom's https://issues.apache.org/jira/browse/OOZIE-2937 OOZIE-2934 Fix "Exceptional return value of java.io.File.mkdirs() ignored" Findbugs error in oozie-sharelib-spark https://issues.apache.org/jira/browse/OOZIE-2934 OOZIE-2927 Append new line character for Hive2 query using query tag https://issues.apache.org/jira/browse/OOZIE-2927 OOZIE-2914 Consolidate Trim https://issues.apache.org/jira/browse/OOZIE-2914 OOZIE-2883 OOZIE throw the error "Missing [oozie.service.ProxyUserService.proxyuser.oozie.service.ProxyUserService.proxyuser.mr.groups] property" https://issues.apache.org/jira/browse/OOZIE-2883 OOZIE-2877 Oozie Git Action https://issues.apache.org/jira/browse/OOZIE-2877 OOZIE-2867 Timezone handling for Coordinators: emphasize "Continent/City" format https://issues.apache.org/jira/browse/OOZIE-2867 OOZIE-2834 ParameterVerifier logging non-useful warning for workflow definition https://issues.apache.org/jira/browse/OOZIE-2834 OOZIE-2833 when using uber mode the regex pattern used in the extractHeapSizeMB method does not allow heap sizes specified in bytes. https://issues.apache.org/jira/browse/OOZIE-2833 OOZIE-2829 Improve sharelib upload to accept multiple source folders https://issues.apache.org/jira/browse/OOZIE-2829 OOZIE-2826 Falcon feed fails to aws s3; Oozie joda time version does not meet required jar version 2.2 or later https://issues.apache.org/jira/browse/OOZIE-2826 OOZIE-2812 SparkConfigurationService should support loading configurations from multiple Spark versions https://issues.apache.org/jira/browse/OOZIE-2812 OOZIE-2795 Create lib directory or symlink for Oozie CLI during packaging https://issues.apache.org/jira/browse/OOZIE-2795 OOZIE-2791 ShareLib installation may fail on busy Hadoop clusters https://issues.apache.org/jira/browse/OOZIE-2791 OOZIE-2784 Include WEEK as a parameter in the Coordinator Expression Language Evaulator https://issues.apache.org/jira/browse/OOZIE-2784 OOZIE-2779 Mask Hive2 action Beeline JDBC password https://issues.apache.org/jira/browse/OOZIE-2779 OOZIE-2775 Oozie server does not stop if there is an exception during service initalization at startup https://issues.apache.org/jira/browse/OOZIE-2775 OOZIE-2736 Reduce the number of
[jira] Subscription: Oozie Patch Available
Issue Subscription Filter: Oozie Patch Available (111 issues) Subscriber: ooziedaily Key Summary OOZIE-3129 Fix test TestConfigurationService.testOozieConfig https://issues-test.apache.org/jira/browse/OOZIE-3129 OOZIE-3127 Remove redundant check for user https://issues-test.apache.org/jira/browse/OOZIE-3127 OOZIE-3126 Add option to allow list of users to access system config https://issues-test.apache.org/jira/browse/OOZIE-3126 OOZIE-3121 bump all maven plugins to latest versions https://issues-test.apache.org/jira/browse/OOZIE-3121 OOZIE-3118 fix for error: self-closing element not allowed https://issues-test.apache.org/jira/browse/OOZIE-3118 OOZIE-3117 fix for warning: no @throws for https://issues-test.apache.org/jira/browse/OOZIE-3117 OOZIE-3116 fix for warning: no description for @throws https://issues-test.apache.org/jira/browse/OOZIE-3116 OOZIE-3115 fix for javadoc warning: empty tag https://issues-test.apache.org/jira/browse/OOZIE-3115 OOZIE-3113 Retry for ZK lock release https://issues-test.apache.org/jira/browse/OOZIE-3113 OOZIE-3112 SparkConfigrationService overwrites properties provided via --properties-file option in SparkAction https://issues-test.apache.org/jira/browse/OOZIE-3112 OOZIE-3105 testJMXInstrumentation from the org.apache.oozie.util.TestMetricsInstrumentation class is flaky https://issues-test.apache.org/jira/browse/OOZIE-3105 OOZIE-3094 fix for grammar mistake https://issues-test.apache.org/jira/browse/OOZIE-3094 OOZIE-3091 Oozie Sqoop Avro Import fails with "java.lang.NoClassDefFoundError: org/apache/avro/mapred/AvroWrapper" https://issues-test.apache.org/jira/browse/OOZIE-3091 OOZIE-3083 Make improved version Info backward compatible https://issues-test.apache.org/jira/browse/OOZIE-3083 OOZIE-3071 Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0 https://issues-test.apache.org/jira/browse/OOZIE-3071 OOZIE-3063 Sanitizing variables that are part of openjpa.ConnectionProperties https://issues-test.apache.org/jira/browse/OOZIE-3063 OOZIE-3062 Set HADOOP_CONF_DIR for spark action https://issues-test.apache.org/jira/browse/OOZIE-3062 OOZIE-3002 address findbugs errors in client lib https://issues-test.apache.org/jira/browse/OOZIE-3002 OOZIE-2975 code clean up in pig sharelib, replace Exception with more explicit, add try with resources, StringBuilder instead of StringBuffer https://issues-test.apache.org/jira/browse/OOZIE-2975 OOZIE-2973 Make sure Oozie works with Hadoop 3 https://issues-test.apache.org/jira/browse/OOZIE-2973 OOZIE-2969 Drop support for Java 1.7 https://issues-test.apache.org/jira/browse/OOZIE-2969 OOZIE-2957 Documentation states that starting a coordinator is possible https://issues-test.apache.org/jira/browse/OOZIE-2957 OOZIE-2956 Fix Findbugs warnings related to reliance on default encoding in oozie-core https://issues-test.apache.org/jira/browse/OOZIE-2956 OOZIE-2955 Fix Findbugs warnings related to reliance on default encoding in oozie-client https://issues-test.apache.org/jira/browse/OOZIE-2955 OOZIE-2954 Fix Checkstyle issues in oozie-client https://issues-test.apache.org/jira/browse/OOZIE-2954 OOZIE-2953 Fix Checkstyle issues in oozie-tools https://issues-test.apache.org/jira/browse/OOZIE-2953 OOZIE-2952 Fix Findbugs warnings in oozie-sharelib-oozie https://issues-test.apache.org/jira/browse/OOZIE-2952 OOZIE-2949 Escape quotes whitespaces in Sqoop field https://issues-test.apache.org/jira/browse/OOZIE-2949 OOZIE-2942 Fix Findbugs warnings in oozie-examples https://issues-test.apache.org/jira/browse/OOZIE-2942 OOZIE-2937 Remove redundant groupId from the child pom's https://issues-test.apache.org/jira/browse/OOZIE-2937 OOZIE-2934 Fix "Exceptional return value of java.io.File.mkdirs() ignored" Findbugs error in oozie-sharelib-spark https://issues-test.apache.org/jira/browse/OOZIE-2934 OOZIE-2927 Append new line character for Hive2 query using query tag https://issues-test.apache.org/jira/browse/OOZIE-2927 OOZIE-2914 Consolidate Trim https://issues-test.apache.org/jira/browse/OOZIE-2914 OOZIE-2883 OOZIE throw the error "Missing [oozie.service.ProxyUserService.proxyuser.oozie.service.ProxyUserService.proxyuser.mr.groups] property" https://issues-test.apache.org/jira/browse/OOZIE-2883 OOZIE-2877 Oozie Git Action https://issues-test.apache.org/jira/browse/OOZIE-2877 OOZIE-2867 Timezone handling for Coordinators: emphasize "Continent/City" format https://issues-test.apache.org/jira/browse/OOZIE-2867 OOZIE-2834 ParameterVerifier logging
[jira] [Commented] (OOZIE-2775) Oozie server does not stop if there is an exception during service initalization at startup
[
https://issues.apache.org/jira/browse/OOZIE-2775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334999#comment-16334999
]
Attila Sasvari commented on OOZIE-2775:
---
{color:#33}[~andras.piros] Thanks for your comments!{color}
* {color:#33}When a {color}ProvisionException is thrown during
{{guiceInjector.getInstance()}} , {{embeddedOozieServer}} will be null. In the
current form, without additional changes: you can't get a reference to the
Jetty server as it does not exist at this point of the execution (it is
actually created after the Oozie services started). The (failed) Oozie services
created by the provider cannot be stopped either. I wonder if it was really
necessary. If so, we can probably handle it in the ServicesProvider. I will try
to address this in a new patch uploaded to *Reviewboard* *first*.
* Regarding a stop() method: can you expand how it should work? Right now,
stopping the Oozie server happens via a SIGTERM signal sent to the process (see
{{bin/oozie-jetty-server.sh}}. This behaviour is pretty similar to the earlier
version (with Tomcat) where this case was not handled by the means of the
[shutdown hook
mechanism|https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/signals006.html]
as far as I remember. I suppose it is OK that we add such a handler, but of
course there might be races (for example what if signal arrives before we
install it). What do you think?
* I believe
[ExceptionInInitializerError|https://docs.oracle.com/javase/7/docs/api/java/lang/ExceptionInInitializerError.html]
is not suitable for handling this sort of service initialization problems,
[ExceptionInInitializerError|https://docs.oracle.com/javase/7/docs/api/java/lang/ExceptionInInitializerError.html]
is a kind of LinkageError. Documentation says "{{ExceptionInInitializerError}}
is thrown to indicate that an exception occurred during evaluation of a static
initializer or the initializer for a static variable.". A more pragmatic answer
is that BoundProviderFactory's {{get()}} by guice only catches Runtime
exceptions.
> Oozie server does not stop if there is an exception during service
> initalization at startup
> ---
>
> Key: OOZIE-2775
> URL: https://issues.apache.org/jira/browse/OOZIE-2775
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Major
> Attachments: OOZIE-2775-01.patch
>
>
> OOZIE-2666 brought in embedded jetty in Oozie. {{com.google.inject.guice}}
> was selected for dependency management. Unfortunately if there is an
> exception at starting Oozie services (e.g. JPAService initialization, called
> via
> https://github.com/apache/oozie/blob/master/server/src/main/java/org/apache/oozie/server/EmbeddedOozieServer.java#L194),
> Oozie won't stop if it cannot destroy Oozie services.
> In OOZIE-2666 a shutdown hook
> (https://github.com/apache/oozie/blob/master/server/src/main/java/org/apache/oozie/server/EmbeddedOozieServer.java#L184)
> was added to print out startup errors and exceptions, destroy pending oozie
> services, and exit eventually, but it does not work.
> Steps to reproduce a startup failure (for example simulate that Oozie cannot
> find required database driver):
> {code}
> $ cd oozie
> $ mvn clean install assembly:single -DskipTests -DjavaVersion=1.8
> -DtargetVersion=1.7 -Phadoop-2 -Dhadoop.version=2.6.0 -Puber;
> $ cd distro/target/oozie-4.4.0-SNAPSHOT-distro/oozie-4.4.0-SNAPSHOT/
> $ export JETTY_OPTS="-Djavax.net.debug=all
> -agentlib:jdwp=transport=dt_socket,server=y,address=8000,suspend=y
> -Dlog4j.logger.org.eclipse.jetty=TRACE
> -Dlog4j.configuration=./conf/oozie-log4j.properties"
> $ bin/oozied.sh run
> $ mv embedded-oozie-server/webapp/WEB-INF/lib/derby-10.10.1.1.jar ..
>
> {code}
> Exception is printed to stdout
> {code}
>
> org.apache.openjpa.persistence.PersistenceException: Cannot load JDBC driver
> class 'org.apache.derby.jdbc.EmbeddedDriver'
> at
> org.apache.openjpa.jdbc.sql.DBDictionaryFactory.newDBDictionary(DBDictionaryFactory.java:106)
> at
> org.apache.openjpa.jdbc.conf.JDBCConfigurationImpl.getDBDictionaryInstance(JDBCConfigurationImpl.java:603)
> at
> org.apache.openjpa.jdbc.meta.MappingRepository.endConfiguration(MappingRepository.java:1520)
> at
> org.apache.openjpa.lib.conf.Configurations.configureInstance(Configurations.java:533)
> at
> org.apache.openjpa.lib.conf.Configurations.configureInstance(Configurations.java:458)
> at
> org.apache.openjpa.lib.conf.PluginValue.instantiate(PluginValue.java:121)
> at
> org.apache.openjpa.conf.MetaDataRepositoryValue.instantiate(MetaDataRepositoryValue.java:68)
> at
>
[jira] [Commented] (OOZIE-1717) Index user_name column
[
https://issues.apache.org/jira/browse/OOZIE-1717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334549#comment-16334549
]
Hadoop QA commented on OOZIE-1717:
--
Testing JIRA OOZIE-1717
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [server].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 2087
.{color:orange}Tests failed at first run:{color}
TestJavaActionExecutor#testCredentialsSkip
TestCoordActionsKillXCommand#testActionKillCommandDate
TestHiveActionExecutor#testHiveAction
.For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:green}*+1 Overall result, good!, no -1s*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/338/
> Index user_name column
> --
>
> Key: OOZIE-1717
> URL: https://issues.apache.org/jira/browse/OOZIE-1717
> Project: Oozie
> Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Attila Sasvari
>Priority: Minor
> Attachments: OOZIE-1717-00.patch
>
>
> User_name is one of the frequently used filter. Adding index should avoid
> full db scan.
> Index need to be done for BUNDLE_JOBS,COORD_JOBS and WF_JOBS.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Success: OOZIE-1717 PreCommit Build #338
Jira: https://issues.apache.org/jira/browse/OOZIE-1717
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/338/
###
## LAST 100 LINES OF THE CONSOLE
###
[...truncated 1.58 MB...]
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/oozie].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/pig].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/hive].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/spark].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [client].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [examples].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [docs].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [server].
[INFO] There are no new bugs found totally].
[TRACE] FindBugs diffs checked and reports created
[TRACE] Summary file size is 2110 bytes
[TRACE] Full summary file size is 1248 bytes
[TRACE] File
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar]
removed
[TRACE] File
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5sum]
removed
Running test-patch task BACKWARDS_COMPATIBILITY
Running test-patch task TESTS
Running test-patch task DISTRO
Testing JIRA OOZIE-1717
Cleaning local git workspace
+1 PATCH_APPLIES
+1 CLEAN
+1 RAW_PATCH_ANALYSIS
+1 the patch does not introduce any @author tags
+1 the patch does not introduce any tabs
+1 the patch does not introduce any trailing spaces
+1 the patch does not introduce any line longer than 132
+1 the patch adds/modifies 1 testcase(s)
+1 RAT
+1 the patch does not seem to introduce new RAT warnings
+1 JAVADOC
+1 the patch does not seem to introduce new Javadoc warnings
+1 COMPILE
+1 HEAD compiles
+1 patch compiles
+1 the patch does not seem to introduce new javac warnings
+1 There are no new bugs found in total.
+1 There are no new bugs found in [core].
+1 There are no new bugs found in [tools].
+1 There are no new bugs found in [sharelib/hive2].
+1 There are no new bugs found in [sharelib/distcp].
+1 There are no new bugs found in [sharelib/hcatalog].
+1 There are no new bugs found in [sharelib/streaming].
+1 There are no new bugs found in [sharelib/sqoop].
+1 There are no new bugs found in [sharelib/oozie].
+1 There are no new bugs found in [sharelib/pig].
+1 There are no new bugs found in [sharelib/hive].
+1 There are no new bugs found in [sharelib/spark].
+1 There are no new bugs found in [client].
+1 There are no new bugs found in [examples].
+1 There are no new bugs found in [docs].
+1 There are no new bugs found in [server].
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient
annotations
+1 the patch does not modify JPA files
+1 TESTS
Tests run: 2087
Tests failed at first run:
TestJavaActionExecutor#testCredentialsSkip
TestCoordActionsKillXCommand#testActionKillCommandDate
TestHiveActionExecutor#testHiveAction
For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
+1 DISTRO
+1 distro tarball builds with the patch
+1 Overall result, good!, no -1s
The full output of the test-patch run is available at
https://builds.apache.org/job/PreCommit-OOZIE-Build/338/
Adding comment to JIRA
% Total% Received % Xferd Average Speed TimeTime Time Current
Dload Upload Total SpentLeft Speed
0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0
0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0100
30090 0 100 3009 0 2418 0:00:01 0:00:01 --:--:--
2416{"self":"https://issues.apache.org/jira/rest/api/2/issue/12697804/comment/16334549","id":"16334549","author":{"self":"https://issues.apache.org/jira/rest/api/2/user?username=hadoopqa","name":"hadoopqa","key":"hadoopqa","emailAddress":"blackhole
at hadoop dot apache dot
[jira] [Commented] (OOZIE-1717) Index user_name column
[
https://issues.apache.org/jira/browse/OOZIE-1717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334430#comment-16334430
]
Attila Sasvari commented on OOZIE-1717:
---
patch 00 adds 6 new indices and changes *db.version* to *3.1*. Here are the
indices from the SQL file generated by the OozieDBCLI tool:
CREATE INDEX I_BNDLTNS_BUNDLE_ID ON BUNDLE_ACTIONS (bundle_id);
CREATE INDEX I_BNDLJBS_CREATED_TIME ON BUNDLE_JOBS (created_time);
{color:#33}*CREATE INDEX I_BNDLJBS_END_TIME ON BUNDLE_JOBS
(end_time);*{color}
CREATE INDEX I_BNDLJBS_LAST_MODIFIED_TIME ON BUNDLE_JOBS (last_modified_time);
CREATE INDEX I_BNDLJBS_STATUS ON BUNDLE_JOBS (status);
CREATE INDEX I_BNDLJBS_SUSPENDED_TIME ON BUNDLE_JOBS (suspended_time);
*CREATE INDEX I_BNDLJBS_USER_NAME ON BUNDLE_JOBS (user_name);*
CREATE INDEX I_CRD_TNS_CREATED_TIME ON COORD_ACTIONS (created_time);
CREATE INDEX I_CRD_TNS_EXTERNAL_ID ON COORD_ACTIONS (external_id);
CREATE INDEX I_CRD_TNS_JOB_ID ON COORD_ACTIONS (job_id);
CREATE INDEX I_CRD_TNS_LAST_MODIFIED_TIME ON COORD_ACTIONS (last_modified_time);
CREATE INDEX I_CRD_TNS_NOMINAL_TIME ON COORD_ACTIONS (nominal_time);
CREATE INDEX I_CRD_TNS_RERUN_TIME ON COORD_ACTIONS (rerun_time);
CREATE INDEX I_CRD_TNS_STATUS ON COORD_ACTIONS (status);
CREATE INDEX I_CRD_JBS_BUNDLE_ID ON COORD_JOBS (bundle_id);
CREATE INDEX I_CRD_JBS_CREATED_TIME ON COORD_JOBS (created_time);
*CREATE INDEX I_CRD_JBS_END_TIME ON COORD_JOBS (end_time);*
CREATE INDEX I_CRD_JBS_LAST_MODIFIED_TIME ON COORD_JOBS (last_modified_time);
CREATE INDEX I_CRD_JBS_NEXT_MATD_TIME ON COORD_JOBS (next_matd_time);
CREATE INDEX I_CRD_JBS_STATUS ON COORD_JOBS (status);
CREATE INDEX I_CRD_JBS_SUSPENDED_TIME ON COORD_JOBS (suspended_time);
*CREATE INDEX I_CRD_JBS_USER_NAME ON COORD_JOBS (user_name);*
CREATE INDEX I_SL_VNTS_DTYPE ON SLA_EVENTS (bean_type);
CREATE INDEX I_SL_RRTN_NOMINAL_TIME ON SLA_REGISTRATION (nominal_time);
CREATE INDEX I_SL_SMRY_APP_NAME ON SLA_SUMMARY (app_name);
CREATE INDEX I_SL_SMRY_EVENT_PROCESSED ON SLA_SUMMARY (event_processed);
CREATE INDEX I_SL_SMRY_LAST_MODIFIED ON SLA_SUMMARY (last_modified);
CREATE INDEX I_SL_SMRY_NOMINAL_TIME ON SLA_SUMMARY (nominal_time);
CREATE INDEX I_SL_SMRY_PARENT_ID ON SLA_SUMMARY (parent_id);
CREATE INDEX I_WF_CTNS_PENDING_AGE ON WF_ACTIONS (pending_age);
CREATE INDEX I_WF_CTNS_STATUS ON WF_ACTIONS (status);
CREATE INDEX I_WF_CTNS_WF_ID ON WF_ACTIONS (wf_id);
*CREATE INDEX I_WF_JOBS_CREATED_TIME ON WF_JOBS (created_time);*
CREATE INDEX I_WF_JOBS_END_TIME ON WF_JOBS (end_time);
CREATE INDEX I_WF_JOBS_EXTERNAL_ID ON WF_JOBS (external_id);
CREATE INDEX I_WF_JOBS_LAST_MODIFIED_TIME ON WF_JOBS (last_modified_time);
CREATE INDEX I_WF_JOBS_PARENT_ID ON WF_JOBS (parent_id);
CREATE INDEX I_WF_JOBS_STATUS ON WF_JOBS (status);
*CREATE INDEX I_WF_JOBS_USER_NAME ON WF_JOBS (user_name);*
Do we want to add anything else?
I am going to run some more tests using Hue / Oozie Web UI (using MariaDB with
slow query logging enabled).
> Index user_name column
> --
>
> Key: OOZIE-1717
> URL: https://issues.apache.org/jira/browse/OOZIE-1717
> Project: Oozie
> Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Attila Sasvari
>Priority: Minor
> Attachments: OOZIE-1717-00.patch
>
>
> User_name is one of the frequently used filter. Adding index should avoid
> full db scan.
> Index need to be done for BUNDLE_JOBS,COORD_JOBS and WF_JOBS.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-1717) Index user_name column
[ https://issues.apache.org/jira/browse/OOZIE-1717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334393#comment-16334393 ] Hadoop QA commented on OOZIE-1717: -- PreCommit-OOZIE-Build started > Index user_name column > -- > > Key: OOZIE-1717 > URL: https://issues.apache.org/jira/browse/OOZIE-1717 > Project: Oozie > Issue Type: Bug >Reporter: Purshotam Shah >Assignee: Attila Sasvari >Priority: Minor > Attachments: OOZIE-1717-00.patch > > > User_name is one of the frequently used filter. Adding index should avoid > full db scan. > Index need to be done for BUNDLE_JOBS,COORD_JOBS and WF_JOBS. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-1717) Index user_name column
[ https://issues.apache.org/jira/browse/OOZIE-1717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Sasvari updated OOZIE-1717: -- Attachment: OOZIE-1717-00.patch > Index user_name column > -- > > Key: OOZIE-1717 > URL: https://issues.apache.org/jira/browse/OOZIE-1717 > Project: Oozie > Issue Type: Bug >Reporter: Purshotam Shah >Assignee: Attila Sasvari >Priority: Minor > Attachments: OOZIE-1717-00.patch > > > User_name is one of the frequently used filter. Adding index should avoid > full db scan. > Index need to be done for BUNDLE_JOBS,COORD_JOBS and WF_JOBS. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (OOZIE-1717) Index user_name column
[ https://issues.apache.org/jira/browse/OOZIE-1717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Sasvari reassigned OOZIE-1717: - Assignee: Attila Sasvari > Index user_name column > -- > > Key: OOZIE-1717 > URL: https://issues.apache.org/jira/browse/OOZIE-1717 > Project: Oozie > Issue Type: Bug >Reporter: Purshotam Shah >Assignee: Attila Sasvari >Priority: Minor > > User_name is one of the frequently used filter. Adding index should avoid > full db scan. > Index need to be done for BUNDLE_JOBS,COORD_JOBS and WF_JOBS. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-3165) Ensure that tests do not create junk in the project folder
[ https://issues.apache.org/jira/browse/OOZIE-3165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Piros updated OOZIE-3165: Summary: Ensure that tests do not create junk in the project folder (was: Ensure that tests does not create junk in the project folder) > Ensure that tests do not create junk in the project folder > -- > > Key: OOZIE-3165 > URL: https://issues.apache.org/jira/browse/OOZIE-3165 > Project: Oozie > Issue Type: Bug > Components: tests >Reporter: Peter Cseh >Priority: Major > > Now that [~kmarton] cleaned up all the junk our tests were creating as part > of OOZIE-3145 we should extend our pre-commit check to make sure no tmp files > will be around after the tests are run. > > We should extend our pre-commit checks with this logic. It should be > something like: > # make sure git s returns nothing (I think this is already part of our > pre-commit) > # apply the patch > # make a git add . or git commit -am "tmp" or something similar so we can > decide whether a change is part of the patch or not. > # run tests > # call git status to check if any of the git tracked files are changed or > any new ones are added during test runs > # add -1 to the pre-commit report if there are files laying around after the > tests -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (OOZIE-3165) Ensure that tests does not create junk in the project folder
Peter Cseh created OOZIE-3165: - Summary: Ensure that tests does not create junk in the project folder Key: OOZIE-3165 URL: https://issues.apache.org/jira/browse/OOZIE-3165 Project: Oozie Issue Type: Bug Components: tests Reporter: Peter Cseh Now that [~kmarton] cleaned up all the junk our tests were creating as part of OOZIE-3145 we should extend our pre-commit check to make sure no tmp files will be around after the tests are run. We should extend our pre-commit checks with this logic. It should be something like: # make sure git s returns nothing (I think this is already part of our pre-commit) # apply the patch # make a git add . or git commit -am "tmp" or something similar so we can decide whether a change is part of the patch or not. # run tests # call git status to check if any of the git tracked files are changed or any new ones are added during test runs # add -1 to the pre-commit report if there are files laying around after the tests -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3145) >git status< should be clean after >mvn test< was called
[
https://issues.apache.org/jira/browse/OOZIE-3145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334256#comment-16334256
]
Peter Cseh commented on OOZIE-3145:
---
+1
> >git status< should be clean after >mvn test< was called
>
>
> Key: OOZIE-3145
> URL: https://issues.apache.org/jira/browse/OOZIE-3145
> Project: Oozie
> Issue Type: Bug
> Components: tests
>Affects Versions: 5.0.0b1
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Major
> Attachments: OOZIE-3145-001.patch, OOZIE-3145-002.patch,
> OOZIE-3145-003.patch, Oozie_test_garbage.png
>
>
> After running TestDistcpMain, local files created are not removed. Subsequent
> test runs will reuse those files and fail:
> {code}
> $ ls -lrt /home/asasvari/workspace/oozie/core/distcp*
> -rw-rw-r-- 1 asasvari asasvari 1025 Dec 21 18:25
> /home/asasvari/workspace/oozie/core/distcp-log4j.properties
> -rw-rw-r-- 1 asasvari asasvari 5756 Dec 21 18:26
> /home/asasvari/workspace/oozie/core/distcp-oozie-1513876931701.log
> {code}
> Those files should be removed after test execution. It would be better to use
> the oozie/core/target directory so that mvn could clean it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-2775) Oozie server does not stop if there is an exception during service initalization at startup
[
https://issues.apache.org/jira/browse/OOZIE-2775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334246#comment-16334246
]
Andras Piros commented on OOZIE-2775:
-
[~asasvari] thanks for the patch!
Couple of thoughts:
* overall fail-fast approach looks good to me
* may be better to have a ReviewBoard review request for discussion
* it may be prudent to emit a log message containing
[{{ProvisionException#getErrorMessages()}}|https://google.github.io/guice/api-docs/latest/javadoc/index.html?com/google/inject/ProvisionException.html]
while on {{EmbeddedOozieServer:225}}
* you're right, calling {{oozieServices#destroy()}} is needed on
{{ServicesProvider:35}}. In that extent, all the Oozie {{Service}} instances
will be cleaned up, including the {{Services}} singleton itself
* I think it's also needed that in {{EmbeddedOozieServer}} we create method
{{stop()}} where the {{org.eclipse.jetty.server.Server}} instance methods
{{stop()}} and {{destroy()}} would also be called, to clean up everything
that's Jetty related. Let's call this new method also from
{{EmbeddedOozieServer#main()}}
* for me it's unclear why {{ExceptionInInitializerError}} was switched to
{{IllegalStateException}} in {{ServicesProvider:35}}
> Oozie server does not stop if there is an exception during service
> initalization at startup
> ---
>
> Key: OOZIE-2775
> URL: https://issues.apache.org/jira/browse/OOZIE-2775
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Major
> Attachments: OOZIE-2775-01.patch
>
>
> OOZIE-2666 brought in embedded jetty in Oozie. {{com.google.inject.guice}}
> was selected for dependency management. Unfortunately if there is an
> exception at starting Oozie services (e.g. JPAService initialization, called
> via
> https://github.com/apache/oozie/blob/master/server/src/main/java/org/apache/oozie/server/EmbeddedOozieServer.java#L194),
> Oozie won't stop if it cannot destroy Oozie services.
> In OOZIE-2666 a shutdown hook
> (https://github.com/apache/oozie/blob/master/server/src/main/java/org/apache/oozie/server/EmbeddedOozieServer.java#L184)
> was added to print out startup errors and exceptions, destroy pending oozie
> services, and exit eventually, but it does not work.
> Steps to reproduce a startup failure (for example simulate that Oozie cannot
> find required database driver):
> {code}
> $ cd oozie
> $ mvn clean install assembly:single -DskipTests -DjavaVersion=1.8
> -DtargetVersion=1.7 -Phadoop-2 -Dhadoop.version=2.6.0 -Puber;
> $ cd distro/target/oozie-4.4.0-SNAPSHOT-distro/oozie-4.4.0-SNAPSHOT/
> $ export JETTY_OPTS="-Djavax.net.debug=all
> -agentlib:jdwp=transport=dt_socket,server=y,address=8000,suspend=y
> -Dlog4j.logger.org.eclipse.jetty=TRACE
> -Dlog4j.configuration=./conf/oozie-log4j.properties"
> $ bin/oozied.sh run
> $ mv embedded-oozie-server/webapp/WEB-INF/lib/derby-10.10.1.1.jar ..
>
> {code}
> Exception is printed to stdout
> {code}
>
> org.apache.openjpa.persistence.PersistenceException: Cannot load JDBC driver
> class 'org.apache.derby.jdbc.EmbeddedDriver'
> at
> org.apache.openjpa.jdbc.sql.DBDictionaryFactory.newDBDictionary(DBDictionaryFactory.java:106)
> at
> org.apache.openjpa.jdbc.conf.JDBCConfigurationImpl.getDBDictionaryInstance(JDBCConfigurationImpl.java:603)
> at
> org.apache.openjpa.jdbc.meta.MappingRepository.endConfiguration(MappingRepository.java:1520)
> at
> org.apache.openjpa.lib.conf.Configurations.configureInstance(Configurations.java:533)
> at
> org.apache.openjpa.lib.conf.Configurations.configureInstance(Configurations.java:458)
> at
> org.apache.openjpa.lib.conf.PluginValue.instantiate(PluginValue.java:121)
> at
> org.apache.openjpa.conf.MetaDataRepositoryValue.instantiate(MetaDataRepositoryValue.java:68)
> at
> org.apache.openjpa.lib.conf.ObjectValue.instantiate(ObjectValue.java:83)
> at
> org.apache.openjpa.conf.OpenJPAConfigurationImpl.newMetaDataRepositoryInstance(OpenJPAConfigurationImpl.java:967)
> at
> org.apache.openjpa.conf.OpenJPAConfigurationImpl.getMetaDataRepositoryInstance(OpenJPAConfigurationImpl.java:958)
> at
> org.apache.openjpa.kernel.AbstractBrokerFactory.makeReadOnly(AbstractBrokerFactory.java:642)
> at
> org.apache.openjpa.kernel.AbstractBrokerFactory.newBroker(AbstractBrokerFactory.java:202)
> at
> org.apache.openjpa.kernel.DelegatingBrokerFactory.newBroker(DelegatingBrokerFactory.java:154)
> at
> org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:226)
> at
> org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:153)
[jira] [Commented] (OOZIE-2775) Oozie server does not stop if there is an exception during service initalization at startup
[
https://issues.apache.org/jira/browse/OOZIE-2775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334196#comment-16334196
]
Attila Sasvari commented on OOZIE-2775:
---
[~andras.piros], [~gezapeti] can you take a look at the first patch? Do you see
any problems with the current fail-fast approach? Would it be enough to call
{{Services.destroy()}} or some extra care is required? If so, could you
recommend a better way to gracefully stop Oozie services that might have been
already initialized and started during startup? Thanks.
> Oozie server does not stop if there is an exception during service
> initalization at startup
> ---
>
> Key: OOZIE-2775
> URL: https://issues.apache.org/jira/browse/OOZIE-2775
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Major
> Attachments: OOZIE-2775-01.patch
>
>
> OOZIE-2666 brought in embedded jetty in Oozie. {{com.google.inject.guice}}
> was selected for dependency management. Unfortunately if there is an
> exception at starting Oozie services (e.g. JPAService initialization, called
> via
> https://github.com/apache/oozie/blob/master/server/src/main/java/org/apache/oozie/server/EmbeddedOozieServer.java#L194),
> Oozie won't stop if it cannot destroy Oozie services.
> In OOZIE-2666 a shutdown hook
> (https://github.com/apache/oozie/blob/master/server/src/main/java/org/apache/oozie/server/EmbeddedOozieServer.java#L184)
> was added to print out startup errors and exceptions, destroy pending oozie
> services, and exit eventually, but it does not work.
> Steps to reproduce a startup failure (for example simulate that Oozie cannot
> find required database driver):
> {code}
> $ cd oozie
> $ mvn clean install assembly:single -DskipTests -DjavaVersion=1.8
> -DtargetVersion=1.7 -Phadoop-2 -Dhadoop.version=2.6.0 -Puber;
> $ cd distro/target/oozie-4.4.0-SNAPSHOT-distro/oozie-4.4.0-SNAPSHOT/
> $ export JETTY_OPTS="-Djavax.net.debug=all
> -agentlib:jdwp=transport=dt_socket,server=y,address=8000,suspend=y
> -Dlog4j.logger.org.eclipse.jetty=TRACE
> -Dlog4j.configuration=./conf/oozie-log4j.properties"
> $ bin/oozied.sh run
> $ mv embedded-oozie-server/webapp/WEB-INF/lib/derby-10.10.1.1.jar ..
>
> {code}
> Exception is printed to stdout
> {code}
>
> org.apache.openjpa.persistence.PersistenceException: Cannot load JDBC driver
> class 'org.apache.derby.jdbc.EmbeddedDriver'
> at
> org.apache.openjpa.jdbc.sql.DBDictionaryFactory.newDBDictionary(DBDictionaryFactory.java:106)
> at
> org.apache.openjpa.jdbc.conf.JDBCConfigurationImpl.getDBDictionaryInstance(JDBCConfigurationImpl.java:603)
> at
> org.apache.openjpa.jdbc.meta.MappingRepository.endConfiguration(MappingRepository.java:1520)
> at
> org.apache.openjpa.lib.conf.Configurations.configureInstance(Configurations.java:533)
> at
> org.apache.openjpa.lib.conf.Configurations.configureInstance(Configurations.java:458)
> at
> org.apache.openjpa.lib.conf.PluginValue.instantiate(PluginValue.java:121)
> at
> org.apache.openjpa.conf.MetaDataRepositoryValue.instantiate(MetaDataRepositoryValue.java:68)
> at
> org.apache.openjpa.lib.conf.ObjectValue.instantiate(ObjectValue.java:83)
> at
> org.apache.openjpa.conf.OpenJPAConfigurationImpl.newMetaDataRepositoryInstance(OpenJPAConfigurationImpl.java:967)
> at
> org.apache.openjpa.conf.OpenJPAConfigurationImpl.getMetaDataRepositoryInstance(OpenJPAConfigurationImpl.java:958)
> at
> org.apache.openjpa.kernel.AbstractBrokerFactory.makeReadOnly(AbstractBrokerFactory.java:642)
> at
> org.apache.openjpa.kernel.AbstractBrokerFactory.newBroker(AbstractBrokerFactory.java:202)
> at
> org.apache.openjpa.kernel.DelegatingBrokerFactory.newBroker(DelegatingBrokerFactory.java:154)
> at
> org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:226)
> at
> org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:153)
> at
> org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:59)
> at
> org.apache.oozie.service.JPAService.getEntityManager(JPAService.java:514)
> at org.apache.oozie.service.JPAService.init(JPAService.java:215)
> at
> org.apache.oozie.service.Services.setServiceInternal(Services.java:387)
> at org.apache.oozie.service.Services.setService(Services.java:373)
> at org.apache.oozie.service.Services.loadServices(Services.java:305)
> at org.apache.oozie.service.Services.init(Services.java:213)
> at
> org.apache.oozie.server.guice.ServicesProvider.get(ServicesProvider.java:31)
> at
