[jira] [Commented] (OOZIE-2775) Oozie server does not stop if there is an exception during service initalization at startup
[
https://issues.apache.org/jira/browse/OOZIE-2775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335913#comment-16335913
]
Hadoop QA commented on OOZIE-2775:
--
PreCommit-OOZIE-Build started
> Oozie server does not stop if there is an exception during service
> initalization at startup
> ---
>
> Key: OOZIE-2775
> URL: https://issues.apache.org/jira/browse/OOZIE-2775
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Major
> Attachments: OOZIE-2775-01.patch, OOZIE-2775-02.patch
>
>
> OOZIE-2666 brought in embedded jetty in Oozie. {{com.google.inject.guice}}
> was selected for dependency management. Unfortunately if there is an
> exception at starting Oozie services (e.g. JPAService initialization, called
> via
> https://github.com/apache/oozie/blob/master/server/src/main/java/org/apache/oozie/server/EmbeddedOozieServer.java#L194),
> Oozie won't stop if it cannot destroy Oozie services.
> In OOZIE-2666 a shutdown hook
> (https://github.com/apache/oozie/blob/master/server/src/main/java/org/apache/oozie/server/EmbeddedOozieServer.java#L184)
> was added to print out startup errors and exceptions, destroy pending oozie
> services, and exit eventually, but it does not work.
> Steps to reproduce a startup failure (for example simulate that Oozie cannot
> find required database driver):
> {code}
> $ cd oozie
> $ mvn clean install assembly:single -DskipTests -DjavaVersion=1.8
> -DtargetVersion=1.7 -Phadoop-2 -Dhadoop.version=2.6.0 -Puber;
> $ cd distro/target/oozie-4.4.0-SNAPSHOT-distro/oozie-4.4.0-SNAPSHOT/
> $ export JETTY_OPTS="-Djavax.net.debug=all
> -agentlib:jdwp=transport=dt_socket,server=y,address=8000,suspend=y
> -Dlog4j.logger.org.eclipse.jetty=TRACE
> -Dlog4j.configuration=./conf/oozie-log4j.properties"
> $ bin/oozied.sh run
> $ mv embedded-oozie-server/webapp/WEB-INF/lib/derby-10.10.1.1.jar ..
>
> {code}
> Exception is printed to stdout
> {code}
>
> org.apache.openjpa.persistence.PersistenceException: Cannot load JDBC driver
> class 'org.apache.derby.jdbc.EmbeddedDriver'
> at
> org.apache.openjpa.jdbc.sql.DBDictionaryFactory.newDBDictionary(DBDictionaryFactory.java:106)
> at
> org.apache.openjpa.jdbc.conf.JDBCConfigurationImpl.getDBDictionaryInstance(JDBCConfigurationImpl.java:603)
> at
> org.apache.openjpa.jdbc.meta.MappingRepository.endConfiguration(MappingRepository.java:1520)
> at
> org.apache.openjpa.lib.conf.Configurations.configureInstance(Configurations.java:533)
> at
> org.apache.openjpa.lib.conf.Configurations.configureInstance(Configurations.java:458)
> at
> org.apache.openjpa.lib.conf.PluginValue.instantiate(PluginValue.java:121)
> at
> org.apache.openjpa.conf.MetaDataRepositoryValue.instantiate(MetaDataRepositoryValue.java:68)
> at
> org.apache.openjpa.lib.conf.ObjectValue.instantiate(ObjectValue.java:83)
> at
> org.apache.openjpa.conf.OpenJPAConfigurationImpl.newMetaDataRepositoryInstance(OpenJPAConfigurationImpl.java:967)
> at
> org.apache.openjpa.conf.OpenJPAConfigurationImpl.getMetaDataRepositoryInstance(OpenJPAConfigurationImpl.java:958)
> at
> org.apache.openjpa.kernel.AbstractBrokerFactory.makeReadOnly(AbstractBrokerFactory.java:642)
> at
> org.apache.openjpa.kernel.AbstractBrokerFactory.newBroker(AbstractBrokerFactory.java:202)
> at
> org.apache.openjpa.kernel.DelegatingBrokerFactory.newBroker(DelegatingBrokerFactory.java:154)
> at
> org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:226)
> at
> org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:153)
> at
> org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:59)
> at
> org.apache.oozie.service.JPAService.getEntityManager(JPAService.java:514)
> at org.apache.oozie.service.JPAService.init(JPAService.java:215)
> at
> org.apache.oozie.service.Services.setServiceInternal(Services.java:387)
> at org.apache.oozie.service.Services.setService(Services.java:373)
> at org.apache.oozie.service.Services.loadServices(Services.java:305)
> at org.apache.oozie.service.Services.init(Services.java:213)
> at
> org.apache.oozie.server.guice.ServicesProvider.get(ServicesProvider.java:31)
> at
> org.apache.oozie.server.guice.ServicesProvider.get(ServicesProvider.java:25)
> at
> com.google.inject.internal.BoundProviderFactory.get(BoundProviderFactory.java:55)
> at
> com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)
> at
>
[jira] [Commented] (OOZIE-2775) Oozie server does not stop if there is an exception during service initalization at startup
[
https://issues.apache.org/jira/browse/OOZIE-2775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335854#comment-16335854
]
Attila Sasvari commented on OOZIE-2775:
---
[~andras.piros] thanks for the review! Regarding the failure:
junit.framework.AssertionFailedError: YARN App state for app
application_1516711234396_0003 expected: but was: at
org.apache.oozie.action.hadoop.TestHiveActionExecutor.testHiveAction(TestHiveActionExecutor.java:158)
it happened twice (rerun).
Failed test passes locally and I am pretty sure it is unrelated to my changes:
the patch does not touches oozie-core and a seperate embedded jetty server and
miniservers are used for testing in the core module.
> Oozie server does not stop if there is an exception during service
> initalization at startup
> ---
>
> Key: OOZIE-2775
> URL: https://issues.apache.org/jira/browse/OOZIE-2775
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Major
> Attachments: OOZIE-2775-01.patch, OOZIE-2775-02.patch
>
>
> OOZIE-2666 brought in embedded jetty in Oozie. {{com.google.inject.guice}}
> was selected for dependency management. Unfortunately if there is an
> exception at starting Oozie services (e.g. JPAService initialization, called
> via
> https://github.com/apache/oozie/blob/master/server/src/main/java/org/apache/oozie/server/EmbeddedOozieServer.java#L194),
> Oozie won't stop if it cannot destroy Oozie services.
> In OOZIE-2666 a shutdown hook
> (https://github.com/apache/oozie/blob/master/server/src/main/java/org/apache/oozie/server/EmbeddedOozieServer.java#L184)
> was added to print out startup errors and exceptions, destroy pending oozie
> services, and exit eventually, but it does not work.
> Steps to reproduce a startup failure (for example simulate that Oozie cannot
> find required database driver):
> {code}
> $ cd oozie
> $ mvn clean install assembly:single -DskipTests -DjavaVersion=1.8
> -DtargetVersion=1.7 -Phadoop-2 -Dhadoop.version=2.6.0 -Puber;
> $ cd distro/target/oozie-4.4.0-SNAPSHOT-distro/oozie-4.4.0-SNAPSHOT/
> $ export JETTY_OPTS="-Djavax.net.debug=all
> -agentlib:jdwp=transport=dt_socket,server=y,address=8000,suspend=y
> -Dlog4j.logger.org.eclipse.jetty=TRACE
> -Dlog4j.configuration=./conf/oozie-log4j.properties"
> $ bin/oozied.sh run
> $ mv embedded-oozie-server/webapp/WEB-INF/lib/derby-10.10.1.1.jar ..
>
> {code}
> Exception is printed to stdout
> {code}
>
> org.apache.openjpa.persistence.PersistenceException: Cannot load JDBC driver
> class 'org.apache.derby.jdbc.EmbeddedDriver'
> at
> org.apache.openjpa.jdbc.sql.DBDictionaryFactory.newDBDictionary(DBDictionaryFactory.java:106)
> at
> org.apache.openjpa.jdbc.conf.JDBCConfigurationImpl.getDBDictionaryInstance(JDBCConfigurationImpl.java:603)
> at
> org.apache.openjpa.jdbc.meta.MappingRepository.endConfiguration(MappingRepository.java:1520)
> at
> org.apache.openjpa.lib.conf.Configurations.configureInstance(Configurations.java:533)
> at
> org.apache.openjpa.lib.conf.Configurations.configureInstance(Configurations.java:458)
> at
> org.apache.openjpa.lib.conf.PluginValue.instantiate(PluginValue.java:121)
> at
> org.apache.openjpa.conf.MetaDataRepositoryValue.instantiate(MetaDataRepositoryValue.java:68)
> at
> org.apache.openjpa.lib.conf.ObjectValue.instantiate(ObjectValue.java:83)
> at
> org.apache.openjpa.conf.OpenJPAConfigurationImpl.newMetaDataRepositoryInstance(OpenJPAConfigurationImpl.java:967)
> at
> org.apache.openjpa.conf.OpenJPAConfigurationImpl.getMetaDataRepositoryInstance(OpenJPAConfigurationImpl.java:958)
> at
> org.apache.openjpa.kernel.AbstractBrokerFactory.makeReadOnly(AbstractBrokerFactory.java:642)
> at
> org.apache.openjpa.kernel.AbstractBrokerFactory.newBroker(AbstractBrokerFactory.java:202)
> at
> org.apache.openjpa.kernel.DelegatingBrokerFactory.newBroker(DelegatingBrokerFactory.java:154)
> at
> org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:226)
> at
> org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:153)
> at
> org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:59)
> at
> org.apache.oozie.service.JPAService.getEntityManager(JPAService.java:514)
> at org.apache.oozie.service.JPAService.init(JPAService.java:215)
> at
> org.apache.oozie.service.Services.setServiceInternal(Services.java:387)
> at org.apache.oozie.service.Services.setService(Services.java:373)
> at
[jira] [Commented] (OOZIE-2775) Oozie server does not stop if there is an exception during service initalization at startup
[
https://issues.apache.org/jira/browse/OOZIE-2775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335824#comment-16335824
]
Andras Piros commented on OOZIE-2775:
-
Thanks for the new patch [~asasvari]! Your new patch looks good to me.
Some details:
* {{Exception}} type: let it be. Thanks for looking at it
* I think the current implementation and usage patterns of {{stop()}} are
sufficient and easy to maintain
+1 (pending the last failing test triage
{{TestHiveActionExecutor#testHiveAction}})
> Oozie server does not stop if there is an exception during service
> initalization at startup
> ---
>
> Key: OOZIE-2775
> URL: https://issues.apache.org/jira/browse/OOZIE-2775
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Major
> Attachments: OOZIE-2775-01.patch, OOZIE-2775-02.patch
>
>
> OOZIE-2666 brought in embedded jetty in Oozie. {{com.google.inject.guice}}
> was selected for dependency management. Unfortunately if there is an
> exception at starting Oozie services (e.g. JPAService initialization, called
> via
> https://github.com/apache/oozie/blob/master/server/src/main/java/org/apache/oozie/server/EmbeddedOozieServer.java#L194),
> Oozie won't stop if it cannot destroy Oozie services.
> In OOZIE-2666 a shutdown hook
> (https://github.com/apache/oozie/blob/master/server/src/main/java/org/apache/oozie/server/EmbeddedOozieServer.java#L184)
> was added to print out startup errors and exceptions, destroy pending oozie
> services, and exit eventually, but it does not work.
> Steps to reproduce a startup failure (for example simulate that Oozie cannot
> find required database driver):
> {code}
> $ cd oozie
> $ mvn clean install assembly:single -DskipTests -DjavaVersion=1.8
> -DtargetVersion=1.7 -Phadoop-2 -Dhadoop.version=2.6.0 -Puber;
> $ cd distro/target/oozie-4.4.0-SNAPSHOT-distro/oozie-4.4.0-SNAPSHOT/
> $ export JETTY_OPTS="-Djavax.net.debug=all
> -agentlib:jdwp=transport=dt_socket,server=y,address=8000,suspend=y
> -Dlog4j.logger.org.eclipse.jetty=TRACE
> -Dlog4j.configuration=./conf/oozie-log4j.properties"
> $ bin/oozied.sh run
> $ mv embedded-oozie-server/webapp/WEB-INF/lib/derby-10.10.1.1.jar ..
>
> {code}
> Exception is printed to stdout
> {code}
>
> org.apache.openjpa.persistence.PersistenceException: Cannot load JDBC driver
> class 'org.apache.derby.jdbc.EmbeddedDriver'
> at
> org.apache.openjpa.jdbc.sql.DBDictionaryFactory.newDBDictionary(DBDictionaryFactory.java:106)
> at
> org.apache.openjpa.jdbc.conf.JDBCConfigurationImpl.getDBDictionaryInstance(JDBCConfigurationImpl.java:603)
> at
> org.apache.openjpa.jdbc.meta.MappingRepository.endConfiguration(MappingRepository.java:1520)
> at
> org.apache.openjpa.lib.conf.Configurations.configureInstance(Configurations.java:533)
> at
> org.apache.openjpa.lib.conf.Configurations.configureInstance(Configurations.java:458)
> at
> org.apache.openjpa.lib.conf.PluginValue.instantiate(PluginValue.java:121)
> at
> org.apache.openjpa.conf.MetaDataRepositoryValue.instantiate(MetaDataRepositoryValue.java:68)
> at
> org.apache.openjpa.lib.conf.ObjectValue.instantiate(ObjectValue.java:83)
> at
> org.apache.openjpa.conf.OpenJPAConfigurationImpl.newMetaDataRepositoryInstance(OpenJPAConfigurationImpl.java:967)
> at
> org.apache.openjpa.conf.OpenJPAConfigurationImpl.getMetaDataRepositoryInstance(OpenJPAConfigurationImpl.java:958)
> at
> org.apache.openjpa.kernel.AbstractBrokerFactory.makeReadOnly(AbstractBrokerFactory.java:642)
> at
> org.apache.openjpa.kernel.AbstractBrokerFactory.newBroker(AbstractBrokerFactory.java:202)
> at
> org.apache.openjpa.kernel.DelegatingBrokerFactory.newBroker(DelegatingBrokerFactory.java:154)
> at
> org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:226)
> at
> org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:153)
> at
> org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:59)
> at
> org.apache.oozie.service.JPAService.getEntityManager(JPAService.java:514)
> at org.apache.oozie.service.JPAService.init(JPAService.java:215)
> at
> org.apache.oozie.service.Services.setServiceInternal(Services.java:387)
> at org.apache.oozie.service.Services.setService(Services.java:373)
> at org.apache.oozie.service.Services.loadServices(Services.java:305)
> at org.apache.oozie.service.Services.init(Services.java:213)
> at
> org.apache.oozie.server.guice.ServicesProvider.get(ServicesProvider.java:31)
> at
>
[jira] [Commented] (OOZIE-2775) Oozie server does not stop if there is an exception during service initalization at startup
[
https://issues.apache.org/jira/browse/OOZIE-2775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335746#comment-16335746
]
Hadoop QA commented on OOZIE-2775:
--
Testing JIRA OOZIE-2775
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:orange}0{color} There are [2] new bugs found in total that would be nice
to have fixed.
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:orange}0{color} There are [2] new bugs found in [server] that would be
nice to have fixed.
. You can find the FindBugs diff here: server/findbugs-new.html
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 2087
.Tests failed: 1
.Tests errors: 0
.The patch failed the following testcases:
testHiveAction(org.apache.oozie.action.hadoop.TestHiveActionExecutor)
.Tests failing with errors:
.{color:orange}Tests failed at first run:{color}
TestJavaActionExecutor#testCredentialsSkip
TestCoordActionsKillXCommand#testActionKillCommandDate
.For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/340/
> Oozie server does not stop if there is an exception during service
> initalization at startup
> ---
>
> Key: OOZIE-2775
> URL: https://issues.apache.org/jira/browse/OOZIE-2775
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Major
> Attachments: OOZIE-2775-01.patch, OOZIE-2775-02.patch
>
>
> OOZIE-2666 brought in embedded jetty in Oozie. {{com.google.inject.guice}}
> was selected for dependency management. Unfortunately if there is an
> exception at starting Oozie services (e.g. JPAService initialization, called
> via
> https://github.com/apache/oozie/blob/master/server/src/main/java/org/apache/oozie/server/EmbeddedOozieServer.java#L194),
> Oozie won't stop if it cannot destroy Oozie services.
> In OOZIE-2666 a shutdown hook
> (https://github.com/apache/oozie/blob/master/server/src/main/java/org/apache/oozie/server/EmbeddedOozieServer.java#L184)
> was added to print out startup errors and exceptions, destroy pending oozie
> services, and exit eventually, but it does not work.
> Steps to reproduce a startup failure (for example simulate that Oozie cannot
> find required database driver):
> {code}
> $ cd oozie
> $ mvn clean install assembly:single -DskipTests -DjavaVersion=1.8
> -DtargetVersion=1.7 -Phadoop-2 -Dhadoop.version=2.6.0 -Puber;
> $ cd
[jira] [Commented] (OOZIE-2775) Oozie server does not stop if there is an exception during service initalization at startup
[
https://issues.apache.org/jira/browse/OOZIE-2775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335651#comment-16335651
]
Hadoop QA commented on OOZIE-2775:
--
PreCommit-OOZIE-Build started
> Oozie server does not stop if there is an exception during service
> initalization at startup
> ---
>
> Key: OOZIE-2775
> URL: https://issues.apache.org/jira/browse/OOZIE-2775
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Major
> Attachments: OOZIE-2775-01.patch, OOZIE-2775-02.patch
>
>
> OOZIE-2666 brought in embedded jetty in Oozie. {{com.google.inject.guice}}
> was selected for dependency management. Unfortunately if there is an
> exception at starting Oozie services (e.g. JPAService initialization, called
> via
> https://github.com/apache/oozie/blob/master/server/src/main/java/org/apache/oozie/server/EmbeddedOozieServer.java#L194),
> Oozie won't stop if it cannot destroy Oozie services.
> In OOZIE-2666 a shutdown hook
> (https://github.com/apache/oozie/blob/master/server/src/main/java/org/apache/oozie/server/EmbeddedOozieServer.java#L184)
> was added to print out startup errors and exceptions, destroy pending oozie
> services, and exit eventually, but it does not work.
> Steps to reproduce a startup failure (for example simulate that Oozie cannot
> find required database driver):
> {code}
> $ cd oozie
> $ mvn clean install assembly:single -DskipTests -DjavaVersion=1.8
> -DtargetVersion=1.7 -Phadoop-2 -Dhadoop.version=2.6.0 -Puber;
> $ cd distro/target/oozie-4.4.0-SNAPSHOT-distro/oozie-4.4.0-SNAPSHOT/
> $ export JETTY_OPTS="-Djavax.net.debug=all
> -agentlib:jdwp=transport=dt_socket,server=y,address=8000,suspend=y
> -Dlog4j.logger.org.eclipse.jetty=TRACE
> -Dlog4j.configuration=./conf/oozie-log4j.properties"
> $ bin/oozied.sh run
> $ mv embedded-oozie-server/webapp/WEB-INF/lib/derby-10.10.1.1.jar ..
>
> {code}
> Exception is printed to stdout
> {code}
>
> org.apache.openjpa.persistence.PersistenceException: Cannot load JDBC driver
> class 'org.apache.derby.jdbc.EmbeddedDriver'
> at
> org.apache.openjpa.jdbc.sql.DBDictionaryFactory.newDBDictionary(DBDictionaryFactory.java:106)
> at
> org.apache.openjpa.jdbc.conf.JDBCConfigurationImpl.getDBDictionaryInstance(JDBCConfigurationImpl.java:603)
> at
> org.apache.openjpa.jdbc.meta.MappingRepository.endConfiguration(MappingRepository.java:1520)
> at
> org.apache.openjpa.lib.conf.Configurations.configureInstance(Configurations.java:533)
> at
> org.apache.openjpa.lib.conf.Configurations.configureInstance(Configurations.java:458)
> at
> org.apache.openjpa.lib.conf.PluginValue.instantiate(PluginValue.java:121)
> at
> org.apache.openjpa.conf.MetaDataRepositoryValue.instantiate(MetaDataRepositoryValue.java:68)
> at
> org.apache.openjpa.lib.conf.ObjectValue.instantiate(ObjectValue.java:83)
> at
> org.apache.openjpa.conf.OpenJPAConfigurationImpl.newMetaDataRepositoryInstance(OpenJPAConfigurationImpl.java:967)
> at
> org.apache.openjpa.conf.OpenJPAConfigurationImpl.getMetaDataRepositoryInstance(OpenJPAConfigurationImpl.java:958)
> at
> org.apache.openjpa.kernel.AbstractBrokerFactory.makeReadOnly(AbstractBrokerFactory.java:642)
> at
> org.apache.openjpa.kernel.AbstractBrokerFactory.newBroker(AbstractBrokerFactory.java:202)
> at
> org.apache.openjpa.kernel.DelegatingBrokerFactory.newBroker(DelegatingBrokerFactory.java:154)
> at
> org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:226)
> at
> org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:153)
> at
> org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:59)
> at
> org.apache.oozie.service.JPAService.getEntityManager(JPAService.java:514)
> at org.apache.oozie.service.JPAService.init(JPAService.java:215)
> at
> org.apache.oozie.service.Services.setServiceInternal(Services.java:387)
> at org.apache.oozie.service.Services.setService(Services.java:373)
> at org.apache.oozie.service.Services.loadServices(Services.java:305)
> at org.apache.oozie.service.Services.init(Services.java:213)
> at
> org.apache.oozie.server.guice.ServicesProvider.get(ServicesProvider.java:31)
> at
> org.apache.oozie.server.guice.ServicesProvider.get(ServicesProvider.java:25)
> at
> com.google.inject.internal.BoundProviderFactory.get(BoundProviderFactory.java:55)
> at
> com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)
> at
>
[jira] [Commented] (OOZIE-2775) Oozie server does not stop if there is an exception during service initalization at startup
[
https://issues.apache.org/jira/browse/OOZIE-2775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334999#comment-16334999
]
Attila Sasvari commented on OOZIE-2775:
---
{color:#33}[~andras.piros] Thanks for your comments!{color}
* {color:#33}When a {color}ProvisionException is thrown during
{{guiceInjector.getInstance()}} , {{embeddedOozieServer}} will be null. In the
current form, without additional changes: you can't get a reference to the
Jetty server as it does not exist at this point of the execution (it is
actually created after the Oozie services started). The (failed) Oozie services
created by the provider cannot be stopped either. I wonder if it was really
necessary. If so, we can probably handle it in the ServicesProvider. I will try
to address this in a new patch uploaded to *Reviewboard* *first*.
* Regarding a stop() method: can you expand how it should work? Right now,
stopping the Oozie server happens via a SIGTERM signal sent to the process (see
{{bin/oozie-jetty-server.sh}}. This behaviour is pretty similar to the earlier
version (with Tomcat) where this case was not handled by the means of the
[shutdown hook
mechanism|https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/signals006.html]
as far as I remember. I suppose it is OK that we add such a handler, but of
course there might be races (for example what if signal arrives before we
install it). What do you think?
* I believe
[ExceptionInInitializerError|https://docs.oracle.com/javase/7/docs/api/java/lang/ExceptionInInitializerError.html]
is not suitable for handling this sort of service initialization problems,
[ExceptionInInitializerError|https://docs.oracle.com/javase/7/docs/api/java/lang/ExceptionInInitializerError.html]
is a kind of LinkageError. Documentation says "{{ExceptionInInitializerError}}
is thrown to indicate that an exception occurred during evaluation of a static
initializer or the initializer for a static variable.". A more pragmatic answer
is that BoundProviderFactory's {{get()}} by guice only catches Runtime
exceptions.
> Oozie server does not stop if there is an exception during service
> initalization at startup
> ---
>
> Key: OOZIE-2775
> URL: https://issues.apache.org/jira/browse/OOZIE-2775
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Major
> Attachments: OOZIE-2775-01.patch
>
>
> OOZIE-2666 brought in embedded jetty in Oozie. {{com.google.inject.guice}}
> was selected for dependency management. Unfortunately if there is an
> exception at starting Oozie services (e.g. JPAService initialization, called
> via
> https://github.com/apache/oozie/blob/master/server/src/main/java/org/apache/oozie/server/EmbeddedOozieServer.java#L194),
> Oozie won't stop if it cannot destroy Oozie services.
> In OOZIE-2666 a shutdown hook
> (https://github.com/apache/oozie/blob/master/server/src/main/java/org/apache/oozie/server/EmbeddedOozieServer.java#L184)
> was added to print out startup errors and exceptions, destroy pending oozie
> services, and exit eventually, but it does not work.
> Steps to reproduce a startup failure (for example simulate that Oozie cannot
> find required database driver):
> {code}
> $ cd oozie
> $ mvn clean install assembly:single -DskipTests -DjavaVersion=1.8
> -DtargetVersion=1.7 -Phadoop-2 -Dhadoop.version=2.6.0 -Puber;
> $ cd distro/target/oozie-4.4.0-SNAPSHOT-distro/oozie-4.4.0-SNAPSHOT/
> $ export JETTY_OPTS="-Djavax.net.debug=all
> -agentlib:jdwp=transport=dt_socket,server=y,address=8000,suspend=y
> -Dlog4j.logger.org.eclipse.jetty=TRACE
> -Dlog4j.configuration=./conf/oozie-log4j.properties"
> $ bin/oozied.sh run
> $ mv embedded-oozie-server/webapp/WEB-INF/lib/derby-10.10.1.1.jar ..
>
> {code}
> Exception is printed to stdout
> {code}
>
> org.apache.openjpa.persistence.PersistenceException: Cannot load JDBC driver
> class 'org.apache.derby.jdbc.EmbeddedDriver'
> at
> org.apache.openjpa.jdbc.sql.DBDictionaryFactory.newDBDictionary(DBDictionaryFactory.java:106)
> at
> org.apache.openjpa.jdbc.conf.JDBCConfigurationImpl.getDBDictionaryInstance(JDBCConfigurationImpl.java:603)
> at
> org.apache.openjpa.jdbc.meta.MappingRepository.endConfiguration(MappingRepository.java:1520)
> at
> org.apache.openjpa.lib.conf.Configurations.configureInstance(Configurations.java:533)
> at
> org.apache.openjpa.lib.conf.Configurations.configureInstance(Configurations.java:458)
> at
> org.apache.openjpa.lib.conf.PluginValue.instantiate(PluginValue.java:121)
> at
> org.apache.openjpa.conf.MetaDataRepositoryValue.instantiate(MetaDataRepositoryValue.java:68)
> at
>
[jira] [Commented] (OOZIE-2775) Oozie server does not stop if there is an exception during service initalization at startup
[
https://issues.apache.org/jira/browse/OOZIE-2775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334246#comment-16334246
]
Andras Piros commented on OOZIE-2775:
-
[~asasvari] thanks for the patch!
Couple of thoughts:
* overall fail-fast approach looks good to me
* may be better to have a ReviewBoard review request for discussion
* it may be prudent to emit a log message containing
[{{ProvisionException#getErrorMessages()}}|https://google.github.io/guice/api-docs/latest/javadoc/index.html?com/google/inject/ProvisionException.html]
while on {{EmbeddedOozieServer:225}}
* you're right, calling {{oozieServices#destroy()}} is needed on
{{ServicesProvider:35}}. In that extent, all the Oozie {{Service}} instances
will be cleaned up, including the {{Services}} singleton itself
* I think it's also needed that in {{EmbeddedOozieServer}} we create method
{{stop()}} where the {{org.eclipse.jetty.server.Server}} instance methods
{{stop()}} and {{destroy()}} would also be called, to clean up everything
that's Jetty related. Let's call this new method also from
{{EmbeddedOozieServer#main()}}
* for me it's unclear why {{ExceptionInInitializerError}} was switched to
{{IllegalStateException}} in {{ServicesProvider:35}}
> Oozie server does not stop if there is an exception during service
> initalization at startup
> ---
>
> Key: OOZIE-2775
> URL: https://issues.apache.org/jira/browse/OOZIE-2775
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Major
> Attachments: OOZIE-2775-01.patch
>
>
> OOZIE-2666 brought in embedded jetty in Oozie. {{com.google.inject.guice}}
> was selected for dependency management. Unfortunately if there is an
> exception at starting Oozie services (e.g. JPAService initialization, called
> via
> https://github.com/apache/oozie/blob/master/server/src/main/java/org/apache/oozie/server/EmbeddedOozieServer.java#L194),
> Oozie won't stop if it cannot destroy Oozie services.
> In OOZIE-2666 a shutdown hook
> (https://github.com/apache/oozie/blob/master/server/src/main/java/org/apache/oozie/server/EmbeddedOozieServer.java#L184)
> was added to print out startup errors and exceptions, destroy pending oozie
> services, and exit eventually, but it does not work.
> Steps to reproduce a startup failure (for example simulate that Oozie cannot
> find required database driver):
> {code}
> $ cd oozie
> $ mvn clean install assembly:single -DskipTests -DjavaVersion=1.8
> -DtargetVersion=1.7 -Phadoop-2 -Dhadoop.version=2.6.0 -Puber;
> $ cd distro/target/oozie-4.4.0-SNAPSHOT-distro/oozie-4.4.0-SNAPSHOT/
> $ export JETTY_OPTS="-Djavax.net.debug=all
> -agentlib:jdwp=transport=dt_socket,server=y,address=8000,suspend=y
> -Dlog4j.logger.org.eclipse.jetty=TRACE
> -Dlog4j.configuration=./conf/oozie-log4j.properties"
> $ bin/oozied.sh run
> $ mv embedded-oozie-server/webapp/WEB-INF/lib/derby-10.10.1.1.jar ..
>
> {code}
> Exception is printed to stdout
> {code}
>
> org.apache.openjpa.persistence.PersistenceException: Cannot load JDBC driver
> class 'org.apache.derby.jdbc.EmbeddedDriver'
> at
> org.apache.openjpa.jdbc.sql.DBDictionaryFactory.newDBDictionary(DBDictionaryFactory.java:106)
> at
> org.apache.openjpa.jdbc.conf.JDBCConfigurationImpl.getDBDictionaryInstance(JDBCConfigurationImpl.java:603)
> at
> org.apache.openjpa.jdbc.meta.MappingRepository.endConfiguration(MappingRepository.java:1520)
> at
> org.apache.openjpa.lib.conf.Configurations.configureInstance(Configurations.java:533)
> at
> org.apache.openjpa.lib.conf.Configurations.configureInstance(Configurations.java:458)
> at
> org.apache.openjpa.lib.conf.PluginValue.instantiate(PluginValue.java:121)
> at
> org.apache.openjpa.conf.MetaDataRepositoryValue.instantiate(MetaDataRepositoryValue.java:68)
> at
> org.apache.openjpa.lib.conf.ObjectValue.instantiate(ObjectValue.java:83)
> at
> org.apache.openjpa.conf.OpenJPAConfigurationImpl.newMetaDataRepositoryInstance(OpenJPAConfigurationImpl.java:967)
> at
> org.apache.openjpa.conf.OpenJPAConfigurationImpl.getMetaDataRepositoryInstance(OpenJPAConfigurationImpl.java:958)
> at
> org.apache.openjpa.kernel.AbstractBrokerFactory.makeReadOnly(AbstractBrokerFactory.java:642)
> at
> org.apache.openjpa.kernel.AbstractBrokerFactory.newBroker(AbstractBrokerFactory.java:202)
> at
> org.apache.openjpa.kernel.DelegatingBrokerFactory.newBroker(DelegatingBrokerFactory.java:154)
> at
> org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:226)
> at
> org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:153)
[jira] [Commented] (OOZIE-2775) Oozie server does not stop if there is an exception during service initalization at startup
[
https://issues.apache.org/jira/browse/OOZIE-2775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334196#comment-16334196
]
Attila Sasvari commented on OOZIE-2775:
---
[~andras.piros], [~gezapeti] can you take a look at the first patch? Do you see
any problems with the current fail-fast approach? Would it be enough to call
{{Services.destroy()}} or some extra care is required? If so, could you
recommend a better way to gracefully stop Oozie services that might have been
already initialized and started during startup? Thanks.
> Oozie server does not stop if there is an exception during service
> initalization at startup
> ---
>
> Key: OOZIE-2775
> URL: https://issues.apache.org/jira/browse/OOZIE-2775
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Major
> Attachments: OOZIE-2775-01.patch
>
>
> OOZIE-2666 brought in embedded jetty in Oozie. {{com.google.inject.guice}}
> was selected for dependency management. Unfortunately if there is an
> exception at starting Oozie services (e.g. JPAService initialization, called
> via
> https://github.com/apache/oozie/blob/master/server/src/main/java/org/apache/oozie/server/EmbeddedOozieServer.java#L194),
> Oozie won't stop if it cannot destroy Oozie services.
> In OOZIE-2666 a shutdown hook
> (https://github.com/apache/oozie/blob/master/server/src/main/java/org/apache/oozie/server/EmbeddedOozieServer.java#L184)
> was added to print out startup errors and exceptions, destroy pending oozie
> services, and exit eventually, but it does not work.
> Steps to reproduce a startup failure (for example simulate that Oozie cannot
> find required database driver):
> {code}
> $ cd oozie
> $ mvn clean install assembly:single -DskipTests -DjavaVersion=1.8
> -DtargetVersion=1.7 -Phadoop-2 -Dhadoop.version=2.6.0 -Puber;
> $ cd distro/target/oozie-4.4.0-SNAPSHOT-distro/oozie-4.4.0-SNAPSHOT/
> $ export JETTY_OPTS="-Djavax.net.debug=all
> -agentlib:jdwp=transport=dt_socket,server=y,address=8000,suspend=y
> -Dlog4j.logger.org.eclipse.jetty=TRACE
> -Dlog4j.configuration=./conf/oozie-log4j.properties"
> $ bin/oozied.sh run
> $ mv embedded-oozie-server/webapp/WEB-INF/lib/derby-10.10.1.1.jar ..
>
> {code}
> Exception is printed to stdout
> {code}
>
> org.apache.openjpa.persistence.PersistenceException: Cannot load JDBC driver
> class 'org.apache.derby.jdbc.EmbeddedDriver'
> at
> org.apache.openjpa.jdbc.sql.DBDictionaryFactory.newDBDictionary(DBDictionaryFactory.java:106)
> at
> org.apache.openjpa.jdbc.conf.JDBCConfigurationImpl.getDBDictionaryInstance(JDBCConfigurationImpl.java:603)
> at
> org.apache.openjpa.jdbc.meta.MappingRepository.endConfiguration(MappingRepository.java:1520)
> at
> org.apache.openjpa.lib.conf.Configurations.configureInstance(Configurations.java:533)
> at
> org.apache.openjpa.lib.conf.Configurations.configureInstance(Configurations.java:458)
> at
> org.apache.openjpa.lib.conf.PluginValue.instantiate(PluginValue.java:121)
> at
> org.apache.openjpa.conf.MetaDataRepositoryValue.instantiate(MetaDataRepositoryValue.java:68)
> at
> org.apache.openjpa.lib.conf.ObjectValue.instantiate(ObjectValue.java:83)
> at
> org.apache.openjpa.conf.OpenJPAConfigurationImpl.newMetaDataRepositoryInstance(OpenJPAConfigurationImpl.java:967)
> at
> org.apache.openjpa.conf.OpenJPAConfigurationImpl.getMetaDataRepositoryInstance(OpenJPAConfigurationImpl.java:958)
> at
> org.apache.openjpa.kernel.AbstractBrokerFactory.makeReadOnly(AbstractBrokerFactory.java:642)
> at
> org.apache.openjpa.kernel.AbstractBrokerFactory.newBroker(AbstractBrokerFactory.java:202)
> at
> org.apache.openjpa.kernel.DelegatingBrokerFactory.newBroker(DelegatingBrokerFactory.java:154)
> at
> org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:226)
> at
> org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:153)
> at
> org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:59)
> at
> org.apache.oozie.service.JPAService.getEntityManager(JPAService.java:514)
> at org.apache.oozie.service.JPAService.init(JPAService.java:215)
> at
> org.apache.oozie.service.Services.setServiceInternal(Services.java:387)
> at org.apache.oozie.service.Services.setService(Services.java:373)
> at org.apache.oozie.service.Services.loadServices(Services.java:305)
> at org.apache.oozie.service.Services.init(Services.java:213)
> at
> org.apache.oozie.server.guice.ServicesProvider.get(ServicesProvider.java:31)
> at
[jira] [Commented] (OOZIE-2775) Oozie server does not stop if there is an exception during service initalization at startup
[
https://issues.apache.org/jira/browse/OOZIE-2775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16200015#comment-16200015
]
Attila Sasvari commented on OOZIE-2775:
---
There is an easier way to reproduce it:
- start the Oozie server, and stop it. Change permission so that
{{/tmp/embedded-jetty-jsp}} (or the location set by {{oozie.jsp.tmp.dir}}) is
not writable by the user that starts Oozie.
- try to restart Oozie
{code}
2017-10-10 13:31:46,871 WARN org.eclipse.jetty.webapp.WebAppContext: SERVER[]
Failed startup of context
o.e.j.w.WebAppContext@42507640{/oozie,file:///usr/lib/oozie/embedded-oozie-server/webapp/,null}
java.lang.IllegalStateException: Failed to delete temp dir
/tmp/embedded-jetty-jsp
at
org.eclipse.jetty.webapp.WebInfConfiguration.configureTempDirectory(WebInfConfiguration.java:376)
at
org.eclipse.jetty.webapp.WebInfConfiguration.resolveTempDirectory(WebInfConfiguration.java:275)
at
org.eclipse.jetty.webapp.WebInfConfiguration.preConfigure(WebInfConfiguration.java:69)
at
org.eclipse.jetty.webapp.WebAppContext.preConfigure(WebAppContext.java:485)
at
org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:521)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:113)
at
org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131)
at org.eclipse.jetty.server.Server.start(Server.java:422)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:105)
at
org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
at org.eclipse.jetty.server.Server.doStart(Server.java:389)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at
org.apache.oozie.server.EmbeddedOozieServer.start(EmbeddedOozieServer.java:189)
at
org.apache.oozie.server.EmbeddedOozieServer.main(EmbeddedOozieServer.java:215)
{code}
Oozie services will start but the server itself will return 503. It might
worth to file a new JIRA to handle this startup issue specifically, but it is
up to you [~andras.piros]. What we could change:
- make an extra check at startup before starting the jetty server. If we see
permission problems, we log a meaningful error message and fail fast.
- create & use /tmp/embedded_jetty_jsp_[OOZIE_PID] instead of
/tmp/embedded_jetty_jsp. "Old" jetty jsp directories should be purged though.
> Oozie server does not stop if there is an exception during service
> initalization at startup
> ---
>
> Key: OOZIE-2775
> URL: https://issues.apache.org/jira/browse/OOZIE-2775
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Andras Piros
>
> OOZIE-2666 brought in embedded jetty in Oozie. {{com.google.inject.guice}}
> was selected for dependency management. Unfortunately if there is an
> exception at starting Oozie services (e.g. JPAService initialization, called
> via
> https://github.com/apache/oozie/blob/master/server/src/main/java/org/apache/oozie/server/EmbeddedOozieServer.java#L194),
> Oozie won't stop if it cannot destroy Oozie services.
> In OOZIE-2666 a shutdown hook
> (https://github.com/apache/oozie/blob/master/server/src/main/java/org/apache/oozie/server/EmbeddedOozieServer.java#L184)
> was added to print out startup errors and exceptions, destroy pending oozie
> services, and exit eventually, but it does not work.
> Steps to reproduce a startup failure (for example simulate that Oozie cannot
> find required database driver):
> {code}
> $ cd oozie
> $ mvn clean install assembly:single -DskipTests -DjavaVersion=1.8
> -DtargetVersion=1.7 -Phadoop-2 -Dhadoop.version=2.6.0 -Puber;
> $ cd distro/target/oozie-4.4.0-SNAPSHOT-distro/oozie-4.4.0-SNAPSHOT/
> $ export JETTY_OPTS="-Djavax.net.debug=all
> -agentlib:jdwp=transport=dt_socket,server=y,address=8000,suspend=y
> -Dlog4j.logger.org.eclipse.jetty=TRACE
> -Dlog4j.configuration=./conf/oozie-log4j.properties"
> $ bin/oozied.sh run
> $ mv embedded-oozie-server/webapp/WEB-INF/lib/derby-10.10.1.1.jar ..
>
> {code}
> Exception is printed to stdout
> {code}
>
> org.apache.openjpa.persistence.PersistenceException: Cannot load JDBC driver
> class 'org.apache.derby.jdbc.EmbeddedDriver'
> at
>
