[jira] [Commented] (OOZIE-3134) Potential inconsistency between the in-memory SLA map and the Oozie database
[
https://issues.apache.org/jira/browse/OOZIE-3134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16426600#comment-16426600
]
Andras Piros commented on OOZIE-3134:
-
Thanks [~kmarton] for the contribution! +1
> Potential inconsistency between the in-memory SLA map and the Oozie database
>
>
> Key: OOZIE-3134
> URL: https://issues.apache.org/jira/browse/OOZIE-3134
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.1.0
>
> Attachments: OOZIE-3134-001.patch, OOZIE-3134-002.patch,
> OOZIE-3134-003.patch, OOZIE-3134-004.patch, OOZIE-3134-005.patch,
> OOZIE-3134-006.patch, OOZIE-3134-007.patch
>
>
> Upon {{SLACalculatorMemory.addRegistration}}, Oozie puts an entry into an
> in-memory concurrent hashmap ("slaMap"), and later [persists
> it|https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L380]
> in the Oozie database.
> However, if there is a failure during the database operation, a
> {{JPAExecutorException}} is thrown, and the entry [is not removed from the
> SLA map|
> https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L393].
> It may introduce inconsistency between the Oozie database and the SLA map.
> To prevent this, a rollback mechanism (with proper logging) should be
> implemented. It would also make sense to do more sanity/consistency check in
> the Oozie server.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3134) Potential inconsistency between the in-memory SLA map and the Oozie database
[
https://issues.apache.org/jira/browse/OOZIE-3134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16425617#comment-16425617
]
Hadoop QA commented on OOZIE-3134:
--
Testing JIRA OOZIE-3134
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:red}-1{color} There are [1] new bugs found below threshold in total that
must be fixed.
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:red}-1{color} There are [1] new bugs found below threshold in [core]
that must be fixed.
. You can find the FindBugs diff here (look for the red and orange ones):
core/findbugs-new.html
. The most important FindBugs errors are:
. At FailingConnectionWrapper.java:[line 176]: This use of
java/sql/Connection.prepareStatement(Ljava/lang/String;II)Ljava/sql/PreparedStatement;
can be vulnerable to SQL injection
. At FailingConnectionWrapper.java:[line 170]
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [client].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 2111
.Tests failed: 1
.Tests errors: 1
.The patch failed the following testcases:
testCoordActionRecoveryServiceForWaitingRegisterPartition(org.apache.oozie.service.TestRecoveryService)
.Tests failing with errors:
testConnectionRetry(org.apache.oozie.service.TestJMSAccessorService)
.{color:orange}Tests failed at first run:{color}
TestJavaActionExecutor#testCredentialsSkip
.For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/477/
> Potential inconsistency between the in-memory SLA map and the Oozie database
>
>
> Key: OOZIE-3134
> URL: https://issues.apache.org/jira/browse/OOZIE-3134
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.1.0
>
> Attachments: OOZIE-3134-001.patch, OOZIE-3134-002.patch,
> OOZIE-3134-003.patch, OOZIE-3134-004.patch, OOZIE-3134-005.patch,
> OOZIE-3134-006.patch, OOZIE-3134-007.patch
>
>
> Upon {{SLACalculatorMemory.addRegistration}}, Oozie puts an entry into an
> in-memory concurrent hashmap ("slaMap"), and later [persists
> it|https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L380]
> in the Oozie database.
> However, if there is a failure during the database operation, a
> {{JPAExecutorException}} is thrown, and the en
[jira] [Commented] (OOZIE-3134) Potential inconsistency between the in-memory SLA map and the Oozie database
[
https://issues.apache.org/jira/browse/OOZIE-3134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16425461#comment-16425461
]
Julia Kinga Marton commented on OOZIE-3134:
---
Thank you, [~andras.piros], I have corrected the long line. The bug found by
findbugs existed until now as well, but it was in a different row.
> Potential inconsistency between the in-memory SLA map and the Oozie database
>
>
> Key: OOZIE-3134
> URL: https://issues.apache.org/jira/browse/OOZIE-3134
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.1.0
>
> Attachments: OOZIE-3134-001.patch, OOZIE-3134-002.patch,
> OOZIE-3134-003.patch, OOZIE-3134-004.patch, OOZIE-3134-005.patch,
> OOZIE-3134-006.patch, OOZIE-3134-007.patch
>
>
> Upon {{SLACalculatorMemory.addRegistration}}, Oozie puts an entry into an
> in-memory concurrent hashmap ("slaMap"), and later [persists
> it|https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L380]
> in the Oozie database.
> However, if there is a failure during the database operation, a
> {{JPAExecutorException}} is thrown, and the entry [is not removed from the
> SLA map|
> https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L393].
> It may introduce inconsistency between the Oozie database and the SLA map.
> To prevent this, a rollback mechanism (with proper logging) should be
> implemented. It would also make sense to do more sanity/consistency check in
> the Oozie server.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3134) Potential inconsistency between the in-memory SLA map and the Oozie database
[
https://issues.apache.org/jira/browse/OOZIE-3134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16425459#comment-16425459
]
Hadoop QA commented on OOZIE-3134:
--
PreCommit-OOZIE-Build started
> Potential inconsistency between the in-memory SLA map and the Oozie database
>
>
> Key: OOZIE-3134
> URL: https://issues.apache.org/jira/browse/OOZIE-3134
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.1.0
>
> Attachments: OOZIE-3134-001.patch, OOZIE-3134-002.patch,
> OOZIE-3134-003.patch, OOZIE-3134-004.patch, OOZIE-3134-005.patch,
> OOZIE-3134-006.patch, OOZIE-3134-007.patch
>
>
> Upon {{SLACalculatorMemory.addRegistration}}, Oozie puts an entry into an
> in-memory concurrent hashmap ("slaMap"), and later [persists
> it|https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L380]
> in the Oozie database.
> However, if there is a failure during the database operation, a
> {{JPAExecutorException}} is thrown, and the entry [is not removed from the
> SLA map|
> https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L393].
> It may introduce inconsistency between the Oozie database and the SLA map.
> To prevent this, a rollback mechanism (with proper logging) should be
> implemented. It would also make sense to do more sanity/consistency check in
> the Oozie server.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3134) Potential inconsistency between the in-memory SLA map and the Oozie database
[
https://issues.apache.org/jira/browse/OOZIE-3134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16411208#comment-16411208
]
Andras Piros commented on OOZIE-3134:
-
[~kmarton] please address {{RAW_PATCH_ANALYSIS}} and {{FINDBUGS_DIFF}} errors,
warnings. Thanks!
> Potential inconsistency between the in-memory SLA map and the Oozie database
>
>
> Key: OOZIE-3134
> URL: https://issues.apache.org/jira/browse/OOZIE-3134
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.1.0
>
> Attachments: OOZIE-3134-001.patch, OOZIE-3134-002.patch,
> OOZIE-3134-003.patch, OOZIE-3134-004.patch, OOZIE-3134-005.patch,
> OOZIE-3134-006.patch
>
>
> Upon {{SLACalculatorMemory.addRegistration}}, Oozie puts an entry into an
> in-memory concurrent hashmap ("slaMap"), and later [persists
> it|https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L380]
> in the Oozie database.
> However, if there is a failure during the database operation, a
> {{JPAExecutorException}} is thrown, and the entry [is not removed from the
> SLA map|
> https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L393].
> It may introduce inconsistency between the Oozie database and the SLA map.
> To prevent this, a rollback mechanism (with proper logging) should be
> implemented. It would also make sense to do more sanity/consistency check in
> the Oozie server.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3134) Potential inconsistency between the in-memory SLA map and the Oozie database
[
https://issues.apache.org/jira/browse/OOZIE-3134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16411175#comment-16411175
]
Hadoop QA commented on OOZIE-3134:
--
Testing JIRA OOZIE-3134
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:red}-1{color} the patch contains 1 line(s) longer than 132
characters
.{color:green}+1{color} the patch adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
.{color:red}WARNING{color}: the current HEAD has 100 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:red}-1{color} There are [1] new bugs found below threshold in total that
must be fixed.
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:red}-1{color} There are [1] new bugs found below threshold in [core]
that must be fixed.
. You can find the FindBugs diff here (look for the red and orange ones):
core/findbugs-new.html
. The most important FindBugs errors are:
. At FailingConnectionWrapper.java:[line 175]: This use of
java/sql/Connection.prepareStatement(Ljava/lang/String;II)Ljava/sql/PreparedStatement;
can be vulnerable to SQL injection
. At FailingConnectionWrapper.java:[line 169]
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [server].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 2100
.Tests failed: 1
.Tests errors: 0
.The patch failed the following testcases:
testHiveAction(org.apache.oozie.action.hadoop.TestHiveActionExecutor)
.Tests failing with errors:
.{color:orange}Tests failed at first run:{color}
TestJavaActionExecutor#testCredentialsSkip
TestCoordActionsKillXCommand#testActionKillCommandActionNumbers
.For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
{color:red}. There is at least one warning, please check{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/460/
> Potential inconsistency between the in-memory SLA map and the Oozie database
>
>
> Key: OOZIE-3134
> URL: https://issues.apache.org/jira/browse/OOZIE-3134
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.1.0
>
> Attachments: OOZIE-3134-001.patch, OOZIE-3134-002.patch,
> OOZIE-3134-003.patch, OOZIE-3134-004.patch, OOZIE-3134-005.patch,
> OOZIE-3134-006.patch
>
>
> Upon {{SLACalculatorMemory.addRegistration}}, Oozie puts an entry into an
> in-memory concurrent hashmap ("slaMap"), and later [persists
> it|https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L380]
> in the Oozie database.
> However, if there is a failure duri
[jira] [Commented] (OOZIE-3134) Potential inconsistency between the in-memory SLA map and the Oozie database
[
https://issues.apache.org/jira/browse/OOZIE-3134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16411013#comment-16411013
]
Hadoop QA commented on OOZIE-3134:
--
PreCommit-OOZIE-Build started
> Potential inconsistency between the in-memory SLA map and the Oozie database
>
>
> Key: OOZIE-3134
> URL: https://issues.apache.org/jira/browse/OOZIE-3134
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.1.0
>
> Attachments: OOZIE-3134-001.patch, OOZIE-3134-002.patch,
> OOZIE-3134-003.patch, OOZIE-3134-004.patch, OOZIE-3134-005.patch,
> OOZIE-3134-006.patch
>
>
> Upon {{SLACalculatorMemory.addRegistration}}, Oozie puts an entry into an
> in-memory concurrent hashmap ("slaMap"), and later [persists
> it|https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L380]
> in the Oozie database.
> However, if there is a failure during the database operation, a
> {{JPAExecutorException}} is thrown, and the entry [is not removed from the
> SLA map|
> https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L393].
> It may introduce inconsistency between the Oozie database and the SLA map.
> To prevent this, a rollback mechanism (with proper logging) should be
> implemented. It would also make sense to do more sanity/consistency check in
> the Oozie server.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3134) Potential inconsistency between the in-memory SLA map and the Oozie database
[
https://issues.apache.org/jira/browse/OOZIE-3134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16411007#comment-16411007
]
Andras Piros commented on OOZIE-3134:
-
Thanks for the contribution [~kmarton]! +1 (pending Jenkins)
> Potential inconsistency between the in-memory SLA map and the Oozie database
>
>
> Key: OOZIE-3134
> URL: https://issues.apache.org/jira/browse/OOZIE-3134
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.1.0
>
> Attachments: OOZIE-3134-001.patch, OOZIE-3134-002.patch,
> OOZIE-3134-003.patch, OOZIE-3134-004.patch, OOZIE-3134-005.patch,
> OOZIE-3134-006.patch
>
>
> Upon {{SLACalculatorMemory.addRegistration}}, Oozie puts an entry into an
> in-memory concurrent hashmap ("slaMap"), and later [persists
> it|https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L380]
> in the Oozie database.
> However, if there is a failure during the database operation, a
> {{JPAExecutorException}} is thrown, and the entry [is not removed from the
> SLA map|
> https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L393].
> It may introduce inconsistency between the Oozie database and the SLA map.
> To prevent this, a rollback mechanism (with proper logging) should be
> implemented. It would also make sense to do more sanity/consistency check in
> the Oozie server.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3134) Potential inconsistency between the in-memory SLA map and the Oozie database
[
https://issues.apache.org/jira/browse/OOZIE-3134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16410098#comment-16410098
]
Hadoop QA commented on OOZIE-3134:
--
Testing JIRA OOZIE-3134
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:red}-1{color} the patch contains 1 line(s) longer than 132
characters
.{color:green}+1{color} the patch adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
.{color:red}WARNING{color}: the current HEAD has 100 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:red}-1{color} There are [1] new bugs found below threshold in total that
must be fixed.
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:red}-1{color} There are [1] new bugs found below threshold in [core]
that must be fixed.
. You can find the FindBugs diff here (look for the red and orange ones):
core/findbugs-new.html
. The most important FindBugs errors are:
. At FailingConnectionWrapper.java:[line 175]: This use of
java/sql/Connection.prepareStatement(Ljava/lang/String;II)Ljava/sql/PreparedStatement;
can be vulnerable to SQL injection
. At FailingConnectionWrapper.java:[line 169]
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [server].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 2100
.{color:orange}Tests failed at first run:{color}
TestJavaActionExecutor#testCredentialsSkip
TestCoordActionsKillXCommand#testActionKillCommandActionNumbers
TestHiveActionExecutor#testHiveAction
.For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
{color:red}. There is at least one warning, please check{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/459/
> Potential inconsistency between the in-memory SLA map and the Oozie database
>
>
> Key: OOZIE-3134
> URL: https://issues.apache.org/jira/browse/OOZIE-3134
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.1.0
>
> Attachments: OOZIE-3134-001.patch, OOZIE-3134-002.patch,
> OOZIE-3134-003.patch, OOZIE-3134-004.patch, OOZIE-3134-005.patch
>
>
> Upon {{SLACalculatorMemory.addRegistration}}, Oozie puts an entry into an
> in-memory concurrent hashmap ("slaMap"), and later [persists
> it|https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L380]
> in the Oozie database.
> However, if there is a failure during the database operation, a
> {{JPAExecutorException}} is thrown, and the entry [is not removed from the
> SLA map|
> https://github.com/apache/oozie/blob/50f4b5984832941f134158
[jira] [Commented] (OOZIE-3134) Potential inconsistency between the in-memory SLA map and the Oozie database
[
https://issues.apache.org/jira/browse/OOZIE-3134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16409883#comment-16409883
]
Hadoop QA commented on OOZIE-3134:
--
PreCommit-OOZIE-Build started
> Potential inconsistency between the in-memory SLA map and the Oozie database
>
>
> Key: OOZIE-3134
> URL: https://issues.apache.org/jira/browse/OOZIE-3134
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.1.0
>
> Attachments: OOZIE-3134-001.patch, OOZIE-3134-002.patch,
> OOZIE-3134-003.patch, OOZIE-3134-004.patch, OOZIE-3134-005.patch
>
>
> Upon {{SLACalculatorMemory.addRegistration}}, Oozie puts an entry into an
> in-memory concurrent hashmap ("slaMap"), and later [persists
> it|https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L380]
> in the Oozie database.
> However, if there is a failure during the database operation, a
> {{JPAExecutorException}} is thrown, and the entry [is not removed from the
> SLA map|
> https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L393].
> It may introduce inconsistency between the Oozie database and the SLA map.
> To prevent this, a rollback mechanism (with proper logging) should be
> implemented. It would also make sense to do more sanity/consistency check in
> the Oozie server.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3134) Potential inconsistency between the in-memory SLA map and the Oozie database
[
https://issues.apache.org/jira/browse/OOZIE-3134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16369184#comment-16369184
]
Julia Kinga Marton commented on OOZIE-3134:
---
The test failures are unrelated with the patch.
> Potential inconsistency between the in-memory SLA map and the Oozie database
>
>
> Key: OOZIE-3134
> URL: https://issues.apache.org/jira/browse/OOZIE-3134
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.0.0
>
> Attachments: OOZIE-3134-001.patch, OOZIE-3134-002.patch,
> OOZIE-3134-003.patch, OOZIE-3134-004.patch
>
>
> Upon {{SLACalculatorMemory.addRegistration}}, Oozie puts an entry into an
> in-memory concurrent hashmap ("slaMap"), and later [persists
> it|https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L380]
> in the Oozie database.
> However, if there is a failure during the database operation, a
> {{JPAExecutorException}} is thrown, and the entry [is not removed from the
> SLA map|
> https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L393].
> It may introduce inconsistency between the Oozie database and the SLA map.
> To prevent this, a rollback mechanism (with proper logging) should be
> implemented. It would also make sense to do more sanity/consistency check in
> the Oozie server.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3134) Potential inconsistency between the in-memory SLA map and the Oozie database
[
https://issues.apache.org/jira/browse/OOZIE-3134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16369177#comment-16369177
]
Hadoop QA commented on OOZIE-3134:
--
Testing JIRA OOZIE-3134
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:red}-1{color} the patch contains 1 line(s) longer than 132
characters
.{color:green}+1{color} the patch adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
.{color:red}WARNING{color}: the current HEAD has 100 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:red}-1{color} There are [1] new bugs found below threshold in total that
must be fixed.
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:red}-1{color} There are [1] new bugs found below threshold in [core]
that must be fixed.
. You can find the FindBugs diff here (look for the red and orange ones):
core/findbugs-new.html
. The most important FindBugs errors are:
. At FailingConnectionWrapper.java:[line 175]: This use of
java/sql/Connection.prepareStatement(Ljava/lang/String;II)Ljava/sql/PreparedStatement;
can be vulnerable to SQL injection
. At FailingConnectionWrapper.java:[line 169]
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [client].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 2096
.Tests failed: 17
.Tests errors: 12
.The patch failed the following testcases:
testCoordActionRecoveryServiceForWaitingRegisterPartition(org.apache.oozie.service.TestRecoveryService)
testCoordinatorActionSelectors(org.apache.oozie.jms.TestJMSJobEventListener)
testCoordinatorActionSelectorsNegative(org.apache.oozie.jms.TestJMSJobEventListener)
testWorkflowJobSelectors(org.apache.oozie.jms.TestJMSJobEventListener)
testOnCoordinatorJobSuccessEvent(org.apache.oozie.jms.TestJMSJobEventListener)
testOnWorkflowJobSuspendEvent(org.apache.oozie.jms.TestJMSJobEventListener)
testWorkflowJobSelectorsAnd(org.apache.oozie.jms.TestJMSJobEventListener)
testOnCoordinatorJobFailureEvent(org.apache.oozie.jms.TestJMSJobEventListener)
testConnectionDrop(org.apache.oozie.jms.TestJMSJobEventListener)
testWorkflowJobSelectorsNegative(org.apache.oozie.jms.TestJMSJobEventListener)
testWorkflowJobSelectorsOr(org.apache.oozie.jms.TestJMSJobEventListener)
testOnWorkflowJobSuccessEvent(org.apache.oozie.jms.TestJMSJobEventListener)
testOnCoordinatorActionWaitingEvent(org.apache.oozie.jms.TestJMSJobEventListener)
testOnWorkflowJobFailureEvent(org.apache.oozie.jms.TestJMSJobEventListener)
testOnCoordinatorActionStartEvent(org.apache.oozie.jms.TestJMSJobEventListener)
testOnWorkflowJobStartedEvent(org.apache.oozie.jms.TestJMSJobEventListener)
testPartitionDependency(org.apache.oozie.service.TestPartitionDependencyManagerService)
.Tests failing with errors:
testOnSLADurationMetEvent(org.apache.oozie.jms.TestJMSSLAEventListener)
testOnSLAStartMetEvent(org.apache.oozie.jms.TestJMSSLAEventListener)
testSLAJobSelectorsNegative(org.apache.oozie.jms.TestJMSSLAEventListener)
testOnSLAEndMetEvent(org.apache.oozie.jms.TestJMSSLAEventListener)
testOnSLAStartMissEvent(org.apache.oozie.jms.
[jira] [Commented] (OOZIE-3134) Potential inconsistency between the in-memory SLA map and the Oozie database
[
https://issues.apache.org/jira/browse/OOZIE-3134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16369103#comment-16369103
]
Hadoop QA commented on OOZIE-3134:
--
PreCommit-OOZIE-Build started
> Potential inconsistency between the in-memory SLA map and the Oozie database
>
>
> Key: OOZIE-3134
> URL: https://issues.apache.org/jira/browse/OOZIE-3134
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.0.0
>
> Attachments: OOZIE-3134-001.patch, OOZIE-3134-002.patch,
> OOZIE-3134-003.patch, OOZIE-3134-004.patch
>
>
> Upon {{SLACalculatorMemory.addRegistration}}, Oozie puts an entry into an
> in-memory concurrent hashmap ("slaMap"), and later [persists
> it|https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L380]
> in the Oozie database.
> However, if there is a failure during the database operation, a
> {{JPAExecutorException}} is thrown, and the entry [is not removed from the
> SLA map|
> https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L393].
> It may introduce inconsistency between the Oozie database and the SLA map.
> To prevent this, a rollback mechanism (with proper logging) should be
> implemented. It would also make sense to do more sanity/consistency check in
> the Oozie server.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3134) Potential inconsistency between the in-memory SLA map and the Oozie database
[
https://issues.apache.org/jira/browse/OOZIE-3134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16362942#comment-16362942
]
Hadoop QA commented on OOZIE-3134:
--
Testing JIRA OOZIE-3134
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:red}-1{color} the patch contains 1 line(s) longer than 132
characters
.{color:green}+1{color} the patch adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
.{color:red}WARNING{color}: the current HEAD has 100 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:red}-1{color} There are [2] new bugs found below threshold in total that
must be fixed.
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:red}-1{color} There are [2] new bugs found below threshold in [core]
that must be fixed.
. You can find the FindBugs diff here (look for the red and orange ones):
core/findbugs-new.html
. The most important FindBugs errors are:
. At FailingConnectionWrapper.java:[line 175]: This use of
java/sql/Connection.prepareStatement(Ljava/lang/String;II)Ljava/sql/PreparedStatement;
can be vulnerable to SQL injection
. org.apache.oozie.util.db.FailingDBHelperForTest.dbPredicate should be package
protected: At FailingConnectionWrapper.java:[line 169]
. At FailingDBHelperForTest.java:[line 30]
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [server].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 2096
.{color:orange}Tests failed at first run:{color}
TestJavaActionExecutor#testCredentialsSkip
.For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
{color:red}. There is at least one warning, please check{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/395/
> Potential inconsistency between the in-memory SLA map and the Oozie database
>
>
> Key: OOZIE-3134
> URL: https://issues.apache.org/jira/browse/OOZIE-3134
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.0.0
>
> Attachments: OOZIE-3134-001.patch, OOZIE-3134-002.patch,
> OOZIE-3134-003.patch
>
>
> Upon {{SLACalculatorMemory.addRegistration}}, Oozie puts an entry into an
> in-memory concurrent hashmap ("slaMap"), and later [persists
> it|https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L380]
> in the Oozie database.
> However, if there is a failure during the database operation, a
> {{JPAExecutorException}} is thrown, and the entry [is not removed from the
> SLA map|
> https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293
[jira] [Commented] (OOZIE-3134) Potential inconsistency between the in-memory SLA map and the Oozie database
[
https://issues.apache.org/jira/browse/OOZIE-3134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16362778#comment-16362778
]
Hadoop QA commented on OOZIE-3134:
--
PreCommit-OOZIE-Build started
> Potential inconsistency between the in-memory SLA map and the Oozie database
>
>
> Key: OOZIE-3134
> URL: https://issues.apache.org/jira/browse/OOZIE-3134
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.0.0
>
> Attachments: OOZIE-3134-001.patch, OOZIE-3134-002.patch,
> OOZIE-3134-003.patch
>
>
> Upon {{SLACalculatorMemory.addRegistration}}, Oozie puts an entry into an
> in-memory concurrent hashmap ("slaMap"), and later [persists
> it|https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L380]
> in the Oozie database.
> However, if there is a failure during the database operation, a
> {{JPAExecutorException}} is thrown, and the entry [is not removed from the
> SLA map|
> https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L393].
> It may introduce inconsistency between the Oozie database and the SLA map.
> To prevent this, a rollback mechanism (with proper logging) should be
> implemented. It would also make sense to do more sanity/consistency check in
> the Oozie server.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3134) Potential inconsistency between the in-memory SLA map and the Oozie database
[
https://issues.apache.org/jira/browse/OOZIE-3134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16362484#comment-16362484
]
Julia Kinga Marton commented on OOZIE-3134:
---
Junit results with dist-test:
[http://dist-test.cloudera.org/job?job_id=oozie2.martonjuliakinga.1518534651.31839#|http://dist-test.cloudera.org/job?job_id=oozie2.martonjuliakinga.1518534651.31839]
> Potential inconsistency between the in-memory SLA map and the Oozie database
>
>
> Key: OOZIE-3134
> URL: https://issues.apache.org/jira/browse/OOZIE-3134
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.0.0
>
> Attachments: OOZIE-3134-001.patch, OOZIE-3134-002.patch,
> OOZIE-3134-003.patch
>
>
> Upon {{SLACalculatorMemory.addRegistration}}, Oozie puts an entry into an
> in-memory concurrent hashmap ("slaMap"), and later [persists
> it|https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L380]
> in the Oozie database.
> However, if there is a failure during the database operation, a
> {{JPAExecutorException}} is thrown, and the entry [is not removed from the
> SLA map|
> https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L393].
> It may introduce inconsistency between the Oozie database and the SLA map.
> To prevent this, a rollback mechanism (with proper logging) should be
> implemented. It would also make sense to do more sanity/consistency check in
> the Oozie server.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3134) Potential inconsistency between the in-memory SLA map and the Oozie database
[
https://issues.apache.org/jira/browse/OOZIE-3134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16355124#comment-16355124
]
Julia Kinga Marton commented on OOZIE-3134:
---
Thank you [~rohini] for pointing out the OOZIE-1980 issue. I will keep that in
mind and will continue with this small fix for the scenario 2.
> Potential inconsistency between the in-memory SLA map and the Oozie database
>
>
> Key: OOZIE-3134
> URL: https://issues.apache.org/jira/browse/OOZIE-3134
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.0.0
>
> Attachments: OOZIE-3134-001.patch, OOZIE-3134-002.patch
>
>
> Upon {{SLACalculatorMemory.addRegistration}}, Oozie puts an entry into an
> in-memory concurrent hashmap ("slaMap"), and later [persists
> it|https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L380]
> in the Oozie database.
> However, if there is a failure during the database operation, a
> {{JPAExecutorException}} is thrown, and the entry [is not removed from the
> SLA map|
> https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L393].
> It may introduce inconsistency between the Oozie database and the SLA map.
> To prevent this, a rollback mechanism (with proper logging) should be
> implemented. It would also make sense to do more sanity/consistency check in
> the Oozie server.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3134) Potential inconsistency between the in-memory SLA map and the Oozie database
[
https://issues.apache.org/jira/browse/OOZIE-3134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354850#comment-16354850
]
Andras Piros commented on OOZIE-3134:
-
Thanks [~rohini] for the feedback! 1) will then be addressed as part of
OOZIE-1980, and only 2) here.
As for OOZIE-2854 and database retries, there is a short intro as part of
[*AG_Install.twiki*|https://oozie.apache.org/docs/5.0.0-beta1/AG_Install.html#Database_Configuration]
in {{5.0.0-beta1}}.
> Potential inconsistency between the in-memory SLA map and the Oozie database
>
>
> Key: OOZIE-3134
> URL: https://issues.apache.org/jira/browse/OOZIE-3134
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.0.0
>
> Attachments: OOZIE-3134-001.patch, OOZIE-3134-002.patch
>
>
> Upon {{SLACalculatorMemory.addRegistration}}, Oozie puts an entry into an
> in-memory concurrent hashmap ("slaMap"), and later [persists
> it|https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L380]
> in the Oozie database.
> However, if there is a failure during the database operation, a
> {{JPAExecutorException}} is thrown, and the entry [is not removed from the
> SLA map|
> https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L393].
> It may introduce inconsistency between the Oozie database and the SLA map.
> To prevent this, a rollback mechanism (with proper logging) should be
> implemented. It would also make sense to do more sanity/consistency check in
> the Oozie server.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3134) Potential inconsistency between the in-memory SLA map and the Oozie database
[
https://issues.apache.org/jira/browse/OOZIE-3134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354633#comment-16354633
]
Rohini Palaniswamy commented on OOZIE-3134:
---
bq. This issue was originally targeted to fix only scenario 2)
Just looked at the patch. All it does is a minor change of adding the entry to
the map after the database operation. This should be fine to go in. 1) can be
addressed in OOZIE-1980. We started with that jira and patch when he had a
database issue. But never got around to wrapping that up and it did not became
a priority as we have not run into database issues after that for long (Touch
wood). But it is something that has to be fixed.
Was not aware of OOZIE-2854 and database update retries. Will take a look at
that sometime.
> Potential inconsistency between the in-memory SLA map and the Oozie database
>
>
> Key: OOZIE-3134
> URL: https://issues.apache.org/jira/browse/OOZIE-3134
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.0.0
>
> Attachments: OOZIE-3134-001.patch, OOZIE-3134-002.patch
>
>
> Upon {{SLACalculatorMemory.addRegistration}}, Oozie puts an entry into an
> in-memory concurrent hashmap ("slaMap"), and later [persists
> it|https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L380]
> in the Oozie database.
> However, if there is a failure during the database operation, a
> {{JPAExecutorException}} is thrown, and the entry [is not removed from the
> SLA map|
> https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L393].
> It may introduce inconsistency between the Oozie database and the SLA map.
> To prevent this, a rollback mechanism (with proper logging) should be
> implemented. It would also make sense to do more sanity/consistency check in
> the Oozie server.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3134) Potential inconsistency between the in-memory SLA map and the Oozie database
[
https://issues.apache.org/jira/browse/OOZIE-3134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354175#comment-16354175
]
Hadoop QA commented on OOZIE-3134:
--
Testing JIRA OOZIE-3134
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
.{color:red}WARNING{color}: the current HEAD has 100 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [server].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 2093
.Tests failed: 4
.Tests errors: 10
.The patch failed the following testcases:
testPartitionDependency(org.apache.oozie.service.TestPartitionDependencyManagerService)
testCoordActionRecoveryServiceForWaitingRegisterPartition(org.apache.oozie.service.TestRecoveryService)
testConnectionDrop(org.apache.oozie.jms.TestJMSJobEventListener)
testPartitionDependency(org.apache.oozie.service.TestPartitionDependencyManagerEhcache)
.Tests failing with errors:
testConnectionRetry(org.apache.oozie.service.TestJMSAccessorService)
testAddRegistrationDBFAilure(org.apache.oozie.sla.TestSLACalculatorMemory)
testUpdateRegistrationDBFailure(org.apache.oozie.sla.TestSLACalculatorMemory)
testParallelFsAndShellWorkflowCompletesSuccessfully(org.apache.oozie.test.TestWorkflowRetries)
testWorkflowWithStartAndEndCompletesSuccessfully(org.apache.oozie.test.TestWorkflowRetries)
testFsDecisionWorkflowCompletesSuccessfully(org.apache.oozie.test.TestWorkflowRetries)
testParallelFsAndShellWorkflowCompletesSuccessfully(org.apache.oozie.test.TestWorkflow)
testWorkflowWithStartAndEndCompletesSuccessfully(org.apache.oozie.test.TestWorkflow)
testFsDecisionWorkflowCompletesSuccessfully(org.apache.oozie.test.TestWorkflow)
testParallelJPAOperationsOnWorkflowBeansRetryAndSucceed(org.apache.oozie.test.TestParallelJPAOperationRetries)
.{color:orange}Tests failed at first run:{color}
TestJavaActionExecutor#testCredentialsSkip
.For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
{color:red}. There is at least one warning, please check{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/380/
> Potential inconsistency between the in-memory SLA map and the Oozie database
>
>
> Key: OOZIE-3134
> URL: https://issues.apache.org/jira/browse/OOZIE-3134
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Pr
[jira] [Commented] (OOZIE-3134) Potential inconsistency between the in-memory SLA map and the Oozie database
[
https://issues.apache.org/jira/browse/OOZIE-3134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354078#comment-16354078
]
Andras Piros commented on OOZIE-3134:
-
[~rohini] while I see there is added value in addressing also 1), which, to my
understanding, would be covered by OOZIE-1980, we've handled transient database
hickups in OOZIE-2854 (part of {{5.0.0-beta1}}). So the main reason behind
OOZIE-1980 would be resolved in {{5.0.0}} but not in {{4.3.1}}.
This issue was originally targeted to fix only scenario 2). What is your view
upon that? E.g. in subsequent {{4.3}} minor versions, would we still need to
address OOZIE-1980?
> Potential inconsistency between the in-memory SLA map and the Oozie database
>
>
> Key: OOZIE-3134
> URL: https://issues.apache.org/jira/browse/OOZIE-3134
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.0.0
>
> Attachments: OOZIE-3134-001.patch, OOZIE-3134-002.patch
>
>
> Upon {{SLACalculatorMemory.addRegistration}}, Oozie puts an entry into an
> in-memory concurrent hashmap ("slaMap"), and later [persists
> it|https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L380]
> in the Oozie database.
> However, if there is a failure during the database operation, a
> {{JPAExecutorException}} is thrown, and the entry [is not removed from the
> SLA map|
> https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L393].
> It may introduce inconsistency between the Oozie database and the SLA map.
> To prevent this, a rollback mechanism (with proper logging) should be
> implemented. It would also make sense to do more sanity/consistency check in
> the Oozie server.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3134) Potential inconsistency between the in-memory SLA map and the Oozie database
[
https://issues.apache.org/jira/browse/OOZIE-3134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354039#comment-16354039
]
Hadoop QA commented on OOZIE-3134:
--
PreCommit-OOZIE-Build started
> Potential inconsistency between the in-memory SLA map and the Oozie database
>
>
> Key: OOZIE-3134
> URL: https://issues.apache.org/jira/browse/OOZIE-3134
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.0.0
>
> Attachments: OOZIE-3134-001.patch, OOZIE-3134-002.patch
>
>
> Upon {{SLACalculatorMemory.addRegistration}}, Oozie puts an entry into an
> in-memory concurrent hashmap ("slaMap"), and later [persists
> it|https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L380]
> in the Oozie database.
> However, if there is a failure during the database operation, a
> {{JPAExecutorException}} is thrown, and the entry [is not removed from the
> SLA map|
> https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L393].
> It may introduce inconsistency between the Oozie database and the SLA map.
> To prevent this, a rollback mechanism (with proper logging) should be
> implemented. It would also make sense to do more sanity/consistency check in
> the Oozie server.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3134) Potential inconsistency between the in-memory SLA map and the Oozie database
[
https://issues.apache.org/jira/browse/OOZIE-3134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354017#comment-16354017
]
Rohini Palaniswamy commented on OOZIE-3134:
---
There is more to this issue. Please refer OOZIE-1980.
There are two issues to be addressed.
1) Include the SLA inserts as part of the coord action table inserts and make
them one transaction which JPA can rollback on database failure. Currently they
are two separate JPA transactions.
2) Remove the SLA entry from the in memory map on JPA rollback.
> Potential inconsistency between the in-memory SLA map and the Oozie database
>
>
> Key: OOZIE-3134
> URL: https://issues.apache.org/jira/browse/OOZIE-3134
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.0.0
>
> Attachments: OOZIE-3134-001.patch, OOZIE-3134-002.patch
>
>
> Upon {{SLACalculatorMemory.addRegistration}}, Oozie puts an entry into an
> in-memory concurrent hashmap ("slaMap"), and later [persists
> it|https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L380]
> in the Oozie database.
> However, if there is a failure during the database operation, a
> {{JPAExecutorException}} is thrown, and the entry [is not removed from the
> SLA map|
> https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L393].
> It may introduce inconsistency between the Oozie database and the SLA map.
> To prevent this, a rollback mechanism (with proper logging) should be
> implemented. It would also make sense to do more sanity/consistency check in
> the Oozie server.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3134) Potential inconsistency between the in-memory SLA map and the Oozie database
[
https://issues.apache.org/jira/browse/OOZIE-3134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350676#comment-16350676
]
Hadoop QA commented on OOZIE-3134:
--
Testing JIRA OOZIE-3134
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch adds/modifies 2 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
.{color:red}WARNING{color}: the current HEAD has 100 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 2092
.Tests failed: 17
.Tests errors: 12
.The patch failed the following testcases:
testPartitionDependency(org.apache.oozie.service.TestPartitionDependencyManagerEhcache)
testCoordActionRecoveryServiceForWaitingRegisterPartition(org.apache.oozie.service.TestRecoveryService)
testCoordinatorActionSelectors(org.apache.oozie.jms.TestJMSJobEventListener)
testCoordinatorActionSelectorsNegative(org.apache.oozie.jms.TestJMSJobEventListener)
testWorkflowJobSelectors(org.apache.oozie.jms.TestJMSJobEventListener)
testOnCoordinatorJobSuccessEvent(org.apache.oozie.jms.TestJMSJobEventListener)
testOnWorkflowJobSuspendEvent(org.apache.oozie.jms.TestJMSJobEventListener)
testWorkflowJobSelectorsAnd(org.apache.oozie.jms.TestJMSJobEventListener)
testOnCoordinatorJobFailureEvent(org.apache.oozie.jms.TestJMSJobEventListener)
testConnectionDrop(org.apache.oozie.jms.TestJMSJobEventListener)
testWorkflowJobSelectorsNegative(org.apache.oozie.jms.TestJMSJobEventListener)
testWorkflowJobSelectorsOr(org.apache.oozie.jms.TestJMSJobEventListener)
testOnWorkflowJobSuccessEvent(org.apache.oozie.jms.TestJMSJobEventListener)
testOnCoordinatorActionWaitingEvent(org.apache.oozie.jms.TestJMSJobEventListener)
testOnWorkflowJobFailureEvent(org.apache.oozie.jms.TestJMSJobEventListener)
testOnCoordinatorActionStartEvent(org.apache.oozie.jms.TestJMSJobEventListener)
testOnWorkflowJobStartedEvent(org.apache.oozie.jms.TestJMSJobEventListener)
.Tests failing with errors:
testDropEventTypeMessage(org.apache.oozie.jms.TestHCatMessageHandler)
testCacheUpdateByMessage(org.apache.oozie.jms.TestHCatMessageHandler)
testOnSLADurationMetEvent(org.apache.oozie.jms.TestJMSSLAEventListener)
testOnSLAStartMetEvent(org.apache.oozie.jms.TestJMSSLAEventListener)
testSLAJobSelectorsNegative(org.apache.oozie.jms.TestJMSSLAEventListener)
testOnSLAEndMetEvent(org.apache.oozie.jms.TestJMSSLAEventListener)
testOnSLAStartMissEvent(org.apache.oozie.jms.TestJMSSLAEventListener)
testSLAJobSelectors(org.apache.oozie.jms.TestJMSSLAEventListener)
testOnSLAEndMissEvent(org.apache.oozie.jms.TestJMSSLAEventListener)
testOnSLADurationMissEvent(org.apache.oozie.jms.TestJMSSLAEventListener)
testConnectionRetry(org.apache.oozie.service.TestJMSAccessorService)
testThreadLocalSession(org.apache.oozie.jms.TestDefaultConnect
[jira] [Commented] (OOZIE-3134) Potential inconsistency between the in-memory SLA map and the Oozie database
[
https://issues.apache.org/jira/browse/OOZIE-3134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350527#comment-16350527
]
Hadoop QA commented on OOZIE-3134:
--
PreCommit-OOZIE-Build started
> Potential inconsistency between the in-memory SLA map and the Oozie database
>
>
> Key: OOZIE-3134
> URL: https://issues.apache.org/jira/browse/OOZIE-3134
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.0.0
>
> Attachments: OOZIE-3134-001.patch
>
>
> Upon {{SLACalculatorMemory.addRegistration}}, Oozie puts an entry into an
> in-memory concurrent hashmap ("slaMap"), and later [persists
> it|https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L380]
> in the Oozie database.
> However, if there is a failure during the database operation, a
> {{JPAExecutorException}} is thrown, and the entry [is not removed from the
> SLA map|
> https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L393].
> It may introduce inconsistency between the Oozie database and the SLA map.
> To prevent this, a rollback mechanism (with proper logging) should be
> implemented. It would also make sense to do more sanity/consistency check in
> the Oozie server.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3134) Potential inconsistency between the in-memory SLA map and the Oozie database
[
https://issues.apache.org/jira/browse/OOZIE-3134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350522#comment-16350522
]
Julia Kinga Marton commented on OOZIE-3134:
---
The failed tests passed locally, however I will update soon a review-fix patch,
I hope then the results will be fine.
> Potential inconsistency between the in-memory SLA map and the Oozie database
>
>
> Key: OOZIE-3134
> URL: https://issues.apache.org/jira/browse/OOZIE-3134
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.0.0
>
> Attachments: OOZIE-3134-001.patch
>
>
> Upon {{SLACalculatorMemory.addRegistration}}, Oozie puts an entry into an
> in-memory concurrent hashmap ("slaMap"), and later [persists
> it|https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L380]
> in the Oozie database.
> However, if there is a failure during the database operation, a
> {{JPAExecutorException}} is thrown, and the entry [is not removed from the
> SLA map|
> https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L393].
> It may introduce inconsistency between the Oozie database and the SLA map.
> To prevent this, a rollback mechanism (with proper logging) should be
> implemented. It would also make sense to do more sanity/consistency check in
> the Oozie server.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3134) Potential inconsistency between the in-memory SLA map and the Oozie database
[
https://issues.apache.org/jira/browse/OOZIE-3134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350505#comment-16350505
]
Hadoop QA commented on OOZIE-3134:
--
Testing JIRA OOZIE-3134
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch adds/modifies 2 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
.{color:red}WARNING{color}: the current HEAD has 100 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [client].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 2092
.Tests failed: 17
.Tests errors: 12
.The patch failed the following testcases:
testCoordActionRecoveryServiceForWaitingRegisterPartition(org.apache.oozie.service.TestRecoveryService)
testCoordinatorActionSelectors(org.apache.oozie.jms.TestJMSJobEventListener)
testCoordinatorActionSelectorsNegative(org.apache.oozie.jms.TestJMSJobEventListener)
testWorkflowJobSelectors(org.apache.oozie.jms.TestJMSJobEventListener)
testOnCoordinatorJobSuccessEvent(org.apache.oozie.jms.TestJMSJobEventListener)
testOnWorkflowJobSuspendEvent(org.apache.oozie.jms.TestJMSJobEventListener)
testWorkflowJobSelectorsAnd(org.apache.oozie.jms.TestJMSJobEventListener)
testOnCoordinatorJobFailureEvent(org.apache.oozie.jms.TestJMSJobEventListener)
testConnectionDrop(org.apache.oozie.jms.TestJMSJobEventListener)
testWorkflowJobSelectorsNegative(org.apache.oozie.jms.TestJMSJobEventListener)
testWorkflowJobSelectorsOr(org.apache.oozie.jms.TestJMSJobEventListener)
testOnWorkflowJobSuccessEvent(org.apache.oozie.jms.TestJMSJobEventListener)
testOnCoordinatorActionWaitingEvent(org.apache.oozie.jms.TestJMSJobEventListener)
testOnWorkflowJobFailureEvent(org.apache.oozie.jms.TestJMSJobEventListener)
testOnCoordinatorActionStartEvent(org.apache.oozie.jms.TestJMSJobEventListener)
testOnWorkflowJobStartedEvent(org.apache.oozie.jms.TestJMSJobEventListener)
testPartitionDependency(org.apache.oozie.service.TestPartitionDependencyManagerService)
.Tests failing with errors:
testOnSLADurationMetEvent(org.apache.oozie.jms.TestJMSSLAEventListener)
testOnSLAStartMetEvent(org.apache.oozie.jms.TestJMSSLAEventListener)
testSLAJobSelectorsNegative(org.apache.oozie.jms.TestJMSSLAEventListener)
testOnSLAEndMetEvent(org.apache.oozie.jms.TestJMSSLAEventListener)
testOnSLAStartMissEvent(org.apache.oozie.jms.TestJMSSLAEventListener)
testSLAJobSelectors(org.apache.oozie.jms.TestJMSSLAEventListener)
testOnSLAEndMissEvent(org.apache.oozie.jms.TestJMSSLAEventListener)
testOnSLADurationMissEvent(org.apache.oozie.jms.TestJMSSLAEventListener)
testDropEventTypeMessage(org.apache.oozie.jms.TestHCatMessageHandler)
testCacheUpdateByMessage(org.apache.oozie.jms.TestHCatMessageHandler)
testThreadLocalSession(org.apache.oozie.jms.TestDefaultConnectionContext)
testConnectionRetry(org.apache.oozie.service.TestJMSAcces
[jira] [Commented] (OOZIE-3134) Potential inconsistency between the in-memory SLA map and the Oozie database
[
https://issues.apache.org/jira/browse/OOZIE-3134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350341#comment-16350341
]
Hadoop QA commented on OOZIE-3134:
--
PreCommit-OOZIE-Build started
> Potential inconsistency between the in-memory SLA map and the Oozie database
>
>
> Key: OOZIE-3134
> URL: https://issues.apache.org/jira/browse/OOZIE-3134
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.0.0
>
> Attachments: OOZIE-3134-001.patch
>
>
> Upon {{SLACalculatorMemory.addRegistration}}, Oozie puts an entry into an
> in-memory concurrent hashmap ("slaMap"), and later [persists
> it|https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L380]
> in the Oozie database.
> However, if there is a failure during the database operation, a
> {{JPAExecutorException}} is thrown, and the entry [is not removed from the
> SLA map|
> https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L393].
> It may introduce inconsistency between the Oozie database and the SLA map.
> To prevent this, a rollback mechanism (with proper logging) should be
> implemented. It would also make sense to do more sanity/consistency check in
> the Oozie server.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3134) Potential inconsistency between the in-memory SLA map and the Oozie database
[
https://issues.apache.org/jira/browse/OOZIE-3134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350328#comment-16350328
]
Andras Piros commented on OOZIE-3134:
-
Thanks for the patch [~kmarton]! Can you please create a ReviewBoard review
request as well?
> Potential inconsistency between the in-memory SLA map and the Oozie database
>
>
> Key: OOZIE-3134
> URL: https://issues.apache.org/jira/browse/OOZIE-3134
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.0.0
>
> Attachments: OOZIE-3134-001.patch
>
>
> Upon {{SLACalculatorMemory.addRegistration}}, Oozie puts an entry into an
> in-memory concurrent hashmap ("slaMap"), and later [persists
> it|https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L380]
> in the Oozie database.
> However, if there is a failure during the database operation, a
> {{JPAExecutorException}} is thrown, and the entry [is not removed from the
> SLA map|
> https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L393].
> It may introduce inconsistency between the Oozie database and the SLA map.
> To prevent this, a rollback mechanism (with proper logging) should be
> implemented. It would also make sense to do more sanity/consistency check in
> the Oozie server.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
