mosche commented on a change in pull request #15910:
URL: https://github.com/apache/beam/pull/15910#discussion_r743781190
##########
File path:
sdks/java/io/amazon-web-services/src/main/java/org/apache/beam/sdk/io/aws/sns/SnsIO.java
##########
@@ -111,25 +111,39 @@ public static Write write() {
@AutoValue.CopyAnnotations
@SuppressWarnings({"rawtypes"})
public abstract static class RetryConfiguration implements Serializable {
+ private static final Duration DEFAULT_INITIAL_DURATION =
Duration.standardSeconds(5);
+
@VisibleForTesting
static final RetryPredicate DEFAULT_RETRY_PREDICATE = new
DefaultRetryPredicate();
abstract int getMaxAttempts();
abstract Duration getMaxDuration();
+ abstract Duration getInitialDuration();
+
abstract RetryPredicate getRetryPredicate();
abstract Builder builder();
- public static RetryConfiguration create(int maxAttempts, Duration
maxDuration) {
+ @VisibleForTesting
+ static RetryConfiguration create(int maxAttempts, Duration maxDuration) {
Review comment:
Thanks, good catch! This one absolutely has to remain public!
Also, `@VisibleForTesting` belongs to the other one ...
##########
File path:
sdks/java/io/amazon-web-services/src/test/java/org/apache/beam/sdk/io/aws/sns/SnsIOTest.java
##########
@@ -114,7 +118,10 @@ public void testDataWritesToSNS() {
@Test
public void testRetries() throws Throwable {
+ thrown.expect(IOException.class);
Review comment:
No, it's just important to not accept any Exception! Otherwise the
AssertionError thrown by `snsWriterFnLogs.verifyWarn` fullfills this, which is
kind of pointless. But that's exactly what happened, because it was checking
the wrong logger
##########
File path:
sdks/java/io/amazon-web-services/src/test/java/org/apache/beam/sdk/io/aws/sns/SnsIOTest.java
##########
@@ -114,7 +118,10 @@ public void testDataWritesToSNS() {
@Test
public void testRetries() throws Throwable {
+ thrown.expect(IOException.class);
thrown.expectMessage("Error writing to SNS");
+ thrown.expectMessage("No more attempts allowed");
Review comment:
Each `expectMessage` is just a `contains` check for the message. In this
case the assertion error thrown if the logs did not exist also contained the
first string :/ So the 2nd one is really the one we are looking for here as it
indicates that we've given up retrying.
##########
File path:
sdks/java/io/amazon-web-services/src/test/java/org/apache/beam/sdk/io/aws/sns/SnsIOTest.java
##########
@@ -123,20 +130,19 @@ public void testRetries() throws Throwable {
SnsIO.write()
.withTopicName(topicName)
.withRetryConfiguration(
- SnsIO.RetryConfiguration.create(4,
org.joda.time.Duration.standardSeconds(10)))
+ SnsIO.RetryConfiguration.create(4, standardSeconds(10),
millis(1)))
.withAWSClientsProvider(new Provider(amazonSnsErrors))
.withResultOutputTag(results));
try {
p.run();
} catch (final Pipeline.PipelineExecutionException e) {
// check 3 retries were initiated by inspecting the log before passing
on the exception
-
expectedLogs.verifyWarn(String.format(SnsIO.Write.SnsWriterFn.RETRY_ATTEMPT_LOG,
1));
-
expectedLogs.verifyWarn(String.format(SnsIO.Write.SnsWriterFn.RETRY_ATTEMPT_LOG,
2));
-
expectedLogs.verifyWarn(String.format(SnsIO.Write.SnsWriterFn.RETRY_ATTEMPT_LOG,
3));
+
snsWriterFnLogs.verifyWarn(String.format(SnsIO.Write.SnsWriterFn.RETRY_ATTEMPT_LOG,
1));
+
snsWriterFnLogs.verifyWarn(String.format(SnsIO.Write.SnsWriterFn.RETRY_ATTEMPT_LOG,
2));
+
snsWriterFnLogs.verifyWarn(String.format(SnsIO.Write.SnsWriterFn.RETRY_ATTEMPT_LOG,
3));
throw e.getCause();
}
- fail("Pipeline is expected to fail because we were unable to write to
SNS.");
Review comment:
The test already sets the expected exception using `thrown`, which fails
the test case if no exception is thrown.
```
@Rule public ExpectedException thrown = ExpectedException.none();
```
##########
File path:
sdks/java/io/amazon-web-services/src/test/java/org/apache/beam/sdk/io/aws/sns/SnsIOTest.java
##########
@@ -123,20 +130,19 @@ public void testRetries() throws Throwable {
SnsIO.write()
.withTopicName(topicName)
.withRetryConfiguration(
- SnsIO.RetryConfiguration.create(4,
org.joda.time.Duration.standardSeconds(10)))
+ SnsIO.RetryConfiguration.create(4, standardSeconds(10),
millis(1)))
.withAWSClientsProvider(new Provider(amazonSnsErrors))
.withResultOutputTag(results));
try {
p.run();
} catch (final Pipeline.PipelineExecutionException e) {
// check 3 retries were initiated by inspecting the log before passing
on the exception
-
expectedLogs.verifyWarn(String.format(SnsIO.Write.SnsWriterFn.RETRY_ATTEMPT_LOG,
1));
-
expectedLogs.verifyWarn(String.format(SnsIO.Write.SnsWriterFn.RETRY_ATTEMPT_LOG,
2));
-
expectedLogs.verifyWarn(String.format(SnsIO.Write.SnsWriterFn.RETRY_ATTEMPT_LOG,
3));
+
snsWriterFnLogs.verifyWarn(String.format(SnsIO.Write.SnsWriterFn.RETRY_ATTEMPT_LOG,
1));
+
snsWriterFnLogs.verifyWarn(String.format(SnsIO.Write.SnsWriterFn.RETRY_ATTEMPT_LOG,
2));
+
snsWriterFnLogs.verifyWarn(String.format(SnsIO.Write.SnsWriterFn.RETRY_ATTEMPT_LOG,
3));
throw e.getCause();
}
- fail("Pipeline is expected to fail because we were unable to write to
SNS.");
Review comment:
The test already sets the expected exception using `thrown.expect*`,
which fails the test case if no exception is thrown.
```
@Rule public ExpectedException thrown = ExpectedException.none();
```
##########
File path:
sdks/java/io/amazon-web-services/src/test/java/org/apache/beam/sdk/io/aws/sns/SnsIOTest.java
##########
@@ -114,7 +118,10 @@ public void testDataWritesToSNS() {
@Test
public void testRetries() throws Throwable {
+ thrown.expect(IOException.class);
Review comment:
No, it's just important to not accept any Exception. Otherwise the
AssertionError thrown by `snsWriterFnLogs.verifyWarn` fullfills this, which is
kind of pointless. But that's exactly what happened, because logs for the wrong
logger were captured.
Even more, the message of the assertion error contained exactly that string
we expected to be thrown because it's also the string we were checking for in
the logs.
##########
File path:
sdks/java/io/amazon-web-services/src/main/java/org/apache/beam/sdk/io/aws/sns/SnsIO.java
##########
@@ -111,25 +111,39 @@ public static Write write() {
@AutoValue.CopyAnnotations
@SuppressWarnings({"rawtypes"})
public abstract static class RetryConfiguration implements Serializable {
+ private static final Duration DEFAULT_INITIAL_DURATION =
Duration.standardSeconds(5);
+
@VisibleForTesting
static final RetryPredicate DEFAULT_RETRY_PREDICATE = new
DefaultRetryPredicate();
abstract int getMaxAttempts();
abstract Duration getMaxDuration();
+ abstract Duration getInitialDuration();
+
abstract RetryPredicate getRetryPredicate();
abstract Builder builder();
- public static RetryConfiguration create(int maxAttempts, Duration
maxDuration) {
+ @VisibleForTesting
+ static RetryConfiguration create(int maxAttempts, Duration maxDuration) {
Review comment:
Thanks, good catch! This one absolutely has to remain public!
Also, `@VisibleForTesting` belongs to the other one ... 🤦
##########
File path:
sdks/java/io/amazon-web-services/src/test/java/org/apache/beam/sdk/io/aws/sns/SnsIOTest.java
##########
@@ -114,7 +118,10 @@ public void testDataWritesToSNS() {
@Test
public void testRetries() throws Throwable {
+ thrown.expect(IOException.class);
Review comment:
No, it's just important to not accept all exceptions. Otherwise the
AssertionError thrown by `snsWriterFnLogs.verifyWarn` fullfills this, which is
kind of pointless. But that's exactly what happened, because logs for the wrong
logger were captured.
Even more, the message of the assertion error contained exactly that string
we expected to be thrown because it's also the string we were checking for in
the logs.
##########
File path:
sdks/java/io/amazon-web-services/src/main/java/org/apache/beam/sdk/io/aws/sns/SnsIO.java
##########
@@ -111,25 +111,39 @@ public static Write write() {
@AutoValue.CopyAnnotations
@SuppressWarnings({"rawtypes"})
public abstract static class RetryConfiguration implements Serializable {
+ private static final Duration DEFAULT_INITIAL_DURATION =
Duration.standardSeconds(5);
+
@VisibleForTesting
static final RetryPredicate DEFAULT_RETRY_PREDICATE = new
DefaultRetryPredicate();
abstract int getMaxAttempts();
abstract Duration getMaxDuration();
+ abstract Duration getInitialDuration();
+
abstract RetryPredicate getRetryPredicate();
abstract Builder builder();
- public static RetryConfiguration create(int maxAttempts, Duration
maxDuration) {
+ @VisibleForTesting
+ static RetryConfiguration create(int maxAttempts, Duration maxDuration) {
Review comment:
Fixed @aromanenko-dev
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
