mosche commented on a change in pull request #15895:
URL: https://github.com/apache/beam/pull/15895#discussion_r742845572
##########
File path:
sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/dynamodb/DynamoDBIO.java
##########
@@ -328,6 +337,8 @@ public static Builder builder() {
public abstract Builder setMaxDuration(Duration maxDuration);
+ abstract Builder setInitialDuration(Duration initialDuration);
Review comment:
This is important for testing, otherwise tests involving retries take
forever.
##########
File path:
sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/dynamodb/DynamoDBIO.java
##########
@@ -504,17 +517,20 @@ public void processElement(ProcessContext context) throws
Exception {
}
private Map<String, AttributeValue>
extractDeduplicateKeyValues(WriteRequest request) {
+ List<String> deduplicationKeys = spec.getDeduplicateKeys();
+ Map<String, AttributeValue> attributes = Collections.emptyMap();
+
if (request.putRequest() != null) {
- return request.putRequest().item().entrySet().stream()
- .filter(entry ->
spec.getDeduplicateKeys().contains(entry.getKey()))
- .collect(Collectors.toMap(Map.Entry::getKey,
Map.Entry::getValue));
+ attributes = request.putRequest().item();
} else if (request.deleteRequest() != null) {
- return request.deleteRequest().key().entrySet().stream()
- .filter(entry ->
spec.getDeduplicateKeys().contains(entry.getKey()))
- .collect(Collectors.toMap(Map.Entry::getKey,
Map.Entry::getValue));
- } else {
- return Collections.emptyMap();
+ attributes = request.deleteRequest().key();
+ }
+
+ if (attributes.isEmpty() || deduplicationKeys.isEmpty()) {
+ return attributes;
}
Review comment:
The fix is to just return the unchanged attributes if
`getDeduplicateKeys()` returns an empty list.
Everything else here is pretty much improving test coverage.
##########
File path:
sdks/java/io/amazon-web-services2/src/test/java/org/apache/beam/sdk/io/aws2/dynamodb/DynamoDBIOTest.java
##########
@@ -64,7 +65,10 @@
/** Test Coverage for the IO. */
public class DynamoDBIOTest implements Serializable {
@Rule public final transient TestPipeline pipeline = TestPipeline.create();
- @Rule public final transient ExpectedLogs expectedLogs =
ExpectedLogs.none(DynamoDBIO.class);
+
+ @Rule
+ public final transient ExpectedLogs writeFnLogs =
+ ExpectedLogs.none(DynamoDBIO.Write.WriteFn.class);
Review comment:
This was previously checking for the wrong logger, tests still succeeded
:/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
