dannycranmer commented on code in PR #1: URL: https://github.com/apache/flink-connector-dynamodb/pull/1#discussion_r999023608
########## flink-connector-aws-dynamodb/src/main/java/org/apache/flink/streaming/connectors/dynamodb/sink/DynamoDbSinkElementConverter.java: ########## @@ -0,0 +1,73 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.streaming.connectors.dynamodb.sink; + +import org.apache.flink.annotation.Internal; +import org.apache.flink.api.connector.sink2.SinkWriter; +import org.apache.flink.connector.base.sink.writer.ElementConverter; +import org.apache.flink.streaming.connectors.dynamodb.util.DynamoDbAttributeValueUtils; +import org.apache.flink.util.Preconditions; + +import software.amazon.awssdk.services.dynamodb.model.WriteRequest; + +/** + * An implementation of the {@link ElementConverter} that uses the AWS DynamoDb SDK v2. The user + * needs to provide a {@link DynamoDbRequestConverter} of the {@code InputT} to transform it into a + * {@link DynamoDbWriteRequest} that may be persisted. + */ +@Internal +public class DynamoDbSinkElementConverter<InputT> Review Comment: @hlteoh37 / @YuriGusev / @nirtsruya there has been discussed regarding support for Conditional Writes and non-batch support. I note that the batch API uses `PutRequest` and non-batch uses `PutItemRequest`. They both internally use the same `Map<String, AttributeValue>`, but the `PutItemRequest` adds additional fields. My concern is that we would not be able to toggle batch mode without implementing a new connector, since the generic RequestItem type will change. Therefore, maybe we need some middle ground here. We could use a POJO that contains an `Map<String, AttributeValue>` that we can transform to either `PutRequest` or `PutItemRequest`. Then we can evolve this element converter to transparently support either API. To summarise, I am considering if we actually need a `RequestEntryT` to be something like: ``` interface DynamoDbWriteRequest<T> { Map<String, AttributeValue> getItems(T record); // And some other method to decide if it is put or delete } ``` Then later we can support for `getConditionalExpression()` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
