[
https://issues.apache.org/jira/browse/FLINK-27142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17564726#comment-17564726
]
Alexander Fedulov edited comment on FLINK-27142 at 7/10/22 10:26 PM:
---------------------------------------------------------------------
[~twalthr] I still do not quite understand what is the added value of the
distinction between the upsert and the append tests in this scope. What seems
to be really tested is whether the key is present in the result rows or not:
[RowElasticsearchEmitter.java#L86-L98|https://github.com/apache/flink/blob/2d517950a9ae4a58b25f59fb6048f4c1a15225ac/flink-connectors/flink-connector-elasticsearch-base/src/main/java/org/apache/flink/connector/elasticsearch/table/RowElasticsearchEmitter.java#L86-L98]
The current UpsertTestSink is not suitable, because it does not extract the
keys at all but rather squashes the results based on the row values.
[UpsertTestSinkWriter.java#L66-L67|https://github.com/apache/flink/blob/d0b7a03432dfb6f9318fc716c15d395cc469e26c/flink-test-utils-parent/flink-test-utils/src/main/java/org/apache/flink/connector/upserttest/sink/UpsertTestSinkWriter.java#L66-L67]
Regarding the upsert/append distinction, one could argue that the
interpretation of the input by the end system is irrelevant for the context of
this test and should be verified in the respective ES connector e2e test
instead.
Am I missing something?
was (Author: afedulov):
[~twalthr] I still do not quite understand what is the added value of the
distinction between the upsert and the append tests in this scope. What seems
to be really tested is whether the key is present in the result rows or not:
[RowElasticsearchEmitter.java#L86-L98|https://github.com/apache/flink/blob/2d517950a9ae4a58b25f59fb6048f4c1a15225ac/flink-connectors/flink-connector-elasticsearch-base/src/main/java/org/apache/flink/connector/elasticsearch/table/RowElasticsearchEmitter.java#L86-L98]
The current UpsertTestSink is not suitable, because it does not extract the
keys at all but rather squashes the results based on the row values.
[UpsertTestSinkWriter.java#L66-L67|https://github.com/apache/flink/blob/d0b7a03432dfb6f9318fc716c15d395cc469e26c/flink-test-utils-parent/flink-test-utils/src/main/java/org/apache/flink/connector/upserttest/sink/UpsertTestSinkWriter.java#L66-L67]
Regarding the upsert/append distinction in this scope, one could argue that the
interpretation of the input by the end system is irrelevant for the context of
this test and should be verified in the respective ES connector e2e test
instead.
Am I missing something?
> Remove bash tests dependencies on the Elasticsearch connector.
> --------------------------------------------------------------
>
> Key: FLINK-27142
> URL: https://issues.apache.org/jira/browse/FLINK-27142
> Project: Flink
> Issue Type: Sub-task
> Reporter: Alexander Fedulov
> Assignee: Alexander Preuss
> Priority: Major
> Labels: pull-request-available
>
> Elasticsearch connector is used in test_quickstart.sh and test_sql_client.sh.
> It is desirable to prevent such cyclic dependency and remove the connector
> usage from within Flink.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
