[
https://issues.apache.org/jira/browse/FLINK-16713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17140231#comment-17140231
]
jackylau edited comment on FLINK-16713 at 6/19/20, 6:24 AM:
------------------------------------------------------------
Hi [~rmetzger], when i coding the es source connector here
[https://github.com/liuyongvs/flink/commit/c397a759d05956629a27bf850458dd4e70330189]
,and i find problem of SearchHits at currentScrollWindowHits =
searchResponse.getHits().getHits()
And the SearchHits is interface in es5 but it is a class in es 6,7 and if i use
it in flink es connector base(the es dependency is 5). it will throw this.
Caused by: java.lang.IncompatibleClassChangeError: Found class
org.elasticsearch.search.SearchHits, but interface was expected.
To fix it : we can do this ways
1) move the logic to ApiCallBridge such as define ElasticsearchResponse, but it
wll make the code weirdly
{code:java}
class ElasticsearchResponse
{
String scroll;
String[] result // convert ervery es connector
searchResponse.getHits().getHits() to result
}
if user want to add some thing, it will need modify this{code}
2) just support es 6,7 and upgrade flink-es-connector-base es dependency
version to 6 and drop flink-es-connector-5. And i found this discussion of
dropping es connector 2 and 5 here
[http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/DISCUSS-Drop-connectors-for-Elasticsearch-2-x-and-5-x-td32662.html
.|http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/DISCUSS-Drop-connectors-for-Elasticsearch-2-x-and-5-x-td32662.html]
Is it possible to drop es5 connector and es5 connector just support DataStream
api currently
was (Author: jackylau):
Hi [~rmetzger], when i coding the es source connector here
[https://github.com/liuyongvs/flink/commit/c397a759d05956629a27bf850458dd4e70330189]
,and i find problem of SearchHits at currentScrollWindowHits =
searchResponse.getHits().getHits()
And the SearchHits is interface in es5 but it is a class in es 6,7 and if i use
it in flink es connector base(the es dependency is 5). it will throw this.
Caused by: java.lang.IncompatibleClassChangeError: Found class
org.elasticsearch.search.SearchHits, but interface was expected.
To fix it : we can do this ways
1) move the logic to ApiCallBridge such as define ElasticsearchResponse, but it
wll make the code weirdly
class ElasticsearchResponse {
Scroll scroll;
String[] result // convert ervery es connector
searchResponse.getHits().getHits() to result
}
2) just support es 6,7 and upgrade flink-es-connector-base es dependency
version to 6 and drop flink-es-connector-5. And i found this discussion of
dropping es connector 2 and 5 here
[http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/DISCUSS-Drop-connectors-for-Elasticsearch-2-x-and-5-x-td32662.html
.|http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/DISCUSS-Drop-connectors-for-Elasticsearch-2-x-and-5-x-td32662.html]
Is it possible to drop es5 connector and es5 connector just support DataStream
api currently
> Support source mode of elasticsearch connector
> ----------------------------------------------
>
> Key: FLINK-16713
> URL: https://issues.apache.org/jira/browse/FLINK-16713
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / ElasticSearch
> Affects Versions: 1.10.0
> Reporter: jackray wang
> Priority: Major
>
> [https://ci.apache.org/projects/flink/flink-docs-release-1.10/dev/table/connect.html#elasticsearch-connector]
> For append-only queries, the connector can also operate in [append
> mode|https://ci.apache.org/projects/flink/flink-docs-release-1.10/dev/table/connect.html#update-modes]
> for exchanging only INSERT messages with the external system. If no key is
> defined by the query, a key is automatically generated by Elasticsearch.
> I want to know ,why the connector of flink with ES just support sink but
> doesn't support source .Which version could add this feature to ?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
