SEPURI-SAI-KRISHNA commented on issue #12988: URL: https://github.com/apache/seatunnel/issues/12988#issuecomment-6570502618
@zhangwen0527 thanks for the config and the log. This is the error #13090 fixes. In 3.0.0, `ElasticsearchVersion` knows only the major versions 2, 5, 6, 7 and 8. The sink looks up the cluster version when the writer starts (`ElasticsearchSinkWriter.initializeSerializer` in your stack trace), so every sink job against 9.x fails this way. The version comes from the cluster, and no sink option skips this check in 3.0.0. I ran your job config against Elasticsearch 9.5.5 with security enabled over HTTP. Only the file path changed, and the file had 20 rows with your schema and 512 floats in each `face_vector`. - With the 3.0.0 version list: the same `ELASTICSEARCH-02 ... version=9` error at `ElasticsearchSinkWriter.java:311`, the same `Restore time 3`, and `t_face_vector` created with 0 documents. - With #13090: the job finished with 20 rows read and 20 written, and `t_face_vector` has 20 documents. Once #13090 is merged, the fix will be in the next release. About `face_vector`: the sink creates the index without a mapping, so Elasticsearch maps the field dynamically. On 9.5.5, a float array of 128 to 4096 values became `dense_vector` (yours: 512 dims, `cosine`), and a kNN query on it worked. A shorter or longer array became `float`. If you need other vector settings, create `t_face_vector` with your mapping before the job. `CREATE_SCHEMA_WHEN_NOT_EXIST` keeps an index that already exists. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
