nodece opened a new pull request, #22654: URL: https://github.com/apache/pulsar/pull/22654
### Motivation https://github.com/apache/pulsar/pull/14383 introduces the event-time-based index name, it works fine. This behavior has been broken by https://github.com/apache/pulsar/pull/14805. Elasticsearch sink yaml: ```yml configs: elasticSearchUrl: "http://localhost:9200"; indexName: "test-index-%{+yyyy-MM-dd}" createIndexIfNeeded: true compatibilityMode: ELASTICSEARCH ``` When the index name equals to the `test-index-%{+yyyy-MM-dd}`, the es index creation fails: ``` co.elastic.clients.elasticsearch._types.ElasticsearchException: [es/index] failed: [invalid_index_name_exception] Invalid index name [test-index-%{+yyyy-MM-dd}], must be lowercase at co.elastic.clients.transport.ElasticsearchTransportBase.getApiResponse(ElasticsearchTransportBase.java:345) ~[elasticsearch-java-8.12.1.jar:?] at co.elastic.clients.transport.ElasticsearchTransportBase.performRequest(ElasticsearchTransportBase.java:147) ~[elasticsearch-java-8.12.1.jar:?] at co.elastic.clients.elasticsearch.ElasticsearchClient.index(ElasticsearchClient.java:1141) ~[elasticsearch-java-8.12.1.jar:?] at org.apache.pulsar.io.elasticsearch.client.elastic.ElasticSearchJavaRestClient.indexDocument(ElasticSearchJavaRestClient.java:163) ~[gurFS0VR_Lh-tuSoWDLguA/:?] at org.apache.pulsar.io.elasticsearch.ElasticSearchClient.indexDocument(ElasticSearchClient.java:190) [gurFS0VR_Lh-tuSoWDLguA/:?] at org.apache.pulsar.io.elasticsearch.ElasticSearchSink.write(ElasticSearchSink.java:139) [gurFS0VR_Lh-tuSoWDLguA/:?] at org.apache.pulsar.functions.instance.JavaInstanceRunnable.sendOutputMessage(JavaInstanceRunnable.java:469) [pulsar-functions-instance.jar:3.3.0-SNAPSHOT] at org.apache.pulsar.functions.instance.JavaInstanceRunnable.handleResult(JavaInstanceRunnable.java:431) [pulsar-functions-instance.jar:3.3.0-SNAPSHOT] at org.apache.pulsar.functions.instance.JavaInstanceRunnable.run(JavaInstanceRunnable.java:352) [pulsar-functions-instance.jar:3.3.0-SNAPSHOT] at java.base/java.lang.Thread.run(Thread.java:840) [?:?] ``` The expected index name should be equal to `test-1-2024-05-06`(eventTime: `System.currentTimeMillis()`). ### Modifications - Use the index passed in instead of reading the index from config. ### Documentation <!-- DO NOT REMOVE THIS SECTION. CHECK THE PROPER BOX ONLY. --> - [ ] `doc` <!-- Your PR contains doc changes. --> - [ ] `doc-required` <!-- Your PR changes impact docs and you will update later --> - [x] `doc-not-needed` <!-- Your PR changes do not impact docs --> - [ ] `doc-complete` <!-- Docs have been already added --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
