stanb commented on PR #38: URL: https://github.com/apache/flink-connector-opensearch/pull/38#issuecomment-2124685703
> It is in voting stage you can find links to artifacts (including jars for os v2) in corresponding email thread https://lists.apache.org/thread/by44cdpfv6p9394vwxhh1vzh3rfskzms I mean the built `jar` file. I tried to build it myself and use with my jobs ``` FROM maven AS build-opensearch-connector RUN git clone https://github.com/apache/flink-connector-opensearch.git WORKDIR /flink-connector-opensearch RUN git checkout v2.0.0-rc1 RUN mvn clean package -U -B --no-transfer-progress -Dflink.version=1.18.1 -DskipTests -Dhttp.keepAlive=false -Dmaven.wagon.http.pool=false -Dmaven.wagon.httpconnectionManager.ttlSeconds=120 FROM flink:1.18.1 # # Add additional connectors not included in the default Flink image # ## Opensearch connector ##ADD https://repo.maven.apache.org/maven2/org/apache/flink/flink-sql-connector-opensearch/1.1.0-1.18/flink-sql-connector-opensearch-1.1.0-1.18.jar /opt/flink/lib/ COPY --from=build-opensearch-connector /flink-connector-opensearch/flink-sql-connector-opensearch2/target/flink-sql-connector-opensearch2-2.0.0.jar /opt/flink/lib/ ## JDBC connector ADD https://repo.maven.apache.org/maven2/org/apache/flink/flink-connector-jdbc/3.1.2-1.18/flink-connector-jdbc-3.1.2-1.18.jar /opt/flink/lib/ ## Postgresql JDBC Driver ADD https://jdbc.postgresql.org/download/postgresql-42.7.1.jar /opt/flink/lib/ RUN chown flink:flink /opt/flink/lib/* ``` Then I submitting my sql job that copies data from postgres table to opensearch index ``` bin/sql-client.sh -f sql/flink.sql ``` and I get following error: ``` 2024-05-22 15:25:47 org.apache.flink.runtime.JobException: Recovery is suppressed by NoRestartBackoffTimeStrategy at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.handleFailure(ExecutionFailureHandler.java:176) at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.getFailureHandlingResult(ExecutionFailureHandler.java:107) at org.apache.flink.runtime.scheduler.DefaultScheduler.recordTaskFailure(DefaultScheduler.java:285) at org.apache.flink.runtime.scheduler.DefaultScheduler.handleTaskFailure(DefaultScheduler.java:276) at org.apache.flink.runtime.scheduler.DefaultScheduler.onTaskFailed(DefaultScheduler.java:269) at org.apache.flink.runtime.scheduler.SchedulerBase.onTaskExecutionStateUpdate(SchedulerBase.java:764) at org.apache.flink.runtime.scheduler.SchedulerBase.updateTaskExecutionState(SchedulerBase.java:741) at org.apache.flink.runtime.scheduler.SchedulerNG.updateTaskExecutionState(SchedulerNG.java:83) at org.apache.flink.runtime.jobmaster.JobMaster.updateTaskExecutionState(JobMaster.java:488) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.apache.flink.runtime.rpc.pekko.PekkoRpcActor.lambda$handleRpcInvocation$1(PekkoRpcActor.java:309) at org.apache.flink.runtime.concurrent.ClassLoadingUtils.runWithContextClassLoader(ClassLoadingUtils.java:83) at org.apache.flink.runtime.rpc.pekko.PekkoRpcActor.handleRpcInvocation(PekkoRpcActor.java:307) at org.apache.flink.runtime.rpc.pekko.PekkoRpcActor.handleRpcMessage(PekkoRpcActor.java:222) at org.apache.flink.runtime.rpc.pekko.FencedPekkoRpcActor.handleRpcMessage(FencedPekkoRpcActor.java:85) at org.apache.flink.runtime.rpc.pekko.PekkoRpcActor.handleMessage(PekkoRpcActor.java:168) at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.PartialFunction.applyOrElse(PartialFunction.scala:127) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:126) at org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:175) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:176) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:176) at org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.apache.pekko.actor.AbstractActor.aroundReceive(AbstractActor.scala:229) at org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(Unknown Source) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(Unknown Source) at java.base/java.util.concurrent.ForkJoinPool.scan(Unknown Source) at java.base/java.util.concurrent.ForkJoinPool.runWorker(Unknown Source) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(Unknown Source) Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.apache.flink.opensearch2.shaded.org.opensearch.node.Node at org.apache.flink.opensearch2.shaded.org.opensearch.common.util.concurrent.OpenSearchExecutors.threadName(OpenSearchExecutors.java:361) at org.apache.flink.opensearch2.shaded.org.opensearch.common.util.concurrent.OpenSearchExecutors.daemonThreadFactory(OpenSearchExecutors.java:375) at org.apache.flink.opensearch2.shaded.org.opensearch.threadpool.Scheduler.initScheduler(Scheduler.java:73) at org.apache.flink.opensearch2.shaded.org.opensearch.action.bulk.BulkProcessor.builder(BulkProcessor.java:263) at org.apache.flink.connector.opensearch.sink.Opensearch2Writer.createBulkProcessor(Opensearch2Writer.java:166) at org.apache.flink.connector.opensearch.sink.Opensearch2Writer.<init>(Opensearch2Writer.java:113) at org.apache.flink.connector.opensearch.sink.Opensearch2Sink.createWriter(Opensearch2Sink.java:95) at org.apache.flink.streaming.runtime.operators.sink.StatelessSinkWriterStateHandler.createWriter(StatelessSinkWriterStateHandler.java:39) at org.apache.flink.streaming.runtime.operators.sink.SinkWriterOperator.initializeState(SinkWriterOperator.java:149) at org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.initializeOperatorState(StreamOperatorStateHandler.java:122) at org.apache.flink.streaming.api.operators.AbstractStreamOperator.initializeState(AbstractStreamOperator.java:274) at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.initializeStateAndOpenOperators(RegularOperatorChain.java:106) at org.apache.flink.streaming.runtime.tasks.StreamTask.restoreGates(StreamTask.java:753) at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.call(StreamTaskActionExecutor.java:55) at org.apache.flink.streaming.runtime.tasks.StreamTask.restoreInternal(StreamTask.java:728) at org.apache.flink.streaming.runtime.tasks.StreamTask.restore(StreamTask.java:693) at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:953) at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:922) at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:746) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:562) at java.base/java.lang.Thread.run(Unknown Source) ``` As I understand, something was not build well and `org.apache.flink.opensearch2.shaded.org.opensearch.node.Node` cant be instantiated. I am not java developer and not familiar with maven. Tried to follow github workflow pipeline to understand if i missing any maven parameter. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
