mkhludnev commented on issue #2997:
URL: https://github.com/apache/hop/issues/2997#issuecomment-1583317601
Ahh. thanks.
1. Right, I miss that fat.jar.
2. Terribly sorry for turning this issue into mailing list thread.
3. I tried to use `flink run` got some issue below.
4. I'm on windows+WSL, I already fixed some incompatibilities. But I'm
giving up and switching to Linux now.
5. Anyway here the question:
```
$ bin/flink run --class org.apache.hop.beam.run.MainBeam -D
PROJECT_HOME=/mnt/c/Users/user/hop/config/projects/samples
/mnt/c/Users/user/hop/config/projects/samples/filename.jar
/mnt/c/Users/user/hop/config/projects/samples/beam/pipelines/generate-synthetic-data.hpl
/mnt/c/Users/user/hop/config/projects/samples/filename.json Flink
```
here's the output
```
>>>>>> Initializing Hop
Hop configuration file not found, not serializing:
/mnt/c/Users/user/flink-1.13.0/config/hop-config.json
```
^ that's wrong. I havent' found a way to pass project config there. How to
do that?
Remaining lines seem fine.
```
Argument 1 : Pipeline filename (.hpl) :
/mnt/c/Users/user/hop/config/projects/samples/beam/pipelines/generate-synthetic-data.hpl
Argument 2 : Environment state filename: (.json) :
/mnt/c/Users/user/hop/config/projects/samples/filename.json
Argument 3 : Pipeline run configuration : Flink
>>>>>> Loading pipeline metadata
>>>>>> Building Apache Beam Pipeline...
>>>>>> Pipeline executing starting...
2023/06/08 14:09:05 - General - Created Apache Beam pipeline with name
'generate-synthetic-data'
```
However, then:
```
023/06/08 14:09:10 - General - Executing this pipeline using the Beam
Pipeline Engine with run configuration 'Flink'
2023/06/08 14:09:10 - General - ERROR: Error starting the Beam pipeline
2023/06/08 14:09:10 - General - ERROR:
org.apache.hop.core.exception.HopException:
2023/06/08 14:09:10 - General - Error executing pipeline with runner Flink
2023/06/08 14:09:10 - General - The RemoteEnvironment cannot be instantiated
when running in a pre-defined context (such as Command Line Client, Scala
Shell, or TestEnvironment)
2023/06/08 14:09:10 - General -
2023/06/08 14:09:10 - General - at
org.apache.hop.beam.engines.BeamPipelineEngine.executePipeline(BeamPipelineEngine.java:319)
2023/06/08 14:09:10 - General - at
org.apache.hop.beam.engines.BeamPipelineEngine.startThreads(BeamPipelineEngine.java:345)
2023/06/08 14:09:10 - General - at
org.apache.hop.beam.engines.BeamPipelineEngine.execute(BeamPipelineEngine.java:635)
2023/06/08 14:09:10 - General - at
org.apache.hop.beam.run.MainBeam.main(MainBeam.java:150)
2023/06/08 14:09:10 - General - Caused by:
org.apache.flink.api.common.InvalidProgramException: The RemoteEnvironment
cannot be instantiated when running in a pre-defined context (such as Command
Line Client, Scala Shell, or TestEnvironment)
2023/06/08 14:09:10 - General - at
org.apache.flink.api.java.RemoteEnvironmentConfigUtils.validate(RemoteEnvironmentConfigUtils.java:52)
2023/06/08 14:09:10 - General - at
org.apache.flink.api.java.RemoteEnvironment.validateAndGetEffectiveConfiguration(RemoteEnvironment.java:116)
2023/06/08 14:09:10 - General - at
org.apache.flink.api.java.RemoteEnvironment.<init>(RemoteEnvironment.java:106)
2023/06/08 14:09:10 - General - at
org.apache.flink.api.java.ExecutionEnvironment.createRemoteEnvironment(ExecutionEnvironment.java:1362)
2023/06/08 14:09:10 - General - at
org.apache.beam.runners.flink.FlinkExecutionEnvironments.createBatchExecutionEnvironment(FlinkExecutionEnvironments.java:103)
2023/06/08 14:09:10 - General - at
org.apache.beam.runners.flink.FlinkExecutionEnvironments.createBatchExecutionEnvironment(FlinkExecutionEnvironments.java:67)
2023/06/08 14:09:10 - General - at
org.apache.beam.runners.flink.FlinkPipelineExecutionEnvironment.translate(FlinkPipelineExecutionEnvironment.java:107)
2023/06/08 14:09:10 - General - at
org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:104)
2023/06/08 14:09:10 - General - at
org.apache.hop.beam.engines.BeamPipelineEngine.executePipeline(BeamPipelineEngine.java:309)
2023/06/08 14:09:10 - General - ... 16 more
```
I read it as: in the command line context we can't connect to another Flink
cluster, but that's what Hop's MainBeam attempts to do. How should it work? Did
I miss something?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
