Hi Thanks for your information. I’m curious to know if a pipeline can be submitted to Flink from the Apache Hop Web, considering that both Flink and Apache Hop Web are running on the same system
Thanks in advance On Fri, 7 Mar 2025 at 23:23, Hans Van Akelyen <[email protected]> wrote: > Hi, > > If the cluster is on the same machine it should be possible. Remote > unfortunately does not work. > > We might have to make that a bit more clear in the docs. > > Cheers, > Hans > > On Fri, 7 Mar 2025 at 18:38, arjun s <[email protected]> wrote: > >> Hi team, >> >> Just to clarify, we can submit and run the pipeline created in Hop to an >> external Flink cluster, correct? >> >> Looking forward to your confirmation. >> >> Thanks in advance >> >> On Fri, 7 Mar 2025 at 04:18, [email protected] <[email protected]> wrote: >> >>> BTW, I know that's an unhelpful error but that's Java complaining rather >>> than hop >>> >>> On Fri, 7 Mar 2025, 9:45 am [email protected], <[email protected]> >>> wrote: >>> >>>> Hello, >>>> >>>> Yes, this is a common error when one of the two following this happen. >>>> >>>> 1. you try to merge two streams where one of the columns have >>>> incompatible data types. E.g. a float and an integer. >>>> >>>> (Going from memory as not near apc) >>>> Start from the end and using the input fields menu validate all fields. >>>> >>>> 2. You are dividing over zero, multiplying by a null or similar. >>>> >>>> On Fri, 7 Mar 2025, 12:55 am arjun s, <[email protected]> wrote: >>>> >>>>> Hi team, >>>>> >>>>> I am encountering an issue while executing the >>>>> generate-synthetic-data.hpl sample pipeline using Apache Flink. Here’s >>>>> what >>>>> I’ve configured: >>>>> >>>>> - Set the Engine Type to Beam Flink pipeline engine >>>>> - Configured the Flink master >>>>> - Mentioned the hop-fat-jar.jar >>>>> >>>>> However, when running the pipeline, I am facing the exception >>>>> mentioned below. I am using Apache Hop version 2.12 on Windows and Apache >>>>> Flink version 1.17.0, which is running on Linux. From the Flink UI, I can >>>>> see that the job is going into a failed state. >>>>> >>>>> org.apache.flink.runtime.client.JobInitializationException: Could not >>>>> start the JobMaster. >>>>> at >>>>> org.apache.flink.runtime.jobmaster.DefaultJobMasterServiceProcess.lambda$new$0(DefaultJobMasterServiceProcess.java:97) >>>>> at >>>>> java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760) >>>>> at >>>>> java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736) >>>>> at >>>>> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) >>>>> at >>>>> java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1595) >>>>> at >>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >>>>> at >>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >>>>> at java.lang.Thread.run(Thread.java:745) >>>>> Caused by: java.util.concurrent.CompletionException: >>>>> java.lang.NullPointerException >>>>> at >>>>> java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:273) >>>>> at >>>>> java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:280) >>>>> at >>>>> java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1592) >>>>> ... 3 more >>>>> Caused by: java.lang.NullPointerException >>>>> at >>>>> org.apache.flink.api.common.ExecutionConfig.getNumberOfExecutionRetries(ExecutionConfig.java:486) >>>>> at >>>>> org.apache.flink.api.common.ExecutionConfig.getRestartStrategy(ExecutionConfig.java:459) >>>>> at >>>>> org.apache.flink.runtime.scheduler.DefaultSchedulerFactory.createInstance(DefaultSchedulerFactory.java:99) >>>>> at >>>>> org.apache.flink.runtime.jobmaster.DefaultSlotPoolServiceSchedulerFactory.createScheduler(DefaultSlotPoolServiceSchedulerFactory.java:119) >>>>> at >>>>> org.apache.flink.runtime.jobmaster.JobMaster.createScheduler(JobMaster.java:371) >>>>> at org.apache.flink.runtime.jobmaster.JobMaster.(JobMaster.java:348) >>>>> at >>>>> org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.internalCreateJobMasterService(DefaultJobMasterServiceFactory.java:123) >>>>> at >>>>> org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.lambda$createJobMasterService$0(DefaultJobMasterServiceFactory.java:95) >>>>> at >>>>> org.apache.flink.util.function.FunctionUtils.lambda$uncheckedSupplier$4(FunctionUtils.java:112) >>>>> at >>>>> java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590) >>>>> >>>>> Has anyone encountered a similar issue or could provide some insights >>>>> on how to resolve this? >>>>> >>>>> Thanks in advance for your help >>>>> >>>>
