Re: Understanding job flow

2019-08-16 Thread Victor Wong
, which cannot be serialized. Best, Victor From: Vishwas Siravara Date: Friday, August 16, 2019 at 11:48 PM To: Steven Nelson , user Subject: Re: Understanding job flow I did not find this to be true. Here is my code snippet. object DruidStreamJob extends Job with SinkFn { private[flink] val

Re: Understanding job flow

2019-08-16 Thread Vishwas Siravara
I did not find this to be true. Here is my code snippet. object DruidStreamJob extends Job with SinkFn { private[flink] val druidConfig = DruidConfig.current private[flink] val decryptMap = ExecutionEnv.loadDecryptionDictionary // //TODO: Add this to sbt jvm, this should be set in sbt

Re: Understanding job flow

2019-08-15 Thread Steven Nelson
The encryptor will be serialized and sent with the rest of your Job Graph when the job is submitted. If it’s not serializable you get an error. Sent from my iPhone > On Aug 15, 2019, at 11:00 AM, Vishwas Siravara wrote: > > Hi guys, > I have a map job where I want to encrypt certain keys . I

Understanding job flow

2019-08-15 Thread Vishwas Siravara
Hi guys, I have a map job where I want to encrypt certain keys . I initialize the encryptor in the main method and apply it in the map function. How is this encryptor shared when I have my job running on multiple task managers with parallelism > 1 ? Thanks, Vishwas