questions about yarn container(taskmanager)memory allocation

2019-05-20 Thread XiangWei Huang
Hi all, Currently i am running my flink application in yarn session mode and using below commnad : *bin/yarn-session.sh -d -s 3 -jm 1024 -tm 4096* when taskmanager complete to started,i found the container launching command is : * bin/java -Xms2765m -Xmx2765m -XX:MaxDirectMemorySize=1331m .

How to Join a dimension table in flink sql

2018-01-15 Thread XiangWei Huang
Hi all, Is it possible to join records read from a kafka stream with one(or more) dimension tables which are saved as mysql table using flink streaming sql.

Re: ExecutionGraph not serializable

2017-11-07 Thread XiangWei Huang
:58 AM, Fabian Hueske wrote: > >> Hi XiangWei, >> >> I don't think this is a public interface, but Till (in CC) might know >> better. >> >> Best, >> Fabian >> >> 2017-11-06 3:27 GMT+01:00 XiangWei Huang : >> >>> Hi Fli

ExecutionGraph not serializable

2017-11-05 Thread XiangWei Huang
Hi Flink users, Flink Jobmanager throw a NotSerializableException when i used JobMasterGateway to get ExecutionGraph of a specific job with message RequestJob(jobID). Blow is the detail of Exception: [ERROR] [akka.remote.EndpointWriter] - Transient association error (association remain

Re: got Warn message - "the expected leader session ID did not equal the received leader session ID " when using LocalFlinkMiniCluster to interpret scala code

2017-09-27 Thread XiangWei Huang
; > Cheers, > Till > > On Wed, Sep 20, 2017 at 5:46 AM, XiangWei Huang <mailto:xw.huang...@gmail.com>> wrote: > Hi Till, > > Thanks for your answer,it worked when i use StandaloneMiniCluster,but > another problem is that i can’t find a way to cancel >

Re: got Warn message - "the expected leader session ID did not equal the received leader session ID " when using LocalFlinkMiniCluster to interpret scala code

2017-09-19 Thread XiangWei Huang
n, I recommend starting a local > standalone cluster and let the code run there. > > Cheers, > Till > > > On Wed, Sep 13, 2017 at 6:21 AM, XiangWei Huang <mailto:xw.huang...@gmail.com>> wrote: > dear all, > > Below is the code i execute: > > im

got Warn message - "the expected leader session ID did not equal the received leader session ID " when using LocalFlinkMiniCluster to interpret scala code

2017-09-12 Thread XiangWei Huang
dear all, Below is the code i execute: import java.io._ import java.net.{URL, URLClassLoader} import java.nio.charset.Charset import java.util.Collections import java.util.concurrent.atomic.AtomicBoolean import com.netease.atom.common.util.logging.Logging import com.netease.atom.interpreter.Code

Re: a lot of connections in state "CLOSE_WAIT"

2017-08-08 Thread XiangWei Huang
It seems so,thanks for your reply Chesnay. > 在 2017年8月8日,22:23,Chesnay Schepler [via Apache Flink User Mailing List > archive.] 写道: > > FLINK-7368 may be the reason for this behavior. > > On 31.07.2017 03:54, XiangWei Huang wrote: >> 1. yes and yes. >>

Re: a lot of connections in state "CLOSE_WAIT"

2017-07-30 Thread XiangWei Huang
> Are the metrics you selected shown correctly? > > Did you modify the "jobmanager.web.refresh-interval" setting? (maybe check > the flink-conf-yaml for the current setting) > > On 26.07.2017 04:57, XiangWei Huang wrote: >> hi, >> >> The browser

Re: a lot of connections in state "CLOSE_WAIT"

2017-07-25 Thread XiangWei Huang
ybe try out if the issue persists with a different one) > > Regards, > Chesnay > > On 25.07.2017 05:20, XiangWei Huang wrote: >> hi, >> >> Sorry for replying so late. >> I have met this issue again and the list is constantly keep growing even if >> i clo

Re: a lot of connections in state "CLOSE_WAIT"

2017-07-24 Thread XiangWei Huang
hi, Sorry for replying so late. I have met this issue again and the list is constantly keep growing even if i close the page ,until the website is been unavailable. This issue appeared each time i add metrics for a job from web ui. by the way ,the version of Flink is 1.3.1 Regards, Xia

Flink will delete all jars uploaded when restart jobmanager

2017-06-13 Thread XiangWei Huang
Hi, When restart flink jobmanager jars which uploaded by user from web ui will be deleted . Is there anyway to avoid this.

Can't kill a job which contains a while loop in the main method before it be submitted

2017-06-13 Thread XiangWei Huang
Hi, I met a problem when use jedis in flink.When using jedis to get a connection to redis if the redis server is not available then jedis will keep trying and never end,the problem is that the job’s status is not set to RUNNING by flink, that means it can’t be killed by flink.The only way to br