6:25 Uhr schrieb Siew Wai Yow
mailto:wai_...@hotmail.com>>:
Hi guys,
May i know flink support ipv6?
Thanks
Yow
Hi guys,
May i know flink support ipv6?
Thanks
Yow
Hi guys,
I have question regarding to the title that need your expertise,
1. I need to build a SFTP SourceFunction, may I know if hadoop
SFTPFileSystem suitable?
2. I need to build a SFTP SinkFunction as well, may I know if per-defined
HDFS rolling file sink accept SFTP connection since
Hi guys,
Anyone can share experience on sftp source? Should i use hadoop sftpfilesystem
or i can simply use any sftp java library in a user-defined source?
Thanks.
Regards,
Yow
Sent: Sunday, January 13, 2019 9:39 AM
To: Siew Wai Yow
Cc: Jamie Grier; user
Subject: Re: Reply: Re: Reply: Re: What happen to state in Flink Task Manager
when crash?
Hi, Yow
I think there is another restart strategy in flink: region failover[1], but I
could not find the documentation, maybe
the same in Flink 1.7? Restart strategy is
for job though not for TM failure.
Thanks!
Hi, Siew Wai Yow
Yes, David is correct, the TM must be recovered, the number of TMs before and
after the crash must be the same.
In my last reply, I want to say that the states may not on the same TM after
vered, and state updates since the last checkpoint
will be lost. Of course that lost state should be recreated as the job rewinds
and resumes."
Hi, Siew Wai Yow
When the job is running, the states are stored in the local RocksDB, Flink will
copy all the needed states to checkpointPath when doing a
nly recover when TM2 being recover?
Thanks.
From: Jamie Grier
Sent: Saturday, January 12, 2019 2:26 AM
To: Siew Wai Yow
Cc: user@flink.apache.org
Subject: Re: What happen to state in Flink Task Manager when crash?
Flink is designed such that local state is backed
Hi,
When one of the task manager is killed, the whole cluster die, is this
something expected? We are using Flink 1.4. Thank you.
Regards,
Yow
Hi Yun Tang,
Thanks to your reply, this is exactly the answer we need.
also thanks @CongXian's reply.
-Siew Wai
From: Tang Cloud
Sent: Monday, July 9, 2018 11:05 AM
To: Siew Wai Yow; user@flink.apache.org
Subject: 答复: flink rocksdb where to configure mount
Hi,
We configure rocksdb as statebackend and checkpoint dir persists to hdfs. When
the job is run, rocksdb automatically mount to tmpfs /tmp, which consume memory.
RocksDBStateBackend rocksdb = new RocksDBStateBackend(new
FsStateBackend(hdfs://), true);
Hi all,
Is there any use case in virtualized environment? Is it a good choice? Please
share your thought. Thank you.
-Yow
a bug. Flink use
Checkpoint path's FileSystem to create the output stream for the Savepoint, but
in your case the checkpoint & savepoint are not using the same file system. A
workaround is to use the same file system for both checkpoint & savepoint.
Best, Sihua
On 06/21/2018 14:07
t requires this, other request allow optional fields to simply
be ommitted...
On 20.06.2018 06:12, Siew Wai Yow wrote:
Hi all,
Seems pass in target-directory is a must now for checkpoints REST API, and the
status will not response with save point directory anymore. I can pass in but
the i
"cancel-job" : {
"type" : "boolean"
}
}
3. GET to /jobs/:jobid/savepoints/:triggerid
On 19.06.2018 17:40, Esteban Serrano wrote:
For #1, you need to use a PATCH request to "/jobs/:jobid"
On Tue, Jun 19, 2018 at 11:35 AM Siew Wai Yow
mailto:wai_.
cancel-job" : {
"type" : "boolean"
}
}
3. GET to /jobs/:jobid/savepoints/:triggerid
On 19.06.2018 17:40, Esteban Serrano wrote:
For #1, you need to use a PATCH request to "/jobs/:jobid"
On Tue, Jun 19, 2018 at 11:35 AM Siew Wai Yow
mailto:wai_...@hotmai
Hi,
Regarding to Flink 1.5.0 REST API breaking change,
* The REST API to cancel a job was changed.
* The REST API to cancel a job with savepoint was changed.
I have few dump questions,
1. Any replacement for cancellation ONLY without save-point? Only found
source to only talk
to mapper 3 and 4.
Cheers,
Till
From: Fabian Hueske
Sent: Tuesday, June 19, 2018 3:55 PM
To: Siew Wai Yow
Cc: Ovidiu-Cristian MARCU; Jörn Franke; user@flink.apache.org;
trohrm...@apache.org
Subject: Re: Flink application does not scale
to replace all commas with some other delimiter.
On 19.06.2018 04:15, Siew Wai Yow wrote:
Hi,
I get the following error when upgrade flink from 1.3.2 to 1.5.0 when using
REST api to upload and run a jar.
{"errors":["Expected only one value [--KAFKA_IN
PREBLN_O@192.168.56
Hi,
I get the following error when upgrade flink from 1.3.2 to 1.5.0 when using
REST api to upload and run a jar.
{"errors":["Expected only one value [--KAFKA_IN PREBLN_O@192.168.56.120:9092,
192.168.56.121:9092, 192.168.56.122:9092/BHARTI_FL_PREBLN_O_124 --KAFKA_OUT
o you mind to share your
thoughts?
Thank you guys!
From: Ovidiu-Cristian MARCU
Sent: Monday, June 18, 2018 6:28 PM
To: Fabian Hueske
Cc: Siew Wai Yow; Jörn Franke; user@flink.apache.org
Subject: Re: Flink application does not scale as expected, please help!
Hi a
*additional info in bold.
From: Siew Wai Yow
Sent: Monday, June 18, 2018 3:57 PM
To: Fabian Hueske
Cc: Jörn Franke; user@flink.apache.org
Subject: Re: Flink application does not scale as expected, please help!
Hi Fabian,
We are using Flink 1.5.0. Any
ecause all happens in same TM. When scale to 32 the
performance drop, not even in par with case of parallelism 16. Is this
something expected? Thank you.
Regards,
Yow
From: Fabian Hueske
Sent: Monday, June 18, 2018 3:47 PM
To: Siew Wai Yow
Cc: Jörn Fr
Hi Jorn, Please find the source @https://github.com/swyow/flink_sample_git
Thank you!
From: Jörn Franke
Sent: Saturday, June 16, 2018 6:03 PM
To: Siew Wai Yow
Cc: user@flink.apache.org
Subject: Re: Flink application does not scale as expected, please help!
Can
the less I get?
From: Siew Wai Yow
Sent: Saturday, June 16, 2018 5:09 PM
To: Jörn Franke
Cc: user@flink.apache.org
Subject: Re: Flink application does not scale as expected, please help!
Hi Jorn, the input data is 1kb per record, in production it will have 10
billions
: to send result to statsd
* psrc: source parallelism
* pJ2R: parallelism of map operator(JsonRecTranslator)
* pAggr: parallelism of process+timer operator(AggregationDuration)
Thank you!
Yow
From: Jörn Franke
Sent: Saturday, June 16, 2018 4:46 PM
To:
26 matches
Mail list logo
