Hi,
I think the documentation is correct. Once the job is stopped with
savepoint, any of its "regular" checkpoints are discarded, and as a
result any shared state gets unreferenced and is also discarded.
Savepoints currently do not have shared state.
Furthermore, the new job should have a new ID
Hi Diwakar,
I'm not sure I fully understand your question.
If event handling in one window depends on some other windows than
TriggerContext.getPartitionedState can not be used. Triggers don't have
access to the global state (only to key-window scoped state).
If that's what you want then please
Hi,
AFAIK there is no direct support for Julia in Flink currently.
However, you may try to call Python from Julia using either Statefun Python
SDK [1] or PyFlink [2]; or implement a remote Statefun module [3].
[1]
Hi,
Deletion of messages in Kafka shouldn't affect Flink state in general.
Probably, some operator in your pipeline is re-reading the topic
and overwrites the state, dropping what was deleted by Kafka.
Could you share the code?
Regards,
Roman
On Tue, Feb 23, 2021 at 7:12 AM bat man wrote:
>
Hi,
Probably another solution would be to register a timer
(using KeyedProcessFunction) once we see an element after keyBy. The timer
will fire in windowIntervalMs. Upon firing, it will emit a dummy element
which will be ignored (or subtracted) in the end.
Upon receiving each new element, the
Hi,
Could you provide the exception stack trace?
Regards,
Roman
On Mon, Feb 8, 2021 at 3:46 PM joris.vanagtmaal <
joris.vanagtm...@wartsila.com> wrote:
> I'm trying to read data from my eventhub in Azure, but i end up with the
> Flink error message 'findAndCreateTableSource failed'
>
> using
Hi,
I'm pulling Yun Tang who is familiar with StateBackends and RocksDB in
particular.
>From what I see, the 2nd snapshot (sp2) is built using the same set of
states obtained from the starting savepoint/checkpoint (sp1) to write its
metadata. This metadata includes serializers snapshots,
Hi ChangZhuo,
Queryable state is exposed on the same address as the TM RPC. You can
change this address by modifying taskmanager.host [1].
However, I'm not sure if setting it to 127.0.0.1 or localhost will not
break connectivity with the other components.
[1]
Hi,
I think Yun Tang is right, HeapStateBackend doesn't (de)serialize the value
on update.
As for "value()", it may (de)serialize it and return a copy if there is an
ongoing async snapshot in progress (to protect from modifications). This
shouldn't happen often though.
Regards,
Roman
On Mon,
Hi,
AFAIK this should be supported in 1.12 via FLINK-19568 [1]
I'm pulling in Timo and Jark who might know better.
https://issues.apache.org/jira/browse/FLINK-19857
Regards,
Roman
On Mon, Feb 8, 2021 at 9:14 AM meneldor wrote:
> Any help please? Is there a way to use the "Last row" from a
Hi,
The open issue you mentioned (FLINK-21053) is about preventing potential
issues in the future.
The issue you are experiencing is most likely FLINK-20992 as Yang Wang said.
So upgrading to 1.12.2 should solve the problem.
Regards,
Roman
On Mon, Feb 8, 2021 at 9:05 AM Lei Wang wrote:
> I
Hi Rex,
Unfortunately not: the same timeout value is used both for savepoints and
checkpoints.
Regards,
Roman
On Sat, Jan 16, 2021 at 9:42 AM Rex Fenley wrote:
> Hello,
>
> I'm wondering if there's a way to set different timeouts for savepoints
> and checkpoints. Our savepoints can take a
I think you're right Till, this is the problem.
In fact, I opened a duplicating jira ticket in parallel :)
I hope we can fix it in the next version of 1.12.
Regards,
Roman
On Fri, Jan 15, 2021 at 2:09 PM Till Rohrmann wrote:
> Thanks for reporting and analyzing this issue Kelly. I think you
Hi Yun,
> b) With unaligned checkpoint enabled, the slower cases might happen if
the downstream task processes very slowly.
I think UC will be the common case with multiple sources each with DoP > 1.
IIUC, waiting for EoP will be needed on each subtask each time one of it's
source subtask
is called
> with
> EndOfPartition) and then taking snapshot for the input channels, as the
> normal unaligned checkpoints does for the InputChannel side. Then
> we would be able to ensure the finished tasks always have an empty state.
>
> I'll also optimize the FLIP to make it mo
Thanks for starting this discussion (and sorry for probably duplicated
questions, I couldn't find them answered in FLIP or this thread).
1. Option 1 is said to be not preferable because it wastes resources and
adds complexity (new event).
However, the resources would be wasted for a relatively
Hello,
Unfortunately, this driver is not currently supported by the Table API [1].
You can implement a dialect for it [2] and construct JdbcTableSource [3]
manually.
Alternatively, you can switch to the DataStream API and use JdbcInputFormat
[4] which doesn't require dialect.
I'm also pulling
Hi Jiazhi,
Could you share table definitions and both queries?
Regards,
Roman
On Fri, Dec 18, 2020 at 4:39 AM ゞ野蠻遊戲χ wrote:
> Hi all
> When I use SQL with UDTF, when I call the tableEnv.sqlQuery ()
> method, I throw the following error: Rowtime attributes must not be in the
> input rows
Hi Jiazhi,
Could you share table definitions and both queries?
Regards,
Roman
On Fri, Dec 18, 2020 at 4:39 AM ゞ野蠻遊戲χ wrote:
> Hi all
> When I use SQL with UDTF, when I call the tableEnv.sqlQuery ()
> method, I throw the following error: Rowtime attributes must not be in the
> input rows
The stacktrace looks similar to
https://issues.apache.org/jira/browse/HIVE-14483
However, it should be fixed in the version used in your setup.
Jingsong Li can you take a look at this error?
Regards,
Roman
On Thu, Dec 17, 2020 at 3:57 PM house－张浩 <312421...@qq.com> wrote:
> when i use pyflink
I believe it was solved in 1.11 by FLINK-15911 [1]
I tried setting taskmanager.rpc.port to 1 for 1.12 and got
tcp6 0 0 :::1:::*LISTEN
13768/java
[1]
https://issues.apache.org/jira/browse/FLINK-15911
Regards,
Roman
On Tue, Dec 8, 2020
Hi Kien,
I am pulling in Yun who might know better.
Regards,
Roman
On Sun, Dec 6, 2020 at 3:52 AM Truong Duc Kien
wrote:
> Hi all,
>
> We are thinking about enabling RocksDB metrics to better monitor our
> pipeline. However, since they will have performance impact, we will have to
> be
Hi Marco,
You can find the list of the supported time units in TimeUtils javadoc [1]:
DAYS： "d", "day"
HOURS： "h", "hour"
MINUTES： "min", "minute"
SECONDS： "s", "sec", "second"
MILLISECONDS： "ms", "milli", "millisecond"
MICROSECONDS： "µs", "micro", "microsecond"
NANOSECONDS： "ns", "nano",
Hi Marco,
Yes, if TTL is not configured then the state will never expire (will stay
forever until deleted explicitly).
Regards,
Roman
On Tue, Dec 8, 2020 at 5:09 PM Marco Villalobos
wrote:
> After reading
>
>
>
Thanks, Randal,
Yes, I think the only way is to partition the stream the same way as
kinesis does (as I wrote before).
Regards,
Roman
On Tue, Dec 8, 2020 at 1:38 PM Randal Pitt wrote:
> Hi Roman,
>
> We're using a custom watermarker that uses a histogram to calculate a "best
> fit" event
Hi Randal,
Can you share the code for the 1st approach
(FlinkKinesisConsumer.setPeriodicWatermarkAssigner))?
I think the 2nd approach (flatMap) can be improved by partitioning the
stream the same way kinesis does (i.e. same partition key).
Regards,
Roman
On Mon, Dec 7, 2020 at 2:44 PM Randal
Hi,
Please verify that:
1. kafka-connector is indeed in the fat jar (e.g. by "jar vtf
your-program.jar | grep KafkaDynamicTableFactory")
2. kafka-connector version matches the version of Flink distribution on EMR.
Regards,
Roman
On Tue, Nov 17, 2020 at 6:47 AM Fanbin Bu wrote:
> Hi,
>
> I
Hi,
I think Robert is right, state handles are deleted first, and then the
directory is deleted non-recursively.
If any exception occurs while removing the files, it will be combined with
the other exception (as suppressed).
So probably Flink failed to delete some files and then directory removal
Hello,
Do both of the types you use have the same nullability?
For a primitive int, the documentation you referred to says: "Output only
if type is not nullable".
Regards,
Roman
On Tue, Nov 17, 2020 at 7:49 AM Rex Fenley wrote:
> Hello,
>
> I'm using the Table API and I have a column which
Hello,
Can you share the full program?
getExecutionPlan call is probably misplaced.
Regards,
Roman
On Tue, Nov 17, 2020 at 8:26 AM Rex Fenley wrote:
> Hello,
>
> I have the following code attempting to print the execution plan for my
> job locally. The job runs fine and Flink UI displays so
gt; In the screenshot appeared that was never assigned taskmanager for the job
> operator.
> I've put the 3 logs( from each node) here.
>
> Thanks and Best Regards.
> Alex
>
>
> On Mon, Oct 19, 2020 at 5:47 PM Khachatryan Roman <
> khachatryan.ro...@gmail.com&g
gt;>
>> Regards,
>> Juha
>>
>> El mié., 21 oct. 2020 a las 10:04, Juha Mynttinen (<
>> juha.myntti...@gmail.com>) escribió:
>>
>>> Hi,
>>>
>>> You're right, I thought about this also after writing the last comment -
>>> fo
m happy to propose a fix if someone is able to assign the ticket to me.
>
> Best,
> Austin
>
> On Mon, Oct 19, 2020 at 6:56 AM Khachatryan Roman <
> khachatryan.ro...@gmail.com> wrote:
>
>> Hey Austin,
>>
>> I think you are right. The problematic row c
VM would allocate
> right away 2048mb (when it starts). If there's not enough memory, the tests
> would fail immediately (JVM couldn't start). The tests would probably fail
> anyways (my case) - better fail fast..
>
> Regards,
> Juha
>
>
>
>
>
>
>
>
> El m
Hello Averell,
I don't think ZK data is stored on a master node. And Flink JM data is
stored usually on DFS - according to "high-availability.storageDir" [1]
In either case, for Flink to be HA, Yarn should also be HA. And I think
this is not the case with a single master node. Please consider
essentially leaving less CPU available to the actual test code. By using
> htop I can also see the garbage collection related threads eating CPU. This
> doesn't seem right. I think it'd make sense to run the tests with less
> parallelism to better utilize the CPUs. Having greatly more threa
To print the result you can create a POJO class matching your projected
fields and use it on line 38 instead of String.class.
For example:
Table result1 = tableA.select("content, content.hashCode() as h1,
hashCode(content) as h2");
tableEnv.toDataSet(result1, HashCodeSelect.class).print();
Hi,
Can you explain what "EMR cluster crashed" means in the 2nd scenario?
Can you also share:
- yarn.application-attempts in Flink
- yarn.resourcemanager.am.max-attempts in Yarn
- number of EMR master nodes (1 or 3)
- EMR version?
Regards,
Roman
On Mon, Oct 19, 2020 at 8:22 AM Averell wrote:
r
>
> On Wed, Oct 14, 2020 at 6:13 PM Khachatryan Roman <
> khachatryan.ro...@gmail.com> wrote:
>
>> Hi,
>> Thanks for sharing the details and sorry for the late reply.
>> You can check the number of free slots in the task manager in the web UI (
>>
Hi,
AFAIK, the features discussed in the threads you mentioned are not yet
implemented. So there is no way to avoid Job restarts in case of ZK rolling
restarts.
I'm pulling in Till as he might know better.
Regards,
Roman
On Fri, Oct 16, 2020 at 7:45 PM Kenzyme wrote:
> Hi,
>
> Related to
>
Hey,
One reason could be that a resource-intensive test was killed by oom
killer. You can inspect /var/log/kern.log for the related messages in your
VM.
Regards,
Roman
On Mon, Oct 19, 2020 at 5:57 PM Juha Mynttinen
wrote:
>
> Hey,
>
> I'm trying to build Flink and failing. I'm running Ubuntu
Hey Austin,
I think you are right. The problematic row contains an odd number of
delimiters in which case skipFields will return -1, which in turn leads to
an exception.
I opened a bug ticket https://issues.apache.org/jira/browse/FLINK-19711 to
fix it.
Regards,
Roman
On Fri, Oct 16, 2020 at
Hi Yuval,
I'm also wondering why do you have such a big metadata file.
Probably, you could reduce it by decreasing
"state.backend.fs.memory-threshold" (if you didn't do so already) [1].
[1]
I assume that before submitting a job you started a cluster with default
settings with ./bin/start-cluster.sh.
Did you submit any other jobs?
Can you share the logs from log folder?
Regards,
Roman
On Wed, Oct 7, 2020 at 11:03 PM Alexander Semeshchenko
wrote:
>
>
erruption is being ignored in the Hadoop / S3
>>> layer. I was looking at the defaults and (if I understood correctly) the
>>> client will retry 20 times. Which would explain why it never gets
>>> cancelled...
>>>
>>> On Thu, Oct 8, 2020 at 1:27 AM Khachatryan Roman
Hi Felipe,
Your source is not parallel so it doesn't make sense to make local group
operator parallel.
If the source implemented ParallelSourceFunction, subsequent operators
would be parallelized too.
Regards,
Roman
On Thu, Oct 8, 2020 at 5:00 PM Felipe Gutierrez <
Hi Dan Diephouse,
>From the logs you provided indeed it looks like 1 causes 2 => 3 => 4, where
2 is a bug.
It's unclear though where the interruption is ignored (Flink/Hadoop FS/S3
client).
What version of Flink are you using?
Regards,
Roman
On Wed, Oct 7, 2020 at 11:16 PM Dan Diephouse
Hi,
1. Yes, StreamingExecutionEnvironment.readFile can be used for files on HDFS
2. I think this is a valid concern. Besides that, there are plans to
deprecate DataSet API [1]
4. Yes, the approach looks good
I'm pulling in Aljoscha for your 3rd question (and probably some
clarifications on
Please use user mailing list for questions related to the use of Flink.
See [1] for the other lists.
[1] https://flink.apache.org/community.html#mailing-lists
Regards,
Roman
On Sun, Sep 27, 2020 at 8:29 AM Dan Hill wrote:
> I'm looking to hire Flink developers (full time or contractors) to
Hi,
Apache Beam [1] and Zeppelin [2] can use Flink.
I don't think there are Flink setups used by open-source projects.
[1] https://beam.apache.org/documentation/runners/flink/
[2] https://zeppelin.apache.org/docs/0.9.0-SNAPSHOT/interpreter/flink.html
Regards,
Roman
On Fri, Sep 25, 2020 at
The closest thing is the backpressure status which you mentioned.
>From there, you can troubleshoot specific subtasks by inspecting their
metrics.
There is no health summary in Flink at the moment.
Regards,
Roman
On Fri, Sep 25, 2020 at 5:35 AM Dan Hill wrote:
> My job has very slow
hanks,
> Alexey
> --
> *From:* Alexey Trenikhun
> *Sent:* Friday, August 28, 2020 11:31 AM
> *To:* Khachatryan Roman
> *Cc:* Flink User Mail List
> *Subject:* Re: FileSystemHaServices and BlobStore
>
> Motivation is to have k8s HA setup without extra component - Zookeeper,
&
Hello Alexey,
I think you need FileSystemBlobStore as you are implementing HA Services,
and BLOBs should be highly available too.
However, I'm a bit concerned about the direction in general: it essentially
means re-implementing ZK functionality on top of FS.
What are the motivation and the use
Hi Anuj Jain,
You need to provide the type parameter when calling
WatermarkStrategy.forBoundedOutOfOrderness like this:
bookingFlowConsumer.assignTimestampsAndWatermarks(WatermarkStrategy.forBoundedOutOfOrderness(Duration.ofMinutes(15))
Regards,
Roman
On Fri, Aug 28, 2020 at 6:49 AM aj
x on roadmap? Thanks!
>
> Best
> Lu
>
> On Tue, Aug 4, 2020 at 1:24 PM Khachatryan Roman <
> khachatryan.ro...@gmail.com> wrote:
>
>> Hi Lu,
>>
>> Yes, this error indicates data loss (unless there were no records in the
>> transactions).
>>
Hi Lu,
Yes, this error indicates data loss (unless there were no records in the
transactions).
Regards,
Roman
On Mon, Aug 3, 2020 at 9:14 PM Lu Niu wrote:
> Hi,
>
> We are using end to end exact-once flink + kafka and encountered belowing
> exception which usually came after checkpoint
Hi Vijay,
The javadoc for
org.apache.flink.streaming.util.serialization.SimpleStringSchema says
you should Use org.apache.flink.api.common.serialization.SimpleStringSchema
instead.
Regards,
Roman
On Mon, Aug 3, 2020 at 5:31 PM Vijayendra Yadav
wrote:
> Hi Team,
>
>
>
Hi Suchithra,
Yes, you need to pass these parameters to standalone-job.sh in Kubernetes
job definition.
I'm pulling in Patrick as he might know this subject better.
Regards,
Roman
On Mon, Aug 3, 2020 at 12:24 PM V N, Suchithra (Nokia - IN/Bangalore) <
suchithra@nokia.com> wrote:
> Hello,
ted too, is this possible in some way?
>
> Thanks
>
> On Mon, Jul 13, 2020 at 8:47 PM Khachatryan Roman <
> khachatryan.ro...@gmail.com> wrote:
>
>> Hi Antonio,
>>
>> Please take a look at broadcast variables:
>> https://ci.apache.org/projec
Hi Antonio,
Please take a look at broadcast variables:
https://ci.apache.org/projects/flink/flink-docs-stable/dev/batch/#broadcast-variables
Regards,
Roman
On Mon, Jul 13, 2020 at 3:49 PM Antonio Martínez Carratalá <
amarti...@alto-analytics.com> wrote:
> Hello
>
> I'm trying to implement the
Hi Manas,
Do you have the same error if you replace
.group_by("five_sec_window, monitorId") \
with
.group_by("five_sec_window") \
?
Regards,
Roman
On Mon, Jul 13, 2020 at 11:16 AM Manas Kale wrote:
> Hi,
> I have the following piece of code (for pyFlink v1.11) :
>
>
Hi Murali Pusala,
Thanks for reporting this issue.
Looks like JavaGcCleanerWrapper should use getDeclaredMethod instead of
getMethod to find Reference.tryHandlePending.
@Andrey, can you confirm this?
Regards,
Roman
On Mon, Jul 13, 2020 at 4:42 AM Murali Krishna Pusala <
, 2020 at 6:38 AM Si-li Liu wrote:
> Hi, Thanks for your help.
>
> The checkpoint configuration is
>
> checkpoint.intervalMS=30
> checkpoint.timeoutMS=30
>
> The error callstack is from JM's log, which happened in every cp.
> Currently I don't have a success cp ye
Hi,
> MessageType schema = reader.getFooter().getFileMetaData().getSchema();
The first thing I'd suggest is to verify that the file contains a valid
schema and can be read by some other program, e.g. parquet-tools schema or
cat [1].
Regards,
Roman
On Thu, Jul 2, 2020 at 11:36 PM Jesse Lord
ext: Evictor.EvictorContext): Unit = {}
>
> override def evictAfter(elements: JIterable[TimestampedValue[RawLog]],
>size: Int,
>window: TimeWindow,
>evictorContext: Evictor.EvictorContext): Unit = {
> val iter = eleme
Thanks, Ori
>From the log, it looks like there IS a memory leak.
At 10:12:53 there was the last "successfull" gc when 13Gb freed in
0.4653809 secs:
[Eden: 17336.0M(17336.0M)->0.0B(2544.0M) Survivors: 40960.0K->2176.0M Heap:
23280.3M(28960.0M)->10047.0M(28960.0M)]
Then the heap grew from 10G to
Thanks for the clarification.
Can you also share the code of other parts, particularly MyFunction?
Regards,
Roman
On Thu, Jul 2, 2020 at 12:49 PM Si-li Liu wrote:
> Rocksdb backend has the same problem
>
> Khachatryan Roman 于2020年7月2日周四 下午6:11写道：
>
>> Thanks for reporting
Thanks for reporting this.
Looks like the window namespace was replaced by VoidNamespace in state
entry.
I've created https://issues.apache.org/jira/browse/FLINK-18464 to further
investigate it.
Regards,
Roman
On Thu, Jul 2, 2020 at 10:52 AM Si-li Liu wrote:
> I'm using flink 1.9 on Mesos
Dear Tom,
This is likely a scala version issue.
Can you post your pom.xml?
Regards,
Roman
On Tue, Jun 2, 2020 at 6:34 PM Tom Burgert wrote:
> Dear all,
>
> I am trying to set up flink and after hours I still fail to make a simple
> program run even though I follow every recommended step in
Hi David,
One option is Ververica Platform which has a notion of Namespaces:
https://docs.ververica.com/administration/namespaces.html
I guess Konstantin can tell you more about it.
Disclaimer: I work for a company that develops this product.
Regards,
Roman
On Tue, Jun 2, 2020 at 5:37 PM
Hi,
Can you check if there are any failures on task manager mention in error
message (ip-10-210-5-104.ap-south-1.compute.internal/10.210.5.104:42317)?
Regards,
Roman
On Tue, Jun 2, 2020 at 10:18 AM ApoorvK
wrote:
> I have a flink job running on version 1.8.2 with parallelism of 12, I took
>
Hi White,
Did you try to increase rest.client.max-content-length [1]?
[1]
https://ci.apache.org/projects/flink/flink-docs-release-1.10/ops/config.html#advanced-options-for-the-rest-endpoint-and-client
Regards,
Roman
On Mon, Jun 1, 2020 at 8:01 AM snack white wrote:
> Hi,
> When I
Influx) as well.
>
> Thanks,
> Hemant
>
> On Tue, May 12, 2020 at 2:08 AM Khachatryan Roman <
> khachatryan.ro...@gmail.com> wrote:
>
>> Hi Hemant,
>>
>> As I understand, each protocol has a distinct set of event types (where
>> event type =
vent message arrives, this operator will append config1, config2
> and config3 to it. Operator 1 will extract and use config1, and output a
> message that has config1 stripped off.
>
> I hope that helps!
>
> Perhaps I am being too pedantic but I would like to know if these tw
Get Outlook for Android <https://aka.ms/ghei36>
>
> ------
> *From:* Khachatryan Roman
> *Sent:* Tuesday, May 12, 2020 1:49:10 AM
> *To:* Jaswin Shah
> *Cc:* user@flink.apache.org
> *Subject:* Re: Not able to implement an usecase
>
> Hi
Hi Hemant,
As I understand, each protocol has a distinct set of event types (where
event type == metrics type); and a distinct set of devices. Is this correct?
> Based on data protocol I have 4-5 topics. Currently the data for a single
event is being pushed to a partition of the kafka
Hi Jaswin,
Currently, DataStream API doesn't support outer joins.
As a workaround, you can use coGroup function [1].
Hive is also not supported by DataStream API though it's supported by Table
API [2].
[1]
Hi Jacky,
Did you try it without -XX:LogFile=${FLINK_LOG_PREFIX}.jit ?
Probably, Flink can't write to this location.
Also, you can try other tools described at
https://ci.apache.org/projects/flink/flink-docs-stable/monitoring/application_profiling.html
Regards,
Roman
On Mon, May 11, 2020 at
Hi Őrhidi,
Can you please provide some details about the errors you get?
Regards,
Roman
On Mon, May 11, 2020 at 9:32 AM Őrhidi Mátyás
wrote:
> Dear Community,
>
> I'm having troubles testing jobs against a secure Hadoop cluster. Is that
> possible? The mini cluster seems to not load any
Hi Yuval,
Thanks for reporting this issue. I'm pulling in Timo and Jark who are
working on the SQL component. They might be able to help you with your
problem.
Regards,
Roman
On Mon, May 11, 2020 at 9:10 AM Yuval Itzchakov wrote:
> Hi,
> While migrating from Flink 1.9 -> 1.10 and from the
Hi Manas,
The approaches you described looks the same:
> each operator only stores what it needs.
> each downstream operator will "strip off" the config parameter that it
needs.
Can you please explain the difference?
Regards,
Roman
On Mon, May 11, 2020 at 8:07 AM Manas Kale wrote:
> Hi,
> I
Hi Aissa,
What is BSONWritable you pass from map to sink?
I guess it's not serializable which causes Flink to use kryo, which fails.
Regards,
Roman
On Sun, May 10, 2020 at 10:42 PM Aissa Elaffani
wrote:
> Hello Guys,
> I am trying to sink my data to MongoDB, But i got some errors. I am
>
Hi Reo,
Please find the answers to your questions below.
> 1, what is the usage of this tmp files?
These files are used by Flink internally for things like caching state
locally, storing jars and so on. They are not intended for the end-user.
> 2, Is there have any mechanism of flink to manage
Hey Aaron,
You can use /jobs/:jobid/savepoints/:triggerid to get the location when the
checkpoint is completed.
Please see
https://ci.apache.org/projects/flink/flink-docs-release-1.10/api/java/index.html?org/apache/flink/runtime/rest/handler/job/savepoints/SavepointHandlers.html
Meanwhile, I've
Hi Gyula,
Could you provide the code of your Flink program, the error with stacktrace
and the Flink version?
Thanks.,
Roman
On Thu, Mar 5, 2020 at 2:17 PM Gyula Fóra wrote:
> Hi All!
>
> Excuse my stupid question, I am pretty new to the Table/SQL API and I am
> trying to play around with it
Hi Felipe,
Please find the answers to your questions below.
> Each "operator_subtask_index" means each instance of the parallel
physical operator, doesn't it?
Yes.
> How can I set a fixed ID for the "operator_id" in my code so I can
identify quickly which operator I am measuring?
You are using
askmanager.Task.doRun(Task.java:707)
>> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:532)
>> at java.lang.Thread.run(Thread.java:748)
>>
>>
>>
>>
>> On Tue, 25 Feb 2020 at 15:50, John Smith wrote:
>>
>>> Ok as soon as I can tom
oint it throws error "state not compatible"
> Previously there were no serializer used.
> I now wish to support state schema Hence need suggestion how can i achieve
> that ?
>
> Regards
>
> On Tue, Feb 25, 2020 at 9:08 PM Khachatryan Roman <
> khachatryan.ro...@gma
Hi,
I think conceptually the pipeline could look something like this:
env
.addSource(...)
.keyBy("device_id")
.window(SlidingEventTimeWindows.of(Time.minutes(15), Time.seconds(10)))
.trigger(new Trigger {
def onElement(el, timestamp, window, ctx) = {
if (window.start ==
As I understand from code, streamMap is a Java map, not Scala. So you can
get NPE while unreferencing the value you got from it.
Also, the approach looks a bit strange.
Can you describe what are you trying to achieve?
Regards,
Roman
On Mon, Feb 24, 2020 at 5:47 PM aj wrote:
>
> I am trying
Hi John,
Seems like this is another instance of
https://issues.apache.org/jira/browse/FLINK-8093
Could you please provide the full stacktrace?
Regards,
Roman
On Mon, Feb 24, 2020 at 10:48 PM John Smith wrote:
> Hi. Just upgraded to 1.10.0 And getting the bellow error when I deploy my
>
Hi ApoorvK,
I understand that you have a savepoint created by Flink 1.6.2 and you want
to use it with Flink 1.8.2. The classes themselves weren't modified.
Is that correct?
Which serializer did you use?
Regards,
Roman
On Tue, Feb 25, 2020 at 8:38 AM ApoorvK
wrote:
> Hi Team,
>
> Earlier we
aturation of certain buffers (looking into this now, however, i am not
> too familiar with flink internals)? It doesn't have to be extremely
> precise. Any hints would be greatly appreciated.
>
> Regards,
> M.
>
> On 25.02.20 13:34, Khachatryan Roman wrote:
>
> H
Hi Morgan,
Regarding backpressure, it can be caused by a number of factors, e.g.
writing to an external system or slow input partitions.
However, if you know that a particular resource is a bottleneck then it
makes sense to monitor its saturation.
It can be done by using Flink metrics. Please
.) 05:19
> To:Khachatryan Roman
> Cc:user
> Subject:Re: Exactly once semantics for hdfs sink
>
> Hi Khachatryan,
> Thanks for your reply. Can you help me understand how it works with hdfs
> specifically , even a link to a document will help.
>
>
> Best,
> Vishwas
>
> O
Hi Kristof,
Flink doesn't have any specific requirements.
You can run Flink on a single node with just one core. The number of
threads is dynamic.
However, you'll probably want to configure memory usage if the default
values are greater than what the actual machine has.
Regards,
Roman
On Mon,
Hi Vishwas,
Yes, Streaming File Sink does support exactly-once semantics and can be
used with HDFS.
Regards,
Roman
On Mon, Feb 10, 2020 at 5:20 PM Vishwas Siravara wrote:
> Hi all,
> I want to use the StreamingFile sink for writing data to hdfs. Can I
> achieve exactly once semantics with
Hi,
Could you please provide a full stacktrace?
Regards,
Roman
On Mon, Feb 10, 2020 at 2:12 PM sunfulin wrote:
> Hi, guys
> I am using Flink 1.10 and test functional cases with hive intergration.
> Hive with 1.1.0-cdh5.3.0 and with hadoop HA enabled.Running flink job I can
> see successful
On Mon, Feb 10, 2020 at 10:10 AM 闫旭 wrote:
> Yes, it’s the one
>
> On Feb 10, 2020, at 5:08 PM, Khachatryan Roman <
> khachatryan.ro...@gmail.com> wrote:
>
> Hi Xu Yan,
>
> Do you mean flink-benchmarks repo?
>
> Regards,
> Roman
>
>
> On Mon, Feb 10, 2
Hi Xu Yan,
Do you mean flink-benchmarks repo?
Regards,
Roman
On Mon, Feb 10, 2020 at 4:18 AM 闫旭 wrote:
> Hi there,
>
> I am just exploring the apache flink git repo and found the performance
> test. I have already test on my local machine, I’m wondering if we got
> online result?
>
> Thanks
1 - 100 of 104 matches
Mail list logo
