I have fixed the issue by increasing the CPU and memory for my JM and TM
pods.
Make sure your instance type can accommodate the required resources.
On Wed, 19 Jul 2023 at 13:35, Orkhan Dadashov
wrote:
> Hi Flink users,
>
> I'm following up on this guide to try the Flink K8S operator (1.5.0
>
Hey!
Please help us understand why you need to delete and recreate the
FlinkDeployment objects in your ecosystem. Maybe we can help suggest some
alternative to make your life easier :)
Of course every prod ecosystem is unique in its own way and larger
platforms generally have a layer on top of
异常栈信息
```
2023-07-20 11:43:01,627 ERROR
org.apache.flink.runtime.taskexecutor.TaskManagerRunner [] - Terminating
TaskManagerRunner with exit code 1.
org.apache.flink.util.FlinkException: Failed to start the TaskManagerRunner.
at
flink 1.17.1 on Yarn实时任务运行了几天出现了Yarn
token过期问题,在1.12未出现。这块具体有什么变化嘛，我是否还需要再配置其他参数。
具体配置:
```
security.kerberos.access.hadoopFileSystems: viewfs://AutoLfCluster;hdfs://ns1
security.kerberos.login.keytab: /xxx/krb5.keytab
security.kerberos.login.principal: flink/xxx
Hi Rajat,
We are using apache zeppelin as our entry point for submitting flink ad-hoc
queries (and spark jobs actually).
It supports interactive queries, data visualization, multiple data query
engines, multiple auth models. You can check out other features on its
official website.
But because
Hi Rajat,
Currently sql-gateway supports REST[1] and Hive[2] endpoints.
For Hive endpoints, you can submit sql jobs with existing Hive clients,
such as hive jdbc, apache superset and other systems.
For REST endpoints, you can use flink sql-client to submit your sql jobs.
We support
Hi Neha,
The HOP window will increase the size of the checkpoint and I'm sorry
that I'm not very familiar with the HOP window.
If the configurations are all right, and you want to confirm if it's a HOP
window issue, I think you can submit a flink job without HOP window but
with regular agg
Hi Gyula,
Got it. Our use case might be unique to our own ecosystem here at
Robinhood, so I will have to look into creating a service that can search
for the latest savepoint / checkpoint in S3 and provide that to the
FlinkDeployment resource.
Will the Flink Community be okay with us adding this
Hi Flink users,
I'm following up on this guide to try the Flink K8S operator (1.5.0 version
):
https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-release-1.5/docs/try-flink-kubernetes-operator/quick-start/
When I try to deploy a basic example, JM and TM start, but TM fails to
Hi!
I don’t understand why you need to delete the deployment to restart. You
can suspend, use the restartNonce or simply upgrade .
These should cover most upgrade/restart scenarios. Like with other
resources in Kubernetes once you delete them the status is gone, so the
FlinkDeployment won’t keep
Hi Gyula,
Thank you for responding so quickly. I went through the page you sent me a
bit more, and I see the following (
https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-release-1.4/docs/custom-resource/job-management/#running-suspending-and-deleting-applications
):
Deleting a
Hey Tony,
Please see:
https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-main/docs/custom-resource/job-management/#stateful-and-stateless-application-upgrades
The operator is made especially to handle stateful application upgrades
robustly. In general any spec change that you make
Hi Flink Community,
My name is Tony Chen, and I am a software engineer at Robinhood. I have
some questions on restarting a Flink Application from a savepoint or
checkpoint.
We currently store our checkpoints and savepoints in S3, and we would like
to use the Apache Flink Kubernetes Operator to
Hi Flink users,
I'm following up on this guide to try the Flink K8S operator (1.5.0 version
):
https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-release-1.5/docs/try-flink-kubernetes-operator/quick-start/
When I try to deploy a basic example, JM and TM start, but TM fails to
Hi team,
I have set up a session cluster on k8s via sql gateway. I am looking for
an open source Flink sql editor that can submit sql queries on top of the
k8s session cluster. Any suggestions for sql editor to submit queries ?
Thanks
Hi,
As documented [1] this option "enables uploading and starting jobs through
the Flink UI (true by default). Please note that even when this is
disabled, session clusters still accept jobs through REST requests (HTTP
calls). This flag only guards the feature to upload jobs in the UI."
It won't
Hi Team,
I am using the upsert-kafka table API connector of Apache Flink to consume
events from a kafka topic, I want to log the kafka payloads that are
consumed. Is there a way to log it?
My code looks as below:
EnvironmentSettings settings =
Thanks for sharing the information.
I also observed the same, S3 (Primary Checkpoint Storage) + EBS (Task Local
Recovery) performs better than EBS as Primary Checkpoint storage.
On Tue, Jul 18, 2023 at 12:21 PM Konstantin Knauf wrote:
> Hi Prabhu,
>
> this should be possible, but is quite
18 matches
Mail list logo
