[
https://issues.apache.org/jira/browse/BEAM-6058?focusedWorklogId=166020&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-166020
]
ASF GitHub Bot logged work on BEAM-6058:
----------------------------------------
Author: ASF GitHub Bot
Created on: 14/Nov/18 18:54
Start Date: 14/Nov/18 18:54
Worklog Time Spent: 10m
Work Description: angoenka edited a comment on issue #7031: [BEAM-6058]
Adding option for flink configuration directory and setting config in
exectution environment
URL: https://github.com/apache/beam/pull/7031#issuecomment-438775803
Properties like taskmanager.runtime.large-record-handler are read from the
submitted JobGraph and are not read from cluster config when using
RemoteEnvironment. Hence we need to read and set the properties in the job
graph. Many of these properties are not even exposed via ExecutionConfig so we
need to read these properties directly.
Reading from a config dir seems be the right way as we are already doing it
for parallelism.
Also this lets us specify all flink properties instead of creating new flags
for individual ones.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 166020)
Time Spent: 2h 10m (was: 2h)
> Support flink config directory for flink runner.
> ------------------------------------------------
>
> Key: BEAM-6058
> URL: https://issues.apache.org/jira/browse/BEAM-6058
> Project: Beam
> Issue Type: Bug
> Components: runner-flink
> Reporter: Ankur Goenka
> Assignee: Ankur Goenka
> Priority: Major
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Give user option to use flink configuration from a config directory.
> Earlier we only used jars to submit flink jobs which by default picks the
> cluster flink properties.
> Portability uses remote environment to submit flink job. In Remote
> execution, flink uses the client config instead of the cluster config. This
> require us to submit the custom flink properties via flink client.
> Suggestion is to read the flink config from a flink configuration folder and
> set them to the environment.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
