[jira] [Commented] (FLINK-11034) Provide "rewriting config” to file system factory
[ https://issues.apache.org/jira/browse/FLINK-11034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17336545#comment-17336545 ] Flink Jira Bot commented on FLINK-11034: This issue was labeled "stale-major" 7 ago and has not received any updates so it is being deprioritized. If this ticket is actually Major, please raise the priority and ask a committer to assign you the issue or revive the public discussion. > Provide "rewriting config” to file system factory > -- > > Key: FLINK-11034 > URL: https://issues.apache.org/jira/browse/FLINK-11034 > Project: Flink > Issue Type: Improvement > Components: Connectors / FileSystem, FileSystems >Reporter: Wei-Che Wei >Priority: Major > Labels: stale-major > > In the discussion in this mailing thread > [http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/how-to-override-s3-key-config-in-flink-job-td24606.html], > it showed that it is not able to overwrite config in file system factory > when submit a flink job. > That means we will share the same config for multiple jobs in a session > cluster. Or user can't use different configuration for checkpointing and file > sink. For example, user might have different s3 buckets for checkpointing and > file sink, but each of the s3 bucket might have different s3 access key for > some management concerns. > We might need to provide a way to overwrite configuration when calling file > system factory "get" method, and let those user facing components, like > checkpointing or file sink, will be able to get overwriting config from user > and create a file system with those changes in the new config. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-11034) Provide "rewriting config” to file system factory
[ https://issues.apache.org/jira/browse/FLINK-11034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17328511#comment-17328511 ] Flink Jira Bot commented on FLINK-11034: This major issue is unassigned and itself and all of its Sub-Tasks have not been updated for 30 days. So, it has been labeled "stale-major". If this ticket is indeed "major", please either assign yourself or give an update. Afterwards, please remove the label. In 7 days the issue will be deprioritized. > Provide "rewriting config” to file system factory > -- > > Key: FLINK-11034 > URL: https://issues.apache.org/jira/browse/FLINK-11034 > Project: Flink > Issue Type: Improvement > Components: Connectors / FileSystem, FileSystems >Reporter: Wei-Che Wei >Priority: Major > Labels: stale-major > > In the discussion in this mailing thread > [http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/how-to-override-s3-key-config-in-flink-job-td24606.html], > it showed that it is not able to overwrite config in file system factory > when submit a flink job. > That means we will share the same config for multiple jobs in a session > cluster. Or user can't use different configuration for checkpointing and file > sink. For example, user might have different s3 buckets for checkpointing and > file sink, but each of the s3 bucket might have different s3 access key for > some management concerns. > We might need to provide a way to overwrite configuration when calling file > system factory "get" method, and let those user facing components, like > checkpointing or file sink, will be able to get overwriting config from user > and create a file system with those changes in the new config. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-11034) Provide "rewriting config” to file system factory
[ https://issues.apache.org/jira/browse/FLINK-11034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704369#comment-16704369 ] Tony Xintong Song commented on FLINK-11034: --- I agree with you that we should allow users to config multiple jobs in the same session differently. However, there are also config options that should remain consistent during the lifecycle of the whole session. I think the first step towards this job-specific config issue is probably separate config options into two categories: job-level configs which should be configured for each job independently, and session-level configs which should remain consistent. And probably we should have two config files for the two categories. Otherwise, it could be troublesome for the user to figure out which config options can be "rewritten" when submitting a new job to a exist session and which can not. > Provide "rewriting config” to file system factory > -- > > Key: FLINK-11034 > URL: https://issues.apache.org/jira/browse/FLINK-11034 > Project: Flink > Issue Type: Improvement >Reporter: Wei-Che Wei >Priority: Major > > In the discussion in this mailing thread > [http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/how-to-override-s3-key-config-in-flink-job-td24606.html], > it showed that it is not able to overwrite config in file system factory > when submit a flink job. > That means we will share the same config for multiple jobs in a session > cluster. Or user can't use different configuration for checkpointing and file > sink. For example, user might have different s3 buckets for checkpointing and > file sink, but each of the s3 bucket might have different s3 access key for > some management concerns. > We might need to provide a way to overwrite configuration when calling file > system factory "get" method, and let those user facing components, like > checkpointing or file sink, will be able to get overwriting config from user > and create a file system with those changes in the new config. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
