[jira] [Commented] (FLINK-1834) Is mapred.output.dir conf parameter really required?
[ https://issues.apache.org/jira/browse/FLINK-1834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17329859#comment-17329859 ] Flink Jira Bot commented on FLINK-1834: --- This issue has been labeled "stale-minor" for 7 days. It is closed now. If you are still affected by this or would like to raise the priority of this ticket please re-open, removing the label "auto-closed" and raise the ticket priority accordingly. > Is mapred.output.dir conf parameter really required? > > > Key: FLINK-1834 > URL: https://issues.apache.org/jira/browse/FLINK-1834 > Project: Flink > Issue Type: Improvement > Components: API / DataSet >Affects Versions: 0.9 >Reporter: Flavio Pompermaier >Priority: Minor > Labels: hadoop, stale-minor > > At the moment if you don't set mapred.output.dir parameter when output to an > hadoop sink Flink throws an error. Probably this parameter could be defaulted > to some tmp dir + jobId (for each output sink). > For example, when output to an HBase sink, the only things you'll find in the > mapred.output.dir will be the SUCCESS files -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-1834) Is mapred.output.dir conf parameter really required?
[ https://issues.apache.org/jira/browse/FLINK-1834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17321766#comment-17321766 ] Flink Jira Bot commented on FLINK-1834: --- This issue and all of its Sub-Tasks have not been updated for 180 days. So, it has been labeled "stale-minor". If you are still affected by this bug or are still interested in this issue, please give an update and remove the label. In 7 days the issue will be closed automatically. > Is mapred.output.dir conf parameter really required? > > > Key: FLINK-1834 > URL: https://issues.apache.org/jira/browse/FLINK-1834 > Project: Flink > Issue Type: Improvement > Components: API / DataSet >Affects Versions: 0.9 >Reporter: Flavio Pompermaier >Priority: Minor > Labels: hadoop, stale-minor > > At the moment if you don't set mapred.output.dir parameter when output to an > hadoop sink Flink throws an error. Probably this parameter could be defaulted > to some tmp dir + jobId (for each output sink). > For example, when output to an HBase sink, the only things you'll find in the > mapred.output.dir will be the SUCCESS files -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-1834) Is mapred.output.dir conf parameter really required?
[ https://issues.apache.org/jira/browse/FLINK-1834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14483249#comment-14483249 ] Fabian Hueske commented on FLINK-1834: -- I totally agree. We should get rid of this mandatory parameter. We try to only set it if it is set by the user and let the HadoopFormat figure out whether it needs it or not. If it does and it is not set, the job should fail. Is mapred.output.dir conf parameter really required? Key: FLINK-1834 URL: https://issues.apache.org/jira/browse/FLINK-1834 Project: Flink Issue Type: Improvement Components: Hadoop Compatibility Affects Versions: 0.9 Reporter: Flavio Pompermaier Priority: Minor Labels: hadoop At the moment if you don't set mapred.output.dir parameter when output to an hadoop sink Flink throws an error. Probably this parameter could be defaulted to some tmp dir + jobId (for each output sink). For example, when output to an HBase sink, the only things you'll find in the mapred.output.dir will be the SUCCESS files -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-1834) Is mapred.output.dir conf parameter really required?
[ https://issues.apache.org/jira/browse/FLINK-1834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14483258#comment-14483258 ] Maximilian Michels commented on FLINK-1834: --- +1 Is mapred.output.dir conf parameter really required? Key: FLINK-1834 URL: https://issues.apache.org/jira/browse/FLINK-1834 Project: Flink Issue Type: Improvement Components: Hadoop Compatibility Affects Versions: 0.9 Reporter: Flavio Pompermaier Priority: Minor Labels: hadoop At the moment if you don't set mapred.output.dir parameter when output to an hadoop sink Flink throws an error. Probably this parameter could be defaulted to some tmp dir + jobId (for each output sink). For example, when output to an HBase sink, the only things you'll find in the mapred.output.dir will be the SUCCESS files -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-1834) Is mapred.output.dir conf parameter really required?
[ https://issues.apache.org/jira/browse/FLINK-1834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14483142#comment-14483142 ] Fabian Hueske commented on FLINK-1834: -- I would be careful with this. Do we know where we should default to? If an OutputFormat does not require an output path we do not need to set it at all and if it does, it should be set by the user, IMO. Is mapred.output.dir conf parameter really required? Key: FLINK-1834 URL: https://issues.apache.org/jira/browse/FLINK-1834 Project: Flink Issue Type: Improvement Components: Hadoop Compatibility Affects Versions: 0.9 Reporter: Flavio Pompermaier Priority: Minor Labels: hadoop At the moment if you don't set mapred.output.dir parameter when output to an hadoop sink Flink throws an error. Probably this parameter could be defaulted to some tmp dir + jobId (for each output sink). For example, when output to an HBase sink, the only things you'll find in the mapred.output.dir will be the SUCCESS files -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-1834) Is mapred.output.dir conf parameter really required?
[
https://issues.apache.org/jira/browse/FLINK-1834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14483125#comment-14483125
]
Maximilian Michels commented on FLINK-1834:
---
Let's check if the user's Hadoop configuration contains {{mapred.output.dir}}
via {{HadoopUtils.getHadoopConfiguration()}}. Otherwise, set a default with a
warning which also contains the output path to which we default.
Do you want to fix that?
Is mapred.output.dir conf parameter really required?
Key: FLINK-1834
URL: https://issues.apache.org/jira/browse/FLINK-1834
Project: Flink
Issue Type: Improvement
Components: Hadoop Compatibility
Affects Versions: 0.9
Reporter: Flavio Pompermaier
Priority: Minor
Labels: hadoop
At the moment if you don't set mapred.output.dir parameter when output to an
hadoop sink Flink throws an error. Probably this parameter could be defaulted
to some tmp dir + jobId (for each output sink).
For example, when output to an HBase sink, the only things you'll find in the
mapred.output.dir will be the SUCCESS files
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
