[jira] [Updated] (MAPREDUCE-6512) FileOutputCommitter tasks unconditionally create parent directories
[ https://issues.apache.org/jira/browse/MAPREDUCE-6512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated MAPREDUCE-6512: -- Resolution: Won't Fix Status: Resolved (was: Patch Available) bq. should we close this as won't fix? Agree that this is probably too risky to fix given all the custom output committers that could be implicitly relying on this behavior. > FileOutputCommitter tasks unconditionally create parent directories > --- > > Key: MAPREDUCE-6512 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6512 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Chang Li >Assignee: Chang Li > Attachments: MAPREDUCE-6512.2.patch, MAPREDUCE-6512.2.patch, > MAPREDUCE-6512.patch, MAPREDUCE-6512.patch > > > If the output directory is deleted then subsequent tasks should fail. Instead > they blindly create the missing parent directories, leading the job to be > "succesful" despite potentially missing almost all of the output. Task > attempts should fail if the parent app attempt directory is missing when they > go to create their task attempt directory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6512) FileOutputCommitter tasks unconditionally create parent directories
[ https://issues.apache.org/jira/browse/MAPREDUCE-6512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Badger updated MAPREDUCE-6512: --- Assignee: Eric Badger (was: Chang Li) > FileOutputCommitter tasks unconditionally create parent directories > --- > > Key: MAPREDUCE-6512 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6512 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Chang Li >Assignee: Eric Badger > Attachments: MAPREDUCE-6512.2.patch, MAPREDUCE-6512.2.patch, > MAPREDUCE-6512.patch, MAPREDUCE-6512.patch > > > If the output directory is deleted then subsequent tasks should fail. Instead > they blindly create the missing parent directories, leading the job to be > "succesful" despite potentially missing almost all of the output. Task > attempts should fail if the parent app attempt directory is missing when they > go to create their task attempt directory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6512) FileOutputCommitter tasks unconditionally create parent directories
[ https://issues.apache.org/jira/browse/MAPREDUCE-6512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chang Li updated MAPREDUCE-6512: Attachment: MAPREDUCE-6512.2.patch > FileOutputCommitter tasks unconditionally create parent directories > --- > > Key: MAPREDUCE-6512 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6512 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Chang Li >Assignee: Chang Li > Attachments: MAPREDUCE-6512.2.patch, MAPREDUCE-6512.2.patch, > MAPREDUCE-6512.patch, MAPREDUCE-6512.patch > > > If the output directory is deleted then subsequent tasks should fail. Instead > they blindly create the missing parent directories, leading the job to be > "succesful" despite potentially missing almost all of the output. Task > attempts should fail if the parent app attempt directory is missing when they > go to create their task attempt directory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6512) FileOutputCommitter tasks unconditionally create parent directories
[ https://issues.apache.org/jira/browse/MAPREDUCE-6512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chang Li updated MAPREDUCE-6512: Attachment: MAPREDUCE-6512.2.patch .2 patch fix broken tests which are caused by missing parent dir > FileOutputCommitter tasks unconditionally create parent directories > --- > > Key: MAPREDUCE-6512 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6512 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Chang Li >Assignee: Chang Li > Attachments: MAPREDUCE-6512.2.patch, MAPREDUCE-6512.patch, > MAPREDUCE-6512.patch > > > If the output directory is deleted then subsequent tasks should fail. Instead > they blindly create the missing parent directories, leading the job to be > "succesful" despite potentially missing almost all of the output. Task > attempts should fail if the parent app attempt directory is missing when they > go to create their task attempt directory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6512) FileOutputCommitter tasks unconditionally create parent directories
[ https://issues.apache.org/jira/browse/MAPREDUCE-6512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chang Li updated MAPREDUCE-6512: Attachment: YARN-4269.2.patch > FileOutputCommitter tasks unconditionally create parent directories > --- > > Key: MAPREDUCE-6512 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6512 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Chang Li >Assignee: Chang Li > Attachments: MAPREDUCE-6512.patch, MAPREDUCE-6512.patch > > > If the output directory is deleted then subsequent tasks should fail. Instead > they blindly create the missing parent directories, leading the job to be > "succesful" despite potentially missing almost all of the output. Task > attempts should fail if the parent app attempt directory is missing when they > go to create their task attempt directory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6512) FileOutputCommitter tasks unconditionally create parent directories
[ https://issues.apache.org/jira/browse/MAPREDUCE-6512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chang Li updated MAPREDUCE-6512: Attachment: MAPREDUCE-6512.patch > FileOutputCommitter tasks unconditionally create parent directories > --- > > Key: MAPREDUCE-6512 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6512 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Chang Li >Assignee: Chang Li > Attachments: MAPREDUCE-6512.patch, MAPREDUCE-6512.patch > > > If the output directory is deleted then subsequent tasks should fail. Instead > they blindly create the missing parent directories, leading the job to be > "succesful" despite potentially missing almost all of the output. Task > attempts should fail if the parent app attempt directory is missing when they > go to create their task attempt directory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6512) FileOutputCommitter tasks unconditionally create parent directories
[ https://issues.apache.org/jira/browse/MAPREDUCE-6512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chang Li updated MAPREDUCE-6512: Attachment: (was: YARN-4269.2.patch) > FileOutputCommitter tasks unconditionally create parent directories > --- > > Key: MAPREDUCE-6512 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6512 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Chang Li >Assignee: Chang Li > Attachments: MAPREDUCE-6512.patch, MAPREDUCE-6512.patch > > > If the output directory is deleted then subsequent tasks should fail. Instead > they blindly create the missing parent directories, leading the job to be > "succesful" despite potentially missing almost all of the output. Task > attempts should fail if the parent app attempt directory is missing when they > go to create their task attempt directory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6512) FileOutputCommitter tasks unconditionally create parent directories
[ https://issues.apache.org/jira/browse/MAPREDUCE-6512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chang Li updated MAPREDUCE-6512: Status: Patch Available (was: Open) > FileOutputCommitter tasks unconditionally create parent directories > --- > > Key: MAPREDUCE-6512 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6512 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Chang Li >Assignee: Chang Li > Attachments: MAPREDUCE-6512.patch > > > If the output directory is deleted then subsequent tasks should fail. Instead > they blindly create the missing parent directories, leading the job to be > "succesful" despite potentially missing almost all of the output. Task > attempts should fail if the parent app attempt directory is missing when they > go to create their task attempt directory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6512) FileOutputCommitter tasks unconditionally create parent directories
[ https://issues.apache.org/jira/browse/MAPREDUCE-6512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chang Li updated MAPREDUCE-6512: Attachment: MAPREDUCE-6512.patch > FileOutputCommitter tasks unconditionally create parent directories > --- > > Key: MAPREDUCE-6512 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6512 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Chang Li >Assignee: Chang Li > Attachments: MAPREDUCE-6512.patch > > > If the output directory is deleted then subsequent tasks should fail. Instead > they blindly create the missing parent directories, leading the job to be > "succesful" despite potentially missing almost all of the output. Task > attempts should fail if the parent app attempt directory is missing when they > go to create their task attempt directory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
