[jira] [Commented] (HADOOP-16613) s3a to set fake directory marker contentType to application/x-directory
[ https://issues.apache.org/jira/browse/HADOOP-16613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17526303#comment-17526303 ] Brais Couce commented on HADOOP-16613: -- You are right. It works fine with 3.3.2. Thanks! > s3a to set fake directory marker contentType to application/x-directory > --- > > Key: HADOOP-16613 > URL: https://issues.apache.org/jira/browse/HADOOP-16613 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.2.1, 3.1.3 >Reporter: Jose Torres >Assignee: Steve Loughran >Priority: Minor > Fix For: 3.3.0 > > > S3AFileSystem doesn't set a contentType for fake directory files, causing it > to be inferred as "application/octet-stream". But fake directory files > created through the S3 web console have content type > "application/x-directory". We may want to adopt the web console behavior as a > standard, since some systems will rely on content type and not size + > trailing slash to determine if an object represents a directory. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16613) s3a to set fake directory marker contentType to application/x-directory
[ https://issues.apache.org/jira/browse/HADOOP-16613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17525717#comment-17525717 ] Steve Loughran commented on HADOOP-16613: - looks like HADOOP-17511 does it properly. try with 3.3.2. > s3a to set fake directory marker contentType to application/x-directory > --- > > Key: HADOOP-16613 > URL: https://issues.apache.org/jira/browse/HADOOP-16613 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.2.1, 3.1.3 >Reporter: Jose Torres >Assignee: Steve Loughran >Priority: Minor > Fix For: 3.3.0 > > > S3AFileSystem doesn't set a contentType for fake directory files, causing it > to be inferred as "application/octet-stream". But fake directory files > created through the S3 web console have content type > "application/x-directory". We may want to adopt the web console behavior as a > standard, since some systems will rely on content type and not size + > trailing slash to determine if an object represents a directory. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16613) s3a to set fake directory marker contentType to application/x-directory
[ https://issues.apache.org/jira/browse/HADOOP-16613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17525095#comment-17525095 ] Brais Couce commented on HADOOP-16613: -- Any update on this? Is this the correct place for the question about the revert? > s3a to set fake directory marker contentType to application/x-directory > --- > > Key: HADOOP-16613 > URL: https://issues.apache.org/jira/browse/HADOOP-16613 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.2.1, 3.1.3 >Reporter: Jose Torres >Assignee: Steve Loughran >Priority: Minor > Fix For: 3.3.0 > > > S3AFileSystem doesn't set a contentType for fake directory files, causing it > to be inferred as "application/octet-stream". But fake directory files > created through the S3 web console have content type > "application/x-directory". We may want to adopt the web console behavior as a > standard, since some systems will rely on content type and not size + > trailing slash to determine if an object represents a directory. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16613) s3a to set fake directory marker contentType to application/x-directory
[ https://issues.apache.org/jira/browse/HADOOP-16613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17508161#comment-17508161 ] Brais Couce commented on HADOOP-16613: -- Should I create a new ticket for this question? > s3a to set fake directory marker contentType to application/x-directory > --- > > Key: HADOOP-16613 > URL: https://issues.apache.org/jira/browse/HADOOP-16613 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.2.1, 3.1.3 >Reporter: Jose Torres >Assignee: Steve Loughran >Priority: Minor > Fix For: 3.3.0 > > > S3AFileSystem doesn't set a contentType for fake directory files, causing it > to be inferred as "application/octet-stream". But fake directory files > created through the S3 web console have content type > "application/x-directory". We may want to adopt the web console behavior as a > standard, since some systems will rely on content type and not size + > trailing slash to determine if an object represents a directory. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16613) s3a to set fake directory marker contentType to application/x-directory
[ https://issues.apache.org/jira/browse/HADOOP-16613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17498806#comment-17498806 ] Brais Couce commented on HADOOP-16613: -- Hi, The information of this ticket says that this is resolved in 3.3.0 version as part of HADOOP-16823. I have tested it and the directory created in S3 does not have contentType = application/x-directory. In [this commit|https://github.com/apache/hadoop/pull/1826/commits/86fd17d3687f93e30e58a8bcb13650e77736965f] the contentType is set but in [this commit|http://https://github.com/apache/hadoop/pull/1826/commits/86fd17d3687f93e30e58a8bcb13650e77736965f] it was reverted. Is there any reason for it? Regards. > s3a to set fake directory marker contentType to application/x-directory > --- > > Key: HADOOP-16613 > URL: https://issues.apache.org/jira/browse/HADOOP-16613 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.2.1, 3.1.3 >Reporter: Jose Torres >Assignee: Steve Loughran >Priority: Minor > Fix For: 3.3.0 > > > S3AFileSystem doesn't set a contentType for fake directory files, causing it > to be inferred as "application/octet-stream". But fake directory files > created through the S3 web console have content type > "application/x-directory". We may want to adopt the web console behavior as a > standard, since some systems will rely on content type and not size + > trailing slash to determine if an object represents a directory. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16613) s3a to set fake directory marker contentType to application/x-directory
[ https://issues.apache.org/jira/browse/HADOOP-16613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17031483#comment-17031483 ] Steve Loughran commented on HADOOP-16613: - Fixing in HADOOP-16823 as I do things with directory deletions > s3a to set fake directory marker contentType to application/x-directory > --- > > Key: HADOOP-16613 > URL: https://issues.apache.org/jira/browse/HADOOP-16613 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.2.1, 3.1.3 >Reporter: Jose Torres >Priority: Minor > > S3AFileSystem doesn't set a contentType for fake directory files, causing it > to be inferred as "application/octet-stream". But fake directory files > created through the S3 web console have content type > "application/x-directory". We may want to adopt the web console behavior as a > standard, since some systems will rely on content type and not size + > trailing slash to determine if an object represents a directory. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16613) s3a to set fake directory marker contentType to application/x-directory
[ https://issues.apache.org/jira/browse/HADOOP-16613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16952213#comment-16952213 ] Jose Torres commented on HADOOP-16613: -- Confirmed that (2) is fine; the S3 web console can't create directory objects that don't end in a slash, and doesn't recognize them as directories if manually created even when they have length 0 and content-type == application/x-directory. > s3a to set fake directory marker contentType to application/x-directory > --- > > Key: HADOOP-16613 > URL: https://issues.apache.org/jira/browse/HADOOP-16613 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.2.1, 3.1.3 >Reporter: Jose Torres >Priority: Minor > > S3AFileSystem doesn't set a contentType for fake directory files, causing it > to be inferred as "application/octet-stream". But fake directory files > created through the S3 web console have content type > "application/x-directory". We may want to adopt the web console behavior as a > standard, since some systems will rely on content type and not size + > trailing slash to determine if an object represents a directory. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16613) s3a to set fake directory marker contentType to application/x-directory
[ https://issues.apache.org/jira/browse/HADOOP-16613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16952194#comment-16952194 ] Jose Torres commented on HADOOP-16613: -- Working on preparing and testing a PR, but I'd propose: # Yes, but we should still consider length 0 things which don't have content-type == application/x-directory to be directories. Otherwise we wouldn't be able to read directory structures from old Hadoop versions. # We should still reject objects that don't end in a / as directories. Changing this would surely introduce weird edge cases in path parsing logic elsewhere. (But I'm gonna check that the web console can't create such objects.) > s3a to set fake directory marker contentType to application/x-directory > --- > > Key: HADOOP-16613 > URL: https://issues.apache.org/jira/browse/HADOOP-16613 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.2.1, 3.1.3 >Reporter: Jose Torres >Priority: Minor > > S3AFileSystem doesn't set a contentType for fake directory files, causing it > to be inferred as "application/octet-stream". But fake directory files > created through the S3 web console have content type > "application/x-directory". We may want to adopt the web console behavior as a > standard, since some systems will rely on content type and not size + > trailing slash to determine if an object represents a directory. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16613) s3a to set fake directory marker contentType to application/x-directory
[ https://issues.apache.org/jira/browse/HADOOP-16613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16949553#comment-16949553 ] Steve Loughran commented on HADOOP-16613: - # should we ourselves say content-type == application/x-directory means it is a dir, irrespective of len # how to react to something without a / which says it is an x-directory? > s3a to set fake directory marker contentType to application/x-directory > --- > > Key: HADOOP-16613 > URL: https://issues.apache.org/jira/browse/HADOOP-16613 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.2.1, 3.1.3 >Reporter: Jose Torres >Priority: Minor > > S3AFileSystem doesn't set a contentType for fake directory files, causing it > to be inferred as "application/octet-stream". But fake directory files > created through the S3 web console have content type > "application/x-directory". We may want to adopt the web console behavior as a > standard, since some systems will rely on content type and not size + > trailing slash to determine if an object represents a directory. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
