[jira] [Updated] (HADOOP-8833) fs -text should make sure to call inputstream.seek(0) before using input stream
[ https://issues.apache.org/jira/browse/HADOOP-8833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh J updated HADOOP-8833: Target Version/s: 3.0.0 (was: 2.0.2-alpha) fs -text should make sure to call inputstream.seek(0) before using input stream --- Key: HADOOP-8833 URL: https://issues.apache.org/jira/browse/HADOOP-8833 Project: Hadoop Common Issue Type: Bug Components: fs Affects Versions: 2.0.2-alpha Reporter: Harsh J Assignee: Harsh J Attachments: HADOOP-8833.patch, HADOOP-8833.patch, HADOOP-8833.patch From Muddy Dixon on HADOOP-8449: Hi We found the changes in order of switch and guard block in {code} private InputStream forMagic(Path p, FileSystem srcFs) throws IOException {code} Because of this change, return value of {code} codec.createInputStream(i) {code} is changed if codec exists. {code} private InputStream forMagic(Path p, FileSystem srcFs) throws IOException { FSDataInputStream i = srcFs.open(p); // check codecs CompressionCodecFactory cf = new CompressionCodecFactory(getConf()); CompressionCodec codec = cf.getCodec(p); if (codec != null) { return codec.createInputStream(i); } switch(i.readShort()) { // cases } {code} New: {code} private InputStream forMagic(Path p, FileSystem srcFs) throws IOException { FSDataInputStream i = srcFs.open(p); switch(i.readShort()) { // === index (or pointer) processes!! // cases default: { // Check the type of compression instead, depending on Codec class's // own detection methods, based on the provided path. CompressionCodecFactory cf = new CompressionCodecFactory(getConf()); CompressionCodec codec = cf.getCodec(p); if (codec != null) { return codec.createInputStream(i); } break; } } // File is non-compressed, or not a file container we know. i.seek(0); return i; } {code} Fix is to use i.seek(0) before we use i anywhere. I missed that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8833) fs -text should make sure to call inputstream.seek(0) before using input stream
[ https://issues.apache.org/jira/browse/HADOOP-8833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh J updated HADOOP-8833: Target Version/s: 3.0.0, 2.0.3-alpha (was: 3.0.0) fs -text should make sure to call inputstream.seek(0) before using input stream --- Key: HADOOP-8833 URL: https://issues.apache.org/jira/browse/HADOOP-8833 Project: Hadoop Common Issue Type: Bug Components: fs Affects Versions: 2.0.2-alpha Reporter: Harsh J Assignee: Harsh J Attachments: HADOOP-8833.patch, HADOOP-8833.patch, HADOOP-8833.patch From Muddy Dixon on HADOOP-8449: Hi We found the changes in order of switch and guard block in {code} private InputStream forMagic(Path p, FileSystem srcFs) throws IOException {code} Because of this change, return value of {code} codec.createInputStream(i) {code} is changed if codec exists. {code} private InputStream forMagic(Path p, FileSystem srcFs) throws IOException { FSDataInputStream i = srcFs.open(p); // check codecs CompressionCodecFactory cf = new CompressionCodecFactory(getConf()); CompressionCodec codec = cf.getCodec(p); if (codec != null) { return codec.createInputStream(i); } switch(i.readShort()) { // cases } {code} New: {code} private InputStream forMagic(Path p, FileSystem srcFs) throws IOException { FSDataInputStream i = srcFs.open(p); switch(i.readShort()) { // === index (or pointer) processes!! // cases default: { // Check the type of compression instead, depending on Codec class's // own detection methods, based on the provided path. CompressionCodecFactory cf = new CompressionCodecFactory(getConf()); CompressionCodec codec = cf.getCodec(p); if (codec != null) { return codec.createInputStream(i); } break; } } // File is non-compressed, or not a file container we know. i.seek(0); return i; } {code} Fix is to use i.seek(0) before we use i anywhere. I missed that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8833) fs -text should make sure to call inputstream.seek(0) before using input stream
[ https://issues.apache.org/jira/browse/HADOOP-8833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh J updated HADOOP-8833: Resolution: Fixed Fix Version/s: 2.0.3-alpha Target Version/s: (was: 3.0.0, 2.0.3-alpha) Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) fs -text should make sure to call inputstream.seek(0) before using input stream --- Key: HADOOP-8833 URL: https://issues.apache.org/jira/browse/HADOOP-8833 Project: Hadoop Common Issue Type: Bug Components: fs Affects Versions: 2.0.2-alpha Reporter: Harsh J Assignee: Harsh J Fix For: 2.0.3-alpha Attachments: HADOOP-8833.patch, HADOOP-8833.patch, HADOOP-8833.patch From Muddy Dixon on HADOOP-8449: Hi We found the changes in order of switch and guard block in {code} private InputStream forMagic(Path p, FileSystem srcFs) throws IOException {code} Because of this change, return value of {code} codec.createInputStream(i) {code} is changed if codec exists. {code} private InputStream forMagic(Path p, FileSystem srcFs) throws IOException { FSDataInputStream i = srcFs.open(p); // check codecs CompressionCodecFactory cf = new CompressionCodecFactory(getConf()); CompressionCodec codec = cf.getCodec(p); if (codec != null) { return codec.createInputStream(i); } switch(i.readShort()) { // cases } {code} New: {code} private InputStream forMagic(Path p, FileSystem srcFs) throws IOException { FSDataInputStream i = srcFs.open(p); switch(i.readShort()) { // === index (or pointer) processes!! // cases default: { // Check the type of compression instead, depending on Codec class's // own detection methods, based on the provided path. CompressionCodecFactory cf = new CompressionCodecFactory(getConf()); CompressionCodec codec = cf.getCodec(p); if (codec != null) { return codec.createInputStream(i); } break; } } // File is non-compressed, or not a file container we know. i.seek(0); return i; } {code} Fix is to use i.seek(0) before we use i anywhere. I missed that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8833) fs -text should make sure to call inputstream.seek(0) before using input stream
[ https://issues.apache.org/jira/browse/HADOOP-8833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh J updated HADOOP-8833: Attachment: HADOOP-8833.patch This should fix it. fs -text should make sure to call inputstream.seek(0) before using input stream --- Key: HADOOP-8833 URL: https://issues.apache.org/jira/browse/HADOOP-8833 Project: Hadoop Common Issue Type: Bug Components: fs Affects Versions: 2.0.2-alpha Reporter: Harsh J Assignee: Harsh J Attachments: HADOOP-8833.patch From Muddy Dixon on HADOOP-8449: Hi We found the changes in order of switch and guard block in {code} private InputStream forMagic(Path p, FileSystem srcFs) throws IOException {code} Because of this change, return value of {code} codec.createInputStream(i) {code} is changed if codec exists. {code} private InputStream forMagic(Path p, FileSystem srcFs) throws IOException { FSDataInputStream i = srcFs.open(p); // check codecs CompressionCodecFactory cf = new CompressionCodecFactory(getConf()); CompressionCodec codec = cf.getCodec(p); if (codec != null) { return codec.createInputStream(i); } switch(i.readShort()) { // cases } {code} New: {code} private InputStream forMagic(Path p, FileSystem srcFs) throws IOException { FSDataInputStream i = srcFs.open(p); switch(i.readShort()) { // === index (or pointer) processes!! // cases default: { // Check the type of compression instead, depending on Codec class's // own detection methods, based on the provided path. CompressionCodecFactory cf = new CompressionCodecFactory(getConf()); CompressionCodec codec = cf.getCodec(p); if (codec != null) { return codec.createInputStream(i); } break; } } // File is non-compressed, or not a file container we know. i.seek(0); return i; } {code} Fix is to use i.seek(0) before we use i anywhere. I missed that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8833) fs -text should make sure to call inputstream.seek(0) before using input stream
[ https://issues.apache.org/jira/browse/HADOOP-8833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh J updated HADOOP-8833: Target Version/s: 2.0.2-alpha Status: Patch Available (was: Open) fs -text should make sure to call inputstream.seek(0) before using input stream --- Key: HADOOP-8833 URL: https://issues.apache.org/jira/browse/HADOOP-8833 Project: Hadoop Common Issue Type: Bug Components: fs Affects Versions: 2.0.2-alpha Reporter: Harsh J Assignee: Harsh J Attachments: HADOOP-8833.patch From Muddy Dixon on HADOOP-8449: Hi We found the changes in order of switch and guard block in {code} private InputStream forMagic(Path p, FileSystem srcFs) throws IOException {code} Because of this change, return value of {code} codec.createInputStream(i) {code} is changed if codec exists. {code} private InputStream forMagic(Path p, FileSystem srcFs) throws IOException { FSDataInputStream i = srcFs.open(p); // check codecs CompressionCodecFactory cf = new CompressionCodecFactory(getConf()); CompressionCodec codec = cf.getCodec(p); if (codec != null) { return codec.createInputStream(i); } switch(i.readShort()) { // cases } {code} New: {code} private InputStream forMagic(Path p, FileSystem srcFs) throws IOException { FSDataInputStream i = srcFs.open(p); switch(i.readShort()) { // === index (or pointer) processes!! // cases default: { // Check the type of compression instead, depending on Codec class's // own detection methods, based on the provided path. CompressionCodecFactory cf = new CompressionCodecFactory(getConf()); CompressionCodec codec = cf.getCodec(p); if (codec != null) { return codec.createInputStream(i); } break; } } // File is non-compressed, or not a file container we know. i.seek(0); return i; } {code} Fix is to use i.seek(0) before we use i anywhere. I missed that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8833) fs -text should make sure to call inputstream.seek(0) before using input stream
[ https://issues.apache.org/jira/browse/HADOOP-8833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom White updated HADOOP-8833: -- Attachment: HADOOP-8833.patch +1 on the fix. I noticed that the test doesn't fail without the fix though. This is because BZip2Codec.BZip2CompressionInputStream.readStreamHeader() tolerates a missing (two-byte) header, so BZip2 files happen to work anyway. I've modified the test slightly to test a deflate-compressed file, and this one does fail without the seek fix. fs -text should make sure to call inputstream.seek(0) before using input stream --- Key: HADOOP-8833 URL: https://issues.apache.org/jira/browse/HADOOP-8833 Project: Hadoop Common Issue Type: Bug Components: fs Affects Versions: 2.0.2-alpha Reporter: Harsh J Assignee: Harsh J Attachments: HADOOP-8833.patch, HADOOP-8833.patch From Muddy Dixon on HADOOP-8449: Hi We found the changes in order of switch and guard block in {code} private InputStream forMagic(Path p, FileSystem srcFs) throws IOException {code} Because of this change, return value of {code} codec.createInputStream(i) {code} is changed if codec exists. {code} private InputStream forMagic(Path p, FileSystem srcFs) throws IOException { FSDataInputStream i = srcFs.open(p); // check codecs CompressionCodecFactory cf = new CompressionCodecFactory(getConf()); CompressionCodec codec = cf.getCodec(p); if (codec != null) { return codec.createInputStream(i); } switch(i.readShort()) { // cases } {code} New: {code} private InputStream forMagic(Path p, FileSystem srcFs) throws IOException { FSDataInputStream i = srcFs.open(p); switch(i.readShort()) { // === index (or pointer) processes!! // cases default: { // Check the type of compression instead, depending on Codec class's // own detection methods, based on the provided path. CompressionCodecFactory cf = new CompressionCodecFactory(getConf()); CompressionCodec codec = cf.getCodec(p); if (codec != null) { return codec.createInputStream(i); } break; } } // File is non-compressed, or not a file container we know. i.seek(0); return i; } {code} Fix is to use i.seek(0) before we use i anywhere. I missed that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8833) fs -text should make sure to call inputstream.seek(0) before using input stream
[ https://issues.apache.org/jira/browse/HADOOP-8833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated HADOOP-8833: - Status: Patch Available (was: Open) fs -text should make sure to call inputstream.seek(0) before using input stream --- Key: HADOOP-8833 URL: https://issues.apache.org/jira/browse/HADOOP-8833 Project: Hadoop Common Issue Type: Bug Components: fs Affects Versions: 2.0.2-alpha Reporter: Harsh J Assignee: Harsh J Attachments: HADOOP-8833.patch, HADOOP-8833.patch From Muddy Dixon on HADOOP-8449: Hi We found the changes in order of switch and guard block in {code} private InputStream forMagic(Path p, FileSystem srcFs) throws IOException {code} Because of this change, return value of {code} codec.createInputStream(i) {code} is changed if codec exists. {code} private InputStream forMagic(Path p, FileSystem srcFs) throws IOException { FSDataInputStream i = srcFs.open(p); // check codecs CompressionCodecFactory cf = new CompressionCodecFactory(getConf()); CompressionCodec codec = cf.getCodec(p); if (codec != null) { return codec.createInputStream(i); } switch(i.readShort()) { // cases } {code} New: {code} private InputStream forMagic(Path p, FileSystem srcFs) throws IOException { FSDataInputStream i = srcFs.open(p); switch(i.readShort()) { // === index (or pointer) processes!! // cases default: { // Check the type of compression instead, depending on Codec class's // own detection methods, based on the provided path. CompressionCodecFactory cf = new CompressionCodecFactory(getConf()); CompressionCodec codec = cf.getCodec(p); if (codec != null) { return codec.createInputStream(i); } break; } } // File is non-compressed, or not a file container we know. i.seek(0); return i; } {code} Fix is to use i.seek(0) before we use i anywhere. I missed that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8833) fs -text should make sure to call inputstream.seek(0) before using input stream
[ https://issues.apache.org/jira/browse/HADOOP-8833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated HADOOP-8833: - Status: Open (was: Patch Available) Cancelling patch to re-submit and kick Jenkins fs -text should make sure to call inputstream.seek(0) before using input stream --- Key: HADOOP-8833 URL: https://issues.apache.org/jira/browse/HADOOP-8833 Project: Hadoop Common Issue Type: Bug Components: fs Affects Versions: 2.0.2-alpha Reporter: Harsh J Assignee: Harsh J Attachments: HADOOP-8833.patch, HADOOP-8833.patch From Muddy Dixon on HADOOP-8449: Hi We found the changes in order of switch and guard block in {code} private InputStream forMagic(Path p, FileSystem srcFs) throws IOException {code} Because of this change, return value of {code} codec.createInputStream(i) {code} is changed if codec exists. {code} private InputStream forMagic(Path p, FileSystem srcFs) throws IOException { FSDataInputStream i = srcFs.open(p); // check codecs CompressionCodecFactory cf = new CompressionCodecFactory(getConf()); CompressionCodec codec = cf.getCodec(p); if (codec != null) { return codec.createInputStream(i); } switch(i.readShort()) { // cases } {code} New: {code} private InputStream forMagic(Path p, FileSystem srcFs) throws IOException { FSDataInputStream i = srcFs.open(p); switch(i.readShort()) { // === index (or pointer) processes!! // cases default: { // Check the type of compression instead, depending on Codec class's // own detection methods, based on the provided path. CompressionCodecFactory cf = new CompressionCodecFactory(getConf()); CompressionCodec codec = cf.getCodec(p); if (codec != null) { return codec.createInputStream(i); } break; } } // File is non-compressed, or not a file container we know. i.seek(0); return i; } {code} Fix is to use i.seek(0) before we use i anywhere. I missed that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8833) fs -text should make sure to call inputstream.seek(0) before using input stream
[ https://issues.apache.org/jira/browse/HADOOP-8833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated HADOOP-8833: - Status: Open (was: Patch Available) fs -text should make sure to call inputstream.seek(0) before using input stream --- Key: HADOOP-8833 URL: https://issues.apache.org/jira/browse/HADOOP-8833 Project: Hadoop Common Issue Type: Bug Components: fs Affects Versions: 2.0.2-alpha Reporter: Harsh J Assignee: Harsh J Attachments: HADOOP-8833.patch, HADOOP-8833.patch, HADOOP-8833.patch From Muddy Dixon on HADOOP-8449: Hi We found the changes in order of switch and guard block in {code} private InputStream forMagic(Path p, FileSystem srcFs) throws IOException {code} Because of this change, return value of {code} codec.createInputStream(i) {code} is changed if codec exists. {code} private InputStream forMagic(Path p, FileSystem srcFs) throws IOException { FSDataInputStream i = srcFs.open(p); // check codecs CompressionCodecFactory cf = new CompressionCodecFactory(getConf()); CompressionCodec codec = cf.getCodec(p); if (codec != null) { return codec.createInputStream(i); } switch(i.readShort()) { // cases } {code} New: {code} private InputStream forMagic(Path p, FileSystem srcFs) throws IOException { FSDataInputStream i = srcFs.open(p); switch(i.readShort()) { // === index (or pointer) processes!! // cases default: { // Check the type of compression instead, depending on Codec class's // own detection methods, based on the provided path. CompressionCodecFactory cf = new CompressionCodecFactory(getConf()); CompressionCodec codec = cf.getCodec(p); if (codec != null) { return codec.createInputStream(i); } break; } } // File is non-compressed, or not a file container we know. i.seek(0); return i; } {code} Fix is to use i.seek(0) before we use i anywhere. I missed that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8833) fs -text should make sure to call inputstream.seek(0) before using input stream
[ https://issues.apache.org/jira/browse/HADOOP-8833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated HADOOP-8833: - Attachment: HADOOP-8833.patch Uploading the same patch again. fs -text should make sure to call inputstream.seek(0) before using input stream --- Key: HADOOP-8833 URL: https://issues.apache.org/jira/browse/HADOOP-8833 Project: Hadoop Common Issue Type: Bug Components: fs Affects Versions: 2.0.2-alpha Reporter: Harsh J Assignee: Harsh J Attachments: HADOOP-8833.patch, HADOOP-8833.patch, HADOOP-8833.patch From Muddy Dixon on HADOOP-8449: Hi We found the changes in order of switch and guard block in {code} private InputStream forMagic(Path p, FileSystem srcFs) throws IOException {code} Because of this change, return value of {code} codec.createInputStream(i) {code} is changed if codec exists. {code} private InputStream forMagic(Path p, FileSystem srcFs) throws IOException { FSDataInputStream i = srcFs.open(p); // check codecs CompressionCodecFactory cf = new CompressionCodecFactory(getConf()); CompressionCodec codec = cf.getCodec(p); if (codec != null) { return codec.createInputStream(i); } switch(i.readShort()) { // cases } {code} New: {code} private InputStream forMagic(Path p, FileSystem srcFs) throws IOException { FSDataInputStream i = srcFs.open(p); switch(i.readShort()) { // === index (or pointer) processes!! // cases default: { // Check the type of compression instead, depending on Codec class's // own detection methods, based on the provided path. CompressionCodecFactory cf = new CompressionCodecFactory(getConf()); CompressionCodec codec = cf.getCodec(p); if (codec != null) { return codec.createInputStream(i); } break; } } // File is non-compressed, or not a file container we know. i.seek(0); return i; } {code} Fix is to use i.seek(0) before we use i anywhere. I missed that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8833) fs -text should make sure to call inputstream.seek(0) before using input stream
[ https://issues.apache.org/jira/browse/HADOOP-8833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated HADOOP-8833: - Status: Patch Available (was: Open) fs -text should make sure to call inputstream.seek(0) before using input stream --- Key: HADOOP-8833 URL: https://issues.apache.org/jira/browse/HADOOP-8833 Project: Hadoop Common Issue Type: Bug Components: fs Affects Versions: 2.0.2-alpha Reporter: Harsh J Assignee: Harsh J Attachments: HADOOP-8833.patch, HADOOP-8833.patch, HADOOP-8833.patch From Muddy Dixon on HADOOP-8449: Hi We found the changes in order of switch and guard block in {code} private InputStream forMagic(Path p, FileSystem srcFs) throws IOException {code} Because of this change, return value of {code} codec.createInputStream(i) {code} is changed if codec exists. {code} private InputStream forMagic(Path p, FileSystem srcFs) throws IOException { FSDataInputStream i = srcFs.open(p); // check codecs CompressionCodecFactory cf = new CompressionCodecFactory(getConf()); CompressionCodec codec = cf.getCodec(p); if (codec != null) { return codec.createInputStream(i); } switch(i.readShort()) { // cases } {code} New: {code} private InputStream forMagic(Path p, FileSystem srcFs) throws IOException { FSDataInputStream i = srcFs.open(p); switch(i.readShort()) { // === index (or pointer) processes!! // cases default: { // Check the type of compression instead, depending on Codec class's // own detection methods, based on the provided path. CompressionCodecFactory cf = new CompressionCodecFactory(getConf()); CompressionCodec codec = cf.getCodec(p); if (codec != null) { return codec.createInputStream(i); } break; } } // File is non-compressed, or not a file container we know. i.seek(0); return i; } {code} Fix is to use i.seek(0) before we use i anywhere. I missed that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira