[jira] [Resolved] (HADOOP-5754) Add link to training from website
[ https://issues.apache.org/jira/browse/HADOOP-5754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom White resolved HADOOP-5754. --- Resolution: Fixed Add link to training from website - Key: HADOOP-5754 URL: https://issues.apache.org/jira/browse/HADOOP-5754 Project: Hadoop Common Issue Type: Improvement Components: documentation Reporter: Tom White Assignee: Tom White Fix For: site Attachments: HADOOP-5754.patch, hadoop-5754-v2.patch, hadoop-5754-v3.patch, hadoop-5754.patch Add a link to training videos from the getting started section (like Pig, see http://hadoop.apache.org/pig/). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HADOOP-5555) JobClient should provide an API to return the job names of jobs
[ https://issues.apache.org/jira/browse/HADOOP-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh J resolved HADOOP-. - Resolution: Not A Problem The JobClient provides both Job and RunningJob returns via some of its cluster-connecting methods, that in turn provide an API to retrieve the Job Name string already. Hence, this has already been fixed. For the 'hadoop job -list' enhancement to show the same, see MAPREDUCE-4424 instead (which I just forked out). Resolving as Not a Problem (anymore). JobClient should provide an API to return the job names of jobs --- Key: HADOOP- URL: https://issues.apache.org/jira/browse/HADOOP- Project: Hadoop Common Issue Type: Improvement Reporter: Runping Qi Currently, there seems to be no way to get the job name of a job from its job id. The JobClient should provide a way to do so. Also, the command line hadoop job -list should also return the job names. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HADOOP-6817) SequenceFile.Reader can't read gzip format compressed sequence file which produce by a mapreduce job without native compression library
[
https://issues.apache.org/jira/browse/HADOOP-6817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J resolved HADOOP-6817.
-
Resolution: Duplicate
This is being addressed via HADOOP-8582.
SequenceFile.Reader can't read gzip format compressed sequence file which
produce by a mapreduce job without native compression library
---
Key: HADOOP-6817
URL: https://issues.apache.org/jira/browse/HADOOP-6817
Project: Hadoop Common
Issue Type: Bug
Components: io
Affects Versions: 0.20.2
Environment: Cluster:CentOS 5,jdk1.6.0_20
Client:Mac SnowLeopard,jdk1.6.0_20
Reporter: Wenjun Huang
An hadoop job output a gzip compressed sequence file(whether record
compressed or block compressed).The client program use SequenceFile.Reader to
read this sequence file,when reading the client program shows the following
exceptions:
2090 [main] WARN org.apache.hadoop.util.NativeCodeLoader - Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
2091 [main] INFO org.apache.hadoop.io.compress.CodecPool - Got brand-new
decompressor
Exception in thread main java.io.EOFException
at java.util.zip.GZIPInputStream.readUByte(GZIPInputStream.java:207)
at java.util.zip.GZIPInputStream.readUShort(GZIPInputStream.java:197)
at java.util.zip.GZIPInputStream.readHeader(GZIPInputStream.java:136)
at java.util.zip.GZIPInputStream.init(GZIPInputStream.java:58)
at java.util.zip.GZIPInputStream.init(GZIPInputStream.java:68)
at
org.apache.hadoop.io.compress.GzipCodec$GzipInputStream$ResetableGZIPInputStream.init(GzipCodec.java:92)
at
org.apache.hadoop.io.compress.GzipCodec$GzipInputStream.init(GzipCodec.java:101)
at
org.apache.hadoop.io.compress.GzipCodec.createInputStream(GzipCodec.java:170)
at
org.apache.hadoop.io.compress.GzipCodec.createInputStream(GzipCodec.java:180)
at org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1520)
at
org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1428)
at
org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1417)
at
org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1412)
at
com.shiningware.intelligenceonline.taobao.mapreduce.HtmlContentSeqOutputView.main(HtmlContentSeqOutputView.java:28)
I studied the code in org.apache.hadoop.io.SequenceFile.Reader.init method
and read:
// Initialize... *not* if this we are constructing a temporary Reader
if (!tempReader) {
valBuffer = new DataInputBuffer();
if (decompress) {
valDecompressor = CodecPool.getDecompressor(codec);
valInFilter = codec.createInputStream(valBuffer, valDecompressor);
valIn = new DataInputStream(valInFilter);
} else {
valIn = valBuffer;
}
the problem seems to be caused by valBuffer = new DataInputBuffer();
,because GzipCodec.createInputStream creates an instance of GzipInputStream
whose constructor creates an instance of ResetableGZIPInputStream class.When
ResetableGZIPInputStream's constructor calls it base class
java.util.zip.GZIPInputStream's constructor ,it trys to read the empty
valBuffer = new DataInputBuffer(); and get no content,so it throws an
EOFException.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-8583) Globbing is not correctly handled in a few cases on Windows
Ramya Sunil created HADOOP-8583:
---
Summary: Globbing is not correctly handled in a few cases on
Windows
Key: HADOOP-8583
URL: https://issues.apache.org/jira/browse/HADOOP-8583
Project: Hadoop Common
Issue Type: Bug
Environment: Windows
Reporter: Ramya Sunil
Glob handling fails in a few cases on a Windows environment.
For example:
{noformat}
c:\ hadoop dfs -ls /
Found 2 items
drwxrwxrwx - Administrator supergroup 0 2012-07-06 15:00 /tmp
drwxr-xr-x - Administrator supergroup 0 2012-07-06 18:52 /user
c:\ hadoop dfs -ls /tmpInvalid*
Found 2 items
drwxr-xr-x - Administrator supergroup 0 2012-07-10 18:50
/user/Administrator/sortInputDir
drwxr-xr-x - Administrator supergroup 0 2012-07-10 18:50
/user/Administrator/sortOutputDir
c:\ hadoop dfs -rmr /tmp/*
Usage: java FsShell [-rmr [-skipTrash] src ]
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
Problem setting up 1st generation Hadoop-0.20 (ANT build) in Eclipse
Hi all, I am trying to setup hadoop 1st generation 0.20 in Eclipse which still uses Ant as its build tool. The build was successful , but when I want to set it up in the Eclipse IDE i.e *File- new Project- Project from existing ANT build file - Select build.xml -Finish* I get this following error : *Problem setting the classpath of the project from the javac classpath: Reference ivy-common.classpath not found.* I have tried finding solutions online but couldn't get a concrete one. Are there any sources or workarounds on setting 1st generation Hadoop in Eclipse.? Also my second question was how to build a binary tar file for hadoop-0.20 which still uses ANT. The wiki pages only have information for maven. Any help is highly appreciated.Thanks -- --With Regards Pavan Kulkarni
Re: Problem setting up 1st generation Hadoop-0.20 (ANT build) in Eclipse
Hey Pavan, The 0.20.x version series was renamed recently to 1.x. Hence, you need to use the branch-1 code path if you want the latest stable branch (MR1, etc.) code. Do these once you have ant 1.8 and a Sun/Oracle JDK 1.6 installed, and you should have it: $ git clone http://github.com/apache/hadoop-common.git hadoop-1 $ cd hadoop-1; git checkout branch-1 $ ant eclipse (Now export this directory into Eclipse as a Java project) HTH. On Wed, Jul 11, 2012 at 12:00 AM, Pavan Kulkarni pavan.babu...@gmail.com wrote: Hi all, I am trying to setup hadoop 1st generation 0.20 in Eclipse which still uses Ant as its build tool. The build was successful , but when I want to set it up in the Eclipse IDE i.e *File- new Project- Project from existing ANT build file - Select build.xml -Finish* I get this following error : *Problem setting the classpath of the project from the javac classpath: Reference ivy-common.classpath not found.* I have tried finding solutions online but couldn't get a concrete one. Are there any sources or workarounds on setting 1st generation Hadoop in Eclipse.? Also my second question was how to build a binary tar file for hadoop-0.20 which still uses ANT. The wiki pages only have information for maven. Any help is highly appreciated.Thanks -- --With Regards Pavan Kulkarni -- Harsh J
[jira] [Created] (HADOOP-8584) test-patch.sh should not immediately exit when no tests are added or modified
Colin Patrick McCabe created HADOOP-8584: Summary: test-patch.sh should not immediately exit when no tests are added or modified Key: HADOOP-8584 URL: https://issues.apache.org/jira/browse/HADOOP-8584 Project: Hadoop Common Issue Type: Bug Reporter: Colin Patrick McCabe Assignee: Colin Patrick McCabe test-patch.sh should not immediately exit when no tests are added or modified. Although it's good to note whether or not a patch introduces or modifies tests, it's not good to abort the Jenkins patch process if it did not. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Problem setting up 1st generation Hadoop-0.20 (ANT build) in Eclipse
Thanks a lot Harsh.I could set it up without any errors. It would be great if you could provide me any pointers on how to build a binary distribution tar file. The information on wiki and in BUILDING.txt only has Maven instructions.Thanks On Tue, Jul 10, 2012 at 2:39 PM, Harsh J ha...@cloudera.com wrote: Hey Pavan, The 0.20.x version series was renamed recently to 1.x. Hence, you need to use the branch-1 code path if you want the latest stable branch (MR1, etc.) code. Do these once you have ant 1.8 and a Sun/Oracle JDK 1.6 installed, and you should have it: $ git clone http://github.com/apache/hadoop-common.git hadoop-1 $ cd hadoop-1; git checkout branch-1 $ ant eclipse (Now export this directory into Eclipse as a Java project) HTH. On Wed, Jul 11, 2012 at 12:00 AM, Pavan Kulkarni pavan.babu...@gmail.com wrote: Hi all, I am trying to setup hadoop 1st generation 0.20 in Eclipse which still uses Ant as its build tool. The build was successful , but when I want to set it up in the Eclipse IDE i.e *File- new Project- Project from existing ANT build file - Select build.xml -Finish* I get this following error : *Problem setting the classpath of the project from the javac classpath: Reference ivy-common.classpath not found.* I have tried finding solutions online but couldn't get a concrete one. Are there any sources or workarounds on setting 1st generation Hadoop in Eclipse.? Also my second question was how to build a binary tar file for hadoop-0.20 which still uses ANT. The wiki pages only have information for maven. Any help is highly appreciated.Thanks -- --With Regards Pavan Kulkarni -- Harsh J -- --With Regards Pavan Kulkarni
Re: Problem setting up 1st generation Hadoop-0.20 (ANT build) in Eclipse
Hey Pavan, Try an ant tar. For more ant targets, read the build.xml at the root of your checkout. On Wed, Jul 11, 2012 at 1:15 AM, Pavan Kulkarni pavan.babu...@gmail.com wrote: Thanks a lot Harsh.I could set it up without any errors. It would be great if you could provide me any pointers on how to build a binary distribution tar file. The information on wiki and in BUILDING.txt only has Maven instructions.Thanks On Tue, Jul 10, 2012 at 2:39 PM, Harsh J ha...@cloudera.com wrote: Hey Pavan, The 0.20.x version series was renamed recently to 1.x. Hence, you need to use the branch-1 code path if you want the latest stable branch (MR1, etc.) code. Do these once you have ant 1.8 and a Sun/Oracle JDK 1.6 installed, and you should have it: $ git clone http://github.com/apache/hadoop-common.git hadoop-1 $ cd hadoop-1; git checkout branch-1 $ ant eclipse (Now export this directory into Eclipse as a Java project) HTH. On Wed, Jul 11, 2012 at 12:00 AM, Pavan Kulkarni pavan.babu...@gmail.com wrote: Hi all, I am trying to setup hadoop 1st generation 0.20 in Eclipse which still uses Ant as its build tool. The build was successful , but when I want to set it up in the Eclipse IDE i.e *File- new Project- Project from existing ANT build file - Select build.xml -Finish* I get this following error : *Problem setting the classpath of the project from the javac classpath: Reference ivy-common.classpath not found.* I have tried finding solutions online but couldn't get a concrete one. Are there any sources or workarounds on setting 1st generation Hadoop in Eclipse.? Also my second question was how to build a binary tar file for hadoop-0.20 which still uses ANT. The wiki pages only have information for maven. Any help is highly appreciated.Thanks -- --With Regards Pavan Kulkarni -- Harsh J -- --With Regards Pavan Kulkarni -- Harsh J
[jira] [Resolved] (HADOOP-8584) test-patch.sh should not immediately exit when no tests are added or modified
[ https://issues.apache.org/jira/browse/HADOOP-8584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Collins resolved HADOOP-8584. - Resolution: Fixed Fix Version/s: 3.0.0 Hadoop Flags: Reviewed I've committed this to trunk. test-patch.sh should not immediately exit when no tests are added or modified - Key: HADOOP-8584 URL: https://issues.apache.org/jira/browse/HADOOP-8584 Project: Hadoop Common Issue Type: Bug Affects Versions: 2.0.0-alpha, 3.0.0 Reporter: Colin Patrick McCabe Assignee: Colin Patrick McCabe Fix For: 3.0.0 Attachments: HADOOP-8584.001.patch test-patch.sh should not immediately exit when no tests are added or modified. Although it's good to note whether or not a patch introduces or modifies tests, it's not good to abort the Jenkins patch process if it did not. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Problem setting up 1st generation Hadoop-0.20 (ANT build) in Eclipse
I tried *ant tar* but it requested a forrest home directory ,so I ran *ant -Dforrest.home=/path/apache-forrest-0.8 compile-core tar* but this gets stuck at an Exception Exception in thread main java.lang.InternalError: Can't connect to X11 window server using ':0' as the value of the DISPLAY variable. No idea what this exception means. How come there isn't a good documentation or a BUILDING.txt file for MR1 releases ? Any help regarding this is appreciated.Thanks On Tue, Jul 10, 2012 at 4:29 PM, Harsh J ha...@cloudera.com wrote: Hey Pavan, Try an ant tar. For more ant targets, read the build.xml at the root of your checkout. On Wed, Jul 11, 2012 at 1:15 AM, Pavan Kulkarni pavan.babu...@gmail.com wrote: Thanks a lot Harsh.I could set it up without any errors. It would be great if you could provide me any pointers on how to build a binary distribution tar file. The information on wiki and in BUILDING.txt only has Maven instructions.Thanks On Tue, Jul 10, 2012 at 2:39 PM, Harsh J ha...@cloudera.com wrote: Hey Pavan, The 0.20.x version series was renamed recently to 1.x. Hence, you need to use the branch-1 code path if you want the latest stable branch (MR1, etc.) code. Do these once you have ant 1.8 and a Sun/Oracle JDK 1.6 installed, and you should have it: $ git clone http://github.com/apache/hadoop-common.git hadoop-1 $ cd hadoop-1; git checkout branch-1 $ ant eclipse (Now export this directory into Eclipse as a Java project) HTH. On Wed, Jul 11, 2012 at 12:00 AM, Pavan Kulkarni pavan.babu...@gmail.com wrote: Hi all, I am trying to setup hadoop 1st generation 0.20 in Eclipse which still uses Ant as its build tool. The build was successful , but when I want to set it up in the Eclipse IDE i.e *File- new Project- Project from existing ANT build file - Select build.xml -Finish* I get this following error : *Problem setting the classpath of the project from the javac classpath: Reference ivy-common.classpath not found.* I have tried finding solutions online but couldn't get a concrete one. Are there any sources or workarounds on setting 1st generation Hadoop in Eclipse.? Also my second question was how to build a binary tar file for hadoop-0.20 which still uses ANT. The wiki pages only have information for maven. Any help is highly appreciated.Thanks -- --With Regards Pavan Kulkarni -- Harsh J -- --With Regards Pavan Kulkarni -- Harsh J -- --With Regards Pavan Kulkarni
Re: Jetty fixes for Hadoop
I'm +1 for adding it. Tom On 7/9/12 6:22 PM, Suresh Srinivas sur...@hortonworks.com wrote: Many folks running Apache Hadoop releases run into Jetty related issues. Some fixes to Jetty alleviates this problem. See Todd's comment: See https://issues.apache.org/jira/browse/MAPREDUCE-2980?focusedCommentId=13291112 page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment -13291112 I propose making this available to Apache Hadoop by: # Hosting the code in shared github repository and publishing the artifacts built to maven central # Update the Apache Hadoop to use the Jetty jars with fixes. Feedback welcome. Regards, Suresh
Re: Jetty fixes for Hadoop
On Tue, Jul 10, 2012 at 2:59 PM, Thomas Graves tgra...@yahoo-inc.comwrote: I'm +1 for adding it. I'm +1 also. -- Owen
Re: Jetty fixes for Hadoop
+1 from me too. We've had this in CDH since Sep '11 and been working much better than the stock 6.1.26. -Todd On Tue, Jul 10, 2012 at 3:14 PM, Owen O'Malley omal...@apache.org wrote: On Tue, Jul 10, 2012 at 2:59 PM, Thomas Graves tgra...@yahoo-inc.comwrote: I'm +1 for adding it. I'm +1 also. -- Owen -- Todd Lipcon Software Engineer, Cloudera
[jira] [Created] (HADOOP-8587) HarFileSystem access of harMetaCache isn't threadsafe
Eli Collins created HADOOP-8587: --- Summary: HarFileSystem access of harMetaCache isn't threadsafe Key: HADOOP-8587 URL: https://issues.apache.org/jira/browse/HADOOP-8587 Project: Hadoop Common Issue Type: Bug Affects Versions: 2.0.0-alpha Reporter: Eli Collins HarFileSystem's use of the static harMetaCache map is not threadsafe. Credit to Todd for pointing this out. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
