[jira] [Created] (HDFS-5283) NN not coming out of startup safemode due to under construction blocks only inside snapshots also counted in safemode threshhold
Vinay created HDFS-5283: --- Summary: NN not coming out of startup safemode due to under construction blocks only inside snapshots also counted in safemode threshhold Key: HDFS-5283 URL: https://issues.apache.org/jira/browse/HDFS-5283 Project: Hadoop HDFS Issue Type: Bug Components: snapshots Affects Versions: 2.1.1-beta, 3.0.0 Reporter: Vinay Assignee: Vinay Priority: Blocker This is observed in one of our env: 1. A MR Job was running which has created some temporary files and was writing to them. 2. Snapshot was taken 3. And Job was killed and temporary files were deleted. 4. Namenode restarted. 5. After restart Namenode was in safemode waiting for blocks Analysis - 1. Since the snapshot taken also includes the temporary files which were open, and later original files are deleted. 2. UnderConstruction blocks count was taken from leases. not considered the UC blocks only inside snapshots 3. So safemode threshold count was more and NN did not come out of safemode -- This message was sent by Atlassian JIRA (v6.1#6144)
Build failed in Jenkins: Hadoop-Hdfs-0.23-Build #747
See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/747/ -- [...truncated 7677 lines...] [ERROR] symbol : class Parser [ERROR] location: package com.google.protobuf [ERROR] https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/ws/trunk/hadoop-hdfs-project/hadoop-hdfs/target/generated-sources/java/org/apache/hadoop/hdfs/protocol/proto/DataTransferProtos.java:[270,37] cannot find symbol [ERROR] symbol : class Parser [ERROR] location: package com.google.protobuf [ERROR] https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/ws/trunk/hadoop-hdfs-project/hadoop-hdfs/target/generated-sources/java/org/apache/hadoop/hdfs/protocol/proto/DataTransferProtos.java:[281,30] cannot find symbol [ERROR] symbol : class Parser [ERROR] location: package com.google.protobuf [ERROR] https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/ws/trunk/hadoop-hdfs-project/hadoop-hdfs/target/generated-sources/java/org/apache/hadoop/hdfs/protocol/proto/DataTransferProtos.java:[10533,37] cannot find symbol [ERROR] symbol : class Parser [ERROR] location: package com.google.protobuf [ERROR] https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/ws/trunk/hadoop-hdfs-project/hadoop-hdfs/target/generated-sources/java/org/apache/hadoop/hdfs/protocol/proto/DataTransferProtos.java:[10544,30] cannot find symbol [ERROR] symbol : class Parser [ERROR] location: package com.google.protobuf [ERROR] https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/ws/trunk/hadoop-hdfs-project/hadoop-hdfs/target/generated-sources/java/org/apache/hadoop/hdfs/protocol/proto/DataTransferProtos.java:[8357,37] cannot find symbol [ERROR] symbol : class Parser [ERROR] location: package com.google.protobuf [ERROR] https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/ws/trunk/hadoop-hdfs-project/hadoop-hdfs/target/generated-sources/java/org/apache/hadoop/hdfs/protocol/proto/DataTransferProtos.java:[8368,30] cannot find symbol [ERROR] symbol : class Parser [ERROR] location: package com.google.protobuf [ERROR] https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/ws/trunk/hadoop-hdfs-project/hadoop-hdfs/target/generated-sources/java/org/apache/hadoop/hdfs/protocol/proto/DataTransferProtos.java:[12641,37] cannot find symbol [ERROR] symbol : class Parser [ERROR] location: package com.google.protobuf [ERROR] https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/ws/trunk/hadoop-hdfs-project/hadoop-hdfs/target/generated-sources/java/org/apache/hadoop/hdfs/protocol/proto/DataTransferProtos.java:[12652,30] cannot find symbol [ERROR] symbol : class Parser [ERROR] location: package com.google.protobuf [ERROR] https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/ws/trunk/hadoop-hdfs-project/hadoop-hdfs/target/generated-sources/java/org/apache/hadoop/hdfs/protocol/proto/DataTransferProtos.java:[9741,37] cannot find symbol [ERROR] symbol : class Parser [ERROR] location: package com.google.protobuf [ERROR] https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/ws/trunk/hadoop-hdfs-project/hadoop-hdfs/target/generated-sources/java/org/apache/hadoop/hdfs/protocol/proto/DataTransferProtos.java:[9752,30] cannot find symbol [ERROR] symbol : class Parser [ERROR] location: package com.google.protobuf [ERROR] https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/ws/trunk/hadoop-hdfs-project/hadoop-hdfs/target/generated-sources/java/org/apache/hadoop/hdfs/protocol/proto/DataTransferProtos.java:[1781,37] cannot find symbol [ERROR] symbol : class Parser [ERROR] location: package com.google.protobuf [ERROR] https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/ws/trunk/hadoop-hdfs-project/hadoop-hdfs/target/generated-sources/java/org/apache/hadoop/hdfs/protocol/proto/DataTransferProtos.java:[1792,30] cannot find symbol [ERROR] symbol : class Parser [ERROR] location: package com.google.protobuf [ERROR] https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/ws/trunk/hadoop-hdfs-project/hadoop-hdfs/target/generated-sources/java/org/apache/hadoop/hdfs/protocol/proto/DataTransferProtos.java:[5338,37] cannot find symbol [ERROR] symbol : class Parser [ERROR] location: package com.google.protobuf [ERROR] https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/ws/trunk/hadoop-hdfs-project/hadoop-hdfs/target/generated-sources/java/org/apache/hadoop/hdfs/protocol/proto/DataTransferProtos.java:[5349,30] cannot find symbol [ERROR] symbol : class Parser [ERROR] location: package com.google.protobuf [ERROR] https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/ws/trunk/hadoop-hdfs-project/hadoop-hdfs/target/generated-sources/java/org/apache/hadoop/hdfs/protocol/proto/DataTransferProtos.java:[6290,37] cannot find symbol [ERROR] symbol : class Parser [ERROR] location: package com.google.protobuf [ERROR] https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/ws/trunk/hadoop-hdfs-project/hadoop-hdfs/target/generated-sources/java/org/apache/hadoop/hdfs/protocol/proto/DataTransferProtos.java:[6301,30] cannot find symbol [ERROR] symbol : class
Hadoop-Hdfs-0.23-Build - Build # 747 - Still Failing
See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/747/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 7870 lines...] [ERROR] location: class com.google.protobuf.InvalidProtocolBufferException [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Hdfs-0.23-Build/trunk/hadoop-hdfs-project/hadoop-hdfs/target/generated-sources/java/org/apache/hadoop/hdfs/protocol/proto/DataTransferProtos.java:[3313,27] cannot find symbol [ERROR] symbol : method setUnfinishedMessage(org.apache.hadoop.hdfs.protocol.proto.DataTransferProtos.OpWriteBlockProto) [ERROR] location: class com.google.protobuf.InvalidProtocolBufferException [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Hdfs-0.23-Build/trunk/hadoop-hdfs-project/hadoop-hdfs/target/generated-sources/java/org/apache/hadoop/hdfs/protocol/proto/DataTransferProtos.java:[3319,8] cannot find symbol [ERROR] symbol : method makeExtensionsImmutable() [ERROR] location: class org.apache.hadoop.hdfs.protocol.proto.DataTransferProtos.OpWriteBlockProto [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Hdfs-0.23-Build/trunk/hadoop-hdfs-project/hadoop-hdfs/target/generated-sources/java/org/apache/hadoop/hdfs/protocol/proto/DataTransferProtos.java:[3330,10] cannot find symbol [ERROR] symbol : method ensureFieldAccessorsInitialized(java.lang.Classorg.apache.hadoop.hdfs.protocol.proto.DataTransferProtos.OpWriteBlockProto,java.lang.Classorg.apache.hadoop.hdfs.protocol.proto.DataTransferProtos.OpWriteBlockProto.Builder) [ERROR] location: class com.google.protobuf.GeneratedMessage.FieldAccessorTable [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Hdfs-0.23-Build/trunk/hadoop-hdfs-project/hadoop-hdfs/target/generated-sources/java/org/apache/hadoop/hdfs/protocol/proto/DataTransferProtos.java:[3335,31] cannot find symbol [ERROR] symbol : class AbstractParser [ERROR] location: package com.google.protobuf [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Hdfs-0.23-Build/trunk/hadoop-hdfs-project/hadoop-hdfs/target/generated-sources/java/org/apache/hadoop/hdfs/protocol/proto/DataTransferProtos.java:[3344,4] method does not override or implement a method from a supertype [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Hdfs-0.23-Build/trunk/hadoop-hdfs-project/hadoop-hdfs/target/generated-sources/java/org/apache/hadoop/hdfs/protocol/proto/DataTransferProtos.java:[4098,12] cannot find symbol [ERROR] symbol : method ensureFieldAccessorsInitialized(java.lang.Classorg.apache.hadoop.hdfs.protocol.proto.DataTransferProtos.OpWriteBlockProto,java.lang.Classorg.apache.hadoop.hdfs.protocol.proto.DataTransferProtos.OpWriteBlockProto.Builder) [ERROR] location: class com.google.protobuf.GeneratedMessage.FieldAccessorTable [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Hdfs-0.23-Build/trunk/hadoop-hdfs-project/hadoop-hdfs/target/generated-sources/java/org/apache/hadoop/hdfs/protocol/proto/DataTransferProtos.java:[4371,104] cannot find symbol [ERROR] symbol : method getUnfinishedMessage() [ERROR] location: class com.google.protobuf.InvalidProtocolBufferException [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Hdfs-0.23-Build/trunk/hadoop-hdfs-project/hadoop-hdfs/target/generated-sources/java/org/apache/hadoop/hdfs/protocol/proto/DataTransferProtos.java:[5264,8] getUnknownFields() in org.apache.hadoop.hdfs.protocol.proto.DataTransferProtos.OpTransferBlockProto cannot override getUnknownFields() in com.google.protobuf.GeneratedMessage; overridden method is final [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Hdfs-0.23-Build/trunk/hadoop-hdfs-project/hadoop-hdfs/target/generated-sources/java/org/apache/hadoop/hdfs/protocol/proto/DataTransferProtos.java:[5284,19] cannot find symbol [ERROR] symbol : method parseUnknownField(com.google.protobuf.CodedInputStream,com.google.protobuf.UnknownFieldSet.Builder,com.google.protobuf.ExtensionRegistryLite,int) [ERROR] location: class org.apache.hadoop.hdfs.protocol.proto.DataTransferProtos.OpTransferBlockProto [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Hdfs-0.23-Build/trunk/hadoop-hdfs-project/hadoop-hdfs/target/generated-sources/java/org/apache/hadoop/hdfs/protocol/proto/DataTransferProtos.java:[5314,15] cannot find symbol [ERROR] symbol : method setUnfinishedMessage(org.apache.hadoop.hdfs.protocol.proto.DataTransferProtos.OpTransferBlockProto) [ERROR] location: class com.google.protobuf.InvalidProtocolBufferException [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Hdfs-0.23-Build/trunk/hadoop-hdfs-project/hadoop-hdfs/target/generated-sources/java/org/apache/hadoop/hdfs/protocol/proto/DataTransferProtos.java:[5317,27] cannot find symbol [ERROR] symbol : method setUnfinishedMessage(org.apache.hadoop.hdfs.protocol.proto.DataTransferProtos.OpTransferBlockProto) [ERROR] location: class com.google.protobuf.InvalidProtocolBufferException [ERROR]
Re: [jira] [Created] (HDFS-5282) Support built in resolution of /~ to user's home directory: /user/{user.name}
This is especially useful for the concept of principal mapping within Knox.
A user that authenticates as foo may be mapped to a principal of bar.
Consequently, a script that logs in then accesses files within their home
directory should be able to do so relative to their home directory.
Without supporting the notion of ~ the user must know what their mapped
to principal is and hardcode file access based on that value.
Which may also change out from under them - or be different across clusters.
On Mon, Sep 30, 2013 at 11:54 PM, Kevin Minder (JIRA) j...@apache.orgwrote:
Kevin Minder created HDFS-5282:
--
Summary: Support built in resolution of /~ to user's home
directory: /user/{user.name}
Key: HDFS-5282
URL: https://issues.apache.org/jira/browse/HDFS-5282
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Kevin Minder
In many cases it would be very convenient for HDFS (and WebHDFS in
particular) to support the Unix notion of /~ representing the user's home
directory. This would allow for some scripts reusable because they would
not contain hard coded home directories.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Hadoop-Hdfs-trunk - Build # 1539 - Failure
See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1539/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 12419 lines...] [mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Hadoop-Hdfs-trunk/trunk/hadoop-hdfs-project/target/test-dir [INFO] Executed tasks [INFO] [INFO] --- maven-source-plugin:2.1.2:jar-no-fork (hadoop-java-sources) @ hadoop-hdfs-project --- [INFO] [INFO] --- maven-source-plugin:2.1.2:test-jar-no-fork (hadoop-java-sources) @ hadoop-hdfs-project --- [INFO] [INFO] --- maven-enforcer-plugin:1.3.1:enforce (dist-enforce) @ hadoop-hdfs-project --- [INFO] [INFO] --- maven-site-plugin:3.0:attach-descriptor (attach-descriptor) @ hadoop-hdfs-project --- [INFO] [INFO] --- maven-javadoc-plugin:2.8.1:jar (module-javadocs) @ hadoop-hdfs-project --- [INFO] Not executing Javadoc as the project is not a Java classpath-capable package [INFO] [INFO] --- maven-enforcer-plugin:1.3.1:enforce (depcheck) @ hadoop-hdfs-project --- [INFO] [INFO] --- maven-checkstyle-plugin:2.6:checkstyle (default-cli) @ hadoop-hdfs-project --- [INFO] [INFO] --- findbugs-maven-plugin:2.3.2:findbugs (default-cli) @ hadoop-hdfs-project --- [INFO] ** FindBugsMojo execute *** [INFO] canGenerate is false [INFO] [INFO] Reactor Summary: [INFO] [INFO] Apache Hadoop HDFS FAILURE [1:43:45.864s] [INFO] Apache Hadoop HttpFS .. SKIPPED [INFO] Apache Hadoop HDFS BookKeeper Journal . SKIPPED [INFO] Apache Hadoop HDFS-NFS SKIPPED [INFO] Apache Hadoop HDFS Project SUCCESS [2.699s] [INFO] [INFO] BUILD FAILURE [INFO] [INFO] Total time: 1:43:49.464s [INFO] Finished at: Tue Oct 01 13:17:41 UTC 2013 [INFO] Final Memory: 35M/402M [INFO] [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.16:test (default-test) on project hadoop-hdfs: There are test failures. [ERROR] [ERROR] Please refer to /home/jenkins/jenkins-slave/workspace/Hadoop-Hdfs-trunk/trunk/hadoop-hdfs-project/hadoop-hdfs/target/surefire-reports for the individual test results. [ERROR] - [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException Build step 'Execute shell' marked build as failure Archiving artifacts Updating HADOOP-9964 Updating YARN-1070 Updating MAPREDUCE-5551 Updating HDFS-4517 Updating HADOOP-1 Updating YARN-1221 Updating YARN-1247 Updating HDFS-5230 Sending e-mails to: hdfs-dev@hadoop.apache.org Email was triggered for: Failure Sending email for trigger: Failure ### ## FAILED TESTS (if any) ## No tests ran.
Build failed in Jenkins: Hadoop-Hdfs-trunk #1539
See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1539/changes Changes: [acmurthy] MAPREDUCE-5551. Fix compat with hadoop-1 in SequenceFileAsBinaryOutputFormat.WritableValueBytes by re-introducing missing constructors. Contributed by Zhijie Shen. [vinodkv] YARN-1070. Fixed race conditions in NodeManager during container-kill. Contributed by Zhijie Shen. [tucu] YARN-1247. test-container-executor has gotten out of sync with the changes to container-executor. (rvs via tucu) [kihwal] HDFS-4517. Cover class RemoteBlockReader with unit tests. Contributed by Vadim Bondarev and Dennis Y. [sandy] YARN-1221. With Fair Scheduler, reserved MB reported in RM web UI increases indefinitely (Siqi Li via Sandy Ryza) [brandonli] HDFS-5230. Introduce RpcInfo to decouple XDR classes from the RPC API. Contributed by Haohui Mai [llu] HADOOP-9964. Fix deadlocks in TestHttpServer by synchronize ReflectionUtils.printThreadInfo. (Junping Du via llu) -- [...truncated 12226 lines...] Running org.apache.hadoop.hdfs.TestParallelShortCircuitRead Tests run: 4, Failures: 0, Errors: 0, Skipped: 4, Time elapsed: 0.163 sec - in org.apache.hadoop.hdfs.TestParallelShortCircuitRead Running org.apache.hadoop.hdfs.TestDFSStorageStateRecovery Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 353.686 sec - in org.apache.hadoop.hdfs.TestDFSStorageStateRecovery Running org.apache.hadoop.hdfs.TestFileCreationEmpty Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.29 sec - in org.apache.hadoop.hdfs.TestFileCreationEmpty Running org.apache.hadoop.hdfs.TestSetrepIncreasing Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 26.737 sec - in org.apache.hadoop.hdfs.TestSetrepIncreasing Running org.apache.hadoop.hdfs.TestEncryptedTransfer Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 79.191 sec - in org.apache.hadoop.hdfs.TestEncryptedTransfer Running org.apache.hadoop.hdfs.TestDFSUpgrade Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 34.026 sec - in org.apache.hadoop.hdfs.TestDFSUpgrade Running org.apache.hadoop.hdfs.TestCrcCorruption Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 34.846 sec - in org.apache.hadoop.hdfs.TestCrcCorruption Running org.apache.hadoop.hdfs.TestHFlush Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 26.303 sec - in org.apache.hadoop.hdfs.TestHFlush Running org.apache.hadoop.hdfs.TestFileAppendRestart Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.77 sec - in org.apache.hadoop.hdfs.TestFileAppendRestart Running org.apache.hadoop.hdfs.TestDatanodeReport Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 19.545 sec - in org.apache.hadoop.hdfs.TestDatanodeReport Running org.apache.hadoop.hdfs.TestShortCircuitLocalRead Tests run: 10, Failures: 0, Errors: 0, Skipped: 10, Time elapsed: 0.195 sec - in org.apache.hadoop.hdfs.TestShortCircuitLocalRead Running org.apache.hadoop.hdfs.TestFileInputStreamCache Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.205 sec - in org.apache.hadoop.hdfs.TestFileInputStreamCache Running org.apache.hadoop.hdfs.TestRestartDFS Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.449 sec - in org.apache.hadoop.hdfs.TestRestartDFS Running org.apache.hadoop.hdfs.TestDFSUpgradeFromImage Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 14.454 sec - in org.apache.hadoop.hdfs.TestDFSUpgradeFromImage Running org.apache.hadoop.hdfs.TestDFSRemove Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 14.334 sec - in org.apache.hadoop.hdfs.TestDFSRemove Running org.apache.hadoop.hdfs.TestHDFSTrash Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.012 sec - in org.apache.hadoop.hdfs.TestHDFSTrash Running org.apache.hadoop.hdfs.TestClientReportBadBlock Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 70.405 sec - in org.apache.hadoop.hdfs.TestClientReportBadBlock Running org.apache.hadoop.hdfs.TestQuota Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.564 sec - in org.apache.hadoop.hdfs.TestQuota Running org.apache.hadoop.hdfs.TestFileLengthOnClusterRestart Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.345 sec - in org.apache.hadoop.hdfs.TestFileLengthOnClusterRestart Running org.apache.hadoop.hdfs.TestDatanodeRegistration Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.075 sec - in org.apache.hadoop.hdfs.TestDatanodeRegistration Running org.apache.hadoop.hdfs.TestAbandonBlock Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.135 sec - in org.apache.hadoop.hdfs.TestAbandonBlock Running org.apache.hadoop.hdfs.TestDFSShell Tests run: 23, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 40.509 sec - in org.apache.hadoop.hdfs.TestDFSShell Running org.apache.hadoop.hdfs.TestListFilesInDFS Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time
[jira] [Created] (HDFS-5284) Flatten INode hierarchy
Tsz Wo (Nicholas), SZE created HDFS-5284: Summary: Flatten INode hierarchy Key: HDFS-5284 URL: https://issues.apache.org/jira/browse/HDFS-5284 Project: Hadoop HDFS Issue Type: Improvement Components: namenode Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Currently, we have a complicated inode hierarchy for representing different states of a file or a directory. For example, when a file is being created, it is represented by an INodeFileUnderConstruction. When a file is being closed, the inode is replaced by an INodeFile. If it is reopened for append, the inode is replaced again by an INodeFileUnderConstruction. This JIRA is to flatten the inode hierarchy. We may also improve the performance by eliminating the inode replacement in runtime. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Created] (HDFS-5286) Flatten INodeDirectory hierarchy
Tsz Wo (Nicholas), SZE created HDFS-5286: Summary: Flatten INodeDirectory hierarchy Key: HDFS-5286 URL: https://issues.apache.org/jira/browse/HDFS-5286 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Similar to the case of INodeFile (HFDS-5285), we should also flatten the INodeDirectory hierarchy. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Created] (HDFS-5285) Flatten INodeFile hierarchy
Tsz Wo (Nicholas), SZE created HDFS-5285: Summary: Flatten INodeFile hierarchy Key: HDFS-5285 URL: https://issues.apache.org/jira/browse/HDFS-5285 Project: Hadoop HDFS Issue Type: Sub-task Components: namenode Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE For files, there are INodeFile, INodeFileUnderConstruction, INodeFileWithSnapshot and INodeFileUnderConstructionWithSnapshot for representing whether a file is under construction or whether it is in some snapshot. The following are two major problems of the current approach: - Java class does not support multiple inheritances so that INodeFileUnderConstructionWithSnapshot cannot extend both INodeFileUnderConstruction and INodeFileWithSnapshot. - The number of classes is exponential to the number of features. Currently, there are only two features, UnderConstruction and WithSnapshot. The number of classes is 2^2 = 4. It is hard to add one more feature since the number of classes will become 2^3 = 8. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Resolved] (HDFS-3983) Hftp should support both SPNEGO and KSSL
[
https://issues.apache.org/jira/browse/HDFS-3983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Owen O'Malley resolved HDFS-3983.
-
Resolution: Won't Fix
Target Version/s: (was: )
KSSL is deprecated and should never be used for secure deployments.
Hftp should support both SPNEGO and KSSL
Key: HDFS-3983
URL: https://issues.apache.org/jira/browse/HDFS-3983
Project: Hadoop HDFS
Issue Type: Bug
Components: security
Affects Versions: 2.0.0-alpha
Reporter: Eli Collins
Assignee: Eli Collins
Priority: Blocker
Attachments: hdfs-3983.txt, hdfs-3983.txt
Hftp currently doesn't work against a secure cluster unless you configure
{{dfs.https.port}} to be the http port, otherwise the client can't fetch
tokens:
{noformat}
$ hadoop fs -ls hftp://c1225.hal.cloudera.com:50070/
12/09/26 18:02:00 INFO fs.FileSystem: Couldn't get a delegation token from
http://c1225.hal.cloudera.com:50470 using http.
ls: Security enabled but user not authenticated by filter
{noformat}
This is due to Hftp still using the https port. Post HDFS-2617 it should use
the regular http port. Hsftp should still use the secure port, however now
that we have HADOOP-8581 it's worth considering removing Hsftp entirely. I'll
start a separate thread about that.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Resolved] (HDFS-3699) HftpFileSystem should try both KSSL and SPNEGO when authentication is required
[ https://issues.apache.org/jira/browse/HDFS-3699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley resolved HDFS-3699. - Resolution: Won't Fix Using KSSL is strongly deprecated and should be avoided in secure clusters. HftpFileSystem should try both KSSL and SPNEGO when authentication is required -- Key: HDFS-3699 URL: https://issues.apache.org/jira/browse/HDFS-3699 Project: Hadoop HDFS Issue Type: Sub-task Reporter: eric baldeschwieler See discussion in HDFS-2617 (Replaced Kerberized SSL for image transfer and fsck with SPNEGO-based solution). To handle the transition from Hadoop1.0 systems running KSSL authentication to Hadoop systems running SPNEGO, it would be good to fix the client in both 1 and 2 to try SPNEGO and then fall back to try KSSL. This will allow organizations that are running a lot of Hadoop 1.0 to gradually transition over, without needing to convert all clusters at the same time. They would first need to update their 1.0 HFTP clients (and 2.0/0.23 if they are already running those) and then they could copy data between clusters without needing to move all clusters to SPNEGO in a big bang. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Created] (HDFS-5287) JN need not validate finalized log segments in newEpoch
Todd Lipcon created HDFS-5287:
-
Summary: JN need not validate finalized log segments in newEpoch
Key: HDFS-5287
URL: https://issues.apache.org/jira/browse/HDFS-5287
Project: Hadoop HDFS
Issue Type: Bug
Components: qjm
Affects Versions: 2.1.1-beta
Reporter: Todd Lipcon
Priority: Minor
In {{scanStorageForLatestEdits}}, the JN will call {{validateLog}} on the last
log segment, regardless of whether it is finalized. If it's finalized, then
this is a needless pass over the logs which can adversely affect failover time
for a graceful failover.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
2.1.2 (Was: Re: [VOTE] Release Apache Hadoop 2.1.1-beta)
Guys, I took a look at the content in 2.1.2-beta so far, other than the critical fixes such as HADOOP-9984 (symlinks) and few others in YARN/MR, there is fairly little content (unit tests fixes etc.) Furthermore, it's standing up well in testing too. Plus, the protocols look good for now (I wrote a gohadoop to try convince myself), let's lock them in. Given that, I'm thinking we can just go ahead rename it 2.2.0 rather than make another 2.1.x release. This will drop a short-lived release (2.1.2) and help us move forward on 2.3 which has a fair bunch of content already... Thoughts? thanks, Arun On Sep 24, 2013, at 4:24 PM, Zhijie Shen zs...@hortonworks.com wrote: I've added MAPREDUCE-5531 to the blocker list. - Zhijie On Tue, Sep 24, 2013 at 3:41 PM, Arun C Murthy a...@hortonworks.com wrote: With 4 +1s (3 binding) and no -1s the vote passes. I'll push it out… I'll make it clear on the release page, that there are some known issues and that we will follow up very shortly with another release. Meanwhile, let's fix the remaining blockers (please mark them as such with Target Version 2.1.2-beta). The current blockers are here: http://s.apache.org/hadoop-2.1.2-beta-blockers thanks, Arun On Sep 16, 2013, at 11:38 PM, Arun C Murthy a...@hortonworks.com wrote: Folks, I've created a release candidate (rc0) for hadoop-2.1.1-beta that I would like to get released - this release fixes a number of bugs on top of hadoop-2.1.0-beta as a result of significant amounts of testing. If things go well, this might be the last of the *beta* releases of hadoop-2.x. The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.1-beta-rc0 The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.1-beta-rc0 The maven artifacts are available via repository.apache.org. Please try the release and vote; the vote will run for the usual 7 days. thanks, Arun -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- Zhijie Shen Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: 2.1.2 (Was: Re: [VOTE] Release Apache Hadoop 2.1.1-beta)
Arun, Does this mean that you want to skip a beta release and go straight to GA with the next release? thx On Tue, Oct 1, 2013 at 4:15 PM, Arun C Murthy a...@hortonworks.com wrote: Guys, I took a look at the content in 2.1.2-beta so far, other than the critical fixes such as HADOOP-9984 (symlinks) and few others in YARN/MR, there is fairly little content (unit tests fixes etc.) Furthermore, it's standing up well in testing too. Plus, the protocols look good for now (I wrote a gohadoop to try convince myself), let's lock them in. Given that, I'm thinking we can just go ahead rename it 2.2.0 rather than make another 2.1.x release. This will drop a short-lived release (2.1.2) and help us move forward on 2.3 which has a fair bunch of content already... Thoughts? thanks, Arun On Sep 24, 2013, at 4:24 PM, Zhijie Shen zs...@hortonworks.com wrote: I've added MAPREDUCE-5531 to the blocker list. - Zhijie On Tue, Sep 24, 2013 at 3:41 PM, Arun C Murthy a...@hortonworks.com wrote: With 4 +1s (3 binding) and no -1s the vote passes. I'll push it out… I'll make it clear on the release page, that there are some known issues and that we will follow up very shortly with another release. Meanwhile, let's fix the remaining blockers (please mark them as such with Target Version 2.1.2-beta). The current blockers are here: http://s.apache.org/hadoop-2.1.2-beta-blockers thanks, Arun On Sep 16, 2013, at 11:38 PM, Arun C Murthy a...@hortonworks.com wrote: Folks, I've created a release candidate (rc0) for hadoop-2.1.1-beta that I would like to get released - this release fixes a number of bugs on top of hadoop-2.1.0-beta as a result of significant amounts of testing. If things go well, this might be the last of the *beta* releases of hadoop-2.x. The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.1-beta-rc0 The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.1-beta-rc0 The maven artifacts are available via repository.apache.org. Please try the release and vote; the vote will run for the usual 7 days. thanks, Arun -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- Zhijie Shen Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- Alejandro
Re: 2.1.2 (Was: Re: [VOTE] Release Apache Hadoop 2.1.1-beta)
(This time copying all the lists) I am +1 for naming the new branch 2.2.0. On Tue, Oct 1, 2013 at 4:15 PM, Arun C Murthy a...@hortonworks.com wrote: Guys, I took a look at the content in 2.1.2-beta so far, other than the critical fixes such as HADOOP-9984 (symlinks) and few others in YARN/MR, there is fairly little content (unit tests fixes etc.) Furthermore, it's standing up well in testing too. Plus, the protocols look good for now (I wrote a gohadoop to try convince myself), let's lock them in. Given that, I'm thinking we can just go ahead rename it 2.2.0 rather than make another 2.1.x release. This will drop a short-lived release (2.1.2) and help us move forward on 2.3 which has a fair bunch of content already... Thoughts? thanks, Arun On Sep 24, 2013, at 4:24 PM, Zhijie Shen zs...@hortonworks.com wrote: I've added MAPREDUCE-5531 to the blocker list. - Zhijie On Tue, Sep 24, 2013 at 3:41 PM, Arun C Murthy a...@hortonworks.com wrote: With 4 +1s (3 binding) and no -1s the vote passes. I'll push it out… I'll make it clear on the release page, that there are some known issues and that we will follow up very shortly with another release. Meanwhile, let's fix the remaining blockers (please mark them as such with Target Version 2.1.2-beta). The current blockers are here: http://s.apache.org/hadoop-2.1.2-beta-blockers thanks, Arun On Sep 16, 2013, at 11:38 PM, Arun C Murthy a...@hortonworks.com wrote: Folks, I've created a release candidate (rc0) for hadoop-2.1.1-beta that I would like to get released - this release fixes a number of bugs on top of hadoop-2.1.0-beta as a result of significant amounts of testing. If things go well, this might be the last of the *beta* releases of hadoop-2.x. The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.1-beta-rc0 The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.1-beta-rc0 The maven artifacts are available via repository.apache.org. Please try the release and vote; the vote will run for the usual 7 days. thanks, Arun -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- Zhijie Shen Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- http://hortonworks.com/download/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: 2.1.2 (Was: Re: [VOTE] Release Apache Hadoop 2.1.1-beta)
+1. We should get an RC as soon as possible so that we can get all the downstream components to sign off. The earlier the better. Thanks, +Vinod On Oct 1, 2013, at 4:15 PM, Arun C Murthy wrote: Guys, I took a look at the content in 2.1.2-beta so far, other than the critical fixes such as HADOOP-9984 (symlinks) and few others in YARN/MR, there is fairly little content (unit tests fixes etc.) Furthermore, it's standing up well in testing too. Plus, the protocols look good for now (I wrote a gohadoop to try convince myself), let's lock them in. Given that, I'm thinking we can just go ahead rename it 2.2.0 rather than make another 2.1.x release. This will drop a short-lived release (2.1.2) and help us move forward on 2.3 which has a fair bunch of content already... Thoughts? thanks, Arun On Sep 24, 2013, at 4:24 PM, Zhijie Shen zs...@hortonworks.com wrote: I've added MAPREDUCE-5531 to the blocker list. - Zhijie On Tue, Sep 24, 2013 at 3:41 PM, Arun C Murthy a...@hortonworks.com wrote: With 4 +1s (3 binding) and no -1s the vote passes. I'll push it out… I'll make it clear on the release page, that there are some known issues and that we will follow up very shortly with another release. Meanwhile, let's fix the remaining blockers (please mark them as such with Target Version 2.1.2-beta). The current blockers are here: http://s.apache.org/hadoop-2.1.2-beta-blockers thanks, Arun On Sep 16, 2013, at 11:38 PM, Arun C Murthy a...@hortonworks.com wrote: Folks, I've created a release candidate (rc0) for hadoop-2.1.1-beta that I would like to get released - this release fixes a number of bugs on top of hadoop-2.1.0-beta as a result of significant amounts of testing. If things go well, this might be the last of the *beta* releases of hadoop-2.x. The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.1-beta-rc0 The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.1-beta-rc0 The maven artifacts are available via repository.apache.org. Please try the release and vote; the vote will run for the usual 7 days. thanks, Arun -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- Zhijie Shen Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: 2.1.2 (Was: Re: [VOTE] Release Apache Hadoop 2.1.1-beta)
+1 for naming the new branch 2.2.0 sanjay On Oct 1, 2013, at 4:55 PM, Suresh Srinivas wrote: (This time copying all the lists) I am +1 for naming the new branch 2.2.0. On Tue, Oct 1, 2013 at 4:15 PM, Arun C Murthy a...@hortonworks.com wrote: Guys, I took a look at the content in 2.1.2-beta so far, other than the critical fixes such as HADOOP-9984 (symlinks) and few others in YARN/MR, there is fairly little content (unit tests fixes etc.) Furthermore, it's standing up well in testing too. Plus, the protocols look good for now (I wrote a gohadoop to try convince myself), let's lock them in. Given that, I'm thinking we can just go ahead rename it 2.2.0 rather than make another 2.1.x release. This will drop a short-lived release (2.1.2) and help us move forward on 2.3 which has a fair bunch of content already... Thoughts? thanks, Arun On Sep 24, 2013, at 4:24 PM, Zhijie Shen zs...@hortonworks.com wrote: I've added MAPREDUCE-5531 to the blocker list. - Zhijie On Tue, Sep 24, 2013 at 3:41 PM, Arun C Murthy a...@hortonworks.com wrote: With 4 +1s (3 binding) and no -1s the vote passes. I'll push it out… I'll make it clear on the release page, that there are some known issues and that we will follow up very shortly with another release. Meanwhile, let's fix the remaining blockers (please mark them as such with Target Version 2.1.2-beta). The current blockers are here: http://s.apache.org/hadoop-2.1.2-beta-blockers thanks, Arun On Sep 16, 2013, at 11:38 PM, Arun C Murthy a...@hortonworks.com wrote: Folks, I've created a release candidate (rc0) for hadoop-2.1.1-beta that I would like to get released - this release fixes a number of bugs on top of hadoop-2.1.0-beta as a result of significant amounts of testing. If things go well, this might be the last of the *beta* releases of hadoop-2.x. The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.1-beta-rc0 The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.1-beta-rc0 The maven artifacts are available via repository.apache.org. Please try the release and vote; the vote will run for the usual 7 days. thanks, Arun -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- Zhijie Shen Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- http://hortonworks.com/download/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is
[jira] [Created] (HDFS-5288) Close idle connections in portmap
Haohui Mai created HDFS-5288: Summary: Close idle connections in portmap Key: HDFS-5288 URL: https://issues.apache.org/jira/browse/HDFS-5288 Project: Hadoop HDFS Issue Type: Improvement Components: nfs Reporter: Haohui Mai Assignee: Haohui Mai Currently the portmap daemon does not close idle connections. The daemon should close any idle connections to save resources. -- This message was sent by Atlassian JIRA (v6.1#6144)
Re: 2.1.2 (Was: Re: [VOTE] Release Apache Hadoop 2.1.1-beta)
HADOOP-9984 is going to break interface compatibility for out-of-tree FileSystems. It'd also be good to let downstream components do some testing before GA. Thanks, Andrew On Tue, Oct 1, 2013 at 5:18 PM, Jagane Sundar jag...@sundar.org wrote: +1 Makes good sense. Jagane -Original Message- From: Arun C Murthy [mailto:a...@hortonworks.com] Sent: Tuesday, October 01, 2013 4:15 PM To: common-...@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-...@hadoop.apache.org; mapreduce-...@hadoop.apache.org Subject: 2.1.2 (Was: Re: [VOTE] Release Apache Hadoop 2.1.1-beta) Guys, I took a look at the content in 2.1.2-beta so far, other than the critical fixes such as HADOOP-9984 (symlinks) and few others in YARN/MR, there is fairly little content (unit tests fixes etc.) Furthermore, it's standing up well in testing too. Plus, the protocols look good for now (I wrote a gohadoop to try convince myself), let's lock them in. Given that, I'm thinking we can just go ahead rename it 2.2.0 rather than make another 2.1.x release. This will drop a short-lived release (2.1.2) and help us move forward on 2.3 which has a fair bunch of content already... Thoughts? thanks, Arun On Sep 24, 2013, at 4:24 PM, Zhijie Shen zs...@hortonworks.com wrote: I've added MAPREDUCE-5531 to the blocker list. - Zhijie On Tue, Sep 24, 2013 at 3:41 PM, Arun C Murthy a...@hortonworks.com wrote: With 4 +1s (3 binding) and no -1s the vote passes. I'll push it out... I'll make it clear on the release page, that there are some known issues and that we will follow up very shortly with another release. Meanwhile, let's fix the remaining blockers (please mark them as such with Target Version 2.1.2-beta). The current blockers are here: http://s.apache.org/hadoop-2.1.2-beta-blockers thanks, Arun On Sep 16, 2013, at 11:38 PM, Arun C Murthy a...@hortonworks.com wrote: Folks, I've created a release candidate (rc0) for hadoop-2.1.1-beta that I would like to get released - this release fixes a number of bugs on top of hadoop-2.1.0-beta as a result of significant amounts of testing. If things go well, this might be the last of the *beta* releases of hadoop-2.x. The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.1-beta-rc0 The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.1-beta -rc0 The maven artifacts are available via repository.apache.org. Please try the release and vote; the vote will run for the usual 7 days. thanks, Arun -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- Zhijie Shen Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: 2.1.2 (Was: Re: [VOTE] Release Apache Hadoop 2.1.1-beta)
Yes, sorry if it wasn't clear. As others seem to agree, I think we'll be better getting a protocol/api stable GA done and then iterating on bugs etc. I'm not super worried about HADOOP-9984 since symlinks just made it to branch-2.1 recently. Currently we only have 2 blockers: HADOOP-9984 MAPREDUCE-5530. Both of which are PA and I've reviewed MR-5530 and is good to go (thanks Robert). Hopefully we can finish up HADOOP-9984 asap and we'll be good. thanks, Arun On Oct 1, 2013, at 4:53 PM, Alejandro Abdelnur t...@cloudera.com wrote: Arun, Does this mean that you want to skip a beta release and go straight to GA with the next release? thx On Tue, Oct 1, 2013 at 4:15 PM, Arun C Murthy a...@hortonworks.com wrote: Guys, I took a look at the content in 2.1.2-beta so far, other than the critical fixes such as HADOOP-9984 (symlinks) and few others in YARN/MR, there is fairly little content (unit tests fixes etc.) Furthermore, it's standing up well in testing too. Plus, the protocols look good for now (I wrote a gohadoop to try convince myself), let's lock them in. Given that, I'm thinking we can just go ahead rename it 2.2.0 rather than make another 2.1.x release. This will drop a short-lived release (2.1.2) and help us move forward on 2.3 which has a fair bunch of content already... Thoughts? thanks, Arun On Sep 24, 2013, at 4:24 PM, Zhijie Shen zs...@hortonworks.com wrote: I've added MAPREDUCE-5531 to the blocker list. - Zhijie On Tue, Sep 24, 2013 at 3:41 PM, Arun C Murthy a...@hortonworks.com wrote: With 4 +1s (3 binding) and no -1s the vote passes. I'll push it out… I'll make it clear on the release page, that there are some known issues and that we will follow up very shortly with another release. Meanwhile, let's fix the remaining blockers (please mark them as such with Target Version 2.1.2-beta). The current blockers are here: http://s.apache.org/hadoop-2.1.2-beta-blockers thanks, Arun On Sep 16, 2013, at 11:38 PM, Arun C Murthy a...@hortonworks.com wrote: Folks, I've created a release candidate (rc0) for hadoop-2.1.1-beta that I would like to get released - this release fixes a number of bugs on top of hadoop-2.1.0-beta as a result of significant amounts of testing. If things go well, this might be the last of the *beta* releases of hadoop-2.x. The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.1-beta-rc0 The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.1-beta-rc0 The maven artifacts are available via repository.apache.org. Please try the release and vote; the vote will run for the usual 7 days. thanks, Arun -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- Zhijie Shen Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- Alejandro -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed
