Re: Don't want to receive the mails again
Hi WangYu and Debashish, To unsubscribe from these mailing lists, send something to mailing list- unsubscr...@hadoop.apache.org. -Sandy On Mon, Jun 23, 2014 at 7:23 PM, Maity, Debashish debashish.ma...@softwareag.com wrote: Yes please do not send me communication again. -Original Message- From: WangYu [mailto:wangyu0...@gmail.com] Sent: Tuesday, June 24, 2014 7:52 AM To: common-dev@hadoop.apache.org Cc: yarn-...@hadoop.apache.org Subject: Don't want to receive the mails again Please don't send me the communication any more.
RE: [DISCUSS] Change by-laws on release votes: 5 days instead of 7
Thanks Arun. +1 Regards, Uma -Original Message- From: Arun C. Murthy [mailto:a...@hortonworks.com] Sent: Saturday, June 21, 2014 11:07 PM To: hdfs-...@hadoop.apache.org Cc: common-dev@hadoop.apache.org; yarn-...@hadoop.apache.org; mapreduce-...@hadoop.apache.org Subject: Re: [DISCUSS] Change by-laws on release votes: 5 days instead of 7 Uma, Voting periods are defined in *minimum* terms, so it already covers what you'd like to see i.e. the vote can continue longer. thanks, Arun On Jun 21, 2014, at 2:19 AM, Gangumalla, Uma uma.ganguma...@intel.com wrote: How about proposing vote for 5days and give chance to RM for extending vote for 2more days( total to 7days) if the rc did not receive enough vote within 5days? If a rc received enough votes in 5days, RM can close vote. I can see an advantage of 7days voting is, that will cover all the week and weekend days. So, if someone wants to test on weekend time(due to the weekday schedules), that will give chance to them. Regards, Uma -Original Message- From: Arun C Murthy [mailto:a...@hortonworks.com] Sent: Saturday, June 21, 2014 11:25 AM To: hdfs-...@hadoop.apache.org; common-dev@hadoop.apache.org; yarn-...@hadoop.apache.org; mapreduce-...@hadoop.apache.org Subject: [DISCUSS] Change by-laws on release votes: 5 days instead of 7 Folks, I'd like to propose we change our by-laws to reduce our voting periods on new releases from 7 days to 5. Currently, it just takes too long to turn around releases; particularly if we have critical security fixes etc. Thoughts? thanks, Arun -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[VOTE] Change by-laws on release votes: 5 days instead of 7
Folks, As discussed, I'd like to call a vote on changing our by-laws to change release votes from 7 days to 5. I've attached the change to by-laws I'm proposing. Please vote, the vote will the usual period of 7 days. thanks, Arun [main]$ svn diff Index: author/src/documentation/content/xdocs/bylaws.xml === --- author/src/documentation/content/xdocs/bylaws.xml (revision 1605015) +++ author/src/documentation/content/xdocs/bylaws.xml (working copy) @@ -344,7 +344,16 @@ pVotes are open for a period of 7 days to allow all active voters time to consider the vote. Votes relating to code changes are not subject to a strict timetable but should be -made as timely as possible./p/li +made as timely as possible./p + + ul + li strongProduct Release - Vote Timeframe/strong + pRelease votes, alone, run for a period of 5 days. All other + votes are subject to the above timeframe of 7 days./p + /li + /ul + /li + /ul /section /body -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Created] (HADOOP-10744) LZ4 Compression fails to recognize PowerPC Little Endian Architecture
Ayappan created HADOOP-10744: Summary: LZ4 Compression fails to recognize PowerPC Little Endian Architecture Key: HADOOP-10744 URL: https://issues.apache.org/jira/browse/HADOOP-10744 Project: Hadoop Common Issue Type: Test Components: io, native Affects Versions: 2.4.0, 2.3.0, 2.2.0 Environment: PowerPC Little Endian (ppc64le) Reporter: Ayappan Lz4 Compression fails to identify the PowerPC Little Endian Architecture. It recognizes it as Big Endian and several testcases( TestCompressorDecompressor, TestCodec, TestLz4CompressorDecompressor) fails due to this. Running org.apache.hadoop.io.compress.TestCompressorDecompressor Tests run: 2, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 0.435 sec FAILURE! - in org.apache.hadoop.io.compress.TestCompressorDecompressor testCompressorDecompressor(org.apache.hadoop.io.compress.TestCompressorDecompressor) Time elapsed: 0.308 sec FAILURE! org.junit.internal.ArrayComparisonFailure: org.apache.hadoop.io.compress.lz4.Lz4Compressor_org.apache.hadoop.io.compress.lz4.Lz4Decompressor- byte arrays not equals error !!!: arrays first differed at element [1428]; expected:4 but was:10 at org.junit.internal.ComparisonCriteria.arrayEquals(ComparisonCriteria.java:50) at org.junit.Assert.internalArrayEquals(Assert.java:473) at org.junit.Assert.assertArrayEquals(Assert.java:294) at org.apache.hadoop.io.compress.CompressDecompressTester$CompressionTestStrategy$2.assertCompression(CompressDecompressTester.java:325) at org.apache.hadoop.io.compress.CompressDecompressTester.test(CompressDecompressTester.java:135) at org.apache.hadoop.io.compress.TestCompressorDecompressor.testCompressorDecompressor(TestCompressorDecompressor.java:58) ... ... . -- This message was sent by Atlassian JIRA (v6.2#6252)
Build failed in Jenkins: Hadoop-Common-trunk #1149
See https://builds.apache.org/job/Hadoop-Common-trunk/1149/changes
Changes:
[wheat9] HDFS-6562. Refactor rename() in FSDirectory. Contributed by Haohui Mai.
[arp] HADOOP-10659. Refactor AccessControlList to reuse utility functions and
to improve performance. (Contributed by Benoy Antony)
[jianhe] YARN-2191. Added a new test to ensure NM will clean up completed
applications in the case of RM restart. Contributed by Wangda Tan
[arp] HDFS-6578. add toString method to DatanodeStorage for easier debugging.
(Contributed by Yongjun Zhang)
[arp] HDFS-6587. Bug in TestBPOfferService can cause test failure. (Contributed
by Zhilei Xu)
--
[...truncated 73936 lines...]
parsing buildfile
jar:file:/home/jenkins/.m2/repository/org/apache/ant/ant/1.8.2/ant-1.8.2.jar!/org/apache/tools/ant/antlib.xml
with URI =
jar:file:/home/jenkins/.m2/repository/org/apache/ant/ant/1.8.2/ant-1.8.2.jar!/org/apache/tools/ant/antlib.xml
from a zip file
Class org.apache.maven.ant.tasks.AttachArtifactTask loaded from parent loader
(parentFirst)
+Datatype attachartifact org.apache.maven.ant.tasks.AttachArtifactTask
Class org.apache.maven.ant.tasks.DependencyFilesetsTask loaded from parent
loader (parentFirst)
+Datatype dependencyfilesets org.apache.maven.ant.tasks.DependencyFilesetsTask
Setting project property: test.build.dir -
https://builds.apache.org/job/Hadoop-Common-trunk/ws/trunk/hadoop-common-project/target/test-dir
Setting project property: test.exclude.pattern - _
Setting project property: zookeeper.version - 3.4.6
Setting project property: hadoop.assemblies.version - 3.0.0-SNAPSHOT
Setting project property: test.exclude - _
Setting project property: distMgmtSnapshotsId - apache.snapshots.https
Setting project property: project.build.sourceEncoding - UTF-8
Setting project property: java.security.egd - file:///dev/urandom
Setting project property: distMgmtSnapshotsUrl -
https://repository.apache.org/content/repositories/snapshots
Setting project property: distMgmtStagingUrl -
https://repository.apache.org/service/local/staging/deploy/maven2
Setting project property: avro.version - 1.7.4
Setting project property: test.build.data -
https://builds.apache.org/job/Hadoop-Common-trunk/ws/trunk/hadoop-common-project/target/test-dir
Setting project property: commons-daemon.version - 1.0.13
Setting project property: hadoop.common.build.dir -
https://builds.apache.org/job/Hadoop-Common-trunk/ws/trunk/hadoop-common-project/../../hadoop-common-project/hadoop-common/target
Setting project property: testsThreadCount - 4
Setting project property: maven.test.redirectTestOutputToFile - true
Setting project property: jdiff.version - 1.0.9
Setting project property: project.reporting.outputEncoding - UTF-8
Setting project property: distMgmtStagingName - Apache Release Distribution
Repository
Setting project property: build.platform - Linux-i386-32
Setting project property: protobuf.version - 2.5.0
Setting project property: failIfNoTests - false
Setting project property: protoc.path - ${env.HADOOP_PROTOC_PATH}
Setting project property: jersey.version - 1.9
Setting project property: distMgmtStagingId - apache.staging.https
Setting project property: distMgmtSnapshotsName - Apache Development Snapshot
Repository
Setting project property: ant.file -
https://builds.apache.org/job/Hadoop-Common-trunk/ws/trunk/hadoop-common-project/pom.xml
[DEBUG] Setting properties with prefix:
Setting project property: project.groupId - org.apache.hadoop
Setting project property: project.artifactId - hadoop-common-project
Setting project property: project.name - Apache Hadoop Common Project
Setting project property: project.description - Apache Hadoop Common Project
Setting project property: project.version - 3.0.0-SNAPSHOT
Setting project property: project.packaging - pom
Setting project property: project.build.directory -
https://builds.apache.org/job/Hadoop-Common-trunk/ws/trunk/hadoop-common-project/target
Setting project property: project.build.outputDirectory -
https://builds.apache.org/job/Hadoop-Common-trunk/ws/trunk/hadoop-common-project/target/classes
Setting project property: project.build.testOutputDirectory -
https://builds.apache.org/job/Hadoop-Common-trunk/ws/trunk/hadoop-common-project/target/test-classes
Setting project property: project.build.sourceDirectory -
https://builds.apache.org/job/Hadoop-Common-trunk/ws/trunk/hadoop-common-project/src/main/java
Setting project property: project.build.testSourceDirectory -
https://builds.apache.org/job/Hadoop-Common-trunk/ws/trunk/hadoop-common-project/src/test/java
Setting project property: localRepository -id: local
url: file:///home/jenkins/.m2/repository/
layout: none
Setting project property: settings.localRepository -
/home/jenkins/.m2/repository
Setting project property: maven.project.dependencies.versions -
[INFO] Executing tasks
Build sequence for target(s) `main' is [main]
Complete build sequence is [main, ]
main:
[jira] [Created] (HADOOP-10745) Improve the delete key rest api of KMS
liyunzhang created HADOOP-10745: --- Summary: Improve the delete key rest api of KMS Key: HADOOP-10745 URL: https://issues.apache.org/jira/browse/HADOOP-10745 Project: Hadoop Common Issue Type: Improvement Components: security Affects Versions: 2.4.0 Reporter: liyunzhang Priority: Minor Deploy KMS on 1 node: create key successfully, No any message are given when deleting a key although the delete action succeeds. #curl -X DELETE http://localhost:16000/kms/v1/key/k2?user.name=1 -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Don't want to receive the mails again
Thanks sandy, but it seems some problems; when i send email to unsubscr...@hadoop.apache.org, i received following message: Delivery to the following recipient failed permanently: unsubscr...@hadoop.apache.org Technical details of permanent failure: Google tried to deliver your message, but it was rejected by the server for the recipient domainhadoop.apache.org by mx1.eu.apache.org. [192.87.106.230]. The error that the other server returned was: 550 mail to unsubscr...@hadoop.apache.org not accepted here 在 2014年6月24日，下午2:56，Sandy Ryza sandy.r...@cloudera.com 写道： Hi WangYu and Debashish, To unsubscribe from these mailing lists, send something to mailing list- unsubscr...@hadoop.apache.org. -Sandy On Mon, Jun 23, 2014 at 7:23 PM, Maity, Debashish debashish.ma...@softwareag.com wrote: Yes please do not send me communication again. -Original Message- From: WangYu [mailto:wangyu0...@gmail.com] Sent: Tuesday, June 24, 2014 7:52 AM To: common-dev@hadoop.apache.org Cc: yarn-...@hadoop.apache.org Subject: Don't want to receive the mails again Please don't send me the communication any more.
RE: Don't want to receive the mails again
Yups same here -Original Message- From: WangYu [mailto:wangyu0...@gmail.com] Sent: Tuesday, June 24, 2014 6:00 PM To: common-dev@hadoop.apache.org Subject: Re: Don't want to receive the mails again Thanks sandy, but it seems some problems; when i send email to unsubscr...@hadoop.apache.org, i received following message: Delivery to the following recipient failed permanently: unsubscr...@hadoop.apache.org Technical details of permanent failure: Google tried to deliver your message, but it was rejected by the server for the recipient domainhadoop.apache.org by mx1.eu.apache.org. [192.87.106.230]. The error that the other server returned was: 550 mail to unsubscr...@hadoop.apache.org not accepted here 在 2014年6月24日，下午2:56，Sandy Ryza sandy.r...@cloudera.com 写道： Hi WangYu and Debashish, To unsubscribe from these mailing lists, send something to mailing list- unsubscr...@hadoop.apache.org. -Sandy On Mon, Jun 23, 2014 at 7:23 PM, Maity, Debashish debashish.ma...@softwareag.com wrote: Yes please do not send me communication again. -Original Message- From: WangYu [mailto:wangyu0...@gmail.com] Sent: Tuesday, June 24, 2014 7:52 AM To: common-dev@hadoop.apache.org Cc: yarn-...@hadoop.apache.org Subject: Don't want to receive the mails again Please don't send me the communication any more.
Re: Don't want to receive the mails again
Please take a look at http://hadoop.apache.org/mailing_lists.html#Developers Use the unsubscribe link. Cheers On Jun 24, 2014, at 5:29 AM, WangYu wangyu0...@gmail.com wrote: Thanks sandy, but it seems some problems; when i send email to unsubscr...@hadoop.apache.org, i received following message: Delivery to the following recipient failed permanently: unsubscr...@hadoop.apache.org Technical details of permanent failure: Google tried to deliver your message, but it was rejected by the server for the recipient domainhadoop.apache.org by mx1.eu.apache.org. [192.87.106.230]. The error that the other server returned was: 550 mail to unsubscr...@hadoop.apache.org not accepted here 在 2014年6月24日，下午2:56，Sandy Ryza sandy.r...@cloudera.com 写道： Hi WangYu and Debashish, To unsubscribe from these mailing lists, send something to mailing list- unsubscr...@hadoop.apache.org. -Sandy On Mon, Jun 23, 2014 at 7:23 PM, Maity, Debashish debashish.ma...@softwareag.com wrote: Yes please do not send me communication again. -Original Message- From: WangYu [mailto:wangyu0...@gmail.com] Sent: Tuesday, June 24, 2014 7:52 AM To: common-dev@hadoop.apache.org Cc: yarn-...@hadoop.apache.org Subject: Don't want to receive the mails again Please don't send me the communication any more.
Re: [VOTE] Change by-laws on release votes: 5 days instead of 7
+1 Thanks Devaraj K On Tue, Jun 24, 2014 at 2:23 PM, Arun C Murthy a...@hortonworks.com wrote: Folks, As discussed, I'd like to call a vote on changing our by-laws to change release votes from 7 days to 5. I've attached the change to by-laws I'm proposing. Please vote, the vote will the usual period of 7 days. thanks, Arun [main]$ svn diff Index: author/src/documentation/content/xdocs/bylaws.xml === --- author/src/documentation/content/xdocs/bylaws.xml (revision 1605015) +++ author/src/documentation/content/xdocs/bylaws.xml (working copy) @@ -344,7 +344,16 @@ pVotes are open for a period of 7 days to allow all active voters time to consider the vote. Votes relating to code changes are not subject to a strict timetable but should be -made as timely as possible./p/li +made as timely as possible./p + + ul + li strongProduct Release - Vote Timeframe/strong + pRelease votes, alone, run for a period of 5 days. All other + votes are subject to the above timeframe of 7 days./p + /li + /ul + /li + /ul /section /body -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- Thanks Devaraj K
Re: [VOTE] Change by-laws on release votes: 5 days instead of 7
+1 (binding) -Sandy On Tue, Jun 24, 2014 at 7:53 AM, Devaraj K deva...@apache.org wrote: +1 Thanks Devaraj K On Tue, Jun 24, 2014 at 2:23 PM, Arun C Murthy a...@hortonworks.com wrote: Folks, As discussed, I'd like to call a vote on changing our by-laws to change release votes from 7 days to 5. I've attached the change to by-laws I'm proposing. Please vote, the vote will the usual period of 7 days. thanks, Arun [main]$ svn diff Index: author/src/documentation/content/xdocs/bylaws.xml === --- author/src/documentation/content/xdocs/bylaws.xml (revision 1605015) +++ author/src/documentation/content/xdocs/bylaws.xml (working copy) @@ -344,7 +344,16 @@ pVotes are open for a period of 7 days to allow all active voters time to consider the vote. Votes relating to code changes are not subject to a strict timetable but should be -made as timely as possible./p/li +made as timely as possible./p + + ul + li strongProduct Release - Vote Timeframe/strong + pRelease votes, alone, run for a period of 5 days. All other + votes are subject to the above timeframe of 7 days./p + /li + /ul + /li + /ul /section /body -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- Thanks Devaraj K
Re: [VOTE] Change by-laws on release votes: 5 days instead of 7
+1 (binding) Jason On 06/24/2014 03:53 AM, Arun C Murthy wrote: Folks, As discussed, I'd like to call a vote on changing our by-laws to change release votes from 7 days to 5. I've attached the change to by-laws I'm proposing. Please vote, the vote will the usual period of 7 days. thanks, Arun [main]$ svn diff Index: author/src/documentation/content/xdocs/bylaws.xml === --- author/src/documentation/content/xdocs/bylaws.xml (revision 1605015) +++ author/src/documentation/content/xdocs/bylaws.xml (working copy) @@ -344,7 +344,16 @@ pVotes are open for a period of 7 days to allow all active voters time to consider the vote. Votes relating to code changes are not subject to a strict timetable but should be -made as timely as possible./p/li +made as timely as possible./p + + ul + li strongProduct Release - Vote Timeframe/strong + pRelease votes, alone, run for a period of 5 days. All other + votes are subject to the above timeframe of 7 days./p + /li + /ul + /li + /ul /section /body
Re: [VOTE] Release Apache Hadoop 0.23.11
+1 (binding) Verified signatures Verified native compilation on Ubuntu and ran some sample jobs. jeagles On Tue, Jun 24, 2014 at 10:02 AM, Devaraj K deva...@apache.org wrote: +1 (non-binding) Deployed in a two node cluster and ran few M/R Jobs, everything works fine. On Thu, Jun 19, 2014 at 8:44 PM, Thomas Graves tgra...@yahoo-inc.com.invalid wrote: Hey Everyone, There have been various bug fixes that have went into branch-0.23 since the 0.23.10 release. We think its time to do a 0.23.11. This is also the last planned release off of branch-0.23 we plan on doing. The RC is available at: http://people.apache.org/~tgraves/hadoop-0.23.11-candidate-0/ The RC Tag in svn is here: http://svn.apache.org/viewvc/hadoop/common/tags/release-0.23.11-rc0/ The maven artifacts are available via repository.apache.org. Please try the release and vote; the vote will run for the usual 7 days til June 26th. I am +1 (binding). thanks, Tom Graves -- Thanks Devaraj K
Re: [VOTE] Change by-laws on release votes: 5 days instead of 7
+1 (non-binding) On Tue, Jun 24, 2014 at 8:54 AM, Jason Lowe jl...@yahoo-inc.com.invalid wrote: +1 (binding) Jason On 06/24/2014 03:53 AM, Arun C Murthy wrote: Folks, As discussed, I'd like to call a vote on changing our by-laws to change release votes from 7 days to 5. I've attached the change to by-laws I'm proposing. Please vote, the vote will the usual period of 7 days. thanks, Arun [main]$ svn diff Index: author/src/documentation/content/xdocs/bylaws.xml === --- author/src/documentation/content/xdocs/bylaws.xml (revision 1605015) +++ author/src/documentation/content/xdocs/bylaws.xml (working copy) @@ -344,7 +344,16 @@ pVotes are open for a period of 7 days to allow all active voters time to consider the vote. Votes relating to code changes are not subject to a strict timetable but should be -made as timely as possible./p/li +made as timely as possible./p + + ul + li strongProduct Release - Vote Timeframe/strong + pRelease votes, alone, run for a period of 5 days. All other + votes are subject to the above timeframe of 7 days./p + /li + /ul + /li + /ul /section /body
Re: [VOTE] Change by-laws on release votes: 5 days instead of 7
+1 (non-binding) I like the idea! Thanks, Mit Desai On 6/24/14, 3:53 AM, Arun C Murthy a...@hortonworks.com wrote: Folks, As discussed, I'd like to call a vote on changing our by-laws to change release votes from 7 days to 5. I've attached the change to by-laws I'm proposing. Please vote, the vote will the usual period of 7 days. thanks, Arun [main]$ svn diff Index: author/src/documentation/content/xdocs/bylaws.xml === --- author/src/documentation/content/xdocs/bylaws.xml (revision 1605015) +++ author/src/documentation/content/xdocs/bylaws.xml (working copy) @@ -344,7 +344,16 @@ pVotes are open for a period of 7 days to allow all active voters time to consider the vote. Votes relating to code changes are not subject to a strict timetable but should be -made as timely as possible./p/li +made as timely as possible./p + + ul + li strongProduct Release - Vote Timeframe/strong + pRelease votes, alone, run for a period of 5 days. All other + votes are subject to the above timeframe of 7 days./p + /li + /ul + /li + /ul /section /body -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: [VOTE] Change by-laws on release votes: 5 days instead of 7
+1 (binding) — Hitesh On Jun 24, 2014, at 1:53 AM, Arun C Murthy a...@hortonworks.com wrote: Folks, As discussed, I'd like to call a vote on changing our by-laws to change release votes from 7 days to 5. I've attached the change to by-laws I'm proposing. Please vote, the vote will the usual period of 7 days. thanks, Arun [main]$ svn diff Index: author/src/documentation/content/xdocs/bylaws.xml === --- author/src/documentation/content/xdocs/bylaws.xml (revision 1605015) +++ author/src/documentation/content/xdocs/bylaws.xml (working copy) @@ -344,7 +344,16 @@ pVotes are open for a period of 7 days to allow all active voters time to consider the vote. Votes relating to code changes are not subject to a strict timetable but should be -made as timely as possible./p/li +made as timely as possible./p + + ul + li strongProduct Release - Vote Timeframe/strong + pRelease votes, alone, run for a period of 5 days. All other + votes are subject to the above timeframe of 7 days./p + /li + /ul + /li + /ul /section /body -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: [VOTE] Change by-laws on release votes: 5 days instead of 7
+1(non-binding) On Wed, Jun 25, 2014 at 1:55 AM, Hitesh Shah hit...@apache.org wrote: +1 (binding) — Hitesh On Jun 24, 2014, at 1:53 AM, Arun C Murthy a...@hortonworks.com wrote: Folks, As discussed, I'd like to call a vote on changing our by-laws to change release votes from 7 days to 5. I've attached the change to by-laws I'm proposing. Please vote, the vote will the usual period of 7 days. thanks, Arun [main]$ svn diff Index: author/src/documentation/content/xdocs/bylaws.xml === --- author/src/documentation/content/xdocs/bylaws.xml (revision 1605015) +++ author/src/documentation/content/xdocs/bylaws.xml (working copy) @@ -344,7 +344,16 @@ pVotes are open for a period of 7 days to allow all active voters time to consider the vote. Votes relating to code changes are not subject to a strict timetable but should be -made as timely as possible./p/li +made as timely as possible./p + + ul + li strongProduct Release - Vote Timeframe/strong + pRelease votes, alone, run for a period of 5 days. All other + votes are subject to the above timeframe of 7 days./p + /li + /ul + /li + /ul /section /body -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- - Tsuyoshi
Re: [VOTE] Change by-laws on release votes: 5 days instead of 7
+1 (binding) -- Aaron T. Myers Software Engineer, Cloudera On Tue, Jun 24, 2014 at 1:53 AM, Arun C Murthy a...@hortonworks.com wrote: Folks, As discussed, I'd like to call a vote on changing our by-laws to change release votes from 7 days to 5. I've attached the change to by-laws I'm proposing. Please vote, the vote will the usual period of 7 days. thanks, Arun [main]$ svn diff Index: author/src/documentation/content/xdocs/bylaws.xml === --- author/src/documentation/content/xdocs/bylaws.xml (revision 1605015) +++ author/src/documentation/content/xdocs/bylaws.xml (working copy) @@ -344,7 +344,16 @@ pVotes are open for a period of 7 days to allow all active voters time to consider the vote. Votes relating to code changes are not subject to a strict timetable but should be -made as timely as possible./p/li +made as timely as possible./p + + ul + li strongProduct Release - Vote Timeframe/strong + pRelease votes, alone, run for a period of 5 days. All other + votes are subject to the above timeframe of 7 days./p + /li + /ul + /li + /ul /section /body -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: [VOTE] Change by-laws on release votes: 5 days instead of 7
+1 (non-binding) On Wed, Jun 25, 2014 at 1:26 AM, Aaron T. Myers a...@cloudera.com wrote: +1 (binding) -- Aaron T. Myers Software Engineer, Cloudera On Tue, Jun 24, 2014 at 1:53 AM, Arun C Murthy a...@hortonworks.com wrote: Folks, As discussed, I'd like to call a vote on changing our by-laws to change release votes from 7 days to 5. I've attached the change to by-laws I'm proposing. Please vote, the vote will the usual period of 7 days. thanks, Arun [main]$ svn diff Index: author/src/documentation/content/xdocs/bylaws.xml === --- author/src/documentation/content/xdocs/bylaws.xml (revision 1605015) +++ author/src/documentation/content/xdocs/bylaws.xml (working copy) @@ -344,7 +344,16 @@ pVotes are open for a period of 7 days to allow all active voters time to consider the vote. Votes relating to code changes are not subject to a strict timetable but should be -made as timely as possible./p/li +made as timely as possible./p + + ul + li strongProduct Release - Vote Timeframe/strong + pRelease votes, alone, run for a period of 5 days. All other + votes are subject to the above timeframe of 7 days./p + /li + /ul + /li + /ul /section /body -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- Zhijie Shen Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Created] (HADOOP-10747) Support configurable retries on SASL connection failures in RPC client.
Chris Nauroth created HADOOP-10747: -- Summary: Support configurable retries on SASL connection failures in RPC client. Key: HADOOP-10747 URL: https://issues.apache.org/jira/browse/HADOOP-10747 Project: Hadoop Common Issue Type: Bug Components: ipc Affects Versions: 2.4.0, 3.0.0 Reporter: Chris Nauroth Assignee: Chris Nauroth The RPC client includes a retry loop around SASL connection failures. Currently, this is hard-coded to a maximum of 5 retries. Let's make this configurable. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: [VOTE] Change by-laws on release votes: 5 days instead of 7
+1 (non-binding) (2014/06/24 10:33), Zhijie Shen wrote: +1 (non-binding) On Wed, Jun 25, 2014 at 1:26 AM, Aaron T. Myers a...@cloudera.com wrote: +1 (binding) -- Aaron T. Myers Software Engineer, Cloudera On Tue, Jun 24, 2014 at 1:53 AM, Arun C Murthy a...@hortonworks.com wrote: Folks, As discussed, I'd like to call a vote on changing our by-laws to change release votes from 7 days to 5. I've attached the change to by-laws I'm proposing. Please vote, the vote will the usual period of 7 days. thanks, Arun [main]$ svn diff Index: author/src/documentation/content/xdocs/bylaws.xml === --- author/src/documentation/content/xdocs/bylaws.xml (revision 1605015) +++ author/src/documentation/content/xdocs/bylaws.xml (working copy) @@ -344,7 +344,16 @@ pVotes are open for a period of 7 days to allow all active voters time to consider the vote. Votes relating to code changes are not subject to a strict timetable but should be -made as timely as possible./p/li +made as timely as possible./p + + ul + li strongProduct Release - Vote Timeframe/strong + pRelease votes, alone, run for a period of 5 days. All other + votes are subject to the above timeframe of 7 days./p + /li + /ul + /li + /ul /section /body -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: [VOTE] Change by-laws on release votes: 5 days instead of 7
+1 (binding) Chris Nauroth Hortonworks http://hortonworks.com/ On Tue, Jun 24, 2014 at 10:58 AM, Jakob Homan jgho...@gmail.com wrote: +1 (binding) On Tue, Jun 24, 2014 at 10:33 AM, Zhijie Shen zs...@hortonworks.com wrote: +1 (non-binding) On Wed, Jun 25, 2014 at 1:26 AM, Aaron T. Myers a...@cloudera.com wrote: +1 (binding) -- Aaron T. Myers Software Engineer, Cloudera On Tue, Jun 24, 2014 at 1:53 AM, Arun C Murthy a...@hortonworks.com wrote: Folks, As discussed, I'd like to call a vote on changing our by-laws to change release votes from 7 days to 5. I've attached the change to by-laws I'm proposing. Please vote, the vote will the usual period of 7 days. thanks, Arun [main]$ svn diff Index: author/src/documentation/content/xdocs/bylaws.xml === --- author/src/documentation/content/xdocs/bylaws.xml (revision 1605015) +++ author/src/documentation/content/xdocs/bylaws.xml (working copy) @@ -344,7 +344,16 @@ pVotes are open for a period of 7 days to allow all active voters time to consider the vote. Votes relating to code changes are not subject to a strict timetable but should be -made as timely as possible./p/li +made as timely as possible./p + + ul + li strongProduct Release - Vote Timeframe/strong + pRelease votes, alone, run for a period of 5 days. All other + votes are subject to the above timeframe of 7 days./p + /li + /ul + /li + /ul /section /body -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- Zhijie Shen Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: [VOTE] Change by-laws on release votes: 5 days instead of 7
+1 -C On Tue, Jun 24, 2014 at 1:53 AM, Arun C Murthy a...@hortonworks.com wrote: Folks, As discussed, I'd like to call a vote on changing our by-laws to change release votes from 7 days to 5. I've attached the change to by-laws I'm proposing. Please vote, the vote will the usual period of 7 days. thanks, Arun [main]$ svn diff Index: author/src/documentation/content/xdocs/bylaws.xml === --- author/src/documentation/content/xdocs/bylaws.xml (revision 1605015) +++ author/src/documentation/content/xdocs/bylaws.xml (working copy) @@ -344,7 +344,16 @@ pVotes are open for a period of 7 days to allow all active voters time to consider the vote. Votes relating to code changes are not subject to a strict timetable but should be -made as timely as possible./p/li +made as timely as possible./p + + ul + li strongProduct Release - Vote Timeframe/strong + pRelease votes, alone, run for a period of 5 days. All other + votes are subject to the above timeframe of 7 days./p + /li + /ul + /li + /ul /section /body -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: [VOTE] Change by-laws on release votes: 5 days instead of 7
+1 On Tue, Jun 24, 2014 at 11:13 AM, Chris Douglas cdoug...@apache.org wrote: +1 -C On Tue, Jun 24, 2014 at 1:53 AM, Arun C Murthy a...@hortonworks.com wrote: Folks, As discussed, I'd like to call a vote on changing our by-laws to change release votes from 7 days to 5. I've attached the change to by-laws I'm proposing. Please vote, the vote will the usual period of 7 days. thanks, Arun [main]$ svn diff Index: author/src/documentation/content/xdocs/bylaws.xml === --- author/src/documentation/content/xdocs/bylaws.xml (revision 1605015) +++ author/src/documentation/content/xdocs/bylaws.xml (working copy) @@ -344,7 +344,16 @@ pVotes are open for a period of 7 days to allow all active voters time to consider the vote. Votes relating to code changes are not subject to a strict timetable but should be -made as timely as possible./p/li +made as timely as possible./p + + ul + li strongProduct Release - Vote Timeframe/strong + pRelease votes, alone, run for a period of 5 days. All other + votes are subject to the above timeframe of 7 days./p + /li + /ul + /li + /ul /section /body -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Moving to JDK7, JDK8 and new major releases
Hi all, Forking this thread as requested by Vinod. To help anyone who's catching up with this thread, I've written up a wiki page containing what I think are the proposals under discussion. I did my very best to make this as fact-based and disinterested as possible; I really appreciate the constructive discussion we've had so far. If you believe you have a proposal pending, please feel free to edit the wiki. https://wiki.apache.org/hadoop/MovingToJdk7and8 I think based on our current compatibility guidelines, Proposal A is the most attractive. We're pretty hamstrung by the requirement to keep the classpath the same, which would be solved by either OSGI or shading our deps (but that's a different discussion). Thanks, Andrew
Re: Moving to JDK7, JDK8 and new major releases
Andrew thanks for writing the proposal. In the proposal you mention: Dropping support for a JDK in a minor release is incompatible, so this would require a change to our compatibility guidelines. Why is dropping a JDK incompatible? sanjay On Jun 24, 2014, at 11:17 AM, Andrew Wang andrew.w...@cloudera.com wrote: Hi all, Forking this thread as requested by Vinod. To help anyone who's catching up with this thread, I've written up a wiki page containing what I think are the proposals under discussion. I did my very best to make this as fact-based and disinterested as possible; I really appreciate the constructive discussion we've had so far. If you believe you have a proposal pending, please feel free to edit the wiki. https://wiki.apache.org/hadoop/MovingToJdk7and8 I think based on our current compatibility guidelines, Proposal A is the most attractive. We're pretty hamstrung by the requirement to keep the classpath the same, which would be solved by either OSGI or shading our deps (but that's a different discussion). Thanks, Andrew -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Reopened] (HADOOP-10468) TestMetricsSystemImpl.testMultiThreadedPublish fails intermediately
[
https://issues.apache.org/jira/browse/HADOOP-10468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Lowe reopened HADOOP-10468:
-
Reopening this issue as it breaks all existing metrics2 property files. Before
this change the properties needed to be lower-cased but now they must be
camel-cased (e.g.: namenode.* now must be NameNode.*).
The release note states that the metrics2 file became case-sensitive, but I
don't believe that's the case. MetricsConfig uses
org.apache.commons.configuration.SubsetConfiguration which I think has always
been case-sensitive.
I'm hoping there's a way we can fix the underlying issue without breaking
existing metrics2 property files, because the way in which they break is
silent. The settings are simply ignored rather than an error being thrown for
unrecognized/unhandled properties.
TestMetricsSystemImpl.testMultiThreadedPublish fails intermediately
---
Key: HADOOP-10468
URL: https://issues.apache.org/jira/browse/HADOOP-10468
Project: Hadoop Common
Issue Type: Bug
Reporter: Haohui Mai
Assignee: Haohui Mai
Fix For: 2.5.0
Attachments: HADOOP-10468.000.patch, HADOOP-10468.001.patch
{{TestMetricsSystemImpl.testMultiThreadedPublish}} can fail intermediately
due to the insufficient size of the sink queue:
{code}
2014-04-06 21:34:55,269 WARN impl.MetricsSinkAdapter
(MetricsSinkAdapter.java:putMetricsImmediate(107)) - Collector has a full
queue and can't consume the given metrics.
2014-04-06 21:34:55,270 WARN impl.MetricsSinkAdapter
(MetricsSinkAdapter.java:putMetricsImmediate(107)) - Collector has a full
queue and can't consume the given metrics.
2014-04-06 21:34:55,271 WARN impl.MetricsSinkAdapter
(MetricsSinkAdapter.java:putMetricsImmediate(107)) - Collector has a full
queue and can't consume the given metrics.
{code}
The unit test should increase the default queue size to avoid intermediate
failure.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
Re: Moving to JDK7, JDK8 and new major releases
Tx for the new thread Andrew, hopefully it can attract more eyes. Here's what I am behind - a modified proposal C. - Overall I wouldn't think about EOL of JDK7 and/or JDK8 specifically given how long it has taken for JDK6 life-cycle to end. We should try to focus on JDK7 only for now. - As we have seen, a lot (majority?) of orgs on Hadoop have moved beyond JDK6 and are already running on JDK7. So upgrading to JDK7 is more of a reflection of reality (to quote Steve) than it in itself being a disruptive change. - We should try decoupling the discussion of major releases from JDK upgrades. We have seen individual libraries getting updated right in the 2.x lines as and when necessary. Given the new reality of JDK7, I don't see the 'JDK change' as much different from the library upgrades. We have seen how long it has taken (and still taking) users and organization to move from Hadoop 1 to Hadoop 2. A Hadoop 3/4 that adds nothing else other than JDK upgrades will be a big source of confusion for users. A major version update is also seen an opportunity for devs to break APIs. Unless we have groundbreaking 'features' (like YARN or wire-compatibility in Hadoop-2) that a majority of users want and that specifically warrant incompatible changes in our APIs or wire protocols, we are better off separating the major-version update discussion into ints own. Irrespective of all this, we should actively get behind better isolation of user classes/jars from MapReduce classpath. This one's been such a long running concern, it's not funny anymore. Thanks, +Vinod On Jun 24, 2014, at 11:17 AM, Andrew Wang andrew.w...@cloudera.com wrote: Hi all, Forking this thread as requested by Vinod. To help anyone who's catching up with this thread, I've written up a wiki page containing what I think are the proposals under discussion. I did my very best to make this as fact-based and disinterested as possible; I really appreciate the constructive discussion we've had so far. If you believe you have a proposal pending, please feel free to edit the wiki. https://wiki.apache.org/hadoop/MovingToJdk7and8 I think based on our current compatibility guidelines, Proposal A is the most attractive. We're pretty hamstrung by the requirement to keep the classpath the same, which would be solved by either OSGI or shading our deps (but that's a different discussion). Thanks, Andrew -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. signature.asc Description: Message signed with OpenPGP using GPGMail
Re: Moving to JDK7, JDK8 and new major releases
While we haven't codified this in our compatibility guidelines, dropping a Java version seems to me like change that needs to happen alongside a major release. In plain talk, it has the ability to break everything for users who aren't doing anything particularly unreasonable. I don't think we should accept Hadoop's compatibility behavior 6 years ago as precedent for what we can do now. That was before Hadoop 1.0. And we probably have several orders of magnitude more production users. I also don't think we should accept library upgrades as precedent. While this may make sense in specific situations, I definitely don't think this is OK in general. I'd be very nervous about updating Guava outside of major version upgrade. Lastly, I think the claim that nobody is running in production on Java 6 is unsubstantiated. We need to think about a JDK upgrade in terms of what its implications are for users, not in terms of what other kinds of compatibility we've broken that's loosely analogous. -Sandy On Tue, Jun 24, 2014 at 3:42 PM, Vinod Kumar Vavilapalli vino...@apache.org wrote: Tx for the new thread Andrew, hopefully it can attract more eyes. Here's what I am behind - a modified proposal C. - Overall I wouldn't think about EOL of JDK7 and/or JDK8 specifically given how long it has taken for JDK6 life-cycle to end. We should try to focus on JDK7 only for now. - As we have seen, a lot (majority?) of orgs on Hadoop have moved beyond JDK6 and are already running on JDK7. So upgrading to JDK7 is more of a reflection of reality (to quote Steve) than it in itself being a disruptive change. - We should try decoupling the discussion of major releases from JDK upgrades. We have seen individual libraries getting updated right in the 2.x lines as and when necessary. Given the new reality of JDK7, I don't see the 'JDK change' as much different from the library upgrades. We have seen how long it has taken (and still taking) users and organization to move from Hadoop 1 to Hadoop 2. A Hadoop 3/4 that adds nothing else other than JDK upgrades will be a big source of confusion for users. A major version update is also seen an opportunity for devs to break APIs. Unless we have groundbreaking 'features' (like YARN or wire-compatibility in Hadoop-2) that a majority of users want and that specifically warrant incompatible changes in our APIs or wire protocols, we are better off separating the major-version update discussion into ints own. Irrespective of all this, we should actively get behind better isolation of user classes/jars from MapReduce classpath. This one's been such a long running concern, it's not funny anymore. Thanks, +Vinod On Jun 24, 2014, at 11:17 AM, Andrew Wang andrew.w...@cloudera.com wrote: Hi all, Forking this thread as requested by Vinod. To help anyone who's catching up with this thread, I've written up a wiki page containing what I think are the proposals under discussion. I did my very best to make this as fact-based and disinterested as possible; I really appreciate the constructive discussion we've had so far. If you believe you have a proposal pending, please feel free to edit the wiki. https://wiki.apache.org/hadoop/MovingToJdk7and8 I think based on our current compatibility guidelines, Proposal A is the most attractive. We're pretty hamstrung by the requirement to keep the classpath the same, which would be solved by either OSGI or shading our deps (but that's a different discussion). Thanks, Andrew -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: Moving to JDK7, JDK8 and new major releases
Hello, I do know scenarios which involve sticking with Java 6. Mostly related to supporting Applications Servers of (Big) 3 Letter companies (both). I dont see that in an Hadoop cluster. Especially given that expressiveness of Java 8 and also Performance of Java 7 are both important in Big Data analytics an das hoc Job platforms. So I would really be Suprised if any user Organisation would wand a major Hadoop upgrade but sticking with an unsupported / outdated JVM. Greetings Bernd (Who still ships 1.4 products)
Re: Moving to JDK7, JDK8 and new major releases
Alejandro, On Tue, Jun 24, 2014 at 4:44 PM, Alejandro Abdelnur t...@cloudera.com wrote: After reading this thread and thinking a bit about it, I think it should be OK such move up to JDK7 in Hadoop 2 for the following reasons: * Existing Hadoop 2 releases and related projects are running on JDK7 in production. * Commercial vendors of Hadoop have already done lot of work to ensure Hadoop on JDK7 works while keeping Hadoop on JDK6 working. * Different from many of the 3rd party libraries used by Hadoop, JDK is much stricter on backwards compatibility. +1 - I think we are all on the same page here. Fully agree. IMPORTANT: I take this as an exception and not as a carte blanche for 3rd party dependencies and for moving from JDK7 to JDK8 (though it could OK for the later if we end up in the same state of affairs) +1. Agree again - let's just wait/watch. From the thread I've become more convinced that (as you've noted before) that since we are at the bottom of the stack, we need to be more conservative. From http://www.oracle.com/technetwork/java/eol-135779.html, it looks like April 2015 is the *earliest* Java7 will EOL. Java6 EOL was Feb 2011 and we are still debating whether we can stop supporting it. So, my guess is that we will support Java7 at least for a year after it's EOL i.e. till sometime in early 2016. It's just practical. Net - We really don't have a good idea when a significant portion of users will actually migrate to Java 8. W.r.t Java7 this took nearly 3 years after Java6 EOL. So for now, let's just wait see how things develop in the field. Even for Hadoop 2.5, I think we could do the move: * Create the Hadoop 2.5 release branch. * Have one nightly Jenkins job that builds Hadoop 2.5 branch with JDK6 to ensure not JDK7 language/API feature creeps out in Hadoop 2.5. Keep this for all Hadoop 2.5.x releases. * Sanity tests for the Hadoop 2.5.x releases should be done with JDK7. * Apply Steve’s patch to require JDK7 on trunk and branch-2. * Move all Apache Jenkins jobs to build/test using JDK7. * Starting from Hadoop 2.6 we support JDK7 language/API features. I think the mechanics make perfect sense to me. I think we should probably think a bit more on whether we drop support for JDK6 in hadoop-2.6 or hadoop-2.7. I'd like to add one more: * Sometime soon (within a release or two) after we actually drop support for Java6 and move branch-2 to JDK7, let's also start testing on Java8. This way we will be ready for Java8 early regardless of when we stop support for Java7. Dropping Java7 is a bridge we can cross when we come to it. thanks, Arun Effectively what we are ensuring that Hadoop 2.5.x builds and test with JDK6 JDK7 and that all tests towards the release are done with JDK7. Users can proactively upgrade to JDK7 before upgrading to Hadoop 2.5.x, or if upgrade to Hadoop 2.5.x and they run into any issue because of JDK6 (which it would be quite unlikely) they can reactively upgrade to JDK7. Thoughts? On Tue, Jun 24, 2014 at 4:22 PM, Andrew Wang andrew.w...@cloudera.com wrote: Hi all, On dependencies, we've bumped library versions when we think it's safe and the APIs in the new version are compatible. Or, it's not leaked to the app classpath (e.g the JUnit version bump). I think the JIRAs Arun mentioned fall into one of those categories. Steve can do a better job explaining this to me, but we haven't bumped things like Jetty or Guava because they are on the classpath and are not compatible. There is this line in the compat guidelines: - Existing MapReduce, YARN HDFS applications and frameworks should work unmodified within a major release i.e. Apache Hadoop ABI is supported. Since Hadoop apps can and do depend on the Hadoop classpath, the classpath is effectively part of our API. I'm sure there are user apps out there that will break if we make incompatible changes to the classpath. I haven't read up on the MR JIRA Arun mentioned, but there MR isn't the only YARN app out there. Sticking to the theme of work unmodified, let's think about the user effort required to upgrade their JDK. This can be a very expensive task. It might need approval up and down the org, meaning lots of certification, testing, and signoff. Considering the amount of user effort involved here, it really seems like dropping a JDK is something that should only happen in a major release. Else, there's the potential for nasty surprises in a supposedly minor release. That said, we are in an unhappy place right now regarding JDK6, and it's true that almost everyone's moved off of JDK6 at this point. So, I'd be okay with an intermediate 2.x release that drops JDK6 support (but no incompatible changes to the classpath like Guava). This is basically free, and we could start using JDK7 idioms like multi-catch and new NIO stuff in Hadoop code (a minor draw I guess). My
Re: Moving to JDK7, JDK8 and new major releases
On Jun 24, 2014, at 4:22 PM, Andrew Wang andrew.w...@cloudera.com wrote: Since Hadoop apps can and do depend on the Hadoop classpath, the classpath is effectively part of our API. I'm sure there are user apps out there that will break if we make incompatible changes to the classpath. I haven't read up on the MR JIRA Arun mentioned, but there MR isn't the only YARN app out there. I think there is a some confusion/misunderstanding here. With hadoop-2 the user is completely in control of his own classpath (we had a similar, but limited capability in hadoop-1 w/ https://issues.apache.org/jira/browse/MAPREDUCE-1938). Furthermore, it's probably not well known that in hadoop-2 the user application (MR or otherwise) can also pick the JDK version by using JAVA_HOME env for the container. So, in effect, MR applications can continue to use java6 while YARN is running java7 - this hasn't been tested extensively though. This capability did not exist in hadoop-1. We've also made some progress with https://issues.apache.org/jira/browse/MAPREDUCE-1700 to defuse user jar-deps from MR system jars. https://issues.apache.org/jira/browse/MAPREDUCE-4421 also helps by ensuring MR applications can pick exact version of MR jars they were compiled against; and not rely on cluster installs. Hope that helps somewhat. thanks, Arun -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
