[jira] [Created] (HDDS-158) DatanodeStateMachine endPoint task throws NullPointerException
Mukul Kumar Singh created HDDS-158:
--
Summary: DatanodeStateMachine endPoint task throws
NullPointerException
Key: HDDS-158
URL: https://issues.apache.org/jira/browse/HDDS-158
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: Ozone Datanode
Affects Versions: 0.2.1
Reporter: Mukul Kumar Singh
Fix For: 0.2.1
While running Ozone, I was able to see the following exception on the datanodes.
{code}
2018-06-09 00:15:09,541 [Datanode State Machine Thread - 0] ERROR - Error
in executing end point task.
java.util.concurrent.ExecutionException: java.lang.NullPointerException
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at
org.apache.hadoop.ozone.container.common.states.datanode.RunningDatanodeState.computeNextContainerState(RunningDatanodeState.java:137)
at
org.apache.hadoop.ozone.container.common.states.datanode.RunningDatanodeState.await(RunningDatanodeState.java:174)
at
org.apache.hadoop.ozone.container.common.states.datanode.RunningDatanodeState.await(RunningDatanodeState.java:46)
at
org.apache.hadoop.ozone.container.common.statemachine.StateContext.execute(StateContext.java:233)
at
org.apache.hadoop.ozone.container.common.statemachine.DatanodeStateMachine.start(DatanodeStateMachine.java:147)
at
org.apache.hadoop.ozone.container.common.statemachine.DatanodeStateMachine.lambda$startDaemon$0(DatanodeStateMachine.java:294)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
at
com.google.protobuf.GeneratedMessage$FieldAccessorTable.getField(GeneratedMessage.java:1535)
at
com.google.protobuf.GeneratedMessage$FieldAccessorTable.access$100(GeneratedMessage.java:1449)
at
com.google.protobuf.GeneratedMessage$Builder.setField(GeneratedMessage.java:366)
at
org.apache.hadoop.ozone.container.common.states.endpoint.HeartbeatEndpointTask.addReports(HeartbeatEndpointTask.java:128)
at
org.apache.hadoop.ozone.container.common.states.endpoint.HeartbeatEndpointTask.call(HeartbeatEndpointTask.java:106)
at
org.apache.hadoop.ozone.container.common.states.endpoint.HeartbeatEndpointTask.call(HeartbeatEndpointTask.java:51)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
... 1 more
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.0.3 (RC0)
Many thanks to all of you! Now we have many +1s including 3 from PMCs, and no -1s. Would like to call it a done vote! I will work on the remaining steps to push forward the release. Wish you all a very nice weekend, --Yongjun On Fri, Jun 8, 2018 at 2:45 PM, John Zhuge wrote: > Thanks Xongjun for the excellent work to drive this release! > > > +1 (binding) > >- Verified checksums and signatures of tarballs >- Built source with native, Oracle Java 1.8.0_152 on Mac OS X 10.13.5 >- Verified cloud connectors: > - ADLS integration tests passed with 1 failure, not a blocker >- Deployed both binary and built source to a pseudo cluster, passed >the following sanity tests in insecure and SSL mode: > - HDFS basic and ACL > - WebHDFS CLI ls and REST LISTSTATUS > - DistCp basic > - MapReduce wordcount > - KMS and HttpFS basic and servlets > - Balancer start/stop > > > ADLS unit test failure: > > > [ERROR] Tests run: 43, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: > 68.889 s <<< FAILURE! - in org.apache.hadoop.fs.adl.live. > TestAdlFileSystemContractLive > > [ERROR] testMkdirsWithUmask(org.apache.hadoop.fs.adl.live. > TestAdlFileSystemContractLive) Time elapsed: 0.851 s <<< FAILURE! > > java.lang.AssertionError: expected:<461> but was:<456> > > > See https://issues.apache.org/jira/browse/HADOOP-14435. I don't think it > is a blocker. > > Thanks, > > On Fri, Jun 8, 2018 at 12:04 PM, Xiao Chen wrote: > >> Thanks for the effort on this Yongjun. >> >> +1 (binding) >> >>- Built from src >>- Deployed a pseudo distributed HDFS with KMS >>- Ran basic hdfs commands with encryption >>- Sanity checked webui and logs >> >> >> >> -Xiao >> >> On Fri, Jun 8, 2018 at 10:34 AM, Brahma Reddy Battula < >> brahmareddy.batt...@hotmail.com> wrote: >> >> > Thanks yongjun zhang for driving this release. >> > >> > +1 (binding). >> > >> > >> > ---Built from the source >> > ---Installed HA cluster >> > ---Execute the basic shell commands >> > ---Browsed the UI's >> > ---Ran sample jobs like pi,wordcount >> > >> > >> > >> > >> > From: Yongjun Zhang >> > Sent: Friday, June 8, 2018 1:04 PM >> > To: Allen Wittenauer >> > Cc: Hadoop Common; Hdfs-dev; mapreduce-...@hadoop.apache.org; >> > yarn-...@hadoop.apache.org >> > Subject: Re: [VOTE] Release Apache Hadoop 3.0.3 (RC0) >> > >> > BTW, thanks Allen and Steve for discussing and suggestion about the site >> > build problem I hit earlier, I did the following step >> > >> > mvn install -DskipTests >> > >> > before doing the steps Nanda listed helped to solve the problems. >> > >> > --Yongjun >> > >> > >> > >> > >> > On Thu, Jun 7, 2018 at 6:15 PM, Yongjun Zhang >> wrote: >> > >> > > Thank you all very much for the testing, feedback and discussion! >> > > >> > > I was able to build outside docker, by following the steps Nanda >> > > described, I saw the same problem; then I tried 3.0.2 released a while >> > > back, it has the same issue. >> > > >> > > As Allen pointed out, it seems the steps to build site are not >> correct. I >> > > have not figured out the correct steps yet. >> > > >> > > At this point, I think this issue should not block the 3.0.3 issue. >> While >> > > at the same time we need to figure out the right steps to build the >> site. >> > > Would you please let me know if you think differently? >> > > >> > > We only have the site build issue reported so far. And we don't have >> > > enough PMC votes yet. So need some more PMCs to help. >> > > >> > > Thanks again, and best regards, >> > > >> > > --Yongjun >> > > >> > > >> > > On Thu, Jun 7, 2018 at 4:15 PM, Allen Wittenauer < >> > a...@effectivemachines.com >> > > > wrote: >> > > >> > >> > On Jun 7, 2018, at 11:47 AM, Steve Loughran < >> ste...@hortonworks.com> >> > >> wrote: >> > >> > >> > >> > Actually, Yongjun has been really good at helping me get set up >> for a >> > >> 2.7.7 release, including "things you need to do to get GPG working in >> > the >> > >> docker image” >> > >> >> > >> *shrugs* I use a different release script after some changes >> > >> broke the in-tree version for building on OS X and I couldn’t get the >> > fixes >> > >> committed upstream. So not sure what the problems are that you are >> > hitting. >> > >> >> > >> > On Jun 7, 2018, at 1:08 PM, Nandakumar Vadivelu < >> > >> nvadiv...@hortonworks.com> wrote: >> > >> > >> > >> > It will be helpful if we can get the correct steps, and also update >> > the >> > >> wiki. >> > >> > https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+ >> > >> Release+Validation >> > >> >> > >> Yup. Looking forward to seeing it. >> > >> >> - >> > >> To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org >> > >> For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org >> > >> >> > >> >> > > >> > >> > > > > -- > John >
Re: [VOTE] Release Apache Hadoop 3.0.3 (RC0)
Thanks Xongjun for the excellent work to drive this release! +1 (binding) - Verified checksums and signatures of tarballs - Built source with native, Oracle Java 1.8.0_152 on Mac OS X 10.13.5 - Verified cloud connectors: - ADLS integration tests passed with 1 failure, not a blocker - Deployed both binary and built source to a pseudo cluster, passed the following sanity tests in insecure and SSL mode: - HDFS basic and ACL - WebHDFS CLI ls and REST LISTSTATUS - DistCp basic - MapReduce wordcount - KMS and HttpFS basic and servlets - Balancer start/stop ADLS unit test failure: [ERROR] Tests run: 43, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 68.889 s <<< FAILURE! - in org.apache.hadoop.fs.adl.live.TestAdlFileSystemContractLive [ERROR] testMkdirsWithUmask(org.apache.hadoop.fs.adl.live.TestAdlFileSystemContractLive) Time elapsed: 0.851 s <<< FAILURE! java.lang.AssertionError: expected:<461> but was:<456> See https://issues.apache.org/jira/browse/HADOOP-14435. I don't think it is a blocker. Thanks, On Fri, Jun 8, 2018 at 12:04 PM, Xiao Chen wrote: > Thanks for the effort on this Yongjun. > > +1 (binding) > >- Built from src >- Deployed a pseudo distributed HDFS with KMS >- Ran basic hdfs commands with encryption >- Sanity checked webui and logs > > > -Xiao > > On Fri, Jun 8, 2018 at 10:34 AM, Brahma Reddy Battula < > brahmareddy.batt...@hotmail.com> wrote: > > > Thanks yongjun zhang for driving this release. > > > > +1 (binding). > > > > > > ---Built from the source > > ---Installed HA cluster > > ---Execute the basic shell commands > > ---Browsed the UI's > > ---Ran sample jobs like pi,wordcount > > > > > > > > > > From: Yongjun Zhang > > Sent: Friday, June 8, 2018 1:04 PM > > To: Allen Wittenauer > > Cc: Hadoop Common; Hdfs-dev; mapreduce-...@hadoop.apache.org; > > yarn-...@hadoop.apache.org > > Subject: Re: [VOTE] Release Apache Hadoop 3.0.3 (RC0) > > > > BTW, thanks Allen and Steve for discussing and suggestion about the site > > build problem I hit earlier, I did the following step > > > > mvn install -DskipTests > > > > before doing the steps Nanda listed helped to solve the problems. > > > > --Yongjun > > > > > > > > > > On Thu, Jun 7, 2018 at 6:15 PM, Yongjun Zhang > wrote: > > > > > Thank you all very much for the testing, feedback and discussion! > > > > > > I was able to build outside docker, by following the steps Nanda > > > described, I saw the same problem; then I tried 3.0.2 released a while > > > back, it has the same issue. > > > > > > As Allen pointed out, it seems the steps to build site are not > correct. I > > > have not figured out the correct steps yet. > > > > > > At this point, I think this issue should not block the 3.0.3 issue. > While > > > at the same time we need to figure out the right steps to build the > site. > > > Would you please let me know if you think differently? > > > > > > We only have the site build issue reported so far. And we don't have > > > enough PMC votes yet. So need some more PMCs to help. > > > > > > Thanks again, and best regards, > > > > > > --Yongjun > > > > > > > > > On Thu, Jun 7, 2018 at 4:15 PM, Allen Wittenauer < > > a...@effectivemachines.com > > > > wrote: > > > > > >> > On Jun 7, 2018, at 11:47 AM, Steve Loughran > > > >> wrote: > > >> > > > >> > Actually, Yongjun has been really good at helping me get set up for > a > > >> 2.7.7 release, including "things you need to do to get GPG working in > > the > > >> docker image” > > >> > > >> *shrugs* I use a different release script after some changes > > >> broke the in-tree version for building on OS X and I couldn’t get the > > fixes > > >> committed upstream. So not sure what the problems are that you are > > hitting. > > >> > > >> > On Jun 7, 2018, at 1:08 PM, Nandakumar Vadivelu < > > >> nvadiv...@hortonworks.com> wrote: > > >> > > > >> > It will be helpful if we can get the correct steps, and also update > > the > > >> wiki. > > >> > https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+ > > >> Release+Validation > > >> > > >> Yup. Looking forward to seeing it. > > >> - > > >> To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org > > >> For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org > > >> > > >> > > > > > > -- John
Apache Hadoop qbt Report: trunk+JDK8 on Windows/x64
For more details, see https://builds.apache.org/job/hadoop-trunk-win/492/ [Jun 7, 2018 2:55:49 PM] (stevel) HADOOP-15512. Clean up Shell from JDK7 workarounds. Contributed by Zsolt [Jun 7, 2018 3:30:06 PM] (weichiu) HDFS-13659. Add more test coverage for contentSummary for snapshottable [Jun 7, 2018 4:01:15 PM] (rohithsharmaks) YARN-8386. App log can not be viewed from Logs tab in secure cluster. [Jun 7, 2018 6:16:17 PM] (jhc) HDFS-13534. libhdfs++: Fix GCC7 build. Contributed by James Clampffer. [Jun 7, 2018 7:47:58 PM] (Bharat) HDDS-119:Skip Apache license header check for some ozone doc scripts. [Jun 7, 2018 8:02:57 PM] (jhc) HDFS-13615. libhdfs++ SaslProtocol hanging while accessing invalid lock. [Jun 7, 2018 8:34:52 PM] (inigoiri) HADOOP-15516. Add test cases to cover FileUtil#readLink. Contributed by [Jun 7, 2018 9:11:20 PM] (arp) HDDS-147. Update Ozone site docs. Contributed by Arpit Agarwal. [Jun 7, 2018 9:35:22 PM] (aengineer) HDDS-127. Add CloseContainerEventHandler in SCM. Contributed by [Jun 7, 2018 11:10:15 PM] (inigoiri) YARN-8400. Fix typos in YARN Federation documentation page. Contributed [Jun 7, 2018 11:38:23 PM] (miklos.szegedi) YARN-6677. Preempt opportunistic containers when root container cgroup [Jun 8, 2018 12:09:34 AM] (rkanter) Disable mounting cgroups by default (miklos.szeg...@cloudera.com via [Jun 8, 2018 12:13:03 AM] (inigoiri) YARN-8359. Exclude containermanager.linux test classes on Windows. [Jun 8, 2018 6:00:26 AM] (jitendra) HADOOP-15482. Upgrade jackson-databind to version 2.9.5. Contributed by -1 overall The following subsystems voted -1: compile mvninstall pathlen unit The following subsystems voted -1 but were configured to be filtered/ignored: cc javac The following subsystems are considered long running: (runtime bigger than 1h 00m 00s) unit Specific tests: Failed junit tests : hadoop.crypto.key.kms.server.TestKMS hadoop.cli.TestAclCLI hadoop.cli.TestAclCLIWithPosixAclInheritance hadoop.cli.TestCacheAdminCLI hadoop.cli.TestCryptoAdminCLI hadoop.cli.TestDeleteCLI hadoop.cli.TestErasureCodingCLI hadoop.cli.TestHDFSCLI hadoop.cli.TestXAttrCLI hadoop.fs.contract.hdfs.TestHDFSContractAppend hadoop.fs.contract.hdfs.TestHDFSContractConcat hadoop.fs.contract.hdfs.TestHDFSContractCreate hadoop.fs.contract.hdfs.TestHDFSContractDelete hadoop.fs.contract.hdfs.TestHDFSContractGetFileStatus hadoop.fs.contract.hdfs.TestHDFSContractMkdir hadoop.fs.contract.hdfs.TestHDFSContractOpen hadoop.fs.contract.hdfs.TestHDFSContractPathHandle hadoop.fs.contract.hdfs.TestHDFSContractRename hadoop.fs.contract.hdfs.TestHDFSContractRootDirectory hadoop.fs.contract.hdfs.TestHDFSContractSeek hadoop.fs.contract.hdfs.TestHDFSContractSetTimes hadoop.fs.loadGenerator.TestLoadGenerator hadoop.fs.permission.TestStickyBit hadoop.fs.shell.TestHdfsTextCommand hadoop.fs.TestEnhancedByteBufferAccess hadoop.fs.TestFcHdfsCreateMkdir hadoop.fs.TestFcHdfsPermission hadoop.fs.TestFcHdfsSetUMask hadoop.fs.TestGlobPaths hadoop.fs.TestHDFSFileContextMainOperations hadoop.fs.TestHdfsNativeCodeLoader hadoop.fs.TestResolveHdfsSymlink hadoop.fs.TestSWebHdfsFileContextMainOperations hadoop.fs.TestSymlinkHdfsDisable hadoop.fs.TestSymlinkHdfsFileContext hadoop.fs.TestSymlinkHdfsFileSystem hadoop.fs.TestUnbuffer hadoop.fs.TestUrlStreamHandler hadoop.fs.TestWebHdfsFileContextMainOperations hadoop.fs.viewfs.TestViewFileSystemAtHdfsRoot hadoop.fs.viewfs.TestViewFileSystemHdfs hadoop.fs.viewfs.TestViewFileSystemLinkFallback hadoop.fs.viewfs.TestViewFileSystemLinkMergeSlash hadoop.fs.viewfs.TestViewFileSystemWithAcls hadoop.fs.viewfs.TestViewFileSystemWithTruncate hadoop.fs.viewfs.TestViewFileSystemWithXAttrs hadoop.fs.viewfs.TestViewFsAtHdfsRoot hadoop.fs.viewfs.TestViewFsDefaultValue hadoop.fs.viewfs.TestViewFsFileStatusHdfs hadoop.fs.viewfs.TestViewFsHdfs hadoop.fs.viewfs.TestViewFsWithAcls hadoop.fs.viewfs.TestViewFsWithXAttrs hadoop.hdfs.client.impl.TestBlockReaderLocal hadoop.hdfs.client.impl.TestBlockReaderLocalLegacy hadoop.hdfs.client.impl.TestBlockReaderRemote hadoop.hdfs.client.impl.TestClientBlockVerification hadoop.hdfs.crypto.TestHdfsCryptoStreams hadoop.hdfs.protocol.datatransfer.sasl.TestSaslDataTransfer hadoop.hdfs.qjournal.client.TestEpochsAreUnique hadoop.hdfs.qjournal.client.TestQJMWithFaults hadoop.hdfs.qjournal.client.TestQuorumJournalManager hadoop.hdfs.qjournal.server.TestJournal hadoop.hdfs.qjournal.server.TestJournalNode
[jira] [Created] (HDFS-13667) Typo: Marking all "datandoes" as stale
Wei-Chiu Chuang created HDFS-13667:
--
Summary: Typo: Marking all "datandoes" as stale
Key: HDFS-13667
URL: https://issues.apache.org/jira/browse/HDFS-13667
Project: Hadoop HDFS
Issue Type: Bug
Components: namenode
Reporter: Wei-Chiu Chuang
{code:title=DatanodeManager#markAllDatanodesStale}
public void markAllDatanodesStale() {
LOG.info("Marking all datandoes as stale");
synchronized (this) {
for (DatanodeDescriptor dn : datanodeMap.values()) {
for(DatanodeStorageInfo storage : dn.getStorageInfos()) {
storage.markStaleAfterFailover();
}
}
}
}
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.0.3 (RC0)
Thanks for the effort on this Yongjun. +1 (binding) - Built from src - Deployed a pseudo distributed HDFS with KMS - Ran basic hdfs commands with encryption - Sanity checked webui and logs -Xiao On Fri, Jun 8, 2018 at 10:34 AM, Brahma Reddy Battula < brahmareddy.batt...@hotmail.com> wrote: > Thanks yongjun zhang for driving this release. > > +1 (binding). > > > ---Built from the source > ---Installed HA cluster > ---Execute the basic shell commands > ---Browsed the UI's > ---Ran sample jobs like pi,wordcount > > > > > From: Yongjun Zhang > Sent: Friday, June 8, 2018 1:04 PM > To: Allen Wittenauer > Cc: Hadoop Common; Hdfs-dev; mapreduce-...@hadoop.apache.org; > yarn-...@hadoop.apache.org > Subject: Re: [VOTE] Release Apache Hadoop 3.0.3 (RC0) > > BTW, thanks Allen and Steve for discussing and suggestion about the site > build problem I hit earlier, I did the following step > > mvn install -DskipTests > > before doing the steps Nanda listed helped to solve the problems. > > --Yongjun > > > > > On Thu, Jun 7, 2018 at 6:15 PM, Yongjun Zhang wrote: > > > Thank you all very much for the testing, feedback and discussion! > > > > I was able to build outside docker, by following the steps Nanda > > described, I saw the same problem; then I tried 3.0.2 released a while > > back, it has the same issue. > > > > As Allen pointed out, it seems the steps to build site are not correct. I > > have not figured out the correct steps yet. > > > > At this point, I think this issue should not block the 3.0.3 issue. While > > at the same time we need to figure out the right steps to build the site. > > Would you please let me know if you think differently? > > > > We only have the site build issue reported so far. And we don't have > > enough PMC votes yet. So need some more PMCs to help. > > > > Thanks again, and best regards, > > > > --Yongjun > > > > > > On Thu, Jun 7, 2018 at 4:15 PM, Allen Wittenauer < > a...@effectivemachines.com > > > wrote: > > > >> > On Jun 7, 2018, at 11:47 AM, Steve Loughran > >> wrote: > >> > > >> > Actually, Yongjun has been really good at helping me get set up for a > >> 2.7.7 release, including "things you need to do to get GPG working in > the > >> docker image” > >> > >> *shrugs* I use a different release script after some changes > >> broke the in-tree version for building on OS X and I couldn’t get the > fixes > >> committed upstream. So not sure what the problems are that you are > hitting. > >> > >> > On Jun 7, 2018, at 1:08 PM, Nandakumar Vadivelu < > >> nvadiv...@hortonworks.com> wrote: > >> > > >> > It will be helpful if we can get the correct steps, and also update > the > >> wiki. > >> > https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+ > >> Release+Validation > >> > >> Yup. Looking forward to seeing it. > >> - > >> To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org > >> For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org > >> > >> > > >
[jira] [Created] (HDFS-13666) Only use existing SASL libraries in libhdfs++
James Clampffer created HDFS-13666: -- Summary: Only use existing SASL libraries in libhdfs++ Key: HDFS-13666 URL: https://issues.apache.org/jira/browse/HDFS-13666 Project: Hadoop HDFS Issue Type: Task Reporter: James Clampffer At one point there were plans to implement SASL support from scratch inside of libhdfs++. Since then we added added Kerberos support through Cyrus SASL and GSasl. I propose removing the partial implementation from libhdfs++ since doing the secure handshake and encryption are still marked TODO but it's unclear what getting that done involves. That way we can focus time on using Cyrus (GSasl uses GPL) to support DIGEST-MD5 auth and wire encryption. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.0.3 (RC0)
Thanks yongjun zhang for driving this release. +1 (binding). ---Built from the source ---Installed HA cluster ---Execute the basic shell commands ---Browsed the UI's ---Ran sample jobs like pi,wordcount From: Yongjun Zhang Sent: Friday, June 8, 2018 1:04 PM To: Allen Wittenauer Cc: Hadoop Common; Hdfs-dev; mapreduce-...@hadoop.apache.org; yarn-...@hadoop.apache.org Subject: Re: [VOTE] Release Apache Hadoop 3.0.3 (RC0) BTW, thanks Allen and Steve for discussing and suggestion about the site build problem I hit earlier, I did the following step mvn install -DskipTests before doing the steps Nanda listed helped to solve the problems. --Yongjun On Thu, Jun 7, 2018 at 6:15 PM, Yongjun Zhang wrote: > Thank you all very much for the testing, feedback and discussion! > > I was able to build outside docker, by following the steps Nanda > described, I saw the same problem; then I tried 3.0.2 released a while > back, it has the same issue. > > As Allen pointed out, it seems the steps to build site are not correct. I > have not figured out the correct steps yet. > > At this point, I think this issue should not block the 3.0.3 issue. While > at the same time we need to figure out the right steps to build the site. > Would you please let me know if you think differently? > > We only have the site build issue reported so far. And we don't have > enough PMC votes yet. So need some more PMCs to help. > > Thanks again, and best regards, > > --Yongjun > > > On Thu, Jun 7, 2018 at 4:15 PM, Allen Wittenauer > wrote: > >> > On Jun 7, 2018, at 11:47 AM, Steve Loughran >> wrote: >> > >> > Actually, Yongjun has been really good at helping me get set up for a >> 2.7.7 release, including "things you need to do to get GPG working in the >> docker image” >> >> *shrugs* I use a different release script after some changes >> broke the in-tree version for building on OS X and I couldn’t get the fixes >> committed upstream. So not sure what the problems are that you are hitting. >> >> > On Jun 7, 2018, at 1:08 PM, Nandakumar Vadivelu < >> nvadiv...@hortonworks.com> wrote: >> > >> > It will be helpful if we can get the correct steps, and also update the >> wiki. >> > https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+ >> Release+Validation >> >> Yup. Looking forward to seeing it. >> - >> To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org >> For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org >> >> >
Re: [VOTE] Release Apache Hadoop 3.0.3 (RC0)
Hello, PMC votes are needed for release, can I get some help from PMCs? Really appreciate it! --Yongjun On Thu, Jun 7, 2018 at 6:15 PM, Yongjun Zhang wrote: > Thank you all very much for the testing, feedback and discussion! > > I was able to build outside docker, by following the steps Nanda > described, I saw the same problem; then I tried 3.0.2 released a while > back, it has the same issue. > > As Allen pointed out, it seems the steps to build site are not correct. I > have not figured out the correct steps yet. > > At this point, I think this issue should not block the 3.0.3 issue. While > at the same time we need to figure out the right steps to build the site. > Would you please let me know if you think differently? > > We only have the site build issue reported so far. And we don't have > enough PMC votes yet. So need some more PMCs to help. > > Thanks again, and best regards, > > --Yongjun > > > On Thu, Jun 7, 2018 at 4:15 PM, Allen Wittenauer > wrote: > >> > On Jun 7, 2018, at 11:47 AM, Steve Loughran >> wrote: >> > >> > Actually, Yongjun has been really good at helping me get set up for a >> 2.7.7 release, including "things you need to do to get GPG working in the >> docker image” >> >> *shrugs* I use a different release script after some changes >> broke the in-tree version for building on OS X and I couldn’t get the fixes >> committed upstream. So not sure what the problems are that you are hitting. >> >> > On Jun 7, 2018, at 1:08 PM, Nandakumar Vadivelu < >> nvadiv...@hortonworks.com> wrote: >> > >> > It will be helpful if we can get the correct steps, and also update the >> wiki. >> > https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+ >> Release+Validation >> >> Yup. Looking forward to seeing it. >> - >> To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org >> For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org >> >> >
[jira] [Created] (HDFS-13665) Move RPC response serialization into Server.doResponse
Plamen Jeliazkov created HDFS-13665: --- Summary: Move RPC response serialization into Server.doResponse Key: HDFS-13665 URL: https://issues.apache.org/jira/browse/HDFS-13665 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Plamen Jeliazkov Assignee: Plamen Jeliazkov In HDFS-13399 we addressed a race condition in AlignmentContext processing where the RPC response would assign a transactionId independently of the transactions own processing, resulting in a stateId response that was lower than expected. See here: https://issues.apache.org/jira/browse/HDFS-13399?focusedCommentId=16464279=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16464279 And here: https://issues.apache.org/jira/browse/HDFS-13399?focusedCommentId=16498660=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16498660 At the end if was agreed upon to move the logic of Server.setupResponse into Server.doResponse directly. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/805/ [Jun 7, 2018 4:46:02 AM] (brahma) HDFS-12950. [oiv] ls will fail in secure cluster. Contributed by [Jun 7, 2018 4:55:56 AM] (rohithsharmaks) YARN-8399. NodeManager is giving 403 GSS exception post upgrade to 3.1 [Jun 7, 2018 2:55:49 PM] (stevel) HADOOP-15512. Clean up Shell from JDK7 workarounds. Contributed by Zsolt [Jun 7, 2018 3:30:06 PM] (weichiu) HDFS-13659. Add more test coverage for contentSummary for snapshottable [Jun 7, 2018 4:01:15 PM] (rohithsharmaks) YARN-8386. App log can not be viewed from Logs tab in secure cluster. [Jun 7, 2018 6:16:17 PM] (jhc) HDFS-13534. libhdfs++: Fix GCC7 build. Contributed by James Clampffer. [Jun 7, 2018 7:47:58 PM] (Bharat) HDDS-119:Skip Apache license header check for some ozone doc scripts. [Jun 7, 2018 8:02:57 PM] (jhc) HDFS-13615. libhdfs++ SaslProtocol hanging while accessing invalid lock. [Jun 7, 2018 8:34:52 PM] (inigoiri) HADOOP-15516. Add test cases to cover FileUtil#readLink. Contributed by [Jun 7, 2018 9:11:20 PM] (arp) HDDS-147. Update Ozone site docs. Contributed by Arpit Agarwal. [Jun 7, 2018 9:35:22 PM] (aengineer) HDDS-127. Add CloseContainerEventHandler in SCM. Contributed by [Jun 7, 2018 11:10:15 PM] (inigoiri) YARN-8400. Fix typos in YARN Federation documentation page. Contributed [Jun 7, 2018 11:38:23 PM] (miklos.szegedi) YARN-6677. Preempt opportunistic containers when root container cgroup [Jun 8, 2018 12:09:34 AM] (rkanter) Disable mounting cgroups by default (miklos.szeg...@cloudera.com via [Jun 8, 2018 12:13:03 AM] (inigoiri) YARN-8359. Exclude containermanager.linux test classes on Windows. -1 overall The following subsystems voted -1: asflicense findbugs pathlen unit xml The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: FindBugs : module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Inconsistent synchronization of org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.AllocationFileLoaderService.reloadListener; locked 75% of time Unsynchronized access at AllocationFileLoaderService.java:75% of time Unsynchronized access at AllocationFileLoaderService.java:[line 117] Failed junit tests : hadoop.hdfs.web.TestWebHdfsTimeouts hadoop.hdfs.TestDFSStripedOutputStreamWithRandomECPolicy hadoop.hdfs.server.namenode.TestDecommissioningStatus hadoop.yarn.server.resourcemanager.reservation.TestCapacityOverTimePolicy hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineStorageEntities hadoop.yarn.server.timelineservice.reader.TestTimelineReaderWebServicesHBaseStorage hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineStorageDomain hadoop.yarn.server.timelineservice.storage.flow.TestHBaseStorageFlowRunCompaction hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineStorageSchema hadoop.yarn.server.timelineservice.storage.flow.TestHBaseStorageFlowRun hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineStorageApps hadoop.yarn.server.timelineservice.storage.flow.TestHBaseStorageFlowActivity hadoop.mapred.TestMRTimelineEventHandling cc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/805/artifact/out/diff-compile-cc-root.txt [4.0K] javac: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/805/artifact/out/diff-compile-javac-root.txt [336K] checkstyle: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/805/artifact/out/diff-checkstyle-root.txt [4.0K] pathlen: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/805/artifact/out/pathlen.txt [12K] pylint: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/805/artifact/out/diff-patch-pylint.txt [24K] shellcheck: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/805/artifact/out/diff-patch-shellcheck.txt [20K] shelldocs: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/805/artifact/out/diff-patch-shelldocs.txt [16K] whitespace: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/805/artifact/out/whitespace-eol.txt [9.4M] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/805/artifact/out/whitespace-tabs.txt [1.1M] xml: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/805/artifact/out/xml.txt [4.0K] findbugs:
