Apache Hadoop qbt Report: trunk+JDK9 on Linux/x86

[Apache Jenkins Server]

For more details, see 
https://builds.apache.org/job/hadoop-qbt-trunk-java9-linux-x86/21/

[Apr 22, 2018 3:07:19 PM] (arp) HDFS-13055. Aggregate usage statistics from 
datanodes. Contributed by
[Apr 23, 2018 2:49:35 AM] (inigoiri) HDFS-13388. RequestHedgingProxyProvider 
calls multiple configured NNs
[Apr 23, 2018 8:02:09 AM] (sunilg) YARN-7956. [UI2] Avoid duplicating 
Components link under
[Apr 23, 2018 8:06:27 AM] (sunilg) YARN-8177. Documentation changes for auto 
creation of Leaf Queues with
[Apr 23, 2018 5:08:53 PM] (arp) HDFS-13433. webhdfs requests can be routed 
incorrectly in federated
[Apr 23, 2018 6:13:18 PM] (cdouglas) HDFS-13408. MiniDFSCluster to support 
being built on randomized base
[Apr 23, 2018 9:14:05 PM] (lei) Update CHANGES, RELEASENOTES, and jdiff for 
3.0.2 release.
[Apr 23, 2018 10:26:53 PM] (fabbri) HADOOP-14756 S3Guard: expose capability 
query in MetadataStore and add
[Apr 23, 2018 10:44:15 PM] (rkanter) HADOOP-15390. Yarn RM logs flooded by 
DelegationTokenRenewer trying to
[Apr 23, 2018 11:09:01 PM] (Bharat) HDFS-13356. Balancer:Set default value of 
minBlockSize to 10mb.
[Apr 24, 2018 2:12:16 AM] (inigoiri) HDFS-13336. Test cases of 
TestWriteToReplica failed in windows.
[Apr 24, 2018 3:25:23 AM] (yqlin) HDFS-13490. RBF: Fix setSafeMode in the 
Router. Contributed by Inigo
[Apr 24, 2018 10:29:14 AM] (junping_du) YARN-7598. Document how to use 
classpath isolation for aux-services in
[Apr 24, 2018 6:30:38 PM] (rkanter) MAPREDUCE-7072. mapred job -history prints 
duplicate counter in human
[Apr 25, 2018 10:50:52 PM] (omalley) HDFS-8456. Introduce 
STORAGE_CONTAINER_SERVICE as a new NodeType.
[Apr 25, 2018 10:51:00 PM] (omalley) HDFS-8614. OzoneHandler : Add Quota 
Support. (Contributed by Anu
[Apr 25, 2018 10:51:00 PM] (omalley) HDFS-8448. Create REST Interface for 
Volumes. Contributed by Anu
[Apr 25, 2018 10:51:00 PM] (omalley) HDFS-8637. OzoneHandler : Add Error Table. 
(Contributed by Anu Engineer)
[Apr 25, 2018 10:51:00 PM] (omalley) HDFS-8634. OzoneHandler: Add userAuth 
Interface and Simple userAuth
[Apr 25, 2018 10:51:00 PM] (omalley) HDFS-8644. OzoneHandler : Add volume 
handler. (Contributed by Anu
[Apr 25, 2018 10:51:00 PM] (omalley) HDFS-8654. OzoneHandler : Add ACL support. 
(Contributed by Anu Engineer)
[Apr 25, 2018 10:51:00 PM] (omalley) HDFS-8680. OzoneHandler : Add Local 
StorageHandler support for volumes.
[Apr 25, 2018 10:51:00 PM] (omalley) HDFS-8717. OzoneHandler : Add common 
bucket objects. (Contributed by Anu
[Apr 25, 2018 10:51:00 PM] (omalley) HDFS-8753. Ozone: Unify 
StorageContainerConfiguration with
[Apr 25, 2018 10:51:00 PM] (omalley) HDFS-8695. OzoneHandler : Add Bucket REST 
Interface. (aengineer)
[Apr 25, 2018 10:51:36 PM] (omalley) HDFS-8527. OzoneHandler: Integration of 
REST interface and container
[Apr 25, 2018 10:51:39 PM] (omalley) HDFS-8757 : OzoneHandler : Add 
localStorageHandler support for Buckets.
[Apr 25, 2018 10:51:39 PM] (omalley) HDFS-9834. OzoneHandler : Enable 
MiniDFSCluster based testing for Ozone.
[Apr 25, 2018 10:51:39 PM] (omalley) HDFS-9845. OzoneHandler : Support List and 
Info Volumes. Contributed by
[Apr 25, 2018 10:51:51 PM] (omalley) HDFS-9853. Ozone: Add container 
definitions. Contributed by Anu
[Apr 25, 2018 10:51:53 PM] (omalley) HDFS-9848. Ozone: Add Ozone Client lib for 
volume handling. Contributed
[Apr 25, 2018 10:51:53 PM] (omalley) HDFS-9907. Exclude Ozone 
protobuf-generated classes from Findbugs
[Apr 25, 2018 10:51:53 PM] (omalley) HDFS-9873. Ozone: Add container transport 
server. Contributed by Anu
[Apr 25, 2018 10:51:58 PM] (omalley) HDFS-9891. Ozone: Add container transport 
client. Contributed by Anu
[Apr 25, 2018 10:52:00 PM] (omalley) HDFS-9920. Stop tracking CHANGES.txt in 
the HDFS-7240 feature branch.
[Apr 25, 2018 10:52:00 PM] (omalley) HDFS-9916. OzoneHandler : Add Key handler. 
Contributed by Anu Engineer.
[Apr 25, 2018 10:52:00 PM] (omalley) HDFS-9916. OzoneHandler : Add Key handler. 
Contributed by Anu Engineer.
[Apr 25, 2018 10:52:00 PM] (omalley) HDFS-9925. Ozone: Add Ozone Client lib for 
bucket handling. Contributed
[Apr 25, 2018 10:52:00 PM] (omalley) HDFS-9926. ozone : Add volume commands to 
CLI. Contributed by Anu
[Apr 25, 2018 10:52:00 PM] (omalley) HDFS-9944. Ozone : Add container 
dispatcher. Contributed by Anu
[Apr 25, 2018 10:52:00 PM] (omalley) HDFS-9961. Ozone: Add buckets commands to 
CLI. Contributed by Anu
[Apr 25, 2018 10:52:00 PM] (omalley) HDFS-10180. Ozone: Refactor container 
Namespace. Contributed by Anu
[Apr 25, 2018 10:52:00 PM] (omalley) HDFS-9960. OzoneHandler : Add localstorage 
support for keys. Contributed
[Apr 25, 2018 10:52:07 PM] (omalley) HDFS-10179. Ozone: Adding logging support. 
Contributed by Anu Engineer.
[Apr 25, 2018 10:52:09 PM] (omalley) HDFS-10196. Ozone : Enable better error 
reporting for failed commands in
[Apr 25, 2018 10:52:25 PM] (omalley) HDFS-10202. ozone : Add key commands to 
CLI. Contributed by Anu
[Apr 25, 2018 10:52:33 PM] 

[jira] [Created] (HDFS-13515) NetUtils#connect should log remote address for NoRouteToHostException

[Ted Yu (JIRA)]

Ted Yu created HDFS-13515:
-

 Summary: NetUtils#connect should log remote address for 
NoRouteToHostException
 Key: HDFS-13515
 URL: https://issues.apache.org/jira/browse/HDFS-13515
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Ted Yu


{code}
hdfs.BlockReaderFactory: I/O error constructing remote block reader.
java.net.NoRouteToHostException: No route to host
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at 
org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:529)
at org.apache.hadoop.hdfs.DFSClient.newConnectedPeer(DFSClient.java:2884)
{code}
In the above stack trace, the remote host was not logged.
This makes troubleshooting a bit hard.

NetUtils#connect should log remote address for NoRouteToHostException .



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

Re: [RESULT][VOTE] Merge HDDS (HDFS-7240) *code* into trunk

[Brahma Reddy Battula]

Sorry for landing late.

Adding separate repo using "git submodule" wn't be good idea here, to get
history/track commits for better maintaince..?

Reference:
http://blog.joncairns.com/2011/10/how-to-use-git-submodules/

On Fri, Apr 27, 2018 at 6:33 AM, Xiaoyu Yao  wrote:

> Thanks Owen for fixing this. Sorry, I was not able to spot the issue
> before the merge is pushed up to trunk.
>
> I compared the changes and the results are the same. I also finish ozone
> testing and verify the basic hdfs and ozone functionality are working fine.
> Moving forward, I will add document on Hadoop wiki to avoid similar issue
> in future.
>
> Regards,
> Xiaoyu
>
> On 4/26/18, 9:59 AM, "Chris Douglas"  wrote:
>
> Thank you for doing this, Owen. If the result is the same, then the
> rebased version is much easier to work with. If not this, then we can
> squash the branch into a single merge commit. The current state would
> be a distant, third choice.
>
> +1 -C
>
> On Thu, Apr 26, 2018 at 5:49 AM, Owen O'Malley 
> wrote:
> > Ok, it was bad enough that I spent yesterday afternoon hacking a fix
> for it.
> >
> > I basically did a git rebase with always picking the current master
> version
> > and then put a new fixup commit to bring the branches into sync.
> >
> > The original - https://github.com/omalley/hadoop/tree/hdfs-7240
> > The rebase hack - https://github.com/omalley/
> hadoop/tree/hdfs-7240-rebase
> >
> > Compare the git log --graph difference for one of the recent commits:
> >
> > | * | | | | | | | | | | | | | | | | | | | | | | | commit
> > 919ae746c3a4a424a77e6359af0abe3d7d2c1541
> > | | | | | | | | | | | | | | | | | | | | | | | | | Author: Weiwei
> Yang <
> > w...@apache.org>
> > | | | | | | | | | | | | | | | | | | | | | | | | | Date:   Wed Nov 1
> > 17:17:03 2017 +0800
> > | | | | | | | | | | | | | | | | | | | | | | | | |
> > | | | | | | | | | | | | | | | | | | | | | | | | | HDFS-12750.
> Ozone:
> > Fix TestStorageContainerManager#testBlockDeletionTransactions.
> Contributed
> > by Xiaoyu Yao.
> > | | | | | | | | | | | | | | | | | | | | | | | | |
> >
> > versus
> >
> > * commit 05592658078acef10c3995940510ff09792e
> > | Author: Weiwei Yang 
> > | Date:   Wed Nov 1 17:17:03 2017 +0800
> > |
> > | HDFS-12750. Ozone: Fix
> > TestStorageContainerManager#testBlockDeletionTransactions.
> Contributed by
> > Xiaoyu Yao.
> > |
> >
> > I'd like permission to edit the hadoop git history to make the
> change.
> >
> > Thoughts?
> >
> > ... Owen
> >
> > On Wed, Apr 25, 2018 at 11:47 AM, Xiaoyu Yao 
> wrote:
> >
> >> We followed the process laid out in a mail thread from 2015 that
> had two
> >> workflows proposed.
> >>
> >> “Git rebase” workflows and “Git merge” workflows.
> >> Ozone had followed git merge workflow, which merged trunk at a
> regular
> >> frequency into ozone, and then ozone was merged back.
> >>
> >> Here is the mail that we followed for the merge process.
> >> https://lists.apache.org/thread.html/43cd65c6b6c3c0e8ac
> >>
> >> Thanks
> >> Xiaoyu
> >>
> >>
> >>
> >>
> >>
> >> *From: *Owen O'Malley 
> >> *Date: *Wednesday, April 25, 2018 at 11:15 AM
> >> *To: *Chris Douglas 
> >> *Cc: *Xiaoyu Yao , Jitendra Pandey <
> >> jiten...@hortonworks.com>, Hdfs-dev 
> >> *Subject: *Re: [RESULT][VOTE] Merge HDDS (HDFS-7240) *code* into
> trunk
> >>
> >>
> >>
> >> *sigh*
> >>
> >>
> >>
> >> I asked you guys to rebase this before merge. It is tempting to
> squash it
> >> into a single commit.
> >>
> >>
> >>
> >> .. Owen
> >>
> >>
> >>
> >> On Wed, Apr 25, 2018 at 11:06 AM, Chris Douglas <
> cdoug...@apache.org>
> >> wrote:
> >>
> >> This really made a mess of trunk. Periodically merging trunk into
> the
> >> HDFS-7240 branch, then merging the whole thing back, created a
> tangle
> >> of references that's very difficult to work with (look at the output
> >> of git log --graph).
> >>
> >> I'm not sure it's even possible to fix this, but this is why feature
> >> branches should rebase and create a merge commit with --no-ff. -C
> >>
> >>
> >>
> >> On Tue, Apr 24, 2018 at 1:20 PM, Xiaoyu Yao 
> wrote:
> >> > I just merged the branch to trunk (834 commits in total)
> >> > Again, thanks for all who contributed to HDDS/Ozone!
> >> >
> >> > Cheers
> >> > -Xiaoyu
> >> >
> >> > On 4/23/18, 7:26 PM, "Jitendra Pandey" 
> wrote:
> >> >
> >> > The