Apache Hadoop qbt Report: trunk+JDK8 on Windows/x64

2018-06-02 Thread Apache Jenkins Server 
For more details, see https://builds.apache.org/job/hadoop-trunk-win/486/

[Jun 1, 2018 7:54:47 PM] (aengineer) HDDS-137. Cleanup Hdds-ozone CheckStyle 
Issues. Contributed by Anu
[Jun 1, 2018 9:21:35 PM] (aengineer) HDDS-142. TestMetadataStore fails on 
Windows. Contributed by  Íñigo
[Jun 1, 2018 9:42:39 PM] (templedf) HDFS-13636. Cross-Site Scripting 
vulnerability in HttpServer2
[Jun 1, 2018 9:49:17 PM] (wangda) YARN-8384. stdout.txt, stderr.txt logs of a 
launched docker container is
[Jun 1, 2018 9:49:18 PM] (wangda) YARN-7962. Race Condition When Stopping 
DelegationTokenRenewer causes RM
[Jun 1, 2018 9:49:18 PM] (wangda) YARN-8372. Distributed shell app master 
should not release containers
[Jun 1, 2018 9:49:18 PM] (wangda) YARN-8349. Remove YARN registry entries when 
a service is killed by the
[Jun 1, 2018 10:36:30 PM] (aengineer) HDDS-144. Fix TestEndPoint#testHeartbeat 
and TestEndPoint#testRegister.
[Jun 1, 2018 10:36:30 PM] (aengineer) HDDS-143. Provide docker compose files to 
measure performance in a
[Jun 1, 2018 11:29:13 PM] (haibochen) YARN-8375. 
TestCGroupElasticMemoryController fails surefire build.
[Jun 1, 2018 11:59:04 PM] (inigoiri) HDFS-13637. RBF: Router fails when 
threadIndex (in ConnectionPool) wraps
[Jun 2, 2018 3:57:03 AM] (rohithsharmaks) MapReduce JHS should honor
[Jun 2, 2018 3:59:06 AM] (rohithsharmaks) More YARN pages need to honor




-1 overall


The following subsystems voted -1:
compile mvninstall pathlen unit


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc javac


The following subsystems are considered long running:
(runtime bigger than 1h 00m 00s)
unit


Specific tests:

Failed junit tests :

   hadoop.crypto.TestCryptoStreamsWithOpensslAesCtrCryptoCodec 
   hadoop.fs.contract.rawlocal.TestRawlocalContractAppend 
   hadoop.fs.TestFsShellCopy 
   hadoop.fs.TestFsShellList 
   hadoop.fs.TestLocalFileSystem 
   hadoop.fs.TestTrash 
   hadoop.fs.viewfs.TestViewFileSystemLocalFileSystem 
   hadoop.http.TestHttpServer 
   hadoop.http.TestHttpServerLogs 
   hadoop.io.compress.TestCodec 
   hadoop.io.nativeio.TestNativeIO 
   hadoop.ipc.TestSocketFactory 
   hadoop.metrics2.impl.TestStatsDMetrics 
   hadoop.security.TestGroupsCaching 
   hadoop.security.TestSecurityUtil 
   hadoop.security.TestShellBasedUnixGroupsMapping 
   hadoop.security.token.TestDtUtilShell 
   hadoop.util.TestDiskCheckerWithDiskIo 
   hadoop.util.TestNativeCodeLoader 
   hadoop.hdfs.client.impl.TestBlockReaderLocal 
   hadoop.hdfs.qjournal.server.TestJournalNode 
   hadoop.hdfs.qjournal.server.TestJournalNodeSync 
   hadoop.hdfs.server.blockmanagement.TestNameNodePrunesMissingStorages 
   hadoop.hdfs.server.datanode.fsdataset.impl.TestProvidedImpl 
   hadoop.hdfs.server.datanode.TestBlockPoolSliceStorage 
   hadoop.hdfs.server.datanode.TestBlockScanner 
   hadoop.hdfs.server.datanode.TestDataNodeFaultInjector 
   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure 
   hadoop.hdfs.server.datanode.TestDirectoryScanner 
   hadoop.hdfs.server.diskbalancer.command.TestDiskBalancerCommand 
   hadoop.hdfs.server.diskbalancer.TestDiskBalancerRPC 
   hadoop.hdfs.server.mover.TestMover 
   hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA 
   hadoop.hdfs.server.namenode.ha.TestPendingCorruptDnMessages 
   hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA 
   hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics 
   hadoop.hdfs.server.namenode.TestEditLogRace 
   hadoop.hdfs.server.namenode.TestReencryptionHandler 
   hadoop.hdfs.server.namenode.TestStartup 
   hadoop.hdfs.TestDatanodeStartupFixesLegacyStorageIDs 
   hadoop.hdfs.TestDFSClientRetries 
   hadoop.hdfs.TestDFSShell 
   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure 
   hadoop.hdfs.TestDFSStripedOutputStreamWithFailureWithRandomECPolicy 
   hadoop.hdfs.TestDFSUpgradeFromImage 
   hadoop.hdfs.TestFetchImage 
   hadoop.hdfs.TestHDFSFileSystemContract 
   hadoop.hdfs.TestLocalDFS 
   hadoop.hdfs.TestMaintenanceState 
   hadoop.hdfs.TestPersistBlocks 
   hadoop.hdfs.TestPread 
   hadoop.hdfs.TestReadStripedFileWithDecodingCorruptData 
   hadoop.hdfs.TestReadStripedFileWithDecodingDeletedData 
   hadoop.hdfs.TestReconstructStripedFile 
   hadoop.hdfs.TestReconstructStripedFileWithRandomECPolicy 
   hadoop.hdfs.TestSecureEncryptionZoneWithKMS 
   hadoop.hdfs.TestTrashWithSecureEncryptionZones 
   hadoop.hdfs.tools.TestDFSAdmin 
   hadoop.hdfs.tools.TestDFSAdminWithHA 
   hadoop.hdfs.tools.TestGetConf 
   hadoop.hdfs.web.TestWebHDFS 
   hadoop.hdfs.web.TestWebHdfsUrl 
   hadoop.fs.http.server.TestHttpFSServerWebServer 
   
hadoop.yarn.server.nodemanager.containermanager.launcher.TestContainerLaunch

Re: [VOTE] Release Apache Hadoop 3.0.3 (RC0)

2018-06-02 Thread Yongjun Zhang 
Hi Gabor,

I got the git tag in, it's release-3.0.3-RC0. Would you please give it a
try?

It should correspond to

commit 37fd7d752db73d984dc31e0cdfd590d252f5e075
Author: Yongjun Zhang 
Date:   Wed May 30 00:07:33 2018 -0700

Update version to 3.0.3 to prepare for 3.0.3 release


Thanks,

--Yongjun

On Fri, Jun 1, 2018 at 4:17 AM, Gabor Bota  wrote:

> Hi Yongjun,
>
> Thank you for working on this release. Is there a git tag in the upstream
> repo which can be checked out? I'd like to build the release from source.
>
> Regards,
> Gabor
>
> On Fri, Jun 1, 2018 at 7:57 AM Shashikant Banerjee <
> sbaner...@hortonworks.com> wrote:
>
>> Looks like the link with the filter seems to be private. I can't see the
>> blocker list.
>> https://issues.apache.org/jira/issues/?filter=12343997
>>
>> Meanwhile , I will be working on testing the release.
>>
>> Thanks
>> Shashi
>> On 6/1/18, 11:18 AM, "Yongjun Zhang"  wrote:
>>
>> Greetings all,
>>
>> I've created the first release candidate (RC0) for Apache Hadoop
>> 3.0.3. This is our next maintenance release to follow up 3.0.2. It
>> includes
>> about 249
>> important fixes and improvements, among which there are 8 blockers.
>> See
>> https://issues.apache.org/jira/issues/?filter=12343997
>>
>> The RC artifacts are available at:
>> https://dist.apache.org/repos/dist/dev/hadoop/3.0.3-RC0/
>>
>> The maven artifacts are available via
>> https://repository.apache.org/content/repositories/
>> orgapachehadoop-1126
>>
>> Please try the release and vote; the vote will run for the usual 5
>> working
>> days, ending on 06/07/2018 PST time. Would really appreciate your
>> participation here.
>>
>> I bumped into quite some issues along the way, many thanks to quite a
>> few
>> people who helped, especially Sammi Chen, Andrew Wang, Junping Du,
>> Eddy Xu.
>>
>> Thanks,
>>
>> --Yongjun
>>
>>
>>

Apache Hadoop qbt Report: trunk+JDK9 on Linux/x86

2018-06-02 Thread Apache Jenkins Server 
For more details, see 
https://builds.apache.org/job/hadoop-qbt-trunk-java9-linux-x86/26/

[May 28, 2018 8:29:59 AM] (aajisaka) HADOOP-15477. Make unjar in RunJar 
overrideable
[May 28, 2018 8:35:49 AM] (rohithsharmaks) MAPREDUCE-7097. MapReduce JHS should 
honor
[May 28, 2018 10:06:34 AM] (aajisaka) HDFS-13628. Update Archival Storage doc 
for Provided Storage
[May 28, 2018 10:41:07 AM] (aajisaka) HADOOP-15449. Increase default timeout of 
ZK session to avoid frequent
[May 28, 2018 11:02:53 AM] (sunilg) YARN-4781. Support intra-queue preemption 
for fairness ordering policy.
[May 28, 2018 5:26:47 PM] (inigoiri) HDFS-13627. TestErasureCodingExerciseAPIs 
fails on Windows. Contributed
[May 28, 2018 11:34:02 PM] (inigoiri) HDFS-13591. TestDFSShell#testSetrepLow 
fails on Windows. Contributed by
[May 28, 2018 11:45:42 PM] (inigoiri) HADOOP-15498. TestHadoopArchiveLogs 
(#testGenerateScript,
[May 29, 2018 12:32:32 AM] (weichiu) HADOOP-15455. Incorrect debug message in 
KMSACL#hasAccess. Contributed
[May 29, 2018 4:00:30 PM] (jlowe) YARN-8338. TimelineService V1.5 doesn't come 
up after HADOOP-15406.
[May 29, 2018 4:11:08 PM] (inigoiri) HADOOP-15497. TestTrash should use proper 
test path to avoid failing on
[May 29, 2018 4:23:11 PM] (wangda) YARN-8339. Service AM should localize 
static/archive resource types to
[May 29, 2018 4:27:36 PM] (wangda) YARN-8369. Javadoc build failed due to 'bad 
use of >'. (Takanobu Asanuma
[May 29, 2018 4:54:06 PM] (aengineer) HDDS-125. Cleanup HDDS CheckStyle issues. 
Contributed by Anu Engineer.
[May 29, 2018 7:15:12 PM] (kihwal) Additional check when unpacking archives. 
Contributed by Wilfred
[May 29, 2018 7:43:17 PM] (jlowe) YARN-8329. Docker client configuration can 
still be set incorrectly.
[May 29, 2018 7:48:50 PM] (aengineer) HDDS-81. Moving ContainerReport inside 
Datanode heartbeat. Contributed
[May 29, 2018 8:38:15 PM] (fabbri) HADOOP-14946 S3Guard testPruneCommandCLI can 
fail. Contributed by Gabor
[May 29, 2018 8:48:55 PM] (aengineer) HDDS-114. Ozone Datanode mbean 
registration fails for StorageLocation.
[May 29, 2018 8:56:58 PM] (eyang) YARN-8362.  Bugfix logic in container retries 
in node manager.  
[May 30, 2018 2:20:22 AM] (fabbri) HADOOP-15480 
AbstractS3GuardToolTestBase.testDiffCommand fails when
[May 30, 2018 8:52:21 AM] (yqlin) HDFS-13626. Fix incorrect username when deny 
the setOwner operation.
[May 30, 2018 3:52:07 PM] (aengineer) HDDS-88. Create separate message 
structure to represent ports in
[May 30, 2018 4:50:19 PM] (ericp) YARN-8377: Javadoc build failed in 
hadoop-yarn-server-nodemanager.
[May 30, 2018 5:13:52 PM] (inigoiri) HDFS-13632. Randomize baseDir for 
MiniJournalCluster in MiniQJMHACluster
[May 30, 2018 5:22:04 PM] (inigoiri) HDFS-13629. Some tests in 
TestDiskBalancerCommand fail on Windows due to
[May 30, 2018 7:37:43 PM] (billie) YARN-8368. yarn app start cli should print 
applicationId. Contributed by
[May 30, 2018 8:19:13 PM] (billie) YARN-8350. NPE in service AM related to 
placement policy. Contributed by
[May 30, 2018 9:21:13 PM] (aengineer) HDDS-128. Support for DN to SCM 
signaling. Contributed by Nanda Kumar.
[May 31, 2018 9:31:10 AM] (aajisaka) MAPREDUCE-7098. Upgrade common-langs 
version to 3.7 in
[May 31, 2018 12:46:39 PM] (wwei) YARN-8367. Fix NPE in 
SingleConstraintAppPlacementAllocator when
[May 31, 2018 1:46:34 PM] (billie) YARN-8333. Load balance YARN services using 
RegistryDNS multiple A
[May 31, 2018 1:59:21 PM] (mackrorysd) HDFS-13582. Improve backward 
compatibility for HDFS-13176 (WebHdfs file
[May 31, 2018 5:12:38 PM] (inigoiri) HDFS-13631. 
TestDFSAdmin#testCheckNumOfBlocksInReportCommand should use
[May 31, 2018 5:56:17 PM] (Bharat) HADOOP-15490:Multiple declaration of 
maven-enforcer-plugin found in
[May 31, 2018 7:05:50 PM] (aengineer) HDDS-112. OzoneShell should support 
commands with url without scheme.
[May 31, 2018 9:56:32 PM] (shv) HDFS-12978. Fine-grained locking while 
consuming journal stream.
[May 31, 2018 10:20:59 PM] (weichiu) HDFS-13440. Log HDFS file name when client 
fails to connect. 
[May 31, 2018 10:26:08 PM] (weichiu) HDFS-13646. DFSAdmin doesn't display 
specialized help for
[May 31, 2018 11:09:33 PM] (weichiu) HADOOP-14783. [KMS] Add missing 
configuration properties into
[May 31, 2018 11:48:33 PM] (vinodkv) YARN-8197. Fixed AM IP Filter and Webapp 
proxy to redirect app
[Jun 1, 2018 12:37:44 AM] (shv) HDFS-13602. Add checkOperation(WRITE) checks in 
FSNamesystem.
[Jun 1, 2018 1:09:36 AM] (eyang) YARN-8308.  Fixed YARN Service AM failure with 
HDFS token renewal.  
[Jun 1, 2018 5:30:12 AM] (yufei) YARN-7340. Fix the missing time stamp in 
exception message in Class
[Jun 1, 2018 7:54:47 PM] (aengineer) HDDS-137. Cleanup Hdds-ozone CheckStyle 
Issues. Contributed by Anu
[Jun 1, 2018 9:21:35 PM] (aengineer) HDDS-142. TestMetadataStore fails on 
Windows. Contributed by  Íñigo
[Jun 1, 2018 9:42:39 PM] (templedf) HDFS-13636. Cross-Site Scripting 
vulnerability in HttpServer2
[Jun

Re: [VOTE] Release Apache Hadoop 3.0.3 (RC0)

2018-06-02 Thread Yongjun Zhang 
Many thanks Shashikant, Zsolt, and Ajay!

Best,

--Yongjun

On Sat, Jun 2, 2018 at 7:34 AM, Shashikant Banerjee <
sbaner...@hortonworks.com> wrote:

> Thanks for working on this Yongjun!
>
> +1 (non-binding)
>  - verified signatures and checksums
>  - built from source and setup single node cluster
>  - ran basic hdfs operations
> - basic sanity check of NN
>
> Thanks
> Shashi
>
> On 6/2/18, 7:31 PM, "Zsolt Venczel"  wrote:
>
> Thanks Yongjun for working on this!
>
> +1 (non-binding)
>
>  - built from source with native library
>  - run hadoop-hdfs-client tests with native library that all passed
>  - set up a cluster with 3 nodes and run teragen, terasort and
> teravalidate
> successfully
>  - created two snapshots and produced a snapshot diff successfully
>  - checked out the webui, looked at the file structure and double
> checked
> the created snapshots
>
> Thanks and best regards,
> Zsolt
>
> On Sat, Jun 2, 2018 at 11:33 AM Ajay Kumar  >
> wrote:
>
> > Thanks for working on this Yongjun!!
> >
> > +1 (non-binding)
> > - verified signatures and checksums
> > - built from source and setup single node cluster
> > - ran basic hdfs operations
> > - ran TestDFSIO(read/write), wordcount, pi jobs.
> > - basic sanity check of NN, RM UI
> >
> > Thanks,
> > Ajay
> >
> > On 6/2/18, 12:45 AM, "Shashikant Banerjee" <
> sbaner...@hortonworks.com>
> > wrote:
> >
> > Hi Yongjun,
> >
> > I am able to see the list after logging in now.
> >
> > Thanks
> > Shashi
> >
> > From: Yongjun Zhang 
> > Date: Friday, June 1, 2018 at 9:11 PM
> > To: Gabor Bota 
> > Cc: Shashikant Banerjee , Hadoop
> Common <
> > common-...@hadoop.apache.org>, Hdfs-dev ,
> "
> > mapreduce-...@hadoop.apache.org" ,
> "
> > yarn-...@hadoop.apache.org" 
> > Subject: Re: [VOTE] Release Apache Hadoop 3.0.3 (RC0)
> >
> >
> > Thanks for the feedback!
> >
> > Hi Shashikant, I thought I have made the filter visible to jira
> users,
> > now I changed it to be visible to all logged-in users of jira.
> Please let
> > me know if you can not see it.
> >
> > Hi Gabor,
> >
> > Good question. I forgot to mention, I have tried to add tag
> earlier,
> > as step 7,8 9 in
> > https://wiki.apache.org/hadoop/HowToRelease, but these steps
> seem to
> > not push anything to git. I suspect step 4 should have been run with
> > --rc-label , and checked with Andrew, he said it doesn't matter and
> often
> > people don't use the rc label.
> >
> > I probably should mention that the build is on commit id
> > 37fd7d752db73d984dc31e0cdfd590d252f5e075.
> >
> > The source is also available at
> > https://dist.apache.org/repos/dist/dev/hadoop/3.0.3-RC0/
> >
> > Thanks.
> >
> > --Yongjun
> >
> > On Fri, Jun 1, 2018 at 4:17 AM, Gabor Bota <
> gabor.b...@cloudera.com
> > > wrote:
> > Hi Yongjun,
> >
> > Thank you for working on this release. Is there a git tag in the
> > upstream repo which can be checked out? I'd like to build the
> release from
> > source.
> >
> > Regards,
> > Gabor
> >
> > On Fri, Jun 1, 2018 at 7:57 AM Shashikant Banerjee <
> > sbaner...@hortonworks.com> wrote:
> > Looks like the link with the filter seems to be private. I can't
> see
> > the blocker list.
> > https://issues.apache.org/jira/issues/?filter=12343997
> >
> > Meanwhile , I will be working on testing the release.
> >
> > Thanks
> > Shashi
> > On 6/1/18, 11:18 AM, "Yongjun Zhang"   > yjzhan...@apache.org>> wrote:
> >
> > Greetings all,
> >
> > I've created the first release candidate (RC0) for Apache
> Hadoop
> > 3.0.3. This is our next maintenance release to follow up
> 3.0.2. It
> > includes
> > about 249
> > important fixes and improvements, among which there are 8
> > blockers. See
> > https://issues.apache.org/jira/issues/?filter=12343997
> >
> > The RC artifacts are available at:
> > https://dist.apache.org/repos/dist/dev/hadoop/3.0.3-RC0/
> >
> > The maven artifacts are available via
> >
> > https://repository.apache.org/content/repositories/
> orgapachehadoop-1126
> >
> > Please try the release and vote; the vote will run for the
> usual 5
> > working
> > days, ending on 06/07/2018 PST time. Would really appreciate
> your
> > participation here.
> >
> > I bumped into quite some issues along the way, many thanks to
>

[jira] [Created] (HDFS-13652) Randomize baseDir for MiniDFSCluster in TestBlockScanner

2018-06-02 Thread Anbang Hu (JIRA) 
Anbang Hu created HDFS-13652:


 Summary: Randomize baseDir for MiniDFSCluster in TestBlockScanner
 Key: HDFS-13652
 URL: https://issues.apache.org/jira/browse/HDFS-13652
 Project: Hadoop HDFS
  Issue Type: Test
Reporter: Anbang Hu
Assignee: Anbang Hu






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-13651) TestReencryptionHandler fails on Windows

2018-06-02 Thread Anbang Hu (JIRA) 
Anbang Hu created HDFS-13651:


 Summary: TestReencryptionHandler fails on Windows
 Key: HDFS-13651
 URL: https://issues.apache.org/jira/browse/HDFS-13651
 Project: Hadoop HDFS
  Issue Type: Test
Reporter: Anbang Hu
Assignee: Anbang Hu






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-13649) Randomize baseDir for MiniDFSCluster in TestReconstructStripedFile and TestReconstructStripedFileWithRandomECPolicy

2018-06-02 Thread Anbang Hu (JIRA) 
Anbang Hu created HDFS-13649:


 Summary: Randomize baseDir for MiniDFSCluster in 
TestReconstructStripedFile and TestReconstructStripedFileWithRandomECPolicy
 Key: HDFS-13649
 URL: https://issues.apache.org/jira/browse/HDFS-13649
 Project: Hadoop HDFS
  Issue Type: Test
Reporter: Anbang Hu
Assignee: Anbang Hu






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDDS-146) Refactor the structure of the acceptance tests

2018-06-02 Thread Elek, Marton (JIRA) 
Elek, Marton created HDDS-146:
-

 Summary: Refactor the structure of the acceptance tests
 Key: HDDS-146
 URL: https://issues.apache.org/jira/browse/HDDS-146
 Project: Hadoop Distributed Data Store
  Issue Type: Improvement
Reporter: Elek, Marton
Assignee: Elek, Marton


I started to fix the packaging the ozone file system which requires additional 
acceptance tests.

The original acceptance test is added when it was only one single file but 
(fortunately) now we have multiple files and multiple tests and ozonefs 
requires even more.

To make it easier to handle multplie acceptance tests I propose some changes on 
the projects structure. This patch includes the following changes:
 # all the start/stop/check common keywords are moved out the a common library 
file (commonlib.robot). All the existing files are simplified.
 # ozone-shell tests are simplified with using parametrized tests. We don't 
need to repeat the same steps multiple times.
 # The directories in the projects are simplified. Both compose file and robot 
files are in the same directory. The basedir is handled from the robot files. 
Now it's easier to run the tests localy (go to the dir and do a simple call 
'robot basic.robot') or start the containers (use docker-compose from the base 
directory)
 # I adjusted the logging (warning about the missing native library is not 
required)
 # Decreased the heartbeat intervall (to make the tests faster)
 # I improved the ozone-shell tests with adding a few additional checks.

 

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

Hadoop 2.7.7 Release in progress

2018-06-02 Thread Steve Loughran 


FYI, I'm cutting a new Hadoop 2.7.7 release: It Is Time.

good to get ready for testing as soon as people can...for now I'm doing the 
JIRA moves and the build setup (docker)

-Steve


-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.0.3 (RC0)

2018-06-02 Thread Shashikant Banerjee 
Thanks for working on this Yongjun!

+1 (non-binding)
 - verified signatures and checksums
 - built from source and setup single node cluster
 - ran basic hdfs operations
- basic sanity check of NN

Thanks
Shashi

On 6/2/18, 7:31 PM, "Zsolt Venczel"  wrote:

Thanks Yongjun for working on this!

+1 (non-binding)

 - built from source with native library
 - run hadoop-hdfs-client tests with native library that all passed
 - set up a cluster with 3 nodes and run teragen, terasort and teravalidate
successfully
 - created two snapshots and produced a snapshot diff successfully
 - checked out the webui, looked at the file structure and double checked
the created snapshots

Thanks and best regards,
Zsolt

On Sat, Jun 2, 2018 at 11:33 AM Ajay Kumar 
wrote:

> Thanks for working on this Yongjun!!
>
> +1 (non-binding)
> - verified signatures and checksums
> - built from source and setup single node cluster
> - ran basic hdfs operations
> - ran TestDFSIO(read/write), wordcount, pi jobs.
> - basic sanity check of NN, RM UI
>
> Thanks,
> Ajay
>
> On 6/2/18, 12:45 AM, "Shashikant Banerjee" 
> wrote:
>
> Hi Yongjun,
>
> I am able to see the list after logging in now.
>
> Thanks
> Shashi
>
> From: Yongjun Zhang 
> Date: Friday, June 1, 2018 at 9:11 PM
> To: Gabor Bota 
> Cc: Shashikant Banerjee , Hadoop Common <
> common-...@hadoop.apache.org>, Hdfs-dev , "
> mapreduce-...@hadoop.apache.org" , "
> yarn-...@hadoop.apache.org" 
> Subject: Re: [VOTE] Release Apache Hadoop 3.0.3 (RC0)
>
>
> Thanks for the feedback!
>
> Hi Shashikant, I thought I have made the filter visible to jira users,
> now I changed it to be visible to all logged-in users of jira. Please let
> me know if you can not see it.
>
> Hi Gabor,
>
> Good question. I forgot to mention, I have tried to add tag earlier,
> as step 7,8 9 in
> https://wiki.apache.org/hadoop/HowToRelease, but these steps seem to
> not push anything to git. I suspect step 4 should have been run with
> --rc-label , and checked with Andrew, he said it doesn't matter and often
> people don't use the rc label.
>
> I probably should mention that the build is on commit id
> 37fd7d752db73d984dc31e0cdfd590d252f5e075.
>
> The source is also available at
> https://dist.apache.org/repos/dist/dev/hadoop/3.0.3-RC0/
>
> Thanks.
>
> --Yongjun
>
> On Fri, Jun 1, 2018 at 4:17 AM, Gabor Bota  > wrote:
> Hi Yongjun,
>
> Thank you for working on this release. Is there a git tag in the
> upstream repo which can be checked out? I'd like to build the release from
> source.
>
> Regards,
> Gabor
>
> On Fri, Jun 1, 2018 at 7:57 AM Shashikant Banerjee <
> sbaner...@hortonworks.com> wrote:
> Looks like the link with the filter seems to be private. I can't see
> the blocker list.
> https://issues.apache.org/jira/issues/?filter=12343997
>
> Meanwhile , I will be working on testing the release.
>
> Thanks
> Shashi
> On 6/1/18, 11:18 AM, "Yongjun Zhang"  yjzhan...@apache.org>> wrote:
>
> Greetings all,
>
> I've created the first release candidate (RC0) for Apache Hadoop
> 3.0.3. This is our next maintenance release to follow up 3.0.2. It
> includes
> about 249
> important fixes and improvements, among which there are 8
> blockers. See
> https://issues.apache.org/jira/issues/?filter=12343997
>
> The RC artifacts are available at:
> https://dist.apache.org/repos/dist/dev/hadoop/3.0.3-RC0/
>
> The maven artifacts are available via
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1126
>
> Please try the release and vote; the vote will run for the usual 5
> working
> days, ending on 06/07/2018 PST time. Would really appreciate your
> participation here.
>
> I bumped into quite some issues along the way, many thanks to
> quite a few
> people who helped, especially Sammi Chen, Andrew Wang, Junping Du,
> Eddy Xu.
>
> Thanks,
>
> --Yongjun
>
>
>
>
>

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

2018-06-02 Thread Apache Jenkins Server 
For more details, see 
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/799/

[Jun 1, 2018 5:30:12 AM] (yufei) YARN-7340. Fix the missing time stamp in 
exception message in Class
[Jun 1, 2018 7:54:47 PM] (aengineer) HDDS-137. Cleanup Hdds-ozone CheckStyle 
Issues. Contributed by Anu
[Jun 1, 2018 9:21:35 PM] (aengineer) HDDS-142. TestMetadataStore fails on 
Windows. Contributed by  Íñigo
[Jun 1, 2018 9:42:39 PM] (templedf) HDFS-13636. Cross-Site Scripting 
vulnerability in HttpServer2
[Jun 1, 2018 9:49:17 PM] (wangda) YARN-8384. stdout.txt, stderr.txt logs of a 
launched docker container is
[Jun 1, 2018 9:49:18 PM] (wangda) YARN-7962. Race Condition When Stopping 
DelegationTokenRenewer causes RM
[Jun 1, 2018 9:49:18 PM] (wangda) YARN-8372. Distributed shell app master 
should not release containers
[Jun 1, 2018 9:49:18 PM] (wangda) YARN-8349. Remove YARN registry entries when 
a service is killed by the
[Jun 1, 2018 10:36:30 PM] (aengineer) HDDS-144. Fix TestEndPoint#testHeartbeat 
and TestEndPoint#testRegister.
[Jun 1, 2018 10:36:30 PM] (aengineer) HDDS-143. Provide docker compose files to 
measure performance in a
[Jun 1, 2018 11:29:13 PM] (haibochen) YARN-8375. 
TestCGroupElasticMemoryController fails surefire build.
[Jun 1, 2018 11:59:04 PM] (inigoiri) HDFS-13637. RBF: Router fails when 
threadIndex (in ConnectionPool) wraps




-1 overall


The following subsystems voted -1:
asflicense findbugs pathlen unit xml


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace


The following subsystems are considered long running:
(runtime bigger than 1h  0m  0s)
unit


Specific tests:

FindBugs :

   module:hadoop-hdds/common 
   Found reliance on default encoding in 
org.apache.hadoop.utils.MetadataKeyFilters$KeyPrefixFilter.filterKey(byte[], 
byte[], byte[]):in 
org.apache.hadoop.utils.MetadataKeyFilters$KeyPrefixFilter.filterKey(byte[], 
byte[], byte[]): String.getBytes() At MetadataKeyFilters.java:[line 97] 

Failed junit tests :

   hadoop.hdfs.client.impl.TestBlockReaderLocal 
   hadoop.hdfs.web.TestWebHdfsTimeouts 
   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration 
   
hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineStorageEntities 
   
hadoop.yarn.server.timelineservice.reader.TestTimelineReaderWebServicesHBaseStorage
 
   
hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineStorageDomain 
   
hadoop.yarn.server.timelineservice.storage.flow.TestHBaseStorageFlowRunCompaction
 
   
hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineStorageSchema 
   hadoop.yarn.server.timelineservice.storage.flow.TestHBaseStorageFlowRun 
   hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineStorageApps 
   
hadoop.yarn.server.timelineservice.storage.flow.TestHBaseStorageFlowActivity 
  

   cc:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/799/artifact/out/diff-compile-cc-root.txt
  [4.0K]

   javac:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/799/artifact/out/diff-compile-javac-root.txt
  [336K]

   checkstyle:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/799/artifact/out/diff-checkstyle-root.txt
  [17M]

   pathlen:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/799/artifact/out/pathlen.txt
  [12K]

   pylint:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/799/artifact/out/diff-patch-pylint.txt
  [24K]

   shellcheck:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/799/artifact/out/diff-patch-shellcheck.txt
  [20K]

   shelldocs:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/799/artifact/out/diff-patch-shelldocs.txt
  [16K]

   whitespace:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/799/artifact/out/whitespace-eol.txt
  [9.4M]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/799/artifact/out/whitespace-tabs.txt
  [1.1M]

   xml:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/799/artifact/out/xml.txt
  [4.0K]

   findbugs:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/799/artifact/out/branch-findbugs-hadoop-hdds_client.txt
  [56K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/799/artifact/out/branch-findbugs-hadoop-hdds_common-warnings.html
  [8.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/799/artifact/out/branch-findbugs-hadoop-hdds_container-service.txt
  [52K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/799/artifact/out/branch-findbugs-hadoop-hdds_server-scm.txt
  [56K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/799/artifact/out/branch-findbugs-hadoop-hdds_tools.txt
  [12K]

Re: [VOTE] Release Apache Hadoop 3.0.3 (RC0)

2018-06-02 Thread Zsolt Venczel 
Thanks Yongjun for working on this!

+1 (non-binding)

 - built from source with native library
 - run hadoop-hdfs-client tests with native library that all passed
 - set up a cluster with 3 nodes and run teragen, terasort and teravalidate
successfully
 - created two snapshots and produced a snapshot diff successfully
 - checked out the webui, looked at the file structure and double checked
the created snapshots

Thanks and best regards,
Zsolt

On Sat, Jun 2, 2018 at 11:33 AM Ajay Kumar 
wrote:

> Thanks for working on this Yongjun!!
>
> +1 (non-binding)
> - verified signatures and checksums
> - built from source and setup single node cluster
> - ran basic hdfs operations
> - ran TestDFSIO(read/write), wordcount, pi jobs.
> - basic sanity check of NN, RM UI
>
> Thanks,
> Ajay
>
> On 6/2/18, 12:45 AM, "Shashikant Banerjee" 
> wrote:
>
> Hi Yongjun,
>
> I am able to see the list after logging in now.
>
> Thanks
> Shashi
>
> From: Yongjun Zhang 
> Date: Friday, June 1, 2018 at 9:11 PM
> To: Gabor Bota 
> Cc: Shashikant Banerjee , Hadoop Common <
> common-...@hadoop.apache.org>, Hdfs-dev , "
> mapreduce-...@hadoop.apache.org" , "
> yarn-...@hadoop.apache.org" 
> Subject: Re: [VOTE] Release Apache Hadoop 3.0.3 (RC0)
>
>
> Thanks for the feedback!
>
> Hi Shashikant, I thought I have made the filter visible to jira users,
> now I changed it to be visible to all logged-in users of jira. Please let
> me know if you can not see it.
>
> Hi Gabor,
>
> Good question. I forgot to mention, I have tried to add tag earlier,
> as step 7,8 9 in
> https://wiki.apache.org/hadoop/HowToRelease, but these steps seem to
> not push anything to git. I suspect step 4 should have been run with
> --rc-label , and checked with Andrew, he said it doesn't matter and often
> people don't use the rc label.
>
> I probably should mention that the build is on commit id
> 37fd7d752db73d984dc31e0cdfd590d252f5e075.
>
> The source is also available at
> https://dist.apache.org/repos/dist/dev/hadoop/3.0.3-RC0/
>
> Thanks.
>
> --Yongjun
>
> On Fri, Jun 1, 2018 at 4:17 AM, Gabor Bota  > wrote:
> Hi Yongjun,
>
> Thank you for working on this release. Is there a git tag in the
> upstream repo which can be checked out? I'd like to build the release from
> source.
>
> Regards,
> Gabor
>
> On Fri, Jun 1, 2018 at 7:57 AM Shashikant Banerjee <
> sbaner...@hortonworks.com> wrote:
> Looks like the link with the filter seems to be private. I can't see
> the blocker list.
> https://issues.apache.org/jira/issues/?filter=12343997
>
> Meanwhile , I will be working on testing the release.
>
> Thanks
> Shashi
> On 6/1/18, 11:18 AM, "Yongjun Zhang"  yjzhan...@apache.org>> wrote:
>
> Greetings all,
>
> I've created the first release candidate (RC0) for Apache Hadoop
> 3.0.3. This is our next maintenance release to follow up 3.0.2. It
> includes
> about 249
> important fixes and improvements, among which there are 8
> blockers. See
> https://issues.apache.org/jira/issues/?filter=12343997
>
> The RC artifacts are available at:
> https://dist.apache.org/repos/dist/dev/hadoop/3.0.3-RC0/
>
> The maven artifacts are available via
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1126
>
> Please try the release and vote; the vote will run for the usual 5
> working
> days, ending on 06/07/2018 PST time. Would really appreciate your
> participation here.
>
> I bumped into quite some issues along the way, many thanks to
> quite a few
> people who helped, especially Sammi Chen, Andrew Wang, Junping Du,
> Eddy Xu.
>
> Thanks,
>
> --Yongjun
>
>
>
>
>

Re: [VOTE] Release Apache Hadoop 3.0.3 (RC0)

2018-06-02 Thread Ajay Kumar 
Thanks for working on this Yongjun!!

+1 (non-binding)
- verified signatures and checksums
- built from source and setup single node cluster
- ran basic hdfs operations
- ran TestDFSIO(read/write), wordcount, pi jobs.
- basic sanity check of NN, RM UI

Thanks,
Ajay

On 6/2/18, 12:45 AM, "Shashikant Banerjee"  wrote:

Hi Yongjun,

I am able to see the list after logging in now.

Thanks
Shashi

From: Yongjun Zhang 
Date: Friday, June 1, 2018 at 9:11 PM
To: Gabor Bota 
Cc: Shashikant Banerjee , Hadoop Common 
, Hdfs-dev , 
"mapreduce-...@hadoop.apache.org" , 
"yarn-...@hadoop.apache.org" 
Subject: Re: [VOTE] Release Apache Hadoop 3.0.3 (RC0)


Thanks for the feedback!

Hi Shashikant, I thought I have made the filter visible to jira users, now 
I changed it to be visible to all logged-in users of jira. Please let me know 
if you can not see it.

Hi Gabor,

Good question. I forgot to mention, I have tried to add tag earlier, as 
step 7,8 9 in
https://wiki.apache.org/hadoop/HowToRelease, but these steps seem to not 
push anything to git. I suspect step 4 should have been run with --rc-label , 
and checked with Andrew, he said it doesn't matter and often people don't use 
the rc label.

I probably should mention that the build is on commit id 
37fd7d752db73d984dc31e0cdfd590d252f5e075.

The source is also available at   
https://dist.apache.org/repos/dist/dev/hadoop/3.0.3-RC0/

Thanks.

--Yongjun

On Fri, Jun 1, 2018 at 4:17 AM, Gabor Bota 
mailto:gabor.b...@cloudera.com>> wrote:
Hi Yongjun,

Thank you for working on this release. Is there a git tag in the upstream 
repo which can be checked out? I'd like to build the release from source.

Regards,
Gabor

On Fri, Jun 1, 2018 at 7:57 AM Shashikant Banerjee 
mailto:sbaner...@hortonworks.com>> wrote:
Looks like the link with the filter seems to be private. I can't see the 
blocker list.
https://issues.apache.org/jira/issues/?filter=12343997

Meanwhile , I will be working on testing the release.

Thanks
Shashi
On 6/1/18, 11:18 AM, "Yongjun Zhang" 
mailto:yjzhan...@apache.org>> wrote:

Greetings all,

I've created the first release candidate (RC0) for Apache Hadoop
3.0.3. This is our next maintenance release to follow up 3.0.2. It 
includes
about 249
important fixes and improvements, among which there are 8 blockers. See
https://issues.apache.org/jira/issues/?filter=12343997

The RC artifacts are available at:
https://dist.apache.org/repos/dist/dev/hadoop/3.0.3-RC0/

The maven artifacts are available via
https://repository.apache.org/content/repositories/orgapachehadoop-1126

Please try the release and vote; the vote will run for the usual 5 
working
days, ending on 06/07/2018 PST time. Would really appreciate your
participation here.

I bumped into quite some issues along the way, many thanks to quite a 
few
people who helped, especially Sammi Chen, Andrew Wang, Junping Du, Eddy 
Xu.

Thanks,

--Yongjun