[VOTE] Release Apache Hadoop 3.1.2 - RC0

2019-01-24 Thread Wangda Tan 
Hi folks,

With tons of help from Sunil, we have created RC0 for Apache Hadoop 3.1.2.
The artifacts are available here:

*http://home.apache.org/~sunilg/hadoop-3.1.2-RC0/
*

The RC tag in git is release-3.1.2-RC0:
https://github.com/apache/hadoop/commits/release-3.1.2-RC0

The maven artifacts are available via repository.apache.org at
*https://repository.apache.org/content/repositories/orgapachehadoop-1212/
*

This vote will run 5 days from now.

3.1.2 contains 325 [1] fixed JIRA issues since 3.1.1.

I have done testing with a pseudo cluster and distributed shell job. My +1
to start.

Best,
Wangda Tan and Sunil Govind

[1] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.1.2)
ORDER BY priority DESC

[jira] [Created] (HADOOP-16072) Visual Studio projects are out of date

2019-01-24 Thread Brian Grunkemeyer (JIRA) 
Brian Grunkemeyer created HADOOP-16072:
--

 Summary: Visual Studio projects are out of date
 Key: HADOOP-16072
 URL: https://issues.apache.org/jira/browse/HADOOP-16072
 Project: Hadoop Common
  Issue Type: Bug
  Components: build
Affects Versions: 3.2.0
 Environment: Building on Windows 10 with Visual Studio 2017.
Reporter: Brian Grunkemeyer
 Fix For: 3.2.0


On Windows when you build, a part of the build process updates some Visual 
Studio solution and project files.  We should simply check in the updated 
version, so that everyone who builds doesn't get stuck with extra changes they 
need to manage.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Resolved] (HADOOP-15998) Jar validation bash scripts don't work on Windows due to platform differences (colons in paths, \r\n)

2019-01-24 Thread Giovanni Matteo Fumarola (JIRA) 


 [ 
https://issues.apache.org/jira/browse/HADOOP-15998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Giovanni Matteo Fumarola resolved HADOOP-15998.
---
Resolution: Incomplete

> Jar validation bash scripts don't work on Windows due to platform differences 
> (colons in paths, \r\n)
> -
>
> Key: HADOOP-15998
> URL: https://issues.apache.org/jira/browse/HADOOP-15998
> Project: Hadoop Common
>  Issue Type: Improvement
>  Components: build
>Affects Versions: 3.2.0, 3.3.0
> Environment: Windows 10
> Visual Studio 2017
>Reporter: Brian Grunkemeyer
>Assignee: Brian Grunkemeyer
>Priority: Blocker
>  Labels: build, windows
> Fix For: 3.3.0
>
> Attachments: HADOOP-15998.v2.patch, HADOOP-15998.v3.patch
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> Building Hadoop fails on Windows due to a few shell scripts that make invalid 
> assumptions:
> 1) Colon shouldn't be used to separate multiple paths in command line 
> parameters. Colons occur in Windows paths.
> 2) Shell scripts that rely on running external tools need to deal with 
> carriage return - line feed differences (lines ending in \r\n, not just \n)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Reopened] (HADOOP-15998) Jar validation bash scripts don't work on Windows due to platform differences (colons in paths, \r\n)

2019-01-24 Thread Giovanni Matteo Fumarola (JIRA) 


 [ 
https://issues.apache.org/jira/browse/HADOOP-15998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Giovanni Matteo Fumarola reopened HADOOP-15998:
---

> Jar validation bash scripts don't work on Windows due to platform differences 
> (colons in paths, \r\n)
> -
>
> Key: HADOOP-15998
> URL: https://issues.apache.org/jira/browse/HADOOP-15998
> Project: Hadoop Common
>  Issue Type: Improvement
>  Components: build
>Affects Versions: 3.2.0, 3.3.0
> Environment: Windows 10
> Visual Studio 2017
>Reporter: Brian Grunkemeyer
>Assignee: Brian Grunkemeyer
>Priority: Blocker
>  Labels: build, windows
> Fix For: 3.3.0
>
> Attachments: HADOOP-15998.v2.patch, HADOOP-15998.v3.patch
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> Building Hadoop fails on Windows due to a few shell scripts that make invalid 
> assumptions:
> 1) Colon shouldn't be used to separate multiple paths in command line 
> parameters. Colons occur in Windows paths.
> 2) Shell scripts that rely on running external tools need to deal with 
> carriage return - line feed differences (lines ending in \r\n, not just \n)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

2019-01-24 Thread Apache Jenkins Server 
For more details, see 
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1026/

[Jan 23, 2019 6:14:49 AM] (wwei) YARN-9205. When using custom resource type, 
application will fail to run
[Jan 23, 2019 8:35:49 AM] (shashikant) HDDS-932. Add blockade Tests for Network 
partition. Contributed by
[Jan 23, 2019 9:59:36 AM] (wwei) YARN-8101. Add UT to verify node-attributes in 
RM nodes rest API.
[Jan 23, 2019 10:31:07 AM] (elek) HDDS-982. Fix 
TestContainerDataYaml#testIncorrectContainerFile.
[Jan 23, 2019 11:30:37 AM] (surendralilhore) HDFS-14153. [SPS] : Add Support 
for Storage Policy Satisfier in WEBHDFS.
[Jan 23, 2019 7:37:49 PM] (bharat) HDDS-764. Run S3 smoke tests with 
replication STANDARD. (#462)
[Jan 23, 2019 10:40:57 PM] (weichiu) HDFS-14061. Check if the cluster topology 
supports the EC policy before
[Jan 23, 2019 11:34:20 PM] (ajay) HDDS-975. Manage ozone security tokens with 
ozone shell cli. Contributed
[Jan 23, 2019 11:57:39 PM] (templedf) HDFS-14185. Cleanup method calls to 
static Assert methods in




-1 overall


The following subsystems voted -1:
asflicense findbugs hadolint pathlen unit


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace


The following subsystems are considered long running:
(runtime bigger than 1h  0m  0s)
unit


Specific tests:

Failed junit tests :

   hadoop.hdfs.web.TestWebHdfsTimeouts 
   hadoop.hdfs.TestDFSClientRetries 
   hadoop.yarn.server.resourcemanager.TestCapacitySchedulerMetrics 
  

   cc:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1026/artifact/out/diff-compile-cc-root.txt
  [4.0K]

   javac:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1026/artifact/out/diff-compile-javac-root.txt
  [336K]

   checkstyle:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1026/artifact/out/diff-checkstyle-root.txt
  [17M]

   hadolint:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1026/artifact/out/diff-patch-hadolint.txt
  [8.0K]

   pathlen:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1026/artifact/out/pathlen.txt
  [12K]

   pylint:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1026/artifact/out/diff-patch-pylint.txt
  [88K]

   shellcheck:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1026/artifact/out/diff-patch-shellcheck.txt
  [20K]

   shelldocs:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1026/artifact/out/diff-patch-shelldocs.txt
  [12K]

   whitespace:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1026/artifact/out/whitespace-eol.txt
  [9.3M]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1026/artifact/out/whitespace-tabs.txt
  [1.1M]

   findbugs:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1026/artifact/out/branch-findbugs-hadoop-hdds_client.txt
  [36K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1026/artifact/out/branch-findbugs-hadoop-hdds_container-service.txt
  [8.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1026/artifact/out/branch-findbugs-hadoop-hdds_framework.txt
  [4.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1026/artifact/out/branch-findbugs-hadoop-hdds_server-scm.txt
  [12K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1026/artifact/out/branch-findbugs-hadoop-hdds_tools.txt
  [4.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1026/artifact/out/branch-findbugs-hadoop-ozone_client.txt
  [8.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1026/artifact/out/branch-findbugs-hadoop-ozone_common.txt
  [4.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1026/artifact/out/branch-findbugs-hadoop-ozone_objectstore-service.txt
  [8.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1026/artifact/out/branch-findbugs-hadoop-ozone_ozone-manager.txt
  [4.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1026/artifact/out/branch-findbugs-hadoop-ozone_ozonefs.txt
  [24K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1026/artifact/out/branch-findbugs-hadoop-ozone_s3gateway.txt
  [48K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1026/artifact/out/branch-findbugs-hadoop-ozone_tools.txt
  [8.0K]

   javadoc:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1026/artifact/out/diff-javadoc-javadoc-root.txt
  [752K]

   unit:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1026/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
  [332K]

[jira] [Created] (HADOOP-16071) Fix typo in DistCp Counters - Bandwidth in Bytes

2019-01-24 Thread Siyao Meng (JIRA) 
Siyao Meng created HADOOP-16071:
---

 Summary: Fix typo in DistCp Counters - Bandwidth in Bytes
 Key: HADOOP-16071
 URL: https://issues.apache.org/jira/browse/HADOOP-16071
 Project: Hadoop Common
  Issue Type: Bug
  Components: tools/distcp
Affects Versions: 3.2.0
Reporter: Siyao Meng
Assignee: Siyao Meng


{code:bash|title=DistCp MR Job Counters}
...
DistCp Counters
Bandwidth in Btyes=20971520
Bytes Copied=20971520
Bytes Expected=20971520
Files Copied=1
{code}

{noformat}
Bandwidth in Btyes -> Bandwidth in Bytes
{noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org