[jira] [Created] (HADOOP-14227) S3Guard: ITestS3AConcurrentOps is not cleaning up test data

2017-03-23 Thread Mingliang Liu (JIRA) 
Mingliang Liu created HADOOP-14227:
--

 Summary: S3Guard: ITestS3AConcurrentOps is not cleaning up test 
data
 Key: HADOOP-14227
 URL: https://issues.apache.org/jira/browse/HADOOP-14227
 Project: Hadoop Common
  Issue Type: Bug
  Components: fs/s3
Reporter: Mingliang Liu
Assignee: Mingliang Liu


After running {{ITestS3AConcurrentOps}}, the test data is not cleanup in 
DynamoDB. There are two reasons:
# The {{ITestS3AConcurrentOps::teardown()}} method is not calling super 
teardown() method to clean up the default test directory.
# The {{auxFs}} is not S3Guard aware even though the {{fs}} to test is. That's 
because the {{auxFs}} is creating a new Configuration object without patching 
in S3Guard options (via {{maybeEnableS3Guard(conf);}}).

This JIRA is to clean up the data after test.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)

2017-03-23 Thread Allen Wittenauer 

Just a heads up.  Looks like some removed the Finish Date off of 2.8.0 in JIRA. 
 It needs to be put back to match what is in the artifacts that we voted on.
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Created] (HADOOP-14226) S3Guard: ITestDynamoDBMetadataStoreScale is not cleaning up test data

2017-03-23 Thread Mingliang Liu (JIRA) 
Mingliang Liu created HADOOP-14226:
--

 Summary: S3Guard: ITestDynamoDBMetadataStoreScale is not cleaning 
up test data
 Key: HADOOP-14226
 URL: https://issues.apache.org/jira/browse/HADOOP-14226
 Project: Hadoop Common
  Issue Type: Sub-task
  Components: fs/s3
Affects Versions: HADOOP-13345
Reporter: Mingliang Liu
Assignee: Mingliang Liu


After running {{ITestDynamoDBMetadataStoreScale}}, the test data is not cleaned 
up. There is a call to {{clearMetadataStore(ms, count);}} in the finally clause 
though. The reason is that, the internally called method 
{{DynamoDBMetadataStore::deleteSubtree()}} is assuming there should be an item 
for the parent dest path:
{code}
parent=/fake-bucket, child=moved-here, is_dir=true
{code}

In DynamoDBMetadataStore implementation, we assume that _if a path exists, all 
its ancestors will also exist in the table_. We need to pre-create dest path to 
maintain this invariant so that test data can be cleaned up successfully.

I think there may be other tests with the same problem. Let's identify/address 
them separately.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Created] (HADOOP-14225) Remove xmlenc dependency

2017-03-23 Thread Chris Douglas (JIRA) 
Chris Douglas created HADOOP-14225:
--

 Summary: Remove xmlenc dependency
 Key: HADOOP-14225
 URL: https://issues.apache.org/jira/browse/HADOOP-14225
 Project: Hadoop Common
  Issue Type: Improvement
Reporter: Chris Douglas
Priority: Minor


The xmlenc library is used only in the following two classes:
{noformat}
o.a.h.fs.MD5MD5CRC32FileChecksum
o.a.h.hdfs.server.namenode.DfsServlet
{noformat}

Given that Hadoop already includes other fast XML encoders as dependencies, we 
can lose this one.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Created] (HADOOP-14224) add dnsutils to Dockerfile to aid in debugging maven repo failures

2017-03-23 Thread Allen Wittenauer (JIRA) 
Allen Wittenauer created HADOOP-14224:
-

 Summary: add dnsutils to Dockerfile to aid in debugging maven repo 
failures
 Key: HADOOP-14224
 URL: https://issues.apache.org/jira/browse/HADOOP-14224
 Project: Hadoop Common
  Issue Type: Improvement
  Components: build
Affects Versions: 3.0.0-alpha3
Reporter: Allen Wittenauer
Assignee: Allen Wittenauer
Priority: Trivial


It would be useful to have dig to troubleshoot if the Docker-local resolver is 
working correctly.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

2017-03-23 Thread Apache Jenkins Server 
For more details, see 
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/354/

[Mar 23, 2017 5:15:24 AM] (jzhuge) HDFS-11561. HttpFS doc errors. Contributed 
by Yuanbo Liu.
[Mar 23, 2017 5:22:22 AM] (zhz) HADOOP-9631. ViewFs should use underlying 
FileSystem's server side




-1 overall


The following subsystems voted -1:
asflicense unit


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace


The following subsystems are considered long running:
(runtime bigger than 1h  0m  0s)
unit


Specific tests:

Failed junit tests :

   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting 
   hadoop.hdfs.server.balancer.TestBalancer 
   hadoop.hdfs.TestReadStripedFileWithMissingBlocks 
   hadoop.hdfs.server.namenode.TestAddStripedBlockInFBR 
   hadoop.hdfs.server.datanode.TestDataNodeUUID 
   hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer 
   hadoop.yarn.server.resourcemanager.TestRMEmbeddedElector 
   hadoop.yarn.server.TestContainerManagerSecurity 
   hadoop.yarn.server.TestMiniYarnClusterNodeUtilization 
  

   cc:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/354/artifact/out/diff-compile-cc-root.txt
  [4.0K]

   javac:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/354/artifact/out/diff-compile-javac-root.txt
  [184K]

   checkstyle:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/354/artifact/out/diff-checkstyle-root.txt
  [17M]

   pylint:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/354/artifact/out/diff-patch-pylint.txt
  [20K]

   shellcheck:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/354/artifact/out/diff-patch-shellcheck.txt
  [24K]

   shelldocs:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/354/artifact/out/diff-patch-shelldocs.txt
  [12K]

   whitespace:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/354/artifact/out/whitespace-eol.txt
  [11M]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/354/artifact/out/whitespace-tabs.txt
  [1.3M]

   javadoc:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/354/artifact/out/diff-javadoc-javadoc-root.txt
  [2.2M]

   unit:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/354/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
  [1.4M]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/354/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
  [60K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/354/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt
  [324K]

   asflicense:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/354/artifact/out/patch-asflicense-problems.txt
  [4.0K]

Powered by Apache Yetus 0.5.0-SNAPSHOT   http://yetus.apache.org



-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Apache Hadoop qbt Report: trunk+JDK8 on Linux/ppc64le

2017-03-23 Thread Apache Jenkins Server 
For more details, see 
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/266/

[Mar 23, 2017 5:15:24 AM] (jzhuge) HDFS-11561. HttpFS doc errors. Contributed 
by Yuanbo Liu.
[Mar 23, 2017 5:22:22 AM] (zhz) HADOOP-9631. ViewFs should use underlying 
FileSystem's server side




-1 overall


The following subsystems voted -1:
compile unit


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc javac


The following subsystems are considered long running:
(runtime bigger than 1h  0m  0s)
unit


Specific tests:

Failed junit tests :

   hadoop.hdfs.server.datanode.TestNNHandlesBlockReportPerStorage 
   hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer 
   hadoop.hdfs.server.datanode.checker.TestThrottledAsyncChecker 
   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting 
   hadoop.hdfs.web.TestWebHdfsTimeouts 
   hadoop.yarn.server.timeline.TestRollingLevelDB 
   hadoop.yarn.server.timeline.TestTimelineDataManager 
   hadoop.yarn.server.timeline.TestLeveldbTimelineStore 
   hadoop.yarn.server.timeline.recovery.TestLeveldbTimelineStateStore 
   hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore 
   
hadoop.yarn.server.applicationhistoryservice.TestApplicationHistoryServer 
   hadoop.yarn.server.resourcemanager.recovery.TestLeveldbRMStateStore 
   hadoop.yarn.server.TestMiniYarnClusterNodeUtilization 
   hadoop.yarn.server.TestContainerManagerSecurity 
   hadoop.yarn.client.api.impl.TestAMRMClient 
   hadoop.yarn.server.timeline.TestLevelDBCacheTimelineStore 
   hadoop.yarn.server.timeline.TestOverrideTimelineStoreYarnClient 
   hadoop.yarn.server.timeline.TestEntityGroupFSTimelineStore 
   hadoop.yarn.applications.distributedshell.TestDistributedShell 
   hadoop.mapred.TestShuffleHandler 
   hadoop.mapreduce.v2.hs.TestHistoryServerLeveldbStateStoreService 

Timed out junit tests :

   org.apache.hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean 
   org.apache.hadoop.hdfs.server.datanode.TestFsDatasetCache 
  

   compile:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/266/artifact/out/patch-compile-root.txt
  [136K]

   cc:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/266/artifact/out/patch-compile-root.txt
  [136K]

   javac:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/266/artifact/out/patch-compile-root.txt
  [136K]

   unit:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/266/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
  [440K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/266/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
  [16K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/266/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice.txt
  [52K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/266/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
  [68K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/266/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt
  [324K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/266/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
  [12K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/266/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-timeline-pluginstorage.txt
  [28K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/266/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-applications-distributedshell.txt
  [12K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/266/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-ui.txt
  [8.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/266/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-shuffle.txt
  [8.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/266/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs.txt
  [16K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/266/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-nativetask.txt
  [44K]

Powered by Apache Yetus 0.5.0-SNAPSHOT   http://yetus.apache.org



-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional

[jira] [Created] (HADOOP-14223) Extend FileStatus#toString() to include details like Erasure Coding and Encryption

2017-03-23 Thread Manoj Govindassamy (JIRA) 
Manoj Govindassamy created HADOOP-14223:
---

 Summary: Extend FileStatus#toString() to include details like 
Erasure Coding and Encryption
 Key: HADOOP-14223
 URL: https://issues.apache.org/jira/browse/HADOOP-14223
 Project: Hadoop Common
  Issue Type: Bug
  Components: fs
Affects Versions: 3.0.0-alpha1
Reporter: Manoj Govindassamy
Assignee: Manoj Govindassamy


HDFS-6843 and HADOOP-13715 have enhanced {{FileStatus}} to include details on 
whether the underlying path is Encrypted and Erasure Coded. The additional 
details are embedded in the FsPermission high order bits. It would be really 
helpful for debugging if FileStatus#toString() returns these new bits details 
along with already existing one. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Created] (HADOOP-14221) Add s3guardtool dump command

2017-03-23 Thread Steve Loughran (JIRA) 
Steve Loughran created HADOOP-14221:
---

 Summary: Add s3guardtool dump command
 Key: HADOOP-14221
 URL: https://issues.apache.org/jira/browse/HADOOP-14221
 Project: Hadoop Common
  Issue Type: Sub-task
  Components: fs/s3
Affects Versions: HADOOP-13345
Reporter: Steve Loughran
Priority: Minor


Add a command Dump the database, perhaps matching a pattern, for diagnostics.

e.g 
{code}
s3guard dump s3a://steve4/datasets/queries
{code}
issue: large files, throttling, etc. etc.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Created] (HADOOP-14220) Add S3GuardTool diagnostics command

2017-03-23 Thread Steve Loughran (JIRA) 
Steve Loughran created HADOOP-14220:
---

 Summary: Add S3GuardTool diagnostics command
 Key: HADOOP-14220
 URL: https://issues.apache.org/jira/browse/HADOOP-14220
 Project: Hadoop Common
  Issue Type: Sub-task
  Components: fs/s3
Affects Versions: HADOOP-13345
Reporter: Steve Loughran


Add a diagnostics command to s3guard which does whatever we need to diagnose 
problems for a specific (named) s3a url. This is something which can be 
attached to bug reports as well as used by developers.

* Properties to log (with provenance attribute, which can track bucket 
overrides: s3guard metastore setup, autocreate, capacity, 

* table present/absent
* # of keys in DDB table for that bucket?
* any other stats?






--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[Write access needed] Update of the PoweredBy wiki page for my company

2017-03-23 Thread Rémy Saissy 
Hi,

as explained on https://wiki.apache.org/hadoop/PoweredBy,
PoweredBy - Hadoop Wiki 
wiki.apache.org
This page documents an alphabetical list of institutions that are using
Apache Hadoop for educational or production uses. Companies that offer
services on ...
I would need write access to that wiki page to update Hadoop clusters usage
information about my company, Criteo.

Thanks.

-- 
Rémy Saissy

[jira] [Created] (HADOOP-14219) RumenToSLS: parsing problem with crashed attempts

2017-03-23 Thread Julien Vaudour (JIRA) 
Julien Vaudour created HADOOP-14219:
---

 Summary: RumenToSLS: parsing problem with crashed attempts
 Key: HADOOP-14219
 URL: https://issues.apache.org/jira/browse/HADOOP-14219
 Project: Hadoop Common
  Issue Type: Bug
  Components: tools
Affects Versions: 2.6.0
Reporter: Julien Vaudour
Priority: Minor


In case of crashed task attempts, we may have in rumen logs task attempts with 
null hostName and finishTime defined to -1

for example
{code}
   {
  "resourceUsageMetrics": {
"heapUsage": 0,
"physicalMemoryUsage": 0,
"virtualMemoryUsage": 0,
"cumulativeCpuUsage": 0
  },
  "vmemKbytes": [],
  "physMemKbytes": [],
  "cpuUsages": [],
  "clockSplits": [],
  "location": null,
  "sortFinished": -1,
  "shuffleFinished": -1,
  "spilledRecords": -1,
  "reduceOutputRecords": -1,
  "reduceShuffleBytes": -1,
  "fileBytesRead": -1,
  "hdfsBytesWritten": -1,
  "hdfsBytesRead": -1,
  "hostName": null,
  "finishTime": -1,
  "startTime": 1489619193378,
  "result": null,
  "attemptID": "attempt_1488896259152_410442_r_15_1",
  "fileBytesWritten": -1,
  "mapInputRecords": -1,
  "mapInputBytes": -1,
  "mapOutputBytes": -1,
  "mapOutputRecords": -1,
  "combineInputRecords": -1,
  "reduceInputGroups": -1,
  "reduceInputRecords": -1
}
{code}

Jackson parser will automatically consider -1 as a java.lang.Integer. However 
RumenToSLSConverter make the assumption than jackson has deserialize all 
timstamp as instance of java.lang.Long, resulting in a ClassCastException.

RumenToSLSConverter also make the assumption that hostName is not null, so we 
can also have a NullPointerException.




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Created] (HADOOP-14218) Replace assertThat with assertTrue in MetricsAsserts

2017-03-23 Thread Akira Ajisaka (JIRA) 
Akira Ajisaka created HADOOP-14218:
--

 Summary: Replace assertThat with assertTrue in MetricsAsserts
 Key: HADOOP-14218
 URL: https://issues.apache.org/jira/browse/HADOOP-14218
 Project: Hadoop Common
  Issue Type: Test
Reporter: Akira Ajisaka
Assignee: Akira Ajisaka
Priority: Minor


{code:title=MetricsAsserts.java}
  public static void assertCounterGt(String name, long greater,
 MetricsRecordBuilder rb) {
Assert.assertThat("Bad value for metric " + name, getLongCounter(name, rb),
new GreaterThan(greater));
  }
{code}
The following code cannot be compiled with Mockito 2.1+ because it does not 
depend on org.hamcrest.Matcher anymore. We can simply replace this code with 
{{assertTrue(message, getLongCounter() > greater)}}.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)

2017-03-23 Thread Junping Du 
Thanks again for all who verified and voted!


I give my binding +1 to conclude the vote for 2.8.0 RC3, based on:
- Build from source and verify signatures
- Deploy pseudo-distributed cluster with capacity scheduler
- Verify UI of daemons, like: NameNode, ResourceManager, NodeManager, etc.
- Run some example MR jobs, like: PI, sleep, etc.

Now, we have:

7 binding +1s, from:
 Wangda Tan, Jason Lowe, Akira Ajisaka, Ravi Prakash,
 Karthik Kambatla, Jian He, Junping Du

18 non-binding +1s, from:
Miklos Szegedi, Eric Payne, Daniel Templeton, Mingliang Liu,
Sunil Govind, Marton Elek, Brahma Reddy Battula, Masatake Iwasaki,
Gergo Pasztor, Haibo Chen, Zhihai Xu, John Zhuge, Eric Badger,
Kuhu Shukla, Larry Mccay, Rakesh Radhakrishnan, Naganarasimha Garla,
Varun Saxena

1 binding +0, from:
Steve Loughran

and no -1s.

So I am glad to announce that the vote of 2.8.0 RC3 passes.

Thanks everyone listed above who tried the release candidate and vote.
Also, kudos to all who ever help with 2.8.0 release effort in all kinds of ways.
Without working together in community, we cannot afford so many issues get found
in RC stage (as a minor release with debit of ~2 years commits) and most of them
get fixed so quickly.

I'll push the release bits and send out an announcement for 2.8.0 soon.


Thanks,

Junping​



From: Karthik Kambatla 
Sent: Wednesday, March 22, 2017 2:10 PM
To: varunsax...@apache.org
Cc: Junping Du; common-dev@hadoop.apache.org; hdfs-...@hadoop.apache.org; 
yarn-...@hadoop.apache.org; mapreduce-...@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

+1 (binding)

* Built from source
* Started a pseudo-distributed cluster with fairscheduler.
* Ran sample jobs
* Verified WebUI

On Wed, Mar 22, 2017 at 11:56 AM, 
varunsax...@apache.org 
> wrote:
Thanks Junping for creating the release.

+1 (non-binding)

* Verified signatures.
* Built from source.
* Set up a pseudo-distributed cluster.
* Successfully ran pi and wordcount jobs.
* Navigated the YARN RM and NM UI.

Regards,
Varun Saxena.

On Wed, Mar 22, 2017 at 12:13 AM, Haibo Chen 
> wrote:

> Thanks Junping for working on the new release!
>
> +1 non-binding
>
> 1) Downloaded the source, verified the checksum
> 2) Built natively from source, and deployed it to a pseudo-distributed
> cluster
> 3) Ran sleep and teragen job and checked both YARN and JHS web UI
> 4) Played with yarn + mapreduce command lines
>
> Best,
> Haibo Chen
>
> On Mon, Mar 20, 2017 at 11:18 AM, Junping Du 
> > wrote:
>
> > ?Thanks for update, John. Then we should be OK with fixing this issue in
> > 2.8.1.
> >
> > Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and
> bump
> > up to blocker in case we could miss this in releasing 2.8.1. :)
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> > 
> > From: John Zhuge >
> > Sent: Monday, March 20, 2017 10:31 AM
> > To: Junping Du
> > Cc: common-dev@hadoop.apache.org; 
> > hdfs-...@hadoop.apache.org;
> > yarn-...@hadoop.apache.org; 
> > mapreduce-...@hadoop.apache.org
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Yes, it only affects ADL. There is a workaround of adding these 2
> > properties to core-site.xml:
> >
> >   
> > fs.adl.impl
> > org.apache.hadoop.fs.adl.AdlFileSystem
> >   
> >
> >   
> > fs.AbstractFileSystem.adl.impl
> > org.apache.hadoop.fs.adl.Adl
> >   
> >
> > I have the initial patch ready but hitting these live unit test failures:
> >
> > Failed tests:
> >   TestAdlFileSystemContractLive.runTest:60->FileSystemContract
> BaseTest.testListStatus:257
> > expected:<1> but was:<10>
> >
> > Tests in error:
> >   TestAdlFileContextMainOperationsLive>FileContextMainOperatio
> nsBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
> >   TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
> >
> >
> > Stay tuned...
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Mon, Mar 20, 2017 at 10:02 AM, Junping Du 
> > 
>  > j...@hortonworks.com>> wrote:
> >
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.?
> >
> >
> > Thanks,
> >
> >
> >