[ANNOUNCE] Aaron Fabbri as Hadoop PMC

I am very pleased to announce that Aaron Fabbri has now been added to the Hadoop PMC.  Welcome aboard, Aaron, and Congratulations! Daniel - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional

Re: June Hadoop Community Meetup

. Hope to see you there! Daniel On 5/23/19 10:57 AM, Daniel Templeton wrote: Hi, all!  I want to let you know that Cloudera is planning to host a contributors meetup on June 26 at our Palo Alto headquarters. We're still working out the details, but we're hoping to follow the format that Oath

June Hadoop Community Meetup

Hi, all!  I want to let you know that Cloudera is planning to host a contributors meetup on June 26 at our Palo Alto headquarters.  We're still working out the details, but we're hoping to follow the format that Oath and LinkedIn followed during the last two.  Please feel free to reach out to

[jira] [Created] (HADOOP-16167) "hadoop CLASSFILE" prints error messages on Ubuntu 18

Daniel Templeton created HADOOP-16167: - Summary: "hadoop CLASSFILE" prints error messages on Ubuntu 18 Key: HADOOP-16167 URL: https://issues.apache.org/jira/browse/HADOOP-16167 Proje

Re: ${!var} In Scripts

Do you want to file a JIRA for it, or shall I? Daniel On 3/4/19 9:55 AM, Allen Wittenauer wrote: On Mar 4, 2019, at 9:33 AM, Daniel Templeton wrote: Thanks! That's not even close to what the docs suggest it does--no idea what's up with that. It does. Here’s the paragraph

Re: ${!var} In Scripts

Wittenauer wrote: On Mar 1, 2019, at 3:04 PM, Daniel Templeton wrote: There are a bunch of uses of the bash syntax, "${!var}", in the Hadoop scripts. Can anyone explain to me what that syntax was supposed to achieve? #!/usr/bin/env bash j="hi" m=&quo

${!var} In Scripts

There are a bunch of uses of the bash syntax, "${!var}", in the Hadoop scripts.  Can anyone explain to me what that syntax was supposed to achieve?  According to the user guide: ${!name[@]} ${!name[*]} If name is an array variable, expands to the list of array indices (keys)

Re: Get M/R Job info from Job History Server

Have you tried consuming the REST APIs from Java? https://stackoverflow.com/questions/12916169/how-to-consume-rest-in-java Daniel On 9/24/18 1:55 AM, Mihir Monani wrote: I know common-dev is not supposed to be use for End-User questions. But can someone help me ? I know i can do this task

[jira] [Created] (HADOOP-15706) Typo in compatibility doc: SHOUD -> SHOULD

Daniel Templeton created HADOOP-15706: - Summary: Typo in compatibility doc: SHOUD -> SHOULD Key: HADOOP-15706 URL: https://issues.apache.org/jira/browse/HADOOP-15706 Project: Hadoop Com

Re: YARN SLS improving idea

Yufei, Wangda, Sunil, any comments? Daniel On 8/11/18 8:48 AM, Sichen Zhao wrote: Hi, Is there anyone who can reply my ideas? Best Regards Sichen Zhao From: Sichen Zhao Sent: Friday, August 10, 2018 11:10 To: Hadoop Common Subject: YARN SLS improving

Re: Hadoop 3.1.0 release discussion

code freeze date (Feb 08). + Sunil as well. Thanks, Wangda On Wed, Jan 17, 2018 at 4:31 PM, Daniel Templeton <dan...@cloudera.com <mailto:dan...@cloudera.com>> wrote: What's the status on resource profiles?  I believe there are still a couple of open JIRAs to

Re: Hadoop 3.1.0 release discussion

What's the status on resource profiles?  I believe there are still a couple of open JIRAs to rethink some of the design choices. Daniel On 1/17/18 11:33 AM, Wangda Tan wrote: Hi All, Since we're fast approaching previously proposed feature freeze date (Jan 30, about 13 days from today). If

Re: When are incompatible changes acceptable (HDFS-12990)

The intent of the compat guidelines is to prevent developers from making incompatible "improvements" at inconvenient times.  The guidelines offer some wiggle room for the cases where something truly broken is being fixed, especially for the sake of compatibility.  I would say that in this

Re: Apache Hadoop 2.8.3 Release Plan

Doh.  Mailer dropped some of the lists.  Replying again to avoid fragmenting the discussion... Still +1 to Andrew's comments. Daniel On 11/21/17 7:53 AM, Daniel Templeton wrote: +1 Daniel On 11/20/17 10:22 PM, Andrew Wang wrote: I'm against including new features in maintenance releases

Re: Hadoop Compatability Guide, Part Deux: Developer Docs

We've now gone a couple of rounds of reviews on HADOOP-14876, and a patch is posted for HADOOP-14875.  Feedback is very welcome.  Please take a look. Daniel On 10/14/17 8:46 AM, Daniel Templeton wrote: I just posted a first patch for HADOOP-14876 that adds downstream developer docs based

[jira] [Created] (HADOOP-14968) YarnClientImpl.getRootQueueInfos() should not do a recursive call to rmClient.getQueueInfo()

Daniel Templeton created HADOOP-14968: - Summary: YarnClientImpl.getRootQueueInfos() should not do a recursive call to rmClient.getQueueInfo() Key: HADOOP-14968 URL: https://issues.apache.org/jira/browse

Hadoop Compatability Guide, Part Deux: Developer Docs

I just posted a first patch for HADOOP-14876 that adds downstream developer docs based on the overhauled compatibility guide from HADOOP-13714.  I would really appreciate some critical review of the doc, as it's much more likely to be read by downstream developers than the compatibility spec

[jira] [Created] (HADOOP-14876) Create downstream developer docs from the compatibility guidelines

Daniel Templeton created HADOOP-14876: - Summary: Create downstream developer docs from the compatibility guidelines Key: HADOOP-14876 URL: https://issues.apache.org/jira/browse/HADOOP-14876

[jira] [Created] (HADOOP-14875) Create end user documentation from the compatibility guidelines

Daniel Templeton created HADOOP-14875: - Summary: Create end user documentation from the compatibility guidelines Key: HADOOP-14875 URL: https://issues.apache.org/jira/browse/HADOOP-14875 Project

Re: [VOTE] Release Apache Hadoop 2.8.2 (RC0)

YARN-6622 is now committed to 2.9.  We could backport YARN-5258 and YARN-6622 for 2.8, but it'll take some editing.  We'll have to check to see what features are unsupported in 2.8 and remove those from the docs.  Not a huge effort overall, though.  Probably a hour's work.  I may have time to

Re: DISCUSS: Hadoop Compatability Guidelines

versions and how it should relate to the hadoop release version. Thoughts ? Cheers -Arun On Tue, Sep 5, 2017 at 10:38 AM, Daniel Templeton <dan...@cloudera.com> wrote: Good idea. I should have thought of that. :) Done. Daniel On 9/5/17 10:33 AM, Anu Engineer wrote: Could you

Re: DISCUSS: Hadoop Compatability Guidelines

Good idea.  I should have thought of that. :)  Done. Daniel On 9/5/17 10:33 AM, Anu Engineer wrote: Could you please attach the PDFs to the JIRA. I think the mailer is stripping them off from the mail. Thanks Anu On 9/5/17, 9:44 AM, "Daniel Templeton" <dan...@cloude

Re: DISCUSS: Hadoop Compatability Guidelines

Resending with a broader audience, and reattaching the PDFs. Daniel On 9/4/17 9:01 AM, Daniel Templeton wrote: All, in prep for Hadoop 3 beta 1 I've been working on updating the compatibility guidelines on HADOOP-13714.  I think the initial doc is more or less complete, so I'd like to open

DISCUSS: Hadoop Compatability Guidelines

All, in prep for Hadoop 3 beta 1 I've been working on updating the compatibility guidelines on HADOOP-13714.  I think the initial doc is more or less complete, so I'd like to open the discussion up to the broader Hadoop community. In the new guidelines, I have drawn some lines in the sand

Re: [VOTE] Merge YARN-3926 (resource profile) to trunk

3 (Jenkins gave +1 already). Documentation: YARN-7056 Special thanks to a team of folks who worked hard and contributed towards this effort including design discussion/development/reviews, etc.: Varun Vasudev, Sunil Govind, Daniel Templeton, Vinod Vavilapalli, Yufei Gu, Karthik Kambatla, Jason L

[jira] [Created] (HADOOP-14666) Tests use assertTrue(....equals(...)) instead of assertEquals()

Daniel Templeton created HADOOP-14666: - Summary: Tests use assertTrue(equals(...)) instead of assertEquals() Key: HADOOP-14666 URL: https://issues.apache.org/jira/browse/HADOOP-14666 Project

[jira] [Created] (HADOOP-14611) NetworkTopology.DEFAULT_HOST_LEVEL is unused

Daniel Templeton created HADOOP-14611: - Summary: NetworkTopology.DEFAULT_HOST_LEVEL is unused Key: HADOOP-14611 URL: https://issues.apache.org/jira/browse/HADOOP-14611 Project: Hadoop Common

[jira] [Created] (HADOOP-14314) The OpenSolaris taxonomy link is dead

Daniel Templeton created HADOOP-14314: - Summary: The OpenSolaris taxonomy link is dead Key: HADOOP-14314 URL: https://issues.apache.org/jira/browse/HADOOP-14314 Project: Hadoop Common

[jira] [Created] (HADOOP-14310) RolloverSignerSecretProvider.LOG should be @VisibleForTesting

Daniel Templeton created HADOOP-14310: - Summary: RolloverSignerSecretProvider.LOG should be @VisibleForTesting Key: HADOOP-14310 URL: https://issues.apache.org/jira/browse/HADOOP-14310 Project

[jira] [Resolved] (HADOOP-7847) branch-1 has findbugs warnings

[ https://issues.apache.org/jira/browse/HADOOP-7847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton resolved HADOOP-7847. -- Resolution: Won't Fix Assignee: Daniel Templeton Closing since branch-1

[jira] [Resolved] (HADOOP-8939) Backport HADOOP-7457: remove cn-docs from branch-1

[ https://issues.apache.org/jira/browse/HADOOP-8939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton resolved HADOOP-8939. -- Resolution: Won't Fix Assignee: Daniel Templeton Gonna say we're not doing

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

Thanks for the new RC, Junping. I built from source and tried it out on a 2-node cluster with HA enabled. I ran a pi job and some streaming jobs. I tested that localization and failover work correctly, and I played a little with the YARN and HDFS web UIs. I did encounter an old friend of

[jira] [Resolved] (HADOOP-14074) --idle_query_timeout does not work in Impala

[ https://issues.apache.org/jira/browse/HADOOP-14074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton resolved HADOOP-14074. --- Resolution: Invalid Assignee: Daniel Templeton Please file this issue

Re: shading killing trunk build times

For most purposes, you can get by with "mvn clean process-resources process-test-resources compile -DskipTests -T 1C". (Hat tip to Karthik) Daniel On 1/18/17 6:27 AM, Steve Loughran wrote: a clean build with no tests, javadocs, is now taking 7+ min locally, with most of that time going into

[jira] [Created] (HADOOP-13997) Typo in metrics docs

Daniel Templeton created HADOOP-13997: - Summary: Typo in metrics docs Key: HADOOP-13997 URL: https://issues.apache.org/jira/browse/HADOOP-13997 Project: Hadoop Common Issue Type: Bug

Re: [VOTE] Release cadence and EOL

Thanks for driving this, Sangjin. Quick question, though: the subject line is "Release cadence and EOL," but I don't see anything about cadence in the proposal. Did I miss something? Daniel On 1/17/17 8:35 AM, Sangjin Lee wrote: Following up on the discussion thread on this topic (

[jira] [Resolved] (HADOOP-13932) Fix indefinite article in comments

[ https://issues.apache.org/jira/browse/HADOOP-13932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton resolved HADOOP-13932. --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2

Re: [DISCUSS] Commit log Pattern Unification

On 11/6/16 8:01 PM, Daniel Templeton wrote: It's also how the committer is included in the log by git. OK, git actually shows name and email in the log. It shows the username in the annotations in NetBeans, which is what I was thinking of. :) Daniel

Re: [DISCUSS] Commit log Pattern Unification

On 11/5/16 1:36 AM, Brahma Reddy Battula wrote: When we look at current git log,commit message patters are like following.. I feel, we should have unified one. Might not be very important,but it will be better have unified one. a).(Contributed by XXX via YYY) b). (XXX via YYY) c).

Re: What is the class that launches the reducers?

How are you intending to verify the map output? It's only partially dumped to disk. None of the intermediate data goes into HDFS. Daniel On Aug 25, 2016 4:10 PM, "xeon Mailinglist" wrote: > But then I need to set identity maps to run the reducers. If I suspend a >

Re: [VOTE] Release Apache Hadoop 2.7.3 RC0

I just downloaded the build tarball and deployed it on a 2-node cluster. It looks to me like it's compiled for the wrong platform: # file /usr/lib/hadoop/bin/container-executor /usr/lib/hadoop/bin/container-executor: setuid setgid Mach-O 64-bit executable I'm also seeing the

[jira] [Created] (HADOOP-13296) Cleanup javadoc for Path

Daniel Templeton created HADOOP-13296: - Summary: Cleanup javadoc for Path Key: HADOOP-13296 URL: https://issues.apache.org/jira/browse/HADOOP-13296 Project: Hadoop Common Issue Type

[jira] [Created] (HADOOP-13243) TestRollingFileSystemSink.testSetInitialFlushTime() fails intermittently

Daniel Templeton created HADOOP-13243: - Summary: TestRollingFileSystemSink.testSetInitialFlushTime() fails intermittently Key: HADOOP-13243 URL: https://issues.apache.org/jira/browse/HADOOP-13243

Javadocs JAR

Can someone please explain to me what I'm missing here? I run "mvn clean install -DskipTests -Pdist", and I end up with a bunch of artifacts in the hadoop-dist/target directory. One of them is called hadoop-dist-3.0.0-alpha1-SNAPSHOT-javadoc.jar. The

[jira] [Resolved] (HADOOP-12750) RollingFileSystemBaseTest.ErrorSink should be replaced by Mockito

[ https://issues.apache.org/jira/browse/HADOOP-12750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton resolved HADOOP-12750. --- Resolution: Won't Fix Because the sink is loaded by reflection by the metrics

[jira] [Resolved] (HADOOP-12848) Configuration.ParsedTimeDuration.unitFor(String) should do more careful parsing

[ https://issues.apache.org/jira/browse/HADOOP-12848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton resolved HADOOP-12848. --- Resolution: Invalid I am greatly saddened to report that I was too entertained

[jira] [Created] (HADOOP-12848) Configuration.ParsedTimeDuration.unitFor(String) should do more careful parsing

Daniel Templeton created HADOOP-12848: - Summary: Configuration.ParsedTimeDuration.unitFor(String) should do more careful parsing Key: HADOOP-12848 URL: https://issues.apache.org/jira/browse/HADOOP-12848

[jira] [Created] (HADOOP-12835) RollingFileSystemSink can throw an NPE on non-secure clusters

Daniel Templeton created HADOOP-12835: - Summary: RollingFileSystemSink can throw an NPE on non-secure clusters Key: HADOOP-12835 URL: https://issues.apache.org/jira/browse/HADOOP-12835 Project

[jira] [Created] (HADOOP-12775) RollingFileSystemSink doesn't work on secure clusters

Daniel Templeton created HADOOP-12775: - Summary: RollingFileSystemSink doesn't work on secure clusters Key: HADOOP-12775 URL: https://issues.apache.org/jira/browse/HADOOP-12775 Project: Hadoop

[jira] [Created] (HADOOP-12750) RollingFileSystemBaseTest.ErrorSink should be replaced by Mockito

Daniel Templeton created HADOOP-12750: - Summary: RollingFileSystemBaseTest.ErrorSink should be replaced by Mockito Key: HADOOP-12750 URL: https://issues.apache.org/jira/browse/HADOOP-12750

[jira] [Created] (HADOOP-12732) Filesystem.addDelegationToken() should automatically replace _HOST

Daniel Templeton created HADOOP-12732: - Summary: Filesystem.addDelegationToken() should automatically replace _HOST Key: HADOOP-12732 URL: https://issues.apache.org/jira/browse/HADOOP-12732

[jira] [Created] (HADOOP-12726) Unsupported operations should throw UnsupportedOperationException

Daniel Templeton created HADOOP-12726: - Summary: Unsupported operations should throw UnsupportedOperationException Key: HADOOP-12726 URL: https://issues.apache.org/jira/browse/HADOOP-12726

Hostname Replacement in Principals

I've noticed that if I add an auth_to_local rule that forces all 2-part principals that don't originate from one of my hosts to nobody, things break. Specifically, many non-MR YARN apps break. The reason MR doesn't break is that MR knows to replace the _HOST string in the RM_PRINCIPAL

[jira] [Created] (HADOOP-12702) Add an HDFS metrics sink

Daniel Templeton created HADOOP-12702: - Summary: Add an HDFS metrics sink Key: HADOOP-12702 URL: https://issues.apache.org/jira/browse/HADOOP-12702 Project: Hadoop Common Issue Type

Enabling Lint

Newbish question: when I'm building locally, I'm not getting the lint warnings, and no amount of fiddling with the POMs is having any effect. This feels like something that that should already be a solved issue that I'm just not grokking. If I run javac by hand, I get the warnings. What am

[jira] [Created] (HADOOP-12566) Add NullGroupMapping

Daniel Templeton created HADOOP-12566: - Summary: Add NullGroupMapping Key: HADOOP-12566 URL: https://issues.apache.org/jira/browse/HADOOP-12566 Project: Hadoop Common Issue Type

[jira] [Resolved] (HADOOP-12530) Error when trying to format HDFS by running hdfs namenode -format

[ https://issues.apache.org/jira/browse/HADOOP-12530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton resolved HADOOP-12530. --- Resolution: Not A Problem Assignee: Daniel Templeton This is a configuration

Re: [VOTE] Release Apache Hadoop 2.6.2

Setup a 3-node cluster and ran some Hadoop Streaming jobs against it. +1 (non-binding) Daniel On 10/23/15 5:24 PM, Sangjin Lee wrote: +1 (binding). Downloaded the tarballs, ran a pseudo-distributed cluster, and ran example MR jobs with normal and uber mode. On Thu, Oct 22, 2015 at 6:28 PM,

[jira] [Created] (HADOOP-12359) hdfs dfs -getmerge docs are wrong

Daniel Templeton created HADOOP-12359: - Summary: hdfs dfs -getmerge docs are wrong Key: HADOOP-12359 URL: https://issues.apache.org/jira/browse/HADOOP-12359 Project: Hadoop Common Issue

Re: [DISCUSSION] Release process

From the Java SE 7 JavaDocs: A program element annotated @Deprecated is one that programmers are discouraged from using, typically because it is dangerous, or because a better alternative exists. Compilers warn when a deprecated program element is used or overridden in non-deprecated code.

Re: Hadoop on Sun Solaris

I'm using it on Solaris without any problem. Of course, I'm just using the provided JAR files. As long as you have all the right pieces, e.g. ant, javac, the libraries, you should be able to build. Daniel Palikala, Rajendra (CCL) wrote: Can I build Hadoop on Sun Solaris. The documentation

Re: Hadoop coding style guideline

I have to weigh in strongly on the pro-braces side. I've seen too many instances (not necessarily in Hadoop) where there was something like: if (foo) stmt; otherstmt; It's not about readability. It's about maintainability. Daniel Doug Cutting wrote: Aaron Kimball wrote: First, I've