[jira] [Created] (HADOOP-10404) Some accesses to DomainSocketWatcher#closed are not protected by lock

[Ted Yu (JIRA)]

Ted Yu created HADOOP-10404:
---

 Summary: Some accesses to DomainSocketWatcher#closed are not 
protected by lock
 Key: HADOOP-10404
 URL: https://issues.apache.org/jira/browse/HADOOP-10404
 Project: Hadoop Common
  Issue Type: Bug
Reporter: Ted Yu
Priority: Minor


{code}
   * Lock which protects toAdd, toRemove, and closed.
   */
  private final ReentrantLock lock = new ReentrantLock();
{code}
There're two places, NotificationHandler.handle() and kick(), where access to 
closed is without holding lock.




--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HADOOP-10402) Configuration.getValByRegex doesn't do variable substitution

[Robert Kanter (JIRA)]

Robert Kanter created HADOOP-10402:
--

 Summary: Configuration.getValByRegex doesn't do variable 
substitution
 Key: HADOOP-10402
 URL: https://issues.apache.org/jira/browse/HADOOP-10402
 Project: Hadoop Common
  Issue Type: Bug
Affects Versions: 2.3.0
Reporter: Robert Kanter
Assignee: Robert Kanter


When using Configuration.getValByRegex(...), variables are not resolved.  

For example:
{code:xml}

   bar
   woot


   foo3
   ${bar}

{code}
If you then try to do something like {{Configuration.getValByRegex(foo.*)}}, it 
will return a Map containing "foo3=$\{bar}" instead of "foo3=woot"



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HADOOP-10403) Hadoop2.2 is not fully transfer hostname to IP address on WEB UI

[hawin (JIRA)]

hawin created HADOOP-10403:
--

 Summary: Hadoop2.2 is not fully transfer hostname to IP address on 
WEB UI
 Key: HADOOP-10403
 URL: https://issues.apache.org/jira/browse/HADOOP-10403
 Project: Hadoop Common
  Issue Type: Bug
  Components: conf, fs, util
Affects Versions: 2.2.0
 Environment: OS: Linux 2.6.32-431.el6.x86_64 #1 SMP Fri Nov 22 
03:15:09 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

master: hostname1
slave1: hostname2
slave2: hostname3
Reporter: hawin


I could not get access when I was trying to click links which are include 
hostname in the URL.

For example: 
My master ip address is 192.168.1.110 and hostname is hostname1
http://192.168.1.110:50070/ works fine
but http://hostname1:50070/ does not work. 

My slave servers also have the same issues. some links only display 
http://hostname:port.  and some links used IP address replace of hostname.

I can ping hostname1, hostname2 and hostname3 and also ssh slave servers 
without password from master server. 


cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.1.110 hostname1
192.168.1.111 hostname2
192.168.1.112 hostname3
*

Please help me to check this issue.  I don't know whether I am missing some 
configuration.  Your hands will be appreciated. 





--
This message was sent by Atlassian JIRA
(v6.2#6252)

Re: Thinking ahead to 2.4

[Arun C Murthy]

Sorry, the previous link had a bug, the correct one is: 
http://s.apache.org/hadoop-2.4.0-blockers.

We are currently down to 12 blockers; with several PA.

thanks,
Arun

On Mar 6, 2014, at 1:40 PM, Arun C Murthy  wrote:

> Gang,
> 
>  Most of the big-ticket items are already in, awesome!
> 
>  I'm thinking we could roll out a 2.4 RC in the next 2-3 weeks after we get 
> through the list of blockers. Here is a handy link: 
> http://s.apache.org/hadoop-2.4-blockers
> 
>  If you find more, please set Target Version to 2.4.0 and mark it a blocker. 
> I'll try nudging people to start closing these soon, appreciate any help!
> 
> thanks,
> Arun
> 
>  
> On Feb 20, 2014, at 3:45 PM, Arun C Murthy  wrote:
> 
>> Thanks Azuryy & Suresh. I've updated the roadmap wiki to reflect this.
>> 
>> Arun
>> 
>> On Feb 20, 2014, at 2:01 PM, Suresh Srinivas  wrote:
>> 
>>> Arun,
>>> 
>>> Some of the previously 2.4 targeted features were made available in 2.3:
>>> - Heterogeneous storage support
>>> - Datanode cache
>>> 
>>> The following are being targeted for 2.4:
>>> - Use protobuf for fsimge (already in)
>>> - ACLs (in trunk. In a week or so, this will be merged to branch-2.4)
>>> - Rolling upgrades (last bunch of jiras being worked in feature branch.
>>> Will be in 2.4 in around two weeks. Currently testing is in progress)
>>> 
>>> So HDFS features should be ready in two weeks.
>>> 
>>> 
>>> On Sat, Feb 15, 2014 at 4:47 PM, Azuryy  wrote:
>>> 
 Hi，
 I think you omit some key pieces in 2.4
 
 Protobuf fsimage, rolling upgrade are also targeting 2.4
 
 
 
 Sent from my iPhone5s
 
> On 2014年2月16日, at 6:59, Arun C Murthy  wrote:
> 
> Folks,
> 
> With hadoop-2.3 nearly done, I think it's time to think ahead to
 hadoop-2.4. I think it was a good idea to expedite release of 2.3 while we
 finished up pieces that didn't make it in such as HDFS Caching & Support
 for Heterogenous Storage.
> 
> Now, most of the key pieces incl. Resource Manager Automatic Failover
 (YARN-149), Application History Server (YARN-321) & Application Timeline
 Server (YARN-1530) are either complete or very close to done, and I think
 we will benefit with an extended test-cycle for 2.4 - similar to what
 happened with 2.2. To provide some context: 2.2 went through nearly 6 weeks
 of extended testing and it really helped us push out a very stable release.
> 
> I think it will be good to create a 2.4 branch ASAP and start testing.
 As such, I plan to cut the branch early next week. With this, we should be
 good shape sometime to release 2.4 in mid-March.
> 
> I've updated https://wiki.apache.org/hadoop/Roadmap to reflect this.
> 
> Also, we should start thinking ahead to 2.5 and what folks would like to
 see in it. If we continue our 6-week cycles, we could shoot to get that out
 in April.
> 
> Thoughts?
> 
> thanks,
> Arun
> 
> 
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
> 
> 
> 
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
 to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified
 that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender
 immediately
> and delete it from your system. Thank You.
 
>>> 
>>> 
>>> 
>>> -- 
>>> http://hortonworks.com/download/
>>> 
>>> -- 
>>> CONFIDENTIALITY NOTICE
>>> NOTICE: This message is intended for the use of the individual or entity to 
>>> which it is addressed and may contain information that is confidential, 
>>> privileged and exempt from disclosure under applicable law. If the reader 
>>> of this message is not the intended recipient, you are hereby notified that 
>>> any printing, copying, dissemination, distribution, disclosure or 
>>> forwarding of this communication is strictly prohibited. If you have 
>>> received this communication in error, please contact the sender immediately 
>>> and delete it from your system. Thank You.
>> 
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> http://hortonworks.com/
>> 
>> 
> 
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
> 
> 

--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/



-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any

Re: Release notes for 2.3.0 on website

[Arun C Murthy]

Andrew,

 This needs a fix to the docs in the main tree, not just the site. Agree we 
should fix this going fwd.

thanks,
Arun

On Mar 7, 2014, at 10:51 AM, Andrew Wang  wrote:

> Hi all,
> 
> I found this page when googling for "Hadoop 2.3.0 documentation", looks like 
> it's pretty out of date (references "new" features like federation and YARN).
> 
> Arun, should we update this with your release blurb? I could also try my hand 
> at pushing the site if you're busy.
> 
> http://hadoop.apache.org/docs/r2.3.0/
> 
> Thanks,
> Andrew

--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/



-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Build failed in Jenkins: Hadoop-Common-trunk #1066

[Apache Jenkins Server]

See 

Changes:

[cmccabe] HDFS-6085. Improve CacheReplicationMonitor log messages a bit 
(cmccabe)

[stack] HADOOP-10337 ConcurrentModificationException from 
MetricsDynamicMBeanBase.createMBeanInfo() (Liang Xie via stack)

[vinodkv] MAPREDUCE-5028. Fixed a bug in MapTask that was causing mappers to 
fail when a large value of io.sort.mb is set. Contributed by Karthik Kambatla.

[vinodkv] YARN-1764. Modified YarnClient to correctly handle failover of 
ResourceManager after the submitApplication call goes through. Contributed by 
Xuan Gong.

[wang] Move HDFS-3405 to 2.4.0 section in CHANGES.txt

[szetszwo] Move HDFS-5535 to Release 2.4.0 in CHANGES.txt.

[arp] HADOOP-10394. TestAuthenticationFilter is flaky. (Arpit Agarwal)

[arp] HADOOP-10395. TestCallQueueManager is flaky. (Arpit Agarwal)

[cnauroth] HADOOP-10399. FileContext API for ACLs. Contributed by Vinayakumar B.

[cnauroth] HDFS-6055. Change default configuration to limit file name length in 
HDFS. Contributed by Chris Nauroth.

[jing9] HDFS-6077. running slive with webhdfs on secure HA cluster fails with 
unkown host exception. Contributed by Jing Zhao.

[vinodkv] YARN-1788. Fixed a bug in ResourceManager to set the apps-completed 
and apps-killed metrics correctly for killed applications. Contributed by Varun 
Vasudev.

[wang] HDFS-6070. Cleanup use of ReadStatistics in DFSInputStream.

[wheat9] HDFS-5892. TestDeleteBlockPool fails in branch-2. Contributed by Ted 
Yu.

[jianhe] YARN-1793. Fixed ClientRMService#forceKillApplication not killing 
unmanaged application. Contributed by Karthik Kambatla

--
[...truncated 66479 lines...]
Adding reference: maven.local.repository
[DEBUG] Initialize Maven Ant Tasks
parsing buildfile 
jar:file:/home/jenkins/.m2/repository/org/apache/maven/plugins/maven-antrun-plugin/1.7/maven-antrun-plugin-1.7.jar!/org/apache/maven/ant/tasks/antlib.xml
 with URI = 
jar:file:/home/jenkins/.m2/repository/org/apache/maven/plugins/maven-antrun-plugin/1.7/maven-antrun-plugin-1.7.jar!/org/apache/maven/ant/tasks/antlib.xml
 from a zip file
parsing buildfile 
jar:file:/home/jenkins/.m2/repository/org/apache/ant/ant/1.8.2/ant-1.8.2.jar!/org/apache/tools/ant/antlib.xml
 with URI = 
jar:file:/home/jenkins/.m2/repository/org/apache/ant/ant/1.8.2/ant-1.8.2.jar!/org/apache/tools/ant/antlib.xml
 from a zip file
Class org.apache.maven.ant.tasks.AttachArtifactTask loaded from parent loader 
(parentFirst)
 +Datatype attachartifact org.apache.maven.ant.tasks.AttachArtifactTask
Class org.apache.maven.ant.tasks.DependencyFilesetsTask loaded from parent 
loader (parentFirst)
 +Datatype dependencyfilesets org.apache.maven.ant.tasks.DependencyFilesetsTask
Setting project property: test.build.dir -> 

Setting project property: test.exclude.pattern -> _
Setting project property: hadoop.assemblies.version -> 3.0.0-SNAPSHOT
Setting project property: test.exclude -> _
Setting project property: distMgmtSnapshotsId -> apache.snapshots.https
Setting project property: project.build.sourceEncoding -> UTF-8
Setting project property: java.security.egd -> file:///dev/urandom
Setting project property: distMgmtSnapshotsUrl -> 
https://repository.apache.org/content/repositories/snapshots
Setting project property: distMgmtStagingUrl -> 
https://repository.apache.org/service/local/staging/deploy/maven2
Setting project property: avro.version -> 1.7.4
Setting project property: test.build.data -> 

Setting project property: commons-daemon.version -> 1.0.13
Setting project property: hadoop.common.build.dir -> 

Setting project property: testsThreadCount -> 4
Setting project property: maven.test.redirectTestOutputToFile -> true
Setting project property: jdiff.version -> 1.0.9
Setting project property: build.platform -> Linux-i386-32
Setting project property: project.reporting.outputEncoding -> UTF-8
Setting project property: distMgmtStagingName -> Apache Release Distribution 
Repository
Setting project property: protobuf.version -> 2.5.0
Setting project property: failIfNoTests -> false
Setting project property: protoc.path -> ${env.HADOOP_PROTOC_PATH}
Setting project property: jersey.version -> 1.9
Setting project property: distMgmtStagingId -> apache.staging.https
Setting project property: distMgmtSnapshotsName -> Apache Development Snapshot 
Repository
Setting project property: ant.file -> 

[DEBUG] Setting properties with prefix: 
Setting project property: project.groupId -> org.apache.hadoop
Setting project property: project.artifactId -> hadoop-common-pr