date:20130517

[
https://issues.apache.org/jira/browse/HADOOP-9517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13660819#comment-13660819
]

Steve Loughran commented on HADOOP-9517:

Another one: OS/JVM compatibility. What guarantees/promises are made for OS
JVM.

# I could see OS family statements: Linux, Windows, vendor-backed versions such
as IBM Power, but not specific versions, especially as the OS vendor trails off
security patches. Testing and bug reports welcome.

# JVM? What could be said here? That the min JVM is specified by the code, no
intent to force updates on a point release unless/until that JVM version
becomes unsupported. Major releases (or different OS platform/Hardware releases
may mandate later versions (e.g windows and Arm prefer open-jdk7). Testing and
bug reports welcome.

# Hardware? No plans to make Hadoop specific to any CPU family, even though
there may be some CPU family-specific optimisations at the native code level.
or this is driven by (JVM, OS) support; testing always welcome).

Define Hadoop Compatibility
---

Key: HADOOP-9517
URL: https://issues.apache.org/jira/browse/HADOOP-9517
Project: Hadoop Common
Issue Type: Bug
Components: documentation
Reporter: Arun C Murthy
Assignee: Karthik Kambatla
Attachments: hadoop-9517.patch, hadoop-9517.patch, hadoop-9517.patch,
hadoop-9517.patch

As we get ready to call hadoop-2 stable we need to better define 'Hadoop
Compatibility'.
http://wiki.apache.org/hadoop/Compatibility is a start, let's document
requirements clearly and completely.

[jira] [Commented] (HADOOP-9562) Create REST interface for HDFS health data

[
https://issues.apache.org/jira/browse/HADOOP-9562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13660825#comment-13660825
]

Steve Loughran commented on HADOOP-9562:

Clearly I was confused. My apologies.

Anyway, I think the proposal makes sense, though the liveness tests we did for
the branch-1 Linux HA on vSphere and Linux HA used an escalating set of probes.
# Pid signal -0
# RPC, HTTP ports open
# HTTP request resolving returning 200. (this is where returning an error
code on dfs safe mode would be invaluable, as it would avoid parsing at all)
# DFS ls operation. This is the one to verify that HDFS is really responding to
requests on the RPC channel.
# for downstream JT liveness: safe mode. state. {{setSafeMode()}} API method
has proven brittle across versions as constants for a nominally internal
operation were moved round. This is also why a safe mode HTTP page appeals to
me.

I suggest then
# some JSON/XML view of DSL health, with XML actually my preference.
# a {{hdfs-live.jspx}} page that returns an HTTP error code if DFS is unhappy.

The
[HappyAxis|http://svn.apache.org/repos/asf/webservices/axis/branches/explicitHeaderWork/java/webapps/axis/happyaxis.jsp]
page I wrote for Apache Axis did a lot more internal state check and
diagnostics of dependencies, env vars etc

Create REST interface for HDFS health data
--

Key: HADOOP-9562
URL: https://issues.apache.org/jira/browse/HADOOP-9562
Project: Hadoop Common
Issue Type: Improvement
Components: fs
Affects Versions: 2.0.4-alpha
Reporter: Trevor Lorimer
Priority: Minor

The HDFS health screen (dfshealth.jsp) displays basic Version, Security and
Health information concerning the NameNode, currently this information is
accessible from classes in the org.apache.hadoop,hdfs.server.namenode package
and cannot be accessed outside the NameNode. This becomes prevalent if the
data is required to be displayed using a new user interface.
The proposal is to create a REST interface to expose all the information
displayed on dfshealth.jsp using GET methods. Wrapper classes will be created
to serve the data to the REST root resource within the hadoop-hdfs project.
This will enable the HDFS health screen information to be accessed remotely.

[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing

2013-05-17 Thread Luke Lu (JIRA)

[
https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13660839#comment-13660839
]

Luke Lu commented on HADOOP-9421:
-

bq. I'm curious how RPC v9 testing is blocked by the SASL changes?

The goal of RPC v9 is to make sure every aspect of RPC is covered by a protobuf
consistently, which enables compatible evolution moving forward. We hope that
RPC v10 will not be necessary. The current patch needs cleanup but the scope
addresses this issue adequately AFAICT.

bq. The demo patch appears to propagate the current limited design which will
be very difficult to support in combination with the new design.

I don't see why it's difficult to support arbitrary new designs with the
protobuf change, which is fairly generic in terms of wireformat (sasl (request,
response) tuples that could contain anything fields). The new code can simply
branch out on the sasl rpc version introduced by the patch.

bq. I'm also not sure why the SASL protobuf should contain error messages
instead of leverage the existing error/fatal RPC response.

Because sasl rpc is (and should not be) not part of the rpc engine. It's
essentially a simple prelude to the real rpc exchange. Since we've decided that
sasl exchange will always end with a status from server, it's natural to report
error there. This keep the sasl rpc logic simple (only deals with sasl
(request, response)? protos) and separate from the main rpc logic, which seems
to be a nice separation of concerns here.

Convert SASL to use ProtoBuf and add lengths for non-blocking processing

[jira] [Commented] (HADOOP-9392) Token based authentication and Single Sign On

2013-05-17 Thread Kyle Leckie (JIRA)


[ 
https://issues.apache.org/jira/browse/HADOOP-9392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13660843#comment-13660843
 ] 

Kyle Leckie commented on HADOOP-9392:
-

Kai - Thanks for the proposal. A few questions:

1) Is it correct that the identity token is not restricted to a particular 
service and the same token is valid for all services that trust the token realm?

2) Is is correct that the identity token is utilized as a bearer token and not 
a shared secret?

3) You have repeatedly grouped LDAP and AD as the same form of authentication. 
Can you further clarify on the meaning of this grouping? 

 Token based authentication and Single Sign On
 -

 Key: HADOOP-9392
 URL: https://issues.apache.org/jira/browse/HADOOP-9392
 Project: Hadoop Common
  Issue Type: New Feature
  Components: security
Reporter: Kai Zheng
Assignee: Kai Zheng
 Fix For: 3.0.0

 Attachments: token-based-authn-plus-sso.pdf


 This is an umbrella entry for one of project Rhino’s topic, for details of 
 project Rhino, please refer to 
 https://github.com/intel-hadoop/project-rhino/. The major goal for this entry 
 as described in project Rhino was 
  
 “Core, HDFS, ZooKeeper, and HBase currently support Kerberos authentication 
 at the RPC layer, via SASL. However this does not provide valuable attributes 
 such as group membership, classification level, organizational identity, or 
 support for user defined attributes. Hadoop components must interrogate 
 external resources for discovering these attributes and at scale this is 
 problematic. There is also no consistent delegation model. HDFS has a simple 
 delegation capability, and only Oozie can take limited advantage of it. We 
 will implement a common token based authentication framework to decouple 
 internal user and service authentication from external mechanisms used to 
 support it (like Kerberos)”
  
 We’d like to start our work from Hadoop-Common and try to provide common 
 facilities by extending existing authentication framework which support:
 1.Pluggable token provider interface 
 2.Pluggable token verification protocol and interface
 3.Security mechanism to distribute secrets in cluster nodes
 4.Delegation model of user authentication

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HADOOP-9533) Centralized Hadoop SSO/Token Server

2013-05-17 Thread Kyle Leckie (JIRA)


[ 
https://issues.apache.org/jira/browse/HADOOP-9533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13660855#comment-13660855
 ] 

Kyle Leckie commented on HADOOP-9533:
-

Daryn
Great point on negotiation. One concern I have with SASL or other forms of 
mechanism negotiation is that the authenticity of the mechanism can be 
confirmed in order to avoid downgrade attacks. Would this assume TLS or some 
other mechanism?

 Centralized Hadoop SSO/Token Server
 ---

 Key: HADOOP-9533
 URL: https://issues.apache.org/jira/browse/HADOOP-9533
 Project: Hadoop Common
  Issue Type: New Feature
  Components: security
Reporter: Larry McCay
 Attachments: HSSO-Interaction-Overview-rev-1.docx, 
 HSSO-Interaction-Overview-rev-1.pdf


 This is an umbrella Jira filing to oversee a set of proposals for introducing 
 a new master service for Hadoop Single Sign On (HSSO).
 There is an increasing need for pluggable authentication providers that 
 authenticate both users and services as well as validate tokens in order to 
 federate identities authenticated by trusted IDPs. These IDPs may be deployed 
 within the enterprise or third-party IDPs that are external to the enterprise.
 These needs speak to a specific pain point: which is a narrow integration 
 path into the enterprise identity infrastructure. Kerberos is a fine solution 
 for those that already have it in place or are willing to adopt its use but 
 there remains a class of user that finds this unacceptable and needs to 
 integrate with a wider variety of identity management solutions.
 Another specific pain point is that of rolling and distributing keys. A 
 related and integral part of the HSSO server is library called the Credential 
 Management Framework (CMF), which will be a common library for easing the 
 management of secrets, keys and credentials.
 Initially, the existing delegation, block access and job tokens will continue 
 to be utilized. There may be some changes required to leverage a PKI based 
 signature facility rather than shared secrets. This is a means to simplify 
 the solution for the pain point of distributing shared secrets.
 This project will primarily centralize the responsibility of authentication 
 and federation into a single service that is trusted across the Hadoop 
 cluster and optionally across multiple clusters. This greatly simplifies a 
 number of things in the Hadoop ecosystem:
 1.a single token format that is used across all of Hadoop regardless of 
 authentication method
 2.a single service to have pluggable providers instead of all services
 3.a single token authority that would be trusted across the cluster/s and 
 through PKI encryption be able to easily issue cryptographically verifiable 
 tokens
 4.automatic rolling of the token authority’s keys and publishing of the 
 public key for easy access by those parties that need to verify incoming 
 tokens
 5.use of PKI for signatures eliminates the need for securely sharing and 
 distributing shared secrets
 In addition to serving as the internal Hadoop SSO service this service will 
 be leveraged by the Knox Gateway from the cluster perimeter in order to 
 acquire the Hadoop cluster tokens. The same token mechanism that is used for 
 internal services will be used to represent user identities. Providing for 
 interesting scenarios such as SSO across Hadoop clusters within an enterprise 
 and/or into the cloud.
 The HSSO service will be comprised of three major components and capabilities:
 1.Federating IDP – authenticates users/services and issues the common 
 Hadoop token
 2.Federating SP – validates the token of trusted external IDPs and issues 
 the common Hadoop token
 3.Token Authority – management of the common Hadoop tokens – including: 
 a.Issuance 
 b.Renewal
 c.Revocation
 As this is a meta Jira for tracking this overall effort, the details of the 
 individual efforts will be submitted along with the child Jira filings.
 Hadoop-Common would seem to be the most appropriate home for such a service 
 and its related common facilities. We will also leverage and extend existing 
 common mechanisms as appropriate.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HADOOP-9570) Configuration.addResource() should only parse the new resource

2013-05-17 Thread Gopal V (JIRA)

Gopal V created HADOOP-9570:
---

 Summary: Configuration.addResource() should only parse the new 
resource
 Key: HADOOP-9570
 URL: https://issues.apache.org/jira/browse/HADOOP-9570
 Project: Hadoop Common
  Issue Type: Bug
  Components: conf
 Environment: Ubuntu LXC
Reporter: Gopal V
Assignee: Gopal V
Priority: Minor


Hadoop configuration parsing re-parses all configuration files when a new 
resource is added to the configuration object.

This is wasteful and runs through every deprecation check unnecessarily.

{code}
JobConf clone = new JobConf(job);
clone.addResource(...);
{code}

will reparse every file including core-site, hdfs-site etc.

Resource addition can be taken care of cleanly, if the addResourceObject() call 
applies the new resource, followed by applying the overlay, without re-parsing 
any of the older files already loaded into the hashtable.


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HADOOP-9562) Create REST interface for HDFS health data


[ 
https://issues.apache.org/jira/browse/HADOOP-9562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13660935#comment-13660935
 ] 

Trevor Lorimer commented on HADOOP-9562:


Hi, just to clarify what I was proposing.
All I intend to do is simply serve up all the data currently displayed on the 
dfshealth jsp page, using only REST GETs, providing both XML and JSON 
responses. 
This stems from needing to present the data on a new UI from a different server.

This would be following the same rationale behind the REST APIs of Resource 
Manager, Node Manager, Application Manager and History Server 
http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html

An example REST GET from http://localhost:50070/hdfs/v1/header would return:

header: {
role: NameNode
state: active
hostAndPort: localhost:8020
securityEnabled: false
}

this would correspond to NameNode 'localhost:8020' (active) displayed on the 
dfshealth jsp page.



 Create REST interface for HDFS health data
 --

 Key: HADOOP-9562
 URL: https://issues.apache.org/jira/browse/HADOOP-9562
 Project: Hadoop Common
  Issue Type: Improvement
  Components: fs
Affects Versions: 2.0.4-alpha
Reporter: Trevor Lorimer
Priority: Minor

 The HDFS health screen (dfshealth.jsp) displays basic Version, Security and 
 Health information concerning the NameNode, currently this information is 
 accessible from classes in the org.apache.hadoop,hdfs.server.namenode package 
 and cannot be accessed outside the NameNode. This becomes prevalent if the 
 data is required to be displayed using a new user interface.
 The proposal is to create a REST interface to expose all the information 
 displayed on dfshealth.jsp using GET methods. Wrapper classes will be created 
 to serve the data to the REST root resource within the hadoop-hdfs project.
 This will enable the HDFS health screen information to be accessed remotely.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HADOOP-9517) Define Hadoop Compatibility

2013-05-17 Thread Karthik Kambatla (JIRA)

[
https://issues.apache.org/jira/browse/HADOOP-9517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13660939#comment-13660939
]

Karthik Kambatla commented on HADOOP-9517:
--

Thanks Steve. Will include all those items in the next version of the patch:
# Source code compatibility - when you say the patch will apply (patch -p0), I
suppose you are talking about the directory structure of the code and not that
the patch will apply cleanly. From a contributor's perspective, this limits any
directory changes include any code refactoring - e.g. move an inner class to a
separate class. In that case, do we guarantee compatibility within a minor
release or a major release.
# Configuration: We already have the section - I ll add a note on the default
values.
# Will add user-level file formats, web UI
# Will capture OS, JVM and Hardware under a new section Requirements.

Define Hadoop Compatibility
---

As we get ready to call hadoop-2 stable we need to better define 'Hadoop
Compatibility'.
http://wiki.apache.org/hadoop/Compatibility is a start, let's document
requirements clearly and completely.

[jira] [Commented] (HADOOP-9562) Create REST interface for HDFS health data

[
https://issues.apache.org/jira/browse/HADOOP-9562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13660955#comment-13660955
]

Steve Loughran commented on HADOOP-9562:

OK. I'd like safe mode in there, and the ability to generate a HTTP error if
any value doesn't match expected. I'm not sure {{header}} is the right name,
but that's a detail

{{hdfs/v1/status?format=jsonstate=active}} returns 200 the result listed,
but {{hdfs/v1/status?format=jsonstate=offline}} returns 500+ JSON.

This makes liveness/config probes from curl trivial -you can omit the JSON
parse stage

Create REST interface for HDFS health data
--

[jira] [Commented] (HADOOP-9517) Define Hadoop Compatibility

2013-05-17 Thread Doug Cutting (JIRA)

[
https://issues.apache.org/jira/browse/HADOOP-9517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13660956#comment-13660956
]

Doug Cutting commented on HADOOP-9517:
--

I vote to strengthen the compatibility requirements for user data file formats.
If we permit any user file-format changes that are not *forward* compatible at
all then they must be rare and very well marked as incompatibilities. It's
generally better to create a new format that programs must opt in to. An
unaltered program, when run against a new version, should ideally continue to
generate data that can still be read by older versions and other potential
implementations of the format. Otherwise we break workflows unless every
element of the flow is updated in lockstep. Rather we'd like to permit both
writers or readers of data files to be upgraded independently. Many folks have
multiple clusters that are not updated simultaneously and might move data files
between them.

Define Hadoop Compatibility
---

As we get ready to call hadoop-2 stable we need to better define 'Hadoop
Compatibility'.
http://wiki.apache.org/hadoop/Compatibility is a start, let's document
requirements clearly and completely.

[jira] [Commented] (HADOOP-9517) Define Hadoop Compatibility


[ 
https://issues.apache.org/jira/browse/HADOOP-9517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13660963#comment-13660963
 ] 

Steve Loughran commented on HADOOP-9517:


-Doug, good, though to guarantee it will add new unit tests (pull in old 
release binaries, verify they can still parse newly created artifacts).

I fear for the native binaries -we have to depend on libsnappy c to generate 
compressed content that older versions of their libs can handle. It's also much 
harder to pull in the old libs for regression testing, the way we can with JAR 
files on the mvn repo

 Define Hadoop Compatibility
 ---

 Key: HADOOP-9517
 URL: https://issues.apache.org/jira/browse/HADOOP-9517
 Project: Hadoop Common
  Issue Type: Bug
  Components: documentation
Reporter: Arun C Murthy
Assignee: Karthik Kambatla
 Attachments: hadoop-9517.patch, hadoop-9517.patch, hadoop-9517.patch, 
 hadoop-9517.patch


 As we get ready to call hadoop-2 stable we need to better define 'Hadoop 
 Compatibility'.
 http://wiki.apache.org/hadoop/Compatibility is a start, let's document 
 requirements clearly and completely.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HADOOP-9517) Define Hadoop Compatibility


[ 
https://issues.apache.org/jira/browse/HADOOP-9517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13660965#comment-13660965
 ] 

Steve Loughran commented on HADOOP-9517:


Karthik, -source tree compatibility

Yes, why I say {{patch}} I mean directory layout -we already have differences 
there between 1.x and 2.x; new changes are inevitable in future. Source code 
applicability can change and we must not make any promises there.

A policy here could be
minor versions:
# No planned changes in source tree layout, though it may happen
# Source files may be added, deleted and moved
# Source files may change so that patches no longer cleanly apply.

major versions: 
# all of the above
# a complete refactoring of the source tree, build and test toolings may happen.

 Define Hadoop Compatibility
 ---

 Key: HADOOP-9517
 URL: https://issues.apache.org/jira/browse/HADOOP-9517
 Project: Hadoop Common
  Issue Type: Bug
  Components: documentation
Reporter: Arun C Murthy
Assignee: Karthik Kambatla
 Attachments: hadoop-9517.patch, hadoop-9517.patch, hadoop-9517.patch, 
 hadoop-9517.patch


 As we get ready to call hadoop-2 stable we need to better define 'Hadoop 
 Compatibility'.
 http://wiki.apache.org/hadoop/Compatibility is a start, let's document 
 requirements clearly and completely.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HADOOP-9570) Configuration.addResource() should only parse the new resource

2013-05-17 Thread Gopal V (JIRA)


 [ 
https://issues.apache.org/jira/browse/HADOOP-9570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gopal V updated HADOOP-9570:


Attachment: HADOOP-9750.patch

Patch to add a new resource without reloading entire configuration.

 Configuration.addResource() should only parse the new resource
 --

 Key: HADOOP-9570
 URL: https://issues.apache.org/jira/browse/HADOOP-9570
 Project: Hadoop Common
  Issue Type: Bug
  Components: conf
 Environment: Ubuntu LXC
Reporter: Gopal V
Assignee: Gopal V
Priority: Minor
 Attachments: HADOOP-9750.patch


 Hadoop configuration parsing re-parses all configuration files when a new 
 resource is added to the configuration object.
 This is wasteful and runs through every deprecation check unnecessarily.
 {code}
 JobConf clone = new JobConf(job);
 clone.addResource(...);
 {code}
 will reparse every file including core-site, hdfs-site etc.
 Resource addition can be taken care of cleanly, if the addResourceObject() 
 call applies the new resource, followed by applying the overlay, without 
 re-parsing any of the older files already loaded into the hashtable.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HADOOP-9571) Enable multiple states to to be specified in Resource Manager apps REST call

Trevor Lorimer created HADOOP-9571:
--

 Summary: Enable multiple states to to be specified in Resource 
Manager apps REST call
 Key: HADOOP-9571
 URL: https://issues.apache.org/jira/browse/HADOOP-9571
 Project: Hadoop Common
  Issue Type: Improvement
  Components: util
Affects Versions: 2.0.4-alpha
Reporter: Trevor Lorimer
Priority: Trivial


Within the YARN Resource Manager REST API the GET call which returns all 
Applications can be filtered by a single State query parameter (http://rm http 
address:port/ws/v1/cluster/apps
). There are 8 possible states (New, Submitted, Accepted, Running, Finishing, 
Finished, Failed, Killed), if no state parameter is specified all states are 
returned, however if a sub-set of states is required then multiple REST calls 
are required (max. of 7).

The proposal is to be able to specify multiple states in a single REST call.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HADOOP-9571) Enable multiple states to to be specified in Resource Manager apps REST call

[
https://issues.apache.org/jira/browse/HADOOP-9571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Trevor Lorimer updated HADOOP-9571:
---

Description:
Within the YARN Resource Manager REST API the GET call which returns all
Applications can be filtered by a single State query parameter (http://rm http
address:port/ws/v1/cluster/apps). There are 8 possible states (New, Submitted,
Accepted, Running, Finishing, Finished, Failed, Killed), if no state parameter
is specified all states are returned, however if a sub-set of states is
required then multiple REST calls are required (max. of 7).

The proposal is to be able to specify multiple states in a single REST call.

was:
Within the YARN Resource Manager REST API the GET call which returns all
Applications can be filtered by a single State query parameter (http://rm http
address:port/ws/v1/cluster/apps
). There are 8 possible states (New, Submitted, Accepted, Running, Finishing,
Finished, Failed, Killed), if no state parameter is specified all states are
returned, however if a sub-set of states is required then multiple REST calls
are required (max. of 7).

The proposal is to be able to specify multiple states in a single REST call.

Enable multiple states to to be specified in Resource Manager apps REST call

Key: HADOOP-9571
URL: https://issues.apache.org/jira/browse/HADOOP-9571
Project: Hadoop Common
Issue Type: Improvement
Components: util
Affects Versions: 2.0.4-alpha
Reporter: Trevor Lorimer
Priority: Trivial

Within the YARN Resource Manager REST API the GET call which returns all
Applications can be filtered by a single State query parameter (http://rm
http address:port/ws/v1/cluster/apps). There are 8 possible states (New,
Submitted, Accepted, Running, Finishing, Finished, Failed, Killed), if no
state parameter is specified all states are returned, however if a sub-set of
states is required then multiple REST calls are required (max. of 7).
The proposal is to be able to specify multiple states in a single REST call.

[jira] [Updated] (HADOOP-9571) Enable multiple states to to be specified in Resource Manager apps REST call

[
https://issues.apache.org/jira/browse/HADOOP-9571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Trevor Lorimer updated HADOOP-9571:
---

Description:
Within the YARN Resource Manager REST API the GET call which returns all
Applications can be filtered by a single State query parameter (http://rm http
address:port/ws/v1/cluster/apps).

There are 8 possible states (New, Submitted, Accepted, Running, Finishing,
Finished, Failed, Killed), if no state parameter is specified all states are
returned, however if a sub-set of states is required then multiple REST calls
are required (max. of 7).

The proposal is to be able to specify multiple states in a single REST call.

was:
Within the YARN Resource Manager REST API the GET call which returns all
Applications can be filtered by a single State query parameter (http://rm http
address:port/ws/v1/cluster/apps). There are 8 possible states (New, Submitted,
Accepted, Running, Finishing, Finished, Failed, Killed), if no state parameter
is specified all states are returned, however if a sub-set of states is
required then multiple REST calls are required (max. of 7).

The proposal is to be able to specify multiple states in a single REST call.

Enable multiple states to to be specified in Resource Manager apps REST call

Within the YARN Resource Manager REST API the GET call which returns all
Applications can be filtered by a single State query parameter (http://rm
http address:port/ws/v1/cluster/apps).
There are 8 possible states (New, Submitted, Accepted, Running, Finishing,
Finished, Failed, Killed), if no state parameter is specified all states are
returned, however if a sub-set of states is required then multiple REST calls
are required (max. of 7).
The proposal is to be able to specify multiple states in a single REST call.

[jira] [Created] (HADOOP-9572) Enhance Pre-Commit Admin job to test-patch multiple branches

Giridharan Kesavan created HADOOP-9572:
--

 Summary: Enhance Pre-Commit Admin job to test-patch multiple 
branches
 Key: HADOOP-9572
 URL: https://issues.apache.org/jira/browse/HADOOP-9572
 Project: Hadoop Common
  Issue Type: Bug
Reporter: Giridharan Kesavan
Assignee: Giridharan Kesavan


Currently PreCommit-Admin job supports triggering PreCommit test jobs on trunk 
for a given project. This jira it to enhance the admin job to support running 
test-patch on any branches for a given project based on the uploaded patch 
name. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HADOOP-9573) Fix test-patch script to work with the enhanced PreCommit-Admin script.


 [ 
https://issues.apache.org/jira/browse/HADOOP-9573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Giridharan Kesavan updated HADOOP-9573:
---

Component/s: build

 Fix test-patch script to work with the enhanced PreCommit-Admin script.
 ---

 Key: HADOOP-9573
 URL: https://issues.apache.org/jira/browse/HADOOP-9573
 Project: Hadoop Common
  Issue Type: Sub-task
  Components: build
Reporter: Giridharan Kesavan
Assignee: Giridharan Kesavan

 test-patch script currently take the latest available patch for a given jira 
 and performs the test. This jira is to enhance test-patch script to take 
 attachment-id of a patch as an input and perform the tests using that 
 attachment-id

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HADOOP-9573) Fix test-patch script to work with the enhanced PreCommit-Admin script.

Giridharan Kesavan created HADOOP-9573:
--

 Summary: Fix test-patch script to work with the enhanced 
PreCommit-Admin script.
 Key: HADOOP-9573
 URL: https://issues.apache.org/jira/browse/HADOOP-9573
 Project: Hadoop Common
  Issue Type: Sub-task
Reporter: Giridharan Kesavan
Assignee: Giridharan Kesavan


test-patch script currently take the latest available patch for a given jira 
and performs the test. This jira is to enhance test-patch script to take 
attachment-id of a patch as an input and perform the tests using that 
attachment-id

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HADOOP-9573) Fix test-patch script to work with the enhanced PreCommit-Admin script.


 [ 
https://issues.apache.org/jira/browse/HADOOP-9573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Giridharan Kesavan updated HADOOP-9573:
---

Affects Version/s: 1.0.3

 Fix test-patch script to work with the enhanced PreCommit-Admin script.
 ---

 Key: HADOOP-9573
 URL: https://issues.apache.org/jira/browse/HADOOP-9573
 Project: Hadoop Common
  Issue Type: Sub-task
  Components: build
Affects Versions: 1.0.3
Reporter: Giridharan Kesavan
Assignee: Giridharan Kesavan

 test-patch script currently take the latest available patch for a given jira 
 and performs the test. This jira is to enhance test-patch script to take 
 attachment-id of a patch as an input and perform the tests using that 
 attachment-id

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HADOOP-9572) Enhance Pre-Commit Admin job to test-patch multiple branches

2013-05-17 Thread Aaron T. Myers (JIRA)


[ 
https://issues.apache.org/jira/browse/HADOOP-9572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13661054#comment-13661054
 ] 

Aaron T. Myers commented on HADOOP-9572:


Hi Giri, this seems like it might be a duplicate of HADOOP-7435. Do you agree?

 Enhance Pre-Commit Admin job to test-patch multiple branches
 

 Key: HADOOP-9572
 URL: https://issues.apache.org/jira/browse/HADOOP-9572
 Project: Hadoop Common
  Issue Type: Bug
Reporter: Giridharan Kesavan
Assignee: Giridharan Kesavan

 Currently PreCommit-Admin job supports triggering PreCommit test jobs on 
 trunk for a given project. This jira it to enhance the admin job to support 
 running test-patch on any branches for a given project based on the uploaded 
 patch name. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HADOOP-9574) Adding new methods in AbstractDelegationTokenSecretManager for restoring RMDelegationTokens on RMRestart

Jian He created HADOOP-9574:
---

 Summary: Adding new methods in 
AbstractDelegationTokenSecretManager for restoring RMDelegationTokens on 
RMRestart
 Key: HADOOP-9574
 URL: https://issues.apache.org/jira/browse/HADOOP-9574
 Project: Hadoop Common
  Issue Type: Bug
Reporter: Jian He


we're considering to add the following methods in 
AbstractDelegationTokenSecretManager for restoring RMDelegationTokens, these 
methods can also possibly be reused by hdfsDelegationTokenSecretManager, see 
YARN-638

  protected void storeNewMasterKey(DelegationKey key) throws IOException {
return;
  }
  protected void removeStoredMasterKey(DelegationKey key) {
return;
  }
  protected void storeNewToken(TokenIdent ident, long renewDate) {
return;
  }
  protected void removeStoredToken(TokenIdent ident) throws IOException {

  }
  protected void updateStoredToken(TokenIdent ident, long renewDate) {
return;
  }


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HADOOP-9575) CombineInputFormat isn't thread safe affecting HiveServer

2013-05-17 Thread Vinod Kumar Vavilapalli (JIRA)

Vinod Kumar Vavilapalli created HADOOP-9575:
---

 Summary: CombineInputFormat isn't thread safe affecting HiveServer
 Key: HADOOP-9575
 URL: https://issues.apache.org/jira/browse/HADOOP-9575
 Project: Hadoop Common
  Issue Type: Bug
Affects Versions: 1.2.0
Reporter: Vinod Kumar Vavilapalli
Assignee: Vinod Kumar Vavilapalli


This was originally fixed as part of MAPREDUCE-5038, but that got reverted now. 
Which uncovers this issue, breaking HiveServer. Originally reported by 
[~thejas].

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HADOOP-9574) Add new methods in AbstractDelegationTokenSecretManager for restoring RMDelegationTokens on RMRestart


 [ 
https://issues.apache.org/jira/browse/HADOOP-9574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jian He updated HADOOP-9574:


Summary: Add new methods in AbstractDelegationTokenSecretManager for 
restoring RMDelegationTokens on RMRestart  (was: Adding new methods in 
AbstractDelegationTokenSecretManager for restoring RMDelegationTokens on 
RMRestart)

 Add new methods in AbstractDelegationTokenSecretManager for restoring 
 RMDelegationTokens on RMRestart
 -

 Key: HADOOP-9574
 URL: https://issues.apache.org/jira/browse/HADOOP-9574
 Project: Hadoop Common
  Issue Type: Bug
Reporter: Jian He

 we're considering to add the following methods in 
 AbstractDelegationTokenSecretManager for restoring RMDelegationTokens, these 
 methods can also possibly be reused by hdfsDelegationTokenSecretManager, see 
 YARN-638
   protected void storeNewMasterKey(DelegationKey key) throws IOException {
 return;
   }
   protected void removeStoredMasterKey(DelegationKey key) {
 return;
   }
   protected void storeNewToken(TokenIdent ident, long renewDate) {
 return;
   }
   protected void removeStoredToken(TokenIdent ident) throws IOException {
   }
   protected void updateStoredToken(TokenIdent ident, long renewDate) {
 return;
   }

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HADOOP-8545) Filesystem Implementation for OpenStack Swift


 [ 
https://issues.apache.org/jira/browse/HADOOP-8545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Loughran updated HADOOP-8545:
---

Attachment: HADOOP-8545-027.patch

rebuilding patch and resubmitting

 Filesystem Implementation for OpenStack Swift
 -

 Key: HADOOP-8545
 URL: https://issues.apache.org/jira/browse/HADOOP-8545
 Project: Hadoop Common
  Issue Type: New Feature
  Components: fs
Affects Versions: 2.0.3-alpha, 1.1.2
Reporter: Tim Miller
Assignee: Dmitry Mezhensky
  Labels: hadoop, patch
 Attachments: HADOOP-8545-026.patch, HADOOP-8545-027.patch, 
 HADOOP-8545-10.patch, HADOOP-8545-11.patch, HADOOP-8545-12.patch, 
 HADOOP-8545-13.patch, HADOOP-8545-14.patch, HADOOP-8545-15.patch, 
 HADOOP-8545-16.patch, HADOOP-8545-17.patch, HADOOP-8545-18.patch, 
 HADOOP-8545-19.patch, HADOOP-8545-1.patch, HADOOP-8545-20.patch, 
 HADOOP-8545-21.patch, HADOOP-8545-22.patch, HADOOP-8545-23.patch, 
 HADOOP-8545-24.patch, HADOOP-8545-25.patch, HADOOP-8545-2.patch, 
 HADOOP-8545-3.patch, HADOOP-8545-4.patch, HADOOP-8545-5.patch, 
 HADOOP-8545-6.patch, HADOOP-8545-7.patch, HADOOP-8545-8.patch, 
 HADOOP-8545-9.patch, HADOOP-8545-javaclouds-2.patch, HADOOP-8545.patch, 
 HADOOP-8545.patch


 ,Add a filesystem implementation for OpenStack Swift object store, similar to 
 the one which exists today for S3.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HADOOP-8545) Filesystem Implementation for OpenStack Swift


 [ 
https://issues.apache.org/jira/browse/HADOOP-8545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Loughran updated HADOOP-8545:
---

Affects Version/s: (was: 1.1.2)
   1.2.1

 Filesystem Implementation for OpenStack Swift
 -

 Key: HADOOP-8545
 URL: https://issues.apache.org/jira/browse/HADOOP-8545
 Project: Hadoop Common
  Issue Type: New Feature
  Components: fs
Affects Versions: 2.0.3-alpha, 1.2.1
Reporter: Tim Miller
Assignee: Dmitry Mezhensky
  Labels: hadoop, patch
 Attachments: HADOOP-8545-026.patch, HADOOP-8545-027.patch, 
 HADOOP-8545-10.patch, HADOOP-8545-11.patch, HADOOP-8545-12.patch, 
 HADOOP-8545-13.patch, HADOOP-8545-14.patch, HADOOP-8545-15.patch, 
 HADOOP-8545-16.patch, HADOOP-8545-17.patch, HADOOP-8545-18.patch, 
 HADOOP-8545-19.patch, HADOOP-8545-1.patch, HADOOP-8545-20.patch, 
 HADOOP-8545-21.patch, HADOOP-8545-22.patch, HADOOP-8545-23.patch, 
 HADOOP-8545-24.patch, HADOOP-8545-25.patch, HADOOP-8545-2.patch, 
 HADOOP-8545-3.patch, HADOOP-8545-4.patch, HADOOP-8545-5.patch, 
 HADOOP-8545-6.patch, HADOOP-8545-7.patch, HADOOP-8545-8.patch, 
 HADOOP-8545-9.patch, HADOOP-8545-javaclouds-2.patch, HADOOP-8545.patch, 
 HADOOP-8545.patch


 ,Add a filesystem implementation for OpenStack Swift object store, similar to 
 the one which exists today for S3.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HADOOP-8545) Filesystem Implementation for OpenStack Swift


 [ 
https://issues.apache.org/jira/browse/HADOOP-8545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Loughran updated HADOOP-8545:
---

Affects Version/s: (was: 1.2.1)
   1.2.0

 Filesystem Implementation for OpenStack Swift
 -

 Key: HADOOP-8545
 URL: https://issues.apache.org/jira/browse/HADOOP-8545
 Project: Hadoop Common
  Issue Type: New Feature
  Components: fs
Affects Versions: 1.2.0, 2.0.3-alpha
Reporter: Tim Miller
Assignee: Dmitry Mezhensky
  Labels: hadoop, patch
 Attachments: HADOOP-8545-026.patch, HADOOP-8545-027.patch, 
 HADOOP-8545-10.patch, HADOOP-8545-11.patch, HADOOP-8545-12.patch, 
 HADOOP-8545-13.patch, HADOOP-8545-14.patch, HADOOP-8545-15.patch, 
 HADOOP-8545-16.patch, HADOOP-8545-17.patch, HADOOP-8545-18.patch, 
 HADOOP-8545-19.patch, HADOOP-8545-1.patch, HADOOP-8545-20.patch, 
 HADOOP-8545-21.patch, HADOOP-8545-22.patch, HADOOP-8545-23.patch, 
 HADOOP-8545-24.patch, HADOOP-8545-25.patch, HADOOP-8545-2.patch, 
 HADOOP-8545-3.patch, HADOOP-8545-4.patch, HADOOP-8545-5.patch, 
 HADOOP-8545-6.patch, HADOOP-8545-7.patch, HADOOP-8545-8.patch, 
 HADOOP-8545-9.patch, HADOOP-8545-javaclouds-2.patch, HADOOP-8545.patch, 
 HADOOP-8545.patch


 ,Add a filesystem implementation for OpenStack Swift object store, similar to 
 the one which exists today for S3.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HADOOP-9576) Make NetUtils.wrapException throw EOFException instead of wrapping it as IOException

Jian He created HADOOP-9576:
---

 Summary: Make NetUtils.wrapException throw EOFException instead of 
wrapping it as IOException
 Key: HADOOP-9576
 URL: https://issues.apache.org/jira/browse/HADOOP-9576
 Project: Hadoop Common
  Issue Type: Bug
Reporter: Jian He


In case of EOFException, NetUtils is now wrapping it as IOException, we may 
want to throw EOFException as it is, since EOFException can happen when 
connection is lost in the middle, the client may want to explicitly handle such 
exception

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HADOOP-9576) Make NetUtils.wrapException throw EOFException instead of wrapping it as IOException


 [ 
https://issues.apache.org/jira/browse/HADOOP-9576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jian He reassigned HADOOP-9576:
---

Assignee: Jian He

 Make NetUtils.wrapException throw EOFException instead of wrapping it as 
 IOException
 

 Key: HADOOP-9576
 URL: https://issues.apache.org/jira/browse/HADOOP-9576
 Project: Hadoop Common
  Issue Type: Bug
Reporter: Jian He
Assignee: Jian He

 In case of EOFException, NetUtils is now wrapping it as IOException, we may 
 want to throw EOFException as it is, since EOFException can happen when 
 connection is lost in the middle, the client may want to explicitly handle 
 such exception

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HADOOP-9574) Add new methods in AbstractDelegationTokenSecretManager for restoring RMDelegationTokens on RMRestart


 [ 
https://issues.apache.org/jira/browse/HADOOP-9574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jian He reassigned HADOOP-9574:
---

Assignee: Jian He

 Add new methods in AbstractDelegationTokenSecretManager for restoring 
 RMDelegationTokens on RMRestart
 -

 Key: HADOOP-9574
 URL: https://issues.apache.org/jira/browse/HADOOP-9574
 Project: Hadoop Common
  Issue Type: Bug
Reporter: Jian He
Assignee: Jian He

 we're considering to add the following methods in 
 AbstractDelegationTokenSecretManager for restoring RMDelegationTokens, these 
 methods can also possibly be reused by hdfsDelegationTokenSecretManager, see 
 YARN-638
   protected void storeNewMasterKey(DelegationKey key) throws IOException {
 return;
   }
   protected void removeStoredMasterKey(DelegationKey key) {
 return;
   }
   protected void storeNewToken(TokenIdent ident, long renewDate) {
 return;
   }
   protected void removeStoredToken(TokenIdent ident) throws IOException {
   }
   protected void updateStoredToken(TokenIdent ident, long renewDate) {
 return;
   }

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HADOOP-9574) Add new methods in AbstractDelegationTokenSecretManager for restoring RMDelegationTokens on RMRestart


 [ 
https://issues.apache.org/jira/browse/HADOOP-9574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jian He updated HADOOP-9574:


Description: 
we're considering to add the following methods in 
AbstractDelegationTokenSecretManager for restoring RMDelegationTokens, these 
methods can also possibly be reused by hdfsDelegationTokenSecretManager, see 
YARN-638

  protected void storeNewMasterKey(DelegationKey key) throws IOException {
return;
  }
  protected void removeStoredMasterKey(DelegationKey key) {
return;
  }
  protected void storeNewToken(TokenIdent ident, long renewDate) {
return;
  }
  protected void removeStoredToken(TokenIdent ident) throws IOException {

  }
  protected void updateStoredToken(TokenIdent ident, long renewDate) {
return;
  }

Also move addPersistedDelegationToken in hdfs.DelegationTokenSecretManager, to 
AbstractDelegationTokenSecretManager


  was:
we're considering to add the following methods in 
AbstractDelegationTokenSecretManager for restoring RMDelegationTokens, these 
methods can also possibly be reused by hdfsDelegationTokenSecretManager, see 
YARN-638

  protected void storeNewMasterKey(DelegationKey key) throws IOException {
return;
  }
  protected void removeStoredMasterKey(DelegationKey key) {
return;
  }
  protected void storeNewToken(TokenIdent ident, long renewDate) {
return;
  }
  protected void removeStoredToken(TokenIdent ident) throws IOException {

  }
  protected void updateStoredToken(TokenIdent ident, long renewDate) {
return;
  }



 Add new methods in AbstractDelegationTokenSecretManager for restoring 
 RMDelegationTokens on RMRestart
 -

 Key: HADOOP-9574
 URL: https://issues.apache.org/jira/browse/HADOOP-9574
 Project: Hadoop Common
  Issue Type: Bug
Reporter: Jian He
Assignee: Jian He

 we're considering to add the following methods in 
 AbstractDelegationTokenSecretManager for restoring RMDelegationTokens, these 
 methods can also possibly be reused by hdfsDelegationTokenSecretManager, see 
 YARN-638
   protected void storeNewMasterKey(DelegationKey key) throws IOException {
 return;
   }
   protected void removeStoredMasterKey(DelegationKey key) {
 return;
   }
   protected void storeNewToken(TokenIdent ident, long renewDate) {
 return;
   }
   protected void removeStoredToken(TokenIdent ident) throws IOException {
   }
   protected void updateStoredToken(TokenIdent ident, long renewDate) {
 return;
   }
 Also move addPersistedDelegationToken in hdfs.DelegationTokenSecretManager, 
 to AbstractDelegationTokenSecretManager

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HADOOP-9454) Support multipart uploads for s3native


 [ 
https://issues.apache.org/jira/browse/HADOOP-9454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jordan Mendelson updated HADOOP-9454:
-

Attachment: HADOOP-9454-5.patch

Move away from deprecated methods to reduce warnings.

 Support multipart uploads for s3native
 --

 Key: HADOOP-9454
 URL: https://issues.apache.org/jira/browse/HADOOP-9454
 Project: Hadoop Common
  Issue Type: Improvement
  Components: fs/s3
Reporter: Jordan Mendelson
 Attachments: HADOOP-9454-5.patch


 The s3native filesystem is limited to 5 GB file uploads to S3, however the 
 newest version of jets3t supports multipart uploads to allow storing multi-TB 
 files. While the s3 filesystem lets you bypass this restriction by uploading 
 blocks, it is necessary for us to output our data into Amazon's 
 publicdatasets bucket which is shared with others.
 Amazon has added a similar feature to their distribution of hadoop as has 
 MapR.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HADOOP-9454) Support multipart uploads for s3native


 [ 
https://issues.apache.org/jira/browse/HADOOP-9454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jordan Mendelson updated HADOOP-9454:
-

Attachment: (was: HADOOP-9454-4.patch)

 Support multipart uploads for s3native
 --

 Key: HADOOP-9454
 URL: https://issues.apache.org/jira/browse/HADOOP-9454
 Project: Hadoop Common
  Issue Type: Improvement
  Components: fs/s3
Reporter: Jordan Mendelson
 Attachments: HADOOP-9454-5.patch


 The s3native filesystem is limited to 5 GB file uploads to S3, however the 
 newest version of jets3t supports multipart uploads to allow storing multi-TB 
 files. While the s3 filesystem lets you bypass this restriction by uploading 
 blocks, it is necessary for us to output our data into Amazon's 
 publicdatasets bucket which is shared with others.
 Amazon has added a similar feature to their distribution of hadoop as has 
 MapR.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HADOOP-8545) Filesystem Implementation for OpenStack Swift


[ 
https://issues.apache.org/jira/browse/HADOOP-8545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13661158#comment-13661158
 ] 

Hadoop QA commented on HADOOP-8545:
---

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12583703/HADOOP-8545-027.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 29 new 
or modified test files.

  {color:red}-1 javac{color}.  The applied patch generated 1379 javac 
compiler warnings (more than the trunk's current 1367 warnings).

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:red}-1 findbugs{color}.  The patch appears to introduce 4 new 
Findbugs (version 1.3.9) warnings.

{color:red}-1 release audit{color}.  The applied patch generated 2 
release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in 
hadoop-common-project/hadoop-common hadoop-tools/hadoop-openstack 
hadoop-tools/hadoop-tools-dist.

{color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-HADOOP-Build/2546//testReport/
Release audit warnings: 
https://builds.apache.org/job/PreCommit-HADOOP-Build/2546//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HADOOP-Build/2546//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-openstack.html
Javac warnings: 
https://builds.apache.org/job/PreCommit-HADOOP-Build/2546//artifact/trunk/patchprocess/diffJavacWarnings.txt
Console output: 
https://builds.apache.org/job/PreCommit-HADOOP-Build/2546//console

This message is automatically generated.

 Filesystem Implementation for OpenStack Swift
 -

 Key: HADOOP-8545
 URL: https://issues.apache.org/jira/browse/HADOOP-8545
 Project: Hadoop Common
  Issue Type: New Feature
  Components: fs
Affects Versions: 1.2.0, 2.0.3-alpha
Reporter: Tim Miller
Assignee: Dmitry Mezhensky
  Labels: hadoop, patch
 Attachments: HADOOP-8545-026.patch, HADOOP-8545-027.patch, 
 HADOOP-8545-10.patch, HADOOP-8545-11.patch, HADOOP-8545-12.patch, 
 HADOOP-8545-13.patch, HADOOP-8545-14.patch, HADOOP-8545-15.patch, 
 HADOOP-8545-16.patch, HADOOP-8545-17.patch, HADOOP-8545-18.patch, 
 HADOOP-8545-19.patch, HADOOP-8545-1.patch, HADOOP-8545-20.patch, 
 HADOOP-8545-21.patch, HADOOP-8545-22.patch, HADOOP-8545-23.patch, 
 HADOOP-8545-24.patch, HADOOP-8545-25.patch, HADOOP-8545-2.patch, 
 HADOOP-8545-3.patch, HADOOP-8545-4.patch, HADOOP-8545-5.patch, 
 HADOOP-8545-6.patch, HADOOP-8545-7.patch, HADOOP-8545-8.patch, 
 HADOOP-8545-9.patch, HADOOP-8545-javaclouds-2.patch, HADOOP-8545.patch, 
 HADOOP-8545.patch


 ,Add a filesystem implementation for OpenStack Swift object store, similar to 
 the one which exists today for S3.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HADOOP-9454) Support multipart uploads for s3native


 [ 
https://issues.apache.org/jira/browse/HADOOP-9454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jordan Mendelson updated HADOOP-9454:
-

Attachment: HADOOP-9454-7.patch

Woops, include all the new files.

 Support multipart uploads for s3native
 --

 Key: HADOOP-9454
 URL: https://issues.apache.org/jira/browse/HADOOP-9454
 Project: Hadoop Common
  Issue Type: Improvement
  Components: fs/s3
Reporter: Jordan Mendelson
 Attachments: HADOOP-9454-7.patch


 The s3native filesystem is limited to 5 GB file uploads to S3, however the 
 newest version of jets3t supports multipart uploads to allow storing multi-TB 
 files. While the s3 filesystem lets you bypass this restriction by uploading 
 blocks, it is necessary for us to output our data into Amazon's 
 publicdatasets bucket which is shared with others.
 Amazon has added a similar feature to their distribution of hadoop as has 
 MapR.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HADOOP-9454) Support multipart uploads for s3native


 [ 
https://issues.apache.org/jira/browse/HADOOP-9454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jordan Mendelson updated HADOOP-9454:
-

Attachment: (was: HADOOP-9454-5.patch)

 Support multipart uploads for s3native
 --

 Key: HADOOP-9454
 URL: https://issues.apache.org/jira/browse/HADOOP-9454
 Project: Hadoop Common
  Issue Type: Improvement
  Components: fs/s3
Reporter: Jordan Mendelson
 Attachments: HADOOP-9454-7.patch


 The s3native filesystem is limited to 5 GB file uploads to S3, however the 
 newest version of jets3t supports multipart uploads to allow storing multi-TB 
 files. While the s3 filesystem lets you bypass this restriction by uploading 
 blocks, it is necessary for us to output our data into Amazon's 
 publicdatasets bucket which is shared with others.
 Amazon has added a similar feature to their distribution of hadoop as has 
 MapR.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HADOOP-9454) Support multipart uploads for s3native


 [ 
https://issues.apache.org/jira/browse/HADOOP-9454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jordan Mendelson updated HADOOP-9454:
-

Attachment: (was: HADOOP-9454-7.patch)

 Support multipart uploads for s3native
 --

 Key: HADOOP-9454
 URL: https://issues.apache.org/jira/browse/HADOOP-9454
 Project: Hadoop Common
  Issue Type: Improvement
  Components: fs/s3
Reporter: Jordan Mendelson

 The s3native filesystem is limited to 5 GB file uploads to S3, however the 
 newest version of jets3t supports multipart uploads to allow storing multi-TB 
 files. While the s3 filesystem lets you bypass this restriction by uploading 
 blocks, it is necessary for us to output our data into Amazon's 
 publicdatasets bucket which is shared with others.
 Amazon has added a similar feature to their distribution of hadoop as has 
 MapR.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HADOOP-9454) Support multipart uploads for s3native


[ 
https://issues.apache.org/jira/browse/HADOOP-9454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13661179#comment-13661179
 ] 

Hadoop QA commented on HADOOP-9454:
---

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12583707/HADOOP-9454-5.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 1 new 
or modified test files.

  {color:red}-1 javac{color}.  The applied patch generated 1380 javac 
compiler warnings (more than the trunk's current 1367 warnings).

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 1.3.9) warnings.

{color:red}-1 release audit{color}.  The applied patch generated 1 
release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in 
hadoop-common-project/hadoop-common.

{color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-HADOOP-Build/2547//testReport/
Release audit warnings: 
https://builds.apache.org/job/PreCommit-HADOOP-Build/2547//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
Javac warnings: 
https://builds.apache.org/job/PreCommit-HADOOP-Build/2547//artifact/trunk/patchprocess/diffJavacWarnings.txt
Console output: 
https://builds.apache.org/job/PreCommit-HADOOP-Build/2547//console

This message is automatically generated.

 Support multipart uploads for s3native
 --

 Key: HADOOP-9454
 URL: https://issues.apache.org/jira/browse/HADOOP-9454
 Project: Hadoop Common
  Issue Type: Improvement
  Components: fs/s3
Reporter: Jordan Mendelson

 The s3native filesystem is limited to 5 GB file uploads to S3, however the 
 newest version of jets3t supports multipart uploads to allow storing multi-TB 
 files. While the s3 filesystem lets you bypass this restriction by uploading 
 blocks, it is necessary for us to output our data into Amazon's 
 publicdatasets bucket which is shared with others.
 Amazon has added a similar feature to their distribution of hadoop as has 
 MapR.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HADOOP-9454) Support multipart uploads for s3native


 [ 
https://issues.apache.org/jira/browse/HADOOP-9454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jordan Mendelson updated HADOOP-9454:
-

Attachment: HADOOP-9454-8.patch

 Support multipart uploads for s3native
 --

 Key: HADOOP-9454
 URL: https://issues.apache.org/jira/browse/HADOOP-9454
 Project: Hadoop Common
  Issue Type: Improvement
  Components: fs/s3
Reporter: Jordan Mendelson
 Attachments: HADOOP-9454-8.patch


 The s3native filesystem is limited to 5 GB file uploads to S3, however the 
 newest version of jets3t supports multipart uploads to allow storing multi-TB 
 files. While the s3 filesystem lets you bypass this restriction by uploading 
 blocks, it is necessary for us to output our data into Amazon's 
 publicdatasets bucket which is shared with others.
 Amazon has added a similar feature to their distribution of hadoop as has 
 MapR.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HADOOP-9454) Support multipart uploads for s3native


 [ 
https://issues.apache.org/jira/browse/HADOOP-9454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jordan Mendelson updated HADOOP-9454:
-

Attachment: (was: HADOOP-9454-8.patch)

 Support multipart uploads for s3native
 --

 Key: HADOOP-9454
 URL: https://issues.apache.org/jira/browse/HADOOP-9454
 Project: Hadoop Common
  Issue Type: Improvement
  Components: fs/s3
Reporter: Jordan Mendelson
 Attachments: HADOOP-9454-9.patch


 The s3native filesystem is limited to 5 GB file uploads to S3, however the 
 newest version of jets3t supports multipart uploads to allow storing multi-TB 
 files. While the s3 filesystem lets you bypass this restriction by uploading 
 blocks, it is necessary for us to output our data into Amazon's 
 publicdatasets bucket which is shared with others.
 Amazon has added a similar feature to their distribution of hadoop as has 
 MapR.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HADOOP-9454) Support multipart uploads for s3native


 [ 
https://issues.apache.org/jira/browse/HADOOP-9454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jordan Mendelson updated HADOOP-9454:
-

Attachment: HADOOP-9454-9.patch

 Support multipart uploads for s3native
 --

 Key: HADOOP-9454
 URL: https://issues.apache.org/jira/browse/HADOOP-9454
 Project: Hadoop Common
  Issue Type: Improvement
  Components: fs/s3
Reporter: Jordan Mendelson
 Attachments: HADOOP-9454-9.patch


 The s3native filesystem is limited to 5 GB file uploads to S3, however the 
 newest version of jets3t supports multipart uploads to allow storing multi-TB 
 files. While the s3 filesystem lets you bypass this restriction by uploading 
 blocks, it is necessary for us to output our data into Amazon's 
 publicdatasets bucket which is shared with others.
 Amazon has added a similar feature to their distribution of hadoop as has 
 MapR.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HADOOP-9440) Unit Test: hadoop-common2.0.3 TestIPC fails on protobuf2.5.0


[ 
https://issues.apache.org/jira/browse/HADOOP-9440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13661188#comment-13661188
 ] 

Hadoop QA commented on HADOOP-9440:
---

{color:green}+1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12575838/HADOOP-9440.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 1 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in 
hadoop-common-project/hadoop-common.

{color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-HADOOP-Build/2548//testReport/
Console output: 
https://builds.apache.org/job/PreCommit-HADOOP-Build/2548//console

This message is automatically generated.

 Unit Test: hadoop-common2.0.3 TestIPC fails on protobuf2.5.0
 

 Key: HADOOP-9440
 URL: https://issues.apache.org/jira/browse/HADOOP-9440
 Project: Hadoop Common
  Issue Type: Bug
  Components: test
Affects Versions: 2.0.5-beta
Reporter: Tian Hong Wang
  Labels: patch
 Attachments: HADOOP-9440.patch


 TestIPC runs normally if use protobuf2.4.1 or below version. But if using 
 protobuf2.5.0, TestIPC.testIpcTimeout   TestIPC.testIpcConnectTimeout will 
 fail.
 java.io.IOException: Failed on local exception: 
 com.google.protobuf.InvalidProtocolBufferException: 500 millis timeout while 
 waiting for channel to be ready for read. ch : 
 java.nio.channels.SocketChannel[connected local=/127.0.0.1:50850 
 remote=louis-ThinkPad-T410/127.0.0.1:50353]; Host Details : local host is: 
 louis-ThinkPad-T410/127.0.0.1; destination host is: 
 louis-ThinkPad-T410:50353; 
   at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:761)
   at org.apache.hadoop.ipc.Client.call(Client.java:1239)
   at org.apache.hadoop.ipc.Client.call(Client.java:1163)
   at org.apache.hadoop.ipc.TestIPC.testIpcTimeout(TestIPC.java:492)
 testIpcConnectTimeout(org.apache.hadoop.ipc.TestIPC)  Time elapsed: 2009 sec  
  ERROR!
 java.io.IOException: Failed on local exception: 
 com.google.protobuf.InvalidProtocolBufferException: 2000 millis timeout while 
 waiting for channel to be ready for read. ch : 
 java.nio.channels.SocketChannel[connected local=/127.0.0.1:51304 
 remote=louis-ThinkPad-T410/127.0.0.1:39525]; Host Details : local host is: 
 louis-ThinkPad-T410/127.0.0.1; destination host is: 
 louis-ThinkPad-T410:39525; 
   at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:761)
   at org.apache.hadoop.ipc.Client.call(Client.java:1239)
   at org.apache.hadoop.ipc.Client.call(Client.java:1163)
   at org.apache.hadoop.ipc.TestIPC.testIpcConnectTimeout(TestIPC.java:515)
 TestIPC.testIpcTimeout   TestIPC.testIpcConnectTimeout fails because it 
 catches the  com.google.protobuf.InvalidProtocolBufferException not 
 SocketTimeoutException.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HADOOP-9454) Support multipart uploads for s3native


[ 
https://issues.apache.org/jira/browse/HADOOP-9454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13661197#comment-13661197
 ] 

Hadoop QA commented on HADOOP-9454:
---

{color:green}+1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12583713/HADOOP-9454-9.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 2 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in 
hadoop-common-project/hadoop-common.

{color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-HADOOP-Build/2549//testReport/
Console output: 
https://builds.apache.org/job/PreCommit-HADOOP-Build/2549//console

This message is automatically generated.

 Support multipart uploads for s3native
 --

 Key: HADOOP-9454
 URL: https://issues.apache.org/jira/browse/HADOOP-9454
 Project: Hadoop Common
  Issue Type: Improvement
  Components: fs/s3
Reporter: Jordan Mendelson
 Attachments: HADOOP-9454-9.patch


 The s3native filesystem is limited to 5 GB file uploads to S3, however the 
 newest version of jets3t supports multipart uploads to allow storing multi-TB 
 files. While the s3 filesystem lets you bypass this restriction by uploading 
 blocks, it is necessary for us to output our data into Amazon's 
 publicdatasets bucket which is shared with others.
 Amazon has added a similar feature to their distribution of hadoop as has 
 MapR.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HADOOP-9527) TestLocalFSFileContextSymlink is broken on Windows

2013-05-17 Thread Ivan Mitic (JIRA)

[
https://issues.apache.org/jira/browse/HADOOP-9527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13661239#comment-13661239
]

Ivan Mitic commented on HADOOP-9527:

Thanks Arpit for the patch and the patience! Sorry for not reviewing this
earlier, I was not near my computer for a week.

1. Interestingly, you are going the similar route I initially took in
branch-1-win :) (see HADOOP-8872) Unfortunately, I don't think this will work
in all cases. See my comments from HADOOP-9061 for the reasons why. Quoting the
branch-1-win approach below:
{quote}
- If on Java6, do file copy instead of symlink. We've seen many problems with
symlinks on files on Java6 and it is simply impossible to make all of them
work. Recent examples include apps (like Oozie) built on top of MR that
directly access symlinks using the Java File object (not thru RLFS), in which
case things just don’t work.
- If on Java7, symlinks work as expected.
{quote}
It took a few iterations to get symlink behavior to work properly across the
Hadoop ecosystem with JDK6 on Windows, hence I’d stick with the approach.

The downside would be that the readlink functionality would not work with the
LFS on JDK6. Not sure if there are legit Hadoop scenarios that rely on this
functionality, (I don’t know of any other than in tests). If not, I would be
fine with just throwing the NOT_IMPL exception if readLink is invoked on JDK6
on Windows. Things are moving toward JDK7, so I don’t see this a big gap.

I am interested in hearing your thoughts on this direction.

2. FileUtil#readLink: My preference would be to expose a single {{File
readLink(File)}} public API instead of FileUtil APIs that accept Paths or
Strings. This makes the contract explicit such that the input path must be a
valid cross-platform local file system path.

3. RawLocalFs#createSymlink: IMO this method should delegate to
FileUtil#symlink(). Currently we have two inconsistent implementation for
symlinks in FileUtil and RawLocalFs.

TestLocalFSFileContextSymlink is broken on Windows
--

Key: HADOOP-9527
URL: https://issues.apache.org/jira/browse/HADOOP-9527
Project: Hadoop Common
Issue Type: Bug
Components: test
Affects Versions: 3.0.0
Reporter: Arpit Agarwal
Assignee: Arpit Agarwal
Fix For: 3.0.0

Attachments: HADOOP-9527.001.patch, HADOOP-9527.002.patch,
HADOOP-9527.003.patch, HADOOP-9527.004.patch, HADOOP-9527.005.patch,
HADOOP-9527.006.patch, HADOOP-9527.007.patch, RenameLink.java

Multiple test cases are broken. I didn't look at each failure in detail.
The main cause of the failures appears to be that RawLocalFS.readLink() does
not work on Windows. We need winutils readlink to fix the test.

[jira] [Commented] (HADOOP-9576) Make NetUtils.wrapException throw EOFException instead of wrapping it as IOException