GitHub user chesterxgchen opened a pull request:
https://github.com/apache/spark/pull/2204
SPARK-3177 (on Master Branch)
The JIRA and PR was original created for branch-1.1, and move to master
branch now.
Chester
The Issue is due to that yarn-alpha and yarn have different APIs for
certain class fields. In this particular case, the ClientBase using reflection
to to address this issue, and we need to different way to test the ClientBase's
method. Original ClientBaseSuite using getFieldValue() method to do this. But
it doesn't work for yarn-alpha as the API returns an array of String instead of
just String (which is the case for Yarn-stable API).
To fix the test, I add a new method
def getFieldValue2[A: ClassTag, A1: ClassTag, B](clazz: Class[_], field:
String,
defaults: => B)
(mapTo: A => B)(mapTo1: A1 => B) : B =
Try(clazz.getField(field)).map(_.get(null)).map {
case v: A => mapTo(v)
case v1: A1 => mapTo1(v1)
case _ => defaults
}.toOption.getOrElse(defaults)
to handle the cases where the field type can be either type A or A1. In
this new method the type A or A1 is pattern matched and corresponding mapTo
function (mapTo or mapTo1) is used.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/AlpineNow/spark SPARK-3177-master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/2204.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2204
----
commit 77be5d0640f8a61e525e6ef8b59fc202b14e7798
Author: chesterxgchen <[email protected]>
Date: 2014-08-29T18:48:51Z
The Issue is due to that yarn-alpha and yarn have different APIs for
certain class fields. In this particular case, the ClientBase using reflection
to to address this issue, and we need to different way to test the ClientBase's
method. Original ClientBaseSuite using getFieldValue() method to do this. But
it doesn't work for yarn-alpha as the API returns an array of String instead of
just String (which is the case for Yarn-stable API).
To fix the test, I add a new method
def getFieldValue2[A: ClassTag, A1: ClassTag, B](clazz: Class[_], field:
String,
defaults: => B)
(mapTo: A => B)(mapTo1: A1 => B) : B =
Try(clazz.getField(field)).map(_.get(null)).map {
case v: A => mapTo(v)
case v1: A1 => mapTo1(v1)
case _ => defaults
}.toOption.getOrElse(defaults)
to handle the cases where the field type can be either type A or A1. In
this new method the type A or A1 is pattern matched and corresponding mapTo
function (mapTo or mapTo1) is used.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
