GitHub user shenh062326 opened a pull request:
https://github.com/apache/spark/pull/4150
[SPARK-5347][CORE] Change FileSplit to InputSplit in update inputMetrics
When inputFormatClass is set to CombineFileInputFormat, input metrics show
that input is empty. It don't appear is spark-1.1.0. It's because in HadoopRDD,
inputMetrics only been set when split is instanceOf FileSplit, but
CombineFileInputFormat use InputSplit. It's not nessesary to instanceOf
FileSplit, only have to instanceOf InputSplit.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/shenh062326/spark my_change1
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/4150.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4150
----
commit 9e04a547115bbcb4c19b55b451ca3afe09955e9f
Author: Hong Shen <[email protected]>
Date: 2015-01-22T00:51:15Z
change FileSplit to InputSplit in update inputMetrics
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
