[GitHub] spark issue #22232: [SPARK-25237][SQL]remove updateBytesReadWithFileSize bec...

2018-09-03 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22232 I think that's fine @maropu ; we can always apportion credit appropriately later. --- - To unsubscribe, e-mail:

[GitHub] spark issue #22232: [SPARK-25237][SQL]remove updateBytesReadWithFileSize bec...

2018-09-03 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22232 @srowen I could take this over or do follow-up if the author is still inactive. --- - To unsubscribe, e-mail:

[GitHub] spark issue #22232: [SPARK-25237][SQL]remove updateBytesReadWithFileSize bec...

2018-09-03 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22232 @dujunling I personally think this can be merged, but only if the test is tightened up --- - To unsubscribe, e-mail:

[GitHub] spark issue #22232: [SPARK-25237][SQL]remove updateBytesReadWithFileSize bec...

2018-08-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22232 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95508/ Test PASSed. ---

[GitHub] spark issue #22232: [SPARK-25237][SQL]remove updateBytesReadWithFileSize bec...

2018-08-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22232 **[Test build #95508 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95508/testReport)** for PR 22232 at commit

[GitHub] spark issue #22232: [SPARK-25237][SQL]remove updateBytesReadWithFileSize bec...

2018-08-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22232 **[Test build #95508 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95508/testReport)** for PR 22232 at commit

[GitHub] spark issue #22232: [SPARK-25237][SQL]remove updateBytesReadWithFileSize bec...

2018-08-30 Thread wzhfy
Github user wzhfy commented on the issue: https://github.com/apache/spark/pull/22232 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22232: [SPARK-25237][SQL]remove updateBytesReadWithFileSize bec...

2018-08-26 Thread dujunling
Github user dujunling commented on the issue: https://github.com/apache/spark/pull/22232 @maropu I have added a ut to check the inputMetrics --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #22232: [SPARK-25237][SQL]remove updateBytesReadWithFileSize bec...

2018-08-25 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22232 It's OK to assume Hadoop 2.6+ only. In fact 2.6 is quite old anyway. --- - To unsubscribe, e-mail:

[GitHub] spark issue #22232: [SPARK-25237][SQL]remove updateBytesReadWithFileSize bec...

2018-08-25 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22232 I'm not sure we can test the case though, for example, how about the sequence below? ``` import org.apache.spark.TaskContext spark.range(10).selectExpr("id AS c0", "rand() AS

[GitHub] spark issue #22232: [SPARK-25237][SQL]remove updateBytesReadWithFileSize bec...

2018-08-25 Thread dujunling
Github user dujunling commented on the issue: https://github.com/apache/spark/pull/22232 While metris suites are in core test , fileScanRdd should be in sql test, it is difficult to add tests to check the input metrics in sql module ---

[GitHub] spark issue #22232: [SPARK-25237][SQL]remove updateBytesReadWithFileSize bec...

2018-08-25 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22232 btw, can you clean up the title and the description..? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #22232: [SPARK-25237][SQL]remove updateBytesReadWithFileSize bec...

2018-08-25 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22232 Is it difficult to add tests for checking the metric in the case `select * from t limit 1`? --- - To unsubscribe, e-mail:

[GitHub] spark issue #22232: [SPARK-25237][SQL]remove updateBytesReadWithFileSize bec...

2018-08-25 Thread wzhfy
Github user wzhfy commented on the issue: https://github.com/apache/spark/pull/22232 test this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22232: [SPARK-25237][SQL]remove updateBytesReadWithFileSize bec...

2018-08-25 Thread wzhfy
Github user wzhfy commented on the issue: https://github.com/apache/spark/pull/22232 this seems to be caused by removing support for Hadoop 2.5 and earlier? cc original authors @cloud-fan @srowen to make sure --- -

[GitHub] spark issue #22232: [SPARK-25237][SQL]remove updateBytesReadWithFileSize bec...

2018-08-25 Thread wzhfy
Github user wzhfy commented on the issue: https://github.com/apache/spark/pull/22232 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22232: [SPARK-25237][SQL]remove updateBytesReadWithFileSize bec...

2018-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22232 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22232: [SPARK-25237][SQL]remove updateBytesReadWithFileSize bec...

2018-08-25 Thread dujunling
Github user dujunling commented on the issue: https://github.com/apache/spark/pull/22232 @wzhfy --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22232: [SPARK-25237][SQL]remove updateBytesReadWithFileSize bec...

2018-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22232 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22232: [SPARK-25237][SQL]remove updateBytesReadWithFileSize bec...

2018-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22232 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional