Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/862#issuecomment-44347248
@ueshin Sorry, my bad, misunderstood your PR description. And I think you
are right.
On the other hand, it seems that `AverageFunction` is not used in query
plan unless we use it in DSL queries explicitly (maybe I missed something
related to aggregation functions here):
```
scala> sql("SELECT AVG(key) FROM src1").collect().foreach(println)
== Query Plan ==
Aggregate false, [], [(CAST(SUM(PartialSum#648), DoubleType) /
CAST(SUM(PartialCount#649), DoubleType)) AS c0#644]
Exchange SinglePartition
Aggregate true, [], [COUNT(key#646) AS PartialCount#649,SUM(key#646) AS
PartialSum#648]
HiveTableScan [key#646], (MetastoreRelation default, src1, None), None),
which is now runnable
14/05/28 07:04:33 INFO scheduler.DAGScheduler: Submitting 1 missing tasks
from Stage 8 (SchemaRDD[45] at RDD at SchemaRDD.scala:98
== Query Plan ==
Aggregate false, [], [(CAST(SUM(PartialSum#648), DoubleType) /
CAST(SUM(PartialCount#649), DoubleType)) AS c0#644]
Exchange SinglePartition
Aggregate true, [], [COUNT(key#646) AS PartialCount#649,SUM(key#646) AS
PartialSum#648]
HiveTableScan [key#646], (MetastoreRelation default, src1, None), None)
```
It seems that currently `AVG` is always turned into partial sum divided by
partial count, and leads us to the right answer (null values are ignored). But
I think your fix still makes sense and should be merged.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
