cloud-fan commented on a change in pull request #32301:
URL: https://github.com/apache/spark/pull/32301#discussion_r638921071
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasing.scala
##########
@@ -149,67 +234,52 @@ object NestedColumnAliasing {
}
/**
- * Return two maps in order to replace nested fields to aliases.
- *
- * If `exclusiveAttrs` is given, any nested field accessors of these
attributes
- * won't be considered in nested fields aliasing.
- *
- * 1. ExtractValue -> Alias: A new alias is created for each nested field.
- * 2. ExprId -> Seq[Alias]: A reference attribute has multiple aliases
pointing it.
+ * Creates a map from root [[Attribute]]s to non-redundant nested
[[ExtractValue]]s.
+ * Nested field accessors of `exclusiveAttrs` are not considered in nested
fields aliasing.
*/
- def getAliasSubMap(exprList: Seq[Expression], exclusiveAttrs: Seq[Attribute]
= Seq.empty)
- : Option[(Map[ExtractValue, Alias], Map[ExprId, Seq[Alias]])] = {
- val (nestedFieldReferences, otherRootReferences) =
- exprList.flatMap(collectRootReferenceAndExtractValue).partition {
- case _: ExtractValue => true
- case _ => false
+ def getAttributeToExtractValues(
+ exprList: Seq[Expression],
+ exclusiveAttrs: Seq[Attribute]): Map[Attribute, Seq[ExtractValue]] = {
+
+ val nestedFieldReferences = new mutable.ArrayBuffer[ExtractValue]()
+ val otherRootReferences = new mutable.ArrayBuffer[AttributeReference]()
+ exprList.foreach { e =>
+ collectRootReferenceAndExtractValue(e).foreach {
+ case ev: ExtractValue => nestedFieldReferences.append(ev)
Review comment:
shall we add an assert here to make sure `ev` only referecnes to one
attribute? To make sure
https://github.com/apache/spark/pull/32301/files#diff-966b594a855ef6f4ee560a4b156755243a1734bb1dfb2f6f4bc92a2f7ba0c2c3R257
is reasonable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
