Github user Apo1 commented on a diff in the pull request:
https://github.com/apache/spark/pull/10374#discussion_r48025796
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/rows.scala
---
@@ -201,7 +201,7 @@ class GenericRow(protected[sql] val values: Array[Any])
extends Row {
override def toSeq: Seq[Any] = values.toSeq
- override def copy(): Row = this
+ override def copy(): Row = new GenericRow(values.clone())
--- End diff --
I try to provide a simple example and I could not try the code below right
now. Imagine you want to query a row, copy it, change a value and then build a
new one. Then you want to build a DF out of the new and the old row. I know
that this is a strange example and there are a lot of better method to
implement it, but you could implement it this way:
import org.apache.spark.sql.Row;
val df = sc.parallelize(1,2,3,5,6,7,8,9).toDF.sort
val firstRow = df.first.copy() // <-- HERE the row will not be copied.
val arr = firstRow.toSeq.toArray
arr(0) = arr(0) * 10
val newRow = Row.fromSeq(arr)
sqlContext.createDataframe(List(firstRow, newRow), df.schema) // Both
row will contain the value 10 and share the same value Sequence
If you don't want that someone uses the copy method, because it does not do
what it implies, then i think we should provide another trait/contract which
does not include this method. It's just confusing if the copy method does not
copy the object.
`row.copy()` and just `row` are identical so far!
I don't see the point, why we should'nt provide this copy method for the
users who want to use it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
