srowen commented on a change in pull request #29078:
URL: https://github.com/apache/spark/pull/29078#discussion_r453373000
##########
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/HadoopFileLinesReader.scala
##########
@@ -46,7 +46,7 @@ class HadoopFileLinesReader(
def this(file: PartitionedFile, conf: Configuration) = this(file, None, conf)
- private val iterator = {
+ private val _iterator = {
Review comment:
This is to resolve a weird compile error:
```
[ERROR] [Error]
/Users/seanowen/Documents/spark_2.13/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/HadoopFileLinesReader.scala:49:
weaker access privileges in overriding
final def iterator: Iterator[org.apache.hadoop.io.Text] (defined in trait
Iterator)
override should not be private
[ERROR] [Error]
/Users/seanowen/Documents/spark_2.13/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/HadoopFileWholeTextReader.scala:38:
weaker access privileges in overriding
final def iterator: Iterator[org.apache.hadoop.io.Text] (defined in trait
Iterator)
override should not be private
```
Not sure, but easy to avoid the name conflict entirely.
##########
File path:
sql/core/src/test/scala/org/apache/spark/sql/DatasetPrimitiveSuite.scala
##########
@@ -223,16 +223,6 @@ class DatasetPrimitiveSuite extends QueryTest with
SharedSparkSession {
checkDataset(Seq(Queue(true)).toDS(), Queue(true))
checkDataset(Seq(Queue("test")).toDS(), Queue("test"))
checkDataset(Seq(Queue(Tuple1(1))).toDS(), Queue(Tuple1(1)))
-
- checkDataset(Seq(ArrayBuffer(1)).toDS(), ArrayBuffer(1))
Review comment:
For complex reasons, I don't think these tests / use cases are possible
as-is in Scala 2.13. They are niche, so I jsut removed them.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
