bersprockets commented on code in PR #47982:
URL: https://github.com/apache/spark/pull/47982#discussion_r1742823468
##########
sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala:
##########
@@ -1707,7 +1707,7 @@ class Dataset[T] private[sql](
/** @inheritdoc */
override def toJSON: Dataset[String] = {
- val rowSchema = this.schema
+ val rowSchema = exprEnc.schema
Review Comment:
@hvanhovell I have no idea how to run or use connect, but looking at the
implementation
[here](https://github.com/apache/spark/blob/6165b12c7a64b4d1206696054e9bbcc15d72f8bf/connector/connect/client/jvm/src/main/scala/org/apache/spark/sql/Dataset.scala#L1201),
I don't think it's a problem (assuming, of course, that I am looking at the
right piece of code).
When I mimic that implementation using spark-shell, I get the right answer:
```
scala> case class Data(a: BigDecimal)
class Data
scala> sql("select 123.456bd as
a").as[Data].select(to_json(struct(col("*")))).collect
warning: 1 deprecation (since 2.13.3); for details, enable `:setting
-deprecation` or `:replay -deprecation`
val res13: Array[org.apache.spark.sql.Row] = Array([{"a":123.456}])
```
Also:
```
case class Data(x: Array[Int], y: String)
class Data
scala> sql("select repeat('Hey there', 17) as y, array_repeat(22, 17) as
x").as[Data].select(to_json(struct(col("*")))).collect
warning: 1 deprecation (since 2.13.3); for details, enable `:setting
-deprecation` or `:replay -deprecation`
val res0: Array[org.apache.spark.sql.Row] = Array([{"y":"Hey thereHey
thereHey thereHey thereHey thereHey thereHey thereHey thereHey thereHey
thereHey thereHey thereHey thereHey thereHey thereHey thereHey
there","x":[22,22,22,22,22,22,22,22,22,22,22,22,22,22,22,22,22]}])
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
