maropu commented on a change in pull request #23854: [SPARK-22000][SQL] Use
String.valueOf to assign value to String type of field in Java Bean
URL: https://github.com/apache/spark/pull/23854#discussion_r258902220
##########
File path:
sql/core/src/test/java/test/org/apache/spark/sql/JavaBeanDeserializationSuite.java
##########
@@ -115,6 +115,37 @@ public void testBeanWithMapFieldsDeserialization() {
Assert.assertEquals(records, MAP_RECORDS);
}
+ private static final List<RecordSpark22000> RECORDS_SPARK_22000 = new
ArrayList<>();
+
+ static {
+ RECORDS_SPARK_22000.add(new RecordSpark22000("1", "[email protected]", 2, 11));
+ RECORDS_SPARK_22000.add(new RecordSpark22000("2", "[email protected]", 3, 12));
+ RECORDS_SPARK_22000.add(new RecordSpark22000("3", "[email protected]", 4, 13));
+ RECORDS_SPARK_22000.add(new RecordSpark22000("4", "[email protected]", 5, 14));
+ }
+
+ @Test
+ public void testSpark22000() {
+ // Here we try to convert the type of 'ref' field, from integer to string.
+ // Before applying SPARK-22000, Spark called toString() against variable
which type might be primitive.
+ // SPARK-22000 it calls String.valueOf() which finally calls toString()
but handles boxing
+ // if the type is primitive.
+ Encoder<RecordSpark22000> encoder = Encoders.bean(RecordSpark22000.class);
+
+ Dataset<RecordSpark22000> dataset = spark
+ .read()
+ .format("csv")
+ .option("header", "true")
+ .option("mode", "DROPMALFORMED")
+ .schema("ref int, userId string, x int, y int")
+ .load("src/test/resources/test-data/spark-22000.csv")
Review comment:
Thanks!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
