cxzl25 commented on code in PR #1910:
URL: https://github.com/apache/orc/pull/1910#discussion_r1582038081
##########
java/bench/core/src/resources/taxi.schema:
##########
@@ -9,13 +9,13 @@ struct<
PULocationID: bigint,
DOLocationID: bigint,
payment_type: bigint,
- fare_amount: decimal(8,2),
- extra: decimal(8,2),
- mta_tax: decimal(8,2),
- tip_amount: decimal(8,2),
- tolls_amount: decimal(8,2),
- improvement_surcharge: decimal(8,2),
- total_amount: decimal(8,2),
+ fare_amount: decimal(10,2),
Review Comment:
The logic written in this may not actually be related to Spark, because
Spark will convert out-of-range values into NULL.
```sql
SELECT CAST(82599861 AS decimal(8, 2))
```
```
NULL
```
If in this PR, the taxi.schema is not modified, the implementation is
written directly, and there is no cast involved at this time, so it will fail
to write.
```
java -jar core/target/orc-benchmarks-core-*-uber.jar generate data -format
parquet -data taxi -compress snappy
```
```java
Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException:
arraycopy: destination index -1 out of bounds for byte[4]
at java.base/java.lang.System.arraycopy(Native Method)
at
org.apache.orc.bench.core.convert.avro.AvroWriter.decimalToBinary(AvroWriter.java:381)
at
org.apache.orc.bench.core.convert.avro.AvroWriter$DecimalConverter.convert(AvroWriter.java:182)
at
org.apache.orc.bench.core.convert.parquet.ParquetWriter.writeBatch(ParquetWriter.java:78)
at
org.apache.orc.bench.core.convert.GenerateVariants.run(GenerateVariants.java:157)
at org.apache.orc.bench.core.Driver.main(Driver.java:64)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
