rdblue commented on a change in pull request #3933:
URL: https://github.com/apache/iceberg/pull/3933#discussion_r791090930
##########
File path: api/src/main/java/org/apache/iceberg/types/Conversions.java
##########
@@ -71,6 +71,12 @@ public static Object fromPartitionString(Type type, String
asString) {
return new BigDecimal(asString);
case DATE:
return Literal.of(asString).to(Types.DateType.get()).value();
+ case TIMESTAMP:
+ if (!asString.contains("T")) {
+ return java.sql.Timestamp.valueOf(asString).getTime() * 1000;
Review comment:
We need to be _very_ strict with what we parse and accept here and
delegating this work to `java.sql.Timestamp` seems like a good way to introduce
formats that we don't intend to support. I would rather use what is built into
`StringLiteral` and supply different formats. Can you look at using
`OffsetDateTime.parse` and a format for this instead?
##########
File path:
spark/v3.2/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestAddFilesProcedure.java
##########
@@ -819,6 +839,26 @@ public void
testPartitionedImportFromEmptyPartitionDoesNotThrow() {
RowFactory.create(4, "Will Doe", "facilities",
toDate("2021-01-02"))),
new StructType(dateStruct)).repartition(2);
+ private static final StructField[] timeStampstruct = {
+ new StructField("id", DataTypes.IntegerType, true, Metadata.empty()),
+ new StructField("name", DataTypes.StringType, true, Metadata.empty()),
+ new StructField("dept", DataTypes.StringType, true, Metadata.empty()),
+ new StructField("ts", DataTypes.TimestampType, true, Metadata.empty())
+ };
+
+ private static Timestamp toTimestamp(String value) {
+ return new Timestamp(DateTime.parse(value).getMillis());
Review comment:
This is not a reliable way to convert to a timestamp because Java Date
and SQL Timestamp have internal time zone logic. Instead, please use Literal or
parse values directly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
