liangchen-datanerd commented on issue #11002:
URL: https://github.com/apache/hudi/issues/11002#issuecomment-2054533892
@ad1happy2go Thanks for the reply. Hudi did transform the partition column
timestamp value to the dataformat value based on the
hoodie.keygen.timebased.output.dateformat:yyyy-MM-dd config. At the same time,
the original timestamp value can't be retrieved for the Spark even though it's
persisted in the parquet file. As Hudi already has _hoodie_partition_path to
indicate the partition path, why not keep the original data for the partition
column? For example, when I query the Hudi table, I expect the time column to
be a timestamp value. How can I retrieve the original timestamp value for the
time column?
This is the Hudi table query as I mentioned:
```
+-------------------+---------------------+------------------+----------------------+--------------------------------------------------------------------------+-------------+----------+----------+
|_hoodie_commit_time|_hoodie_commit_seqno
|_hoodie_record_key|_hoodie_partition_path|_hoodie_file_name
|employee_name|department|time |
+-------------------+---------------------+------------------+----------------------+--------------------------------------------------------------------------+-------------+----------+----------+
|20240411142532923 |20240411142532923_1_0|James |2023-01-02
|ea678686-d3d3-4555-b894-30ecb1da2a47-0_1-134-190_20240411142532923.parquet|James
|Sales |2023-01-02|
|20240411142532923 |20240411142532923_1_1|Robert |2023-01-02
|ea678686-d3d3-4555-b894-30ecb1da2a47-0_1-134-190_20240411142532923.parquet|Robert
|Sales |2023-01-02|
|20240411142532923 |20240411142532923_0_0|Michael |2023-01-01
|ec109c4b-723f-46ce-8bb2-5d1e57ecc204-0_0-134-191_20240411142532923.parquet|Michael
|Sales |2023-01-01|
|20240411142532923 |20240411142532923_0_1|Maria |2023-01-01
|ec109c4b-723f-46ce-8bb2-5d1e57ecc204-0_0-134-191_20240411142532923.parquet|Maria
|Finance |2023-01-01|
+-------------------+---------------------+------------------+----------------------+--------------------------------------------------------------------------+-------------+----------+----------+
```
If I have not illustrated this issue well, this ticket
[HUDI-3204](https://issues.apache.org/jira/browse/HUDI-3204) states similar
issue. Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
