KSubramanyaH commented on issue #11335:
URL: https://github.com/apache/hudi/issues/11335#issuecomment-2134325978
Hello @ad1happy2go ...
below is printschema just before running Hudi load
root
|-- op: string (nullable = true)
|-- _acq_load_ts: string (nullable = true)
|-- id: decimal(20,0) (nullable = true)
|-- dataset_id: decimal(20,0) (nullable = true)
|-- deleted_id: decimal(20,0) (nullable = true)
|-- user_modified_at: timestamp (nullable = true)
|-- created_at: timestamp (nullable = true)
|-- modified_at: timestamp (nullable = true)
|-- client_id: string (nullable = true)
|-- opt_lock: long (nullable = true)
|-- parent_id: decimal(20,0) (nullable = true)
|-- name: string (nullable = true)
|-- target_amount: decimal(13,4) (nullable = true)
|-- overwritten_target_amount: decimal(13,4) (nullable = true)
|-- calculated_spent_amount: decimal(13,4) (nullable = true)
|-- filter_id: string (nullable = true)
|-- recurring: byte (nullable = true)
|-- frequency: string (nullable = true)
|-- txn_ids: string (nullable = true)
|-- excluded_txn_ids: string (nullable = true)
|-- rollover_amount: decimal(13,4) (nullable = true)
|-- reset_rollover_amount: decimal(13,4) (nullable = true)
|-- cumulative_rollover_amount: decimal(13,4) (nullable = true)
|-- rollover_type: string (nullable = true)
|-- _acq_stream_position: string (nullable = true)
|-- _acq_change_seq: string (nullable = true)
|-- _job_run_id: string (nullable = true)
|-- _s3_intermediate_location: string (nullable = true)
|-- _partition_year: string (nullable = true)
|-- _partition_month: string (nullable = true)
One more note : We are loading 2 hudi tables with same schema .
1 table is just load all cdc data with bulk insert and is working fine
2 table load is upsert and delete . But it is failing with decimal encode
issue
I am very much sure that incoming data has not crossed prescribed schema
(.i.e decimal(13,4)) .. Maximum that we receive decimal(10,4)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
