Jonathan Vexler created HUDI-6201:
-------------------------------------
Summary: Timeline server sometimes does not send bootstrap base
path for a skeleton file
Key: HUDI-6201
URL: https://issues.apache.org/jira/browse/HUDI-6201
Project: Apache Hudi
Issue Type: Bug
Components: bootstrap, timeline-server
Reporter: Jonathan Vexler
Attachments: TestBootstrapRead.java
[^TestBootstrapRead.java] In the attached file, enable the timeline server
'hoodie.embed.timeline.server'. It will occasionally fail in metadata or mixed
mode because some records will be null besides the metadata columns:
{code:java}
+-------------------+---------------------+------------------------------------+-------------------------+------------------------------------------------------------------------+------------------+------------------------------------+------------------+-------------------+-------------+------------+----------+------------------+----------+-------------------+-------------------+------------------------+-------------------+--------+----------+--------------+---------+--------------------+---------+--------------------------+----------+|_hoodie_commit_time|_hoodie_commit_seqno
|_hoodie_record_key |_hoodie_partition_path
|_hoodie_file_name
|_hoodie_is_deleted|_row_key |begin_lat
|begin_lon
|city_to_state|current_date|current_ts|distance_in_meters|driver |end_lat
|end_lon |fare |height
|nation |partition |partition_path|rider |seconds_since_epoch
|timestamp|tip_history |weight
|+-------------------+---------------------+------------------------------------+-------------------------+------------------------------------------------------------------------+------------------+------------------------------------+------------------+-------------------+-------------+------------+----------+------------------+----------+-------------------+-------------------+------------------------+-------------------+--------+----------+--------------+---------+--------------------+---------+--------------------------+----------+|00000000000001
|00000000000001_4_0
|876743b0-f5e7-4289-b13b-1a0404d94380|partition_path=2015-03-16|7e1dea56-e88c-4072-be61-f4ae01feaaa3_1-138-381_20230510125841762.parquet|null
|null |null |null
|null |null |null |null |null
|null |null |null |null
|null |null |null |null |null |null
|null |null ||00000000000001 |00000000000001_4_1
|00923d1a-58fc-4d42-8953-4a47b47d738f|partition_path=2015-03-16|7e1dea56-e88c-4072-be61-f4ae01feaaa3_1-138-381_20230510125841762.parquet|null
|null |null |null
|null |null |null |null |null
|null |null |null |null
|null |null |null |null |null |null
|null |null ||20230510125841762
|20230510125841762_1_2|b318c482-8e43-4614-bdab-80946d5a9f53|partition_path=2015-03-16|7e1dea56-e88c-4072-be61-f4ae01feaaa3_1-138-381_20230510125841762.parquet|false
|b318c482-8e43-4614-bdab-80946d5a9f53|0.5285807377766387|0.12835359814395741|[CA]
|12 |1047178778|521899450
|driver-001|0.41394620067559684|0.08532822423986208|[42.25978252084417,
USD]|[0, 0, 7, -91, -36]|[Canada]|2015-03-16|2015-03-16
|rider-001|-2845295541651788027|0 |[[32.10533813167099,
USD]]|0.59076524||00000000000001 |00000000000001_4_3
|4dcc72d7-0878-41c2-a85d-3e6374b88bb8|partition_path=2015-03-16|7e1dea56-e88c-4072-be61-f4ae01feaaa3_1-138-381_20230510125841762.parquet|null
|null |null |null
|null |null |null |null |null
|null |null |null |null
|null |null |null |null |null |null
|null |null ||00000000000001 |00000000000001_4_4
|cfa79530-fc9f-42de-a181-34c06e79d9c5|partition_path=2015-03-16|7e1dea56-e88c-4072-be61-f4ae01feaaa3_1-138-381_20230510125841762.parquet|null
|null |null |null
|null |null |null |null |null
|null |null |null |null
|null |null |null |null |null |null
|null |null ||00000000000001 |00000000000001_4_5
|4d2a4755-83a7-4201-9b65-0148752d55b7|partition_path=2015-03-16|7e1dea56-e88c-4072-be61-f4ae01feaaa3_1-138-381_20230510125841762.parquet|null
|null |null |null
|null |null |null |null |null
|null |null |null |null
|null |null |null |null |null |null
|null |null ||00000000000001 |00000000000001_4_6
|76429084-f78e-4c6d-a70d-088cb5d955aa|partition_path=2015-03-16|7e1dea56-e88c-4072-be61-f4ae01feaaa3_1-138-381_20230510125841762.parquet|null
|null |null |null
|null |null |null |null |null
|null |null |null |null
|null |null |null |null |null |null
|null |null ||00000000000001 |00000000000001_4_7
|8436456b-9858-45cb-8a07-bdae536a2d17|partition_path=2015-03-16|7e1dea56-e88c-4072-be61-f4ae01feaaa3_1-138-381_20230510125841762.parquet|null
|null |null |null
|null |null |null |null |null
|null |null |null |null
|null |null |null |null |null |null
|null |null ||00000000000001 |00000000000001_4_8
|b6ca00ed-a4f4-4db5-a269-6be250b64caa|partition_path=2015-03-16|7e1dea56-e88c-4072-be61-f4ae01feaaa3_1-138-381_20230510125841762.parquet|null
|null |null |null
|null |null |null |null |null
|null |null |null |null
|null |null |null |null |null |null
|null |null
|+-------------------+---------------------+------------------------------------+-------------------------+------------------------------------------------------------------------+------------------+------------------------------------+------------------+-------------------+-------------+------------+----------+------------------+----------+-------------------+-------------------+------------------------+-------------------+--------+----------+--------------+---------+--------------------+---------+--------------------------+----------++-------------------+--------------------+------------------------------------+-------------------------+-------------------------------------------------------------------+
{code}
This is due to the bootstrap base file not being set, so HoodieMergeHelper does
not process the file as a bootstrap skeleton file
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
