XiaoHongbo-Hope opened a new pull request, #11074:
URL: https://github.com/apache/paimon/pull/11074

   Merged LeRobot imports currently keep each source's original task, subtask, 
stats, and Episode metadata in Arrow IPC binary columns. These columns cannot 
be inspected as JSON in SQL. This change writes JSON payloads and source 
schemas instead, while keeping common training columns typed and preserving 
source-specific extension fields for future export.
   
   New table groups use `tasks_json`, `subtasks_json`, `stats_json`, and 
`episodes_schema_json` in `__datasets`, plus `source_episode_json` in 
`__episodes`. The training reader excludes these archive fields and still reads 
existing table groups with IPC columns. This PR is independent of #11070.
   
   Validation: 144 LeRobot aggregate tests passed; flake8 and `git diff 
--check` passed. Two customer sample sources (25/30 FPS) imported into a local 
table group, and their task, subtask, and Episode Parquet data round-tripped 
through JSON with schemas and metadata intact. The updated reader also opened a 
table group written by the prior IPC layout.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to