stevenzwu commented on issue #5803: URL: https://github.com/apache/iceberg/issues/5803#issuecomment-1258984740
@zixiao144 yes, saw the logs in taskmanager log file. I forgot the the `StreamingMonitorFunction` runs on taskmanagers. Flink streaming read only discovers `append` snapshot. How did you append data to the table? it seems that Flink source see the snapshots but didn't find any splits from snapshots with `append` operation. Can you run java API to check the snapshot history and the `operation` type of the `Snapshot`? Alternatively, you can run Spark SQL to query the metadata table: https://iceberg.apache.org/docs/latest/spark-queries/#snapshots. ``` 2022-09-27 00:19:28,682 DEBUG org.apache.iceberg.flink.source.StreamingMonitorFunction [] - Start discovering splits from -1 (exclusive) to 6496244702617183271 (inclusive) 2022-09-27 00:19:40,565 DEBUG org.apache.iceberg.flink.source.StreamingMonitorFunction [] - Discovered 0 splits, time elapsed 11883ms 2022-09-27 00:19:40,566 DEBUG org.apache.iceberg.flink.source.StreamingMonitorFunction [] - Forwarded 0 splits, time elapsed 1ms 2022-09-27 00:19:57,703 DEBUG org.apache.iceberg.flink.source.StreamingMonitorFunction [] - Start discovering splits from 6496244702617183271 (exclusive) to 2877339576078753525 (inclusive) 2022-09-27 00:19:57,704 DEBUG org.apache.iceberg.flink.source.StreamingMonitorFunction [] - Discovered 0 splits, time elapsed 1ms 2022-09-27 00:19:57,705 DEBUG org.apache.iceberg.flink.source.StreamingMonitorFunction [] - Forwarded 0 splits, time elapsed 0ms 2022-09-27 00:20:02,759 DEBUG org.apache.iceberg.flink.source.StreamingMonitorFunction [] - Start discovering splits from 2877339576078753525 (exclusive) to 9205452674806088860 (inclusive) 2022-09-27 00:20:02,759 DEBUG org.apache.iceberg.flink.source.StreamingMonitorFunction [] - Discovered 0 splits, time elapsed 0ms 2022-09-27 00:20:02,759 DEBUG org.apache.iceberg.flink.source.StreamingMonitorFunction [] - Forwarded 0 splits, time elapsed 0ms 2022-09-27 00:20:52,053 DEBUG org.apache.iceberg.flink.source.StreamingMonitorFunction [] - Start discovering splits from 9205452674806088860 (exclusive) to 85589348609116719 (inclusive) 2022-09-27 00:20:52,070 DEBUG org.apache.iceberg.flink.source.StreamingMonitorFunction [] - Discovered 0 splits, time elapsed 17ms 2022-09-27 00:20:52,070 DEBUG org.apache.iceberg.flink.source.StreamingMonitorFunction [] - Forwarded 0 splits, time elapsed 0ms 2022-09-27 00:21:42,325 DEBUG org.apache.iceberg.flink.source.StreamingMonitorFunction [] - Start discovering splits from 85589348609116719 (exclusive) to 519180963154444509 (inclusive) 2022-09-27 00:21:42,345 DEBUG org.apache.iceberg.flink.source.StreamingMonitorFunction [] - Discovered 0 splits, time elapsed 20ms 2022-09-27 00:21:42,345 DEBUG org.apache.iceberg.flink.source.StreamingMonitorFunction [] - Forwarded 0 splits, time elapsed 0ms 2022-09-27 00:22:32,575 DEBUG org.apache.iceberg.flink.source.StreamingMonitorFunction [] - Start discovering splits from 519180963154444509 (exclusive) to 2413530983076443232 (inclusive) 2022-09-27 00:22:32,595 DEBUG org.apache.iceberg.flink.source.StreamingMonitorFunction [] - Discovered 0 splits, time elapsed 20ms 2022-09-27 00:22:32,595 DEBUG org.apache.iceberg.flink.source.StreamingMonitorFunction [] - Forwarded 0 splits, time elapsed 0ms 2022-09-27 00:23:21,804 DEBUG org.apache.iceberg.flink.source.StreamingMonitorFunction [] - Start discovering splits from 2413530983076443232 (exclusive) to 5161337709432232128 (inclusive) 2022-09-27 00:23:21,819 DEBUG org.apache.iceberg.flink.source.StreamingMonitorFunction [] - Discovered 0 splits, time elapsed 15ms 2022-09-27 00:23:21,819 DEBUG org.apache.iceberg.flink.source.StreamingMonitorFunction [] - Forwarded 0 splits, time elapsed 0ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
