hangc0276 opened a new issue, #4621: URL: https://github.com/apache/iceberg/issues/4621
### Motivation [Apache Pulsar](https://github.com/apache/pulsar) is doing integration with Iceberg, and take Iceberg as tiered storage to offload topic cold data into Iceberg. When consumers fetch cold data from topic, Pulsar broker will locate the target data is stored in Pulsar or not. If the target data stored in tiered storage (Iceberg), Pulsar broker will fetch data from Iceberg by Java API, and package them into Pulsar format and dispatch to consumer side. For Pulsar Iceberg integration, we first use iceberg writer to streaming write topics messages into iceberg table in one thread. And then we use iceberg reader to read records from iceberg table by streaming. In pulsar's read case, we should ensure the records read by the same order with the write. However, we found current Iceberg Java reader implementation doesn't support read records by order or doesn't support `order by ` operation on reading. ### What we need - We need Iceberg to support read records out which keep the writer order, or support order by specific fields. - We need to read the change log of the iceberg table by streaming. - Does the Iceberg community has plan to support this feature? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
