> I'm reluctant to do this without an explicit call from the user or in a
> service. The problem is when to expire snapshots. Iceberg is called regularly
> to read and write tables. That might seem like a good time to expire
> snapshots, but it doesn't make sense for either one to have a side ef
Replies inline.
On Mon, May 6, 2019 at 3:01 PM Anton Okolnychyi
wrote:
> I am also wondering whether it makes sense to have a config that limits
> the number of snapshot we want to track. This config can be based on the
> number of snapshots (e.g. keep only 1 snapshots) or based on time (e.g
> On Mon, May 6, 2019 at 4:18 AM Anton Okolnychyi
> wrote:
> Hi,
>
> I would like to discuss the support for micro-batch streaming in Iceberg.
>
> First of all, do we think micro-batch use-cases are appropriate for Iceberg?
> What do we consider as "slow-moving da
, we could also use a table's snapshot history to
support reading from an Iceberg table as a streaming source.
On Mon, May 6, 2019 at 4:18 AM Anton Okolnychyi
wrote:
> Hi,
>
> I would like to discuss the support for micro-batch streaming in Iceberg.
>
> First of all, do we t
Hi,
I would like to discuss the support for micro-batch streaming in Iceberg.
First of all, do we think micro-batch use-cases are appropriate for Iceberg?
What do we consider as "slow-moving data"? Do we want to support batch
intervals of 30s? What about intervals of 1m/2m/5m? The la
