Re: Question about SplittableDoFn

Thanks for sharing. I'll comment on the PR. On Tue, May 18, 2021 at 3:44 PM Miguel Anzo Palomo wrote: > Boyuan Zhang, It's about this issue > , the code can be found > here > > On Tue, May 18,

Re: Question about SplittableDoFn

Boyuan Zhang, It's about this issue , the code can be found here On Tue, May 18, 2021 at 5:29 PM Boyuan Zhang wrote: > Would you like to share your draft code? Iterating on the code might be >

Re: Question about SplittableDoFn

Would you like to share your draft code? Iterating on the code might be easier to figure out the issue. On Tue, May 18, 2021 at 3:28 PM Robert Burke wrote: > IIRC the Initial Restrictions method gives you an element and you return > the restrictions relative to that element. > > It's entirely

Re: Question about SplittableDoFn

IIRC the Initial Restrictions method gives you an element and you return the restrictions relative to that element. It's entirely appropriate to stat files or query databases in order to determine the initial restrictions and partitions of the data. On Tue, May 18, 2021, 3:21 PM Miguel Anzo

Question about SplittableDoFn

Hi, I’m looking at how to implement a reader as a SplittableDoFn and I'm having some problems with the initial restriction, specifically, how do you set the initial restriction if you don’t know the size of the data? The DoFn that I'm working on takes a PCollection of Spanner *ReadOperations *and