Re: Support for PaneInfo in Python SDK

2019-05-31 Thread Tanay Tummalapalli
Hi Pablo, Thanks for that example, it would be great to be able to use fileio.WriteToFiles transform to write files with filenames that are based on their PaneInfo. Thanks @Charles Chen, for adding the remaining work on the issue - the emission of PaneInfo in the Python implementation of GBK

Re: Support for PaneInfo in Python SDK

2019-05-30 Thread Pablo Estrada
Hi Tanay, thanks for bringing this to the mailing list. I believe this is certainly useful, and necessary. As an example, the fileio.WriteToFiles transform does not work well without PaneInfo data (since we can't know how many firings there are for each window, and we can't give names to files

Support for PaneInfo in Python SDK

2019-05-30 Thread Tanay Tummalapalli
Hi everyone, The PR linked in [BEAM-3759] - "Add support for PaneInfo descriptor in Python SDK"[1] was merged, but, the issue is still open. There might be some work left on this for full support for PaneInfo. Eg: Although the PaneInfo class exists, it is not accessible in a DoFn via a