On Thu, Oct 29, 2020 at 3:18 AM Ismaël Mejía <[email protected]> wrote: > > Thanks for sharing, > > I was initially confused by the title/terminology, I thought it was > about an end-user transform but this is a 'protocol' for a runner to > get the string representation of an element encoded by a SDK Harness > (potentially in a different language) if I understood correctly. > > Are there use cases where a runner cares about the String > representation of data encoded by the SDK harness apart of the > debugging case?
Yeah, I think this is the intent. E.g. a runner could use this to show, in its UI or logs, particularly expensive elements, or hot keys, or excessive uses of state, or even just a sampling of "typical" elements for a given PCollection. > I ask this because I was imagining that if we care > 'only' about debugging data processed by the harness, we could just > have a new debug-like Instruction that produces the tuple of <encoded > data, string representation> and avoid a round-trip. Could you clarify what you mean by this? We certainly wouldn't want the stringification of all elements, only some of them, often post-hoc. > But well take this with a grain of salt, I am far from an expert on > portability, just curious about finding the simplest approach. > > On Thu, Oct 29, 2020 at 12:02 AM Sam Rohde <[email protected]> wrote: > > > > done! > > > > On Wed, Oct 28, 2020 at 3:54 PM Tyson Hamilton <[email protected]> wrote: > >> > >> Can you open up comment access please? > >> > >> On Wed, Oct 28, 2020 at 3:40 PM Sam Rohde <[email protected]> wrote: > >>> > >>> +Lukasz Cwik > >>> > >>> On Tue, Oct 27, 2020 at 12:04 PM Sam Rohde <[email protected]> wrote: > >>>> > >>>> Hi All, > >>>> > >>>> I'm working on a project in Dataflow that requires the runner to > >>>> translate an element to a human-readable form. To do this, I want to add > >>>> a new well-known transform that allows any runner to ask the SDK to > >>>> stringify (human-readable) an element. Let me know what you think, you > >>>> can find the proposed specification and implementation details here. > >>>> > >>>> If there are no objections, I want to start implementation as soon as I > >>>> can. > >>>> > >>>> Regards, > >>>> Sam
