Here's the doc I've been working on: https://docs.google.com/document/d/1nGGP2sLb5fLamB_dnkHVHC8BVjDD_SE46mQPIPkK5cQ/edit?usp=sharing
The general purpose of the doc is to share high level design thoughts and process for authoring an IO transform. Topics include: * List of example IOs that people can refer to, and for what reasons * Steps for implementers * Read transform design * Write transform design (mostly a TODO for now) Specifics of implementations, notably including the two items that started recent discussions of the IO Authoring doc (the service interface and the fake/mock pattern), are probably best covered by the transform style guide that Eugene's working on and the "testing IO transforms" doc[1]. This is definitely a first draft. There is a lot that could be written on the topic, and I have struggled a bit with how much detail to put in, but since the jira issue has been open a while, this is a simple set of things that I thought wouldn't be too controversial, and that would be forward compatible with SplittableDoFn coming (thus, spending too much time discussing sources isn't useful.) I look forward to contributions from folks like Eugene, JB, and Dan who have a lot more experience in this area. I'd like to get something up on the website sooner rather than later, so if folks want to add large chunks/sections, I might push us towards getting up a smaller, correct version in a first PR, and folks can add more sections/design patterns/etc... in later PRs. Thanks! Stephen [1] Testing IO transforms - https://docs.google.com/document/d/153J9jPQhMCNi_eBzJfhAg-NprQ7vbf1jNVRgdqeEE8I/edit?usp=sharing
