I have a step in my beam pipeline that needs some data from a rest service. The data acquired from the rest service is dependent on the context of the data being processed and relatively large. The rest client I am using isn't serializable - nor is it likely possible to make it so (background threads, etc.).
#1 What are the practical limits to the size of side inputs (e.g., I could try to gather all the data from the rest service and provide it as a side-input)? #2 Assuming that using the rest client is the better option, would a singleton instance be safe way to instantiate the rest client? Thanks, rdm
