scwhittle commented on PR #27867: URL: https://github.com/apache/beam/pull/27867#issuecomment-1669223837
It seems confusing to add the sdk options to dataflow options when there are potentially a lot of things relevant only when running as sdk and not v1 (now or in the future). Things like --maxCacheMemoryUsage would now be in DataflowOptions and perhaps be tempting to try but ineffectual. Another approach would be to extract the memory monitor options so that they can be shared by both runners. We actually have two MemoryMonitor implementations at the moment that differ slightly, it would be nice to unify them as well but I believe that most options would be common and the unique options could remain in current locations or perhaps done via subclasses of a base MemoryManagerOptions Or you could just promote the single GCThrashingPercentagePerPeriod out of the debug dataflow options to dataflow options so it is visible. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
