scwhittle commented on PR #37662: URL: https://github.com/apache/beam/pull/37662#issuecomment-4010593888
Yes i was using that at first but it has some downsides: - formats as string instead of payload - adds labels we don't have in existing logs - we need to modify the MonitoredResource based upon the log as it includes the step name and the only way to do that was to override the protected logentryfor method instead of using enhancers etc - we want to ensure we don't double log which is tricky if there are two separate handlers I was originally delegating to an instance of LoggingHandler but it actually wasn't saving much code and was doing extra things we didn't want. On Fri, Mar 6, 2026, 8:14 AM Arun Pandian ***@***.***> wrote: > ***@***.**** commented on this pull request. > ------------------------------ > > In > runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/logging/DataflowWorkerLoggingHandler.java > <https://github.com/apache/beam/pull/37662#discussion_r2894225141>: > > > + if (isDirectLog) { > + if (directThrottler.shouldAttemptDirectLog()) { > + try { > + LogEntry logEntry = constructDirectLogEntry(record, executionState); > + if (testDirectLogInterceptor != null) { > + // The default labels are applied by write options generally bute we merge them in here > + // so they are visible to the test. > + HashMap<String, String> mergedLabels = new HashMap<>(defaultLabels); > + mergedLabels.putAll(logEntry.getLabels()); > + logEntry = logEntry.toBuilder().setLabels(mergedLabels).build(); > + if (logEntry.getResource() == null) { > + logEntry = logEntry.toBuilder().setResource(steplessMonitoredResource).build(); > + } > + checkNotNull(testDirectLogInterceptor).accept(logEntry); > + } else { > + checkNotNull(directLogging).write(ImmutableList.of(logEntry), directWriteOptions); > > did you consider calling to com.google.cloud.logging.LoggingHandler to do > direct logging? Seems like LoggingHandler has most of the logic already. We > could setup a ErrorManager to collect errors and fallback to disk logging. > > — > Reply to this email directly, view it on GitHub > <https://github.com/apache/beam/pull/37662#pullrequestreview-3901883961>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/ABBZZTBGLCRX7MWYWEJCVFD4PJ3GPAVCNFSM6AAAAACV2T5K7SVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZTSMBRHA4DGOJWGE> > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> > -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
