[
https://issues.apache.org/jira/browse/BEAM-14144?focusedWorklogId=754206&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-754206
]
ASF GitHub Bot logged work on BEAM-14144:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 07/Apr/22 15:56
Start Date: 07/Apr/22 15:56
Worklog Time Spent: 10m
Work Description: lukecwik commented on code in PR #17151:
URL: https://github.com/apache/beam/pull/17151#discussion_r845292705
##########
runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowRunner.java:
##########
@@ -378,6 +378,13 @@ public static DataflowRunner fromOptions(PipelineOptions
options) {
+ "' invalid. Please make sure the value is non-negative.");
}
+ // Verify that if recordJfrOnGcThrashing is set, the pipeline is at least
on java 11
+ if (dataflowOptions.getRecordJfrOnGcThrashing()
+ && Environments.getJavaVersion() == Environments.JavaVersion.java8) {
+ throw new IllegalArgumentException(
+ "recordJfrOnGcThrashing is only supported on java 9 and up.");
Review Comment:
```suggestion
"recordJfrOnGcThrashing is only supported on Java 9 and up.");
```
##########
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/util/MemoryMonitor.java:
##########
@@ -198,20 +227,43 @@ public long totalGCTimeMilliseconds() {
private final File localDumpFolder;
+ private final String workerId;
+
+ private final @Nullable JfrInterop jfrInterop;
+
+ private final Clock clock;
+
public static MemoryMonitor fromOptions(PipelineOptions options) {
DataflowPipelineDebugOptions debugOptions =
options.as(DataflowPipelineDebugOptions.class);
+ DataflowWorkerHarnessOptions workerHarnessOptions =
+ options.as(DataflowWorkerHarnessOptions.class);
String uploadToGCSPath = debugOptions.getSaveHeapDumpsToGcsPath();
+ String workerId = workerHarnessOptions.getWorkerId();
boolean canDumpHeap = uploadToGCSPath != null ||
debugOptions.getDumpHeapOnOOM();
double gcThrashingPercentagePerPeriod =
debugOptions.getGCThrashingPercentagePerPeriod();
+ Duration jfrProfileDuration;
+ if (uploadToGCSPath != null && debugOptions.getRecordJfrOnGcThrashing()) {
+ if (Environments.getJavaVersion() == Environments.JavaVersion.java8) {
+ throw new IllegalArgumentException(
+ "recordJfrOnGcThrashing is only supported on java 9 and up.");
Review Comment:
```suggestion
"recordJfrOnGcThrashing is only supported on Java 9 and up.");
```
##########
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/util/MemoryMonitor.java:
##########
@@ -515,6 +562,10 @@ public void run() {
if (isThrashing.get()) {
currentThrashingCount++;
+ if (currentThrashingCount == 1) {
+ runJfrProfileOnHeapThrashing();
+ }
+
Review Comment:
I thought it was wrong but I misaligned the spacing.
Issue Time Tracking
-------------------
Worklog Id: (was: 754206)
Time Spent: 2h 50m (was: 2h 40m)
> Record JFR profiles when GC thrashing is detected
> -------------------------------------------------
>
> Key: BEAM-14144
> URL: https://issues.apache.org/jira/browse/BEAM-14144
> Project: Beam
> Issue Type: Improvement
> Components: runner-dataflow
> Reporter: Steve Niemitz
> Assignee: Steve Niemitz
> Priority: P2
> Time Spent: 2h 50m
> Remaining Estimate: 0h
>
> It'd be useful for debugging GC issues in jobs to have allocation profiles
> when it was occurring. In java 9+, JFR is included in the JDK, we could use
> that to record profiles when GC thrashing is detected.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
