[
https://issues.apache.org/jira/browse/FLINK-14435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
lining updated FLINK-14435:
---------------------------
Description:
* information from TaskExecutorResourceSpec in flip-49, add it to
TaskExecutorRegistration.
{code:json}
public class TaskManagerResourceInfo {
private final double cpuCores;
private final long frameworkHeap;
private final long frameworkOffHeap;
private final long taskHeap;
private final long taskOffHeap;
private final long shuffleMemory;
private final long managedMemory;
private final long jvmMetaSpace;
private final long jvmOverhead;
private final long totalProcessMemory;
}{code}
* url: /taskmanagers/:taskmanagerid
* response: add
{code:java}
resource: {
cpuCores: 4,
frameworkHeap: 134217728,
frameworkOffHeap: 134217728,
taskHeap: 181193928,
taskOffHeap: 0,
shuffleMemory: 33554432,
managedMemory: 322122552,
jvmMetaSpace: 134217728,
jvmOverhead: 134217728,
totalProcessMemory: 1073741824
}
{code}
was:
h5. data in json
{code:json}
{
"cpuAllocated": -1,
"cpuUsage": -1,
"taskHeapAllocated": 966787072,
"taskHeapUsed": 76071880,
"heapManageMemoryMax": 0,
"heapManageMemoryUsed": 0,
"offHeapManageMemoryMax": 0,
"offHeapManageMemoryUsed": 0,
"networkMemoryMax": 107413504,
"networkMemoryUsed": 0
}{code}
h5. merge information to match Taskmanager’s memory composition
{code:java}
public static TaskManagerResourceInfo create(HardwareDescription
hardwareDescription, TaskManagerMetricsInfo taskManagerMetrics,
TaskManagerResourceConfig taskManagerResourceConfig) {
long javaHeapAllocated = taskManagerMetrics.getHeapCommitted();
long javaHeapUsed = taskManagerMetrics.getHeapUsed();
long pageSize = taskManagerResourceConfig.getSizeOfMemorySegment();
long heapManageMemoryAllocated = 0L;
long heapManageMemoryUsed = 0L;
long offHeapManageMemoryAllocated = 0L;
long offHeapManageMemoryUsed = 0L;
long networkMemoryAllocated =
taskManagerMetrics.getMemorySegmentsTotal() * pageSize;
long networkMemoryUsed = (taskManagerMetrics.getMemorySegmentsTotal() -
taskManagerMetrics.getMemorySegmentsAvailable()) * pageSize;
long manageMemoryAllocated =
taskManagerMetrics.getManageMemorySegmentsTotal() * pageSize;
long manageMemoryUsed =
(taskManagerMetrics.getManageMemorySegmentsTotal() -
taskManagerMetrics.getManageMemorySegmentsAvailable()) * pageSize;
if(taskManagerResourceConfig.getManagedMemoryType().equalsIgnoreCase(MemoryType.HEAP.name(){
heapManageMemoryAllocated = manageMemoryAllocated;
heapManageMemoryUsed = manageMemoryUsed;
javaHeapAllocated = javaHeapAllocated - heapManageMemoryAllocated;
javaHeapUsed = taskManagerMetrics.getHeapUsed() -
heapManageMemoryUsed;
} else {
offHeapManageMemoryAllocated = manageMemoryAllocated;
offHeapManageMemoryUsed = manageMemoryUsed;
}
return new TaskManagerResourceInfo(-1.0d, -1.0d, javaHeapAllocated,
javaHeapUsed,
heapManageMemoryAllocated, heapManageMemoryUsed,
offHeapManageMemoryAllocated,
offHeapManageMemoryUsed, networkMemoryAllocated,
networkMemoryUsed);
}{code}
* cpuAllocated depend on flip-49 （TaskExecutorResourceSpec）
* cpuUsage = （metric
Status.JVM.CPU.Load[from|https://docs.oracle.com/javase/7/docs/jre/api/management/extension/com/sun/management/OperatingSystemMXBean.html#getProcessCpuLoad()]）
* cpuAllocated
> Add TaskManageResourceInfo which match the memory compositions of taskmanager
> -----------------------------------------------------------------------------
>
> Key: FLINK-14435
> URL: https://issues.apache.org/jira/browse/FLINK-14435
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / REST
> Reporter: lining
> Priority: Major
>
> * information from TaskExecutorResourceSpec in flip-49, add it to
> TaskExecutorRegistration.
> {code:json}
> public class TaskManagerResourceInfo {
> private final double cpuCores;
> private final long frameworkHeap;
> private final long frameworkOffHeap;
> private final long taskHeap;
> private final long taskOffHeap;
> private final long shuffleMemory;
> private final long managedMemory;
> private final long jvmMetaSpace;
> private final long jvmOverhead;
> private final long totalProcessMemory;
> }{code}
> * url: /taskmanagers/:taskmanagerid
> * response: add
> {code:java}
> resource: {
> cpuCores: 4,
> frameworkHeap: 134217728,
> frameworkOffHeap: 134217728,
> taskHeap: 181193928,
> taskOffHeap: 0,
> shuffleMemory: 33554432,
> managedMemory: 322122552,
> jvmMetaSpace: 134217728,
> jvmOverhead: 134217728,
> totalProcessMemory: 1073741824
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
