surahman commented on pull request #3710:
URL: https://github.com/apache/incubator-heron/pull/3710#issuecomment-933893882
Hi @nicknezis, I have spun up the K8s Heron locally but only have 4 cores
and 4gb I can allocate. As such the `acking` topology remains in a `pending`
state. I will test further later. The topology appears to be launching just
fine, but I am unsure if the template is being used. No additional `Roles` or
`RoleBindings` were required:
<details>
<summary>Config Map of Pod Template</summary>
```bash
minikube kubectl -- get configmaps configmap-pod-template -o yaml
```
```yaml
apiVersion: v1
data:
pod_template.yaml: |
apiVersion: v1
kind: PodTemplate
metadata:
name: pod-template-example
namespace: default
template:
metadata:
name: acking-pod-template-example
kind: ConfigMap
metadata:
creationTimestamp: "2021-10-04T21:49:11Z"
name: configmap-pod-template
namespace: default
resourceVersion: "1021"
uid: da578cac-27cc-4378-8cf3-664a208fcd96
```
</details>
<details>
<summary>Heron Submit</summary>
```bash
kubernetes ~/.heron/examples/heron-api-examples.jar \
--verbose \
--config-property
heron.kubernetes.pod.template.configmap.name=configmap-pod-template.pod_template.yaml
\
org.apache.heron.examples.api.AckingTopology acking
```
```bash
[2021-10-04 17:54:40 -0400] [DEBUG]: Input Command Line Args:
{'cluster/[role]/[env]': 'kubernetes', 'topology-file-name':
'/home/saad/.heron/examples/heron-api-examples.jar', 'topology-class-name':
'org.apache.heron.examples.api.AckingTopology', 'config_path':
'/home/saad/.heron/conf', 'config_property':
['heron.kubernetes.pod.template.configmap.name=configmap-pod-template.pod_template.yaml'],
'deploy_deactivated': False, 'dry_run': False, 'dry_run_format':
'colored_table', 'extra_launch_classpath': '', 'release_yaml_file': '',
'service_url': '', 'topology_main_jvm_property': [], 'verbose': 'True',
'verbose_gc': False, 'subcommand': 'submit'}
[2021-10-04 17:54:40 -0400] [DEBUG]: Input Command Line Args:
{'cluster/[role]/[env]': 'kubernetes', 'topology-file-name':
'/home/saad/.heron/examples/heron-api-examples.jar', 'topology-class-name':
'org.apache.heron.examples.api.AckingTopology', 'config_path':
'/home/saad/.heron/conf', 'config_property':
['heron.kubernetes.pod.template.configmap.name=configmap-pod-template.pod_template.yaml'],
'deploy_deactivated': False, 'dry_run': False, 'dry_run_format':
'colored_table', 'extra_launch_classpath': '', 'release_yaml_file': '',
'service_url': '', 'topology_main_jvm_property': [], 'verbose': 'True',
'verbose_gc': False, 'subcommand': 'submit'}
[2021-10-04 17:54:40 -0400] [DEBUG]: Using cluster definition from file
/home/saad/.config/heron/kubernetes/cli.yaml
[2021-10-04 17:54:40 -0400] [DEBUG]: Processed Command Line Args:
{'topology-file-name': '/home/saad/.heron/examples/heron-api-examples.jar',
'topology-class-name': 'org.apache.heron.examples.api.AckingTopology',
'config_path': '/home/saad/.heron/conf', 'config_property':
['heron.kubernetes.pod.template.configmap.name=configmap-pod-template.pod_template.yaml'],
'deploy_deactivated': False, 'dry_run': False, 'dry_run_format':
'colored_table', 'extra_launch_classpath': '', 'release_yaml_file': '',
'service_url':
'http://localhost:8001/api/v1/namespaces/default/services/heron-apiserver:9000/proxy',
'topology_main_jvm_property': [], 'verbose': 'True', 'verbose_gc': False,
'subcommand': 'submit', 'cluster': 'kubernetes', 'role': 'saad', 'environ':
'default', 'submit_user': 'saad', 'deploy_mode': 'server'}
[2021-10-04 17:54:40 -0400] [DEBUG]: Submit Args {'topology-file-name':
'/home/saad/.heron/examples/heron-api-examples.jar', 'topology-class-name':
'org.apache.heron.examples.api.AckingTopology', 'config_path':
'/home/saad/.heron/conf', 'config_property':
['heron.kubernetes.pod.template.configmap.name=configmap-pod-template.pod_template.yaml'],
'deploy_deactivated': False, 'dry_run': False, 'dry_run_format':
'colored_table', 'extra_launch_classpath': '', 'release_yaml_file': '',
'service_url':
'http://localhost:8001/api/v1/namespaces/default/services/heron-apiserver:9000/proxy',
'topology_main_jvm_property': [], 'verbose': 'True', 'verbose_gc': False,
'subcommand': 'submit', 'cluster': 'kubernetes', 'role': 'saad', 'environ':
'default', 'submit_user': 'saad', 'deploy_mode': 'server'}
[2021-10-04 17:54:40 -0400] [DEBUG]: Invoking class using command:
`/usr/bin/java -client -Xmx1g -cp
'/home/saad/.heron/examples/heron-api-examples.jar:/home/saad/.heron/lib/third_party/*'
org.apache.heron.examples.api.AckingTopology acking`
[2021-10-04 17:54:40 -0400] [DEBUG]: Heron options:
{cmdline.topologydefn.tmpdirectory=/tmp/tmpvj0pjzzc,cmdline.topology.initial.state=RUNNING,cmdline.topology.role=saad,cmdline.topology.environment=default,cmdline.topology.cluster=kubernetes,cmdline.topology.file_name=/home/saad/.heron/examples/heron-api-examples.jar,cmdline.topology.class_name=org.apache.heron.examples.api.AckingTopology,cmdline.topology.submit_user=saad}
[2021-10-04 17:54:40 -0400] [DEBUG]: Topology config: kvs {
key: "topology.component.rammap"
value: "word:1073741824,exclaim1:1073741824"
type: STRING_VALUE
}
kvs {
key: "topology.team.environment"
serialized_value: "\254\355\000\005t\000\007default"
type: JAVA_SERIALIZED_VALUE
}
kvs {
key: "topology.container.disk"
value: "2147483648"
type: STRING_VALUE
}
kvs {
key: "topology.container.ram"
value: "4294967296"
type: STRING_VALUE
}
kvs {
key: "topology.enable.message.timeouts"
value: "true"
type: STRING_VALUE
}
kvs {
key: "topology.serializer.classname"
value: "org.apache.heron.api.serializer.KryoSerializer"
type: STRING_VALUE
}
kvs {
key: "topology.debug"
value: "true"
type: STRING_VALUE
}
kvs {
key: "topology.max.spout.pending"
value: "1000000000"
type: STRING_VALUE
}
kvs {
key: "topology.container.cpu"
value: "3.0"
type: STRING_VALUE
}
kvs {
key: "topology.stateful.spill.state"
value: "false"
type: STRING_VALUE
}
kvs {
key: "topology.name"
value: "acking"
type: STRING_VALUE
}
kvs {
key: "topology.team.name"
value: "saad"
type: STRING_VALUE
}
kvs {
key: "topology.stateful.spill.state.location"
value: "./spilled-state/"
type: STRING_VALUE
}
kvs {
key: "topology.component.parallelism"
value: "1"
type: STRING_VALUE
}
kvs {
key: "topology.stmgrs"
value: "2"
type: STRING_VALUE
}
kvs {
key: "topology.worker.childopts"
value: "-XX:+HeapDumpOnOutOfMemoryError"
type: STRING_VALUE
}
kvs {
key: "topology.reliability.mode"
value: "ATLEAST_ONCE"
type: STRING_VALUE
}
kvs {
key: "topology.message.timeout.secs"
value: "10"
type: STRING_VALUE
}
[2021-10-04 17:54:40 -0400] [DEBUG]: Component config:
[2021-10-04 17:54:40 -0400] [DEBUG]: word => kvs {
key: "topology.component.parallelism"
value: "2"
type: STRING_VALUE
}
[2021-10-04 17:54:40 -0400] [DEBUG]: exclaim1 => kvs {
key: "topology.component.parallelism"
value: "2"
type: STRING_VALUE
}
[2021-10-04 17:54:40 -0400] [INFO]: Launching topology: 'acking'
[2021-10-04 17:54:40 -0400] [INFO]: {'topology-file-name':
'/home/saad/.heron/examples/heron-api-examples.jar', 'topology-class-name':
'org.apache.heron.examples.api.AckingTopology', 'config_path':
'/home/saad/.heron/conf', 'config_property':
['heron.kubernetes.pod.template.configmap.name=configmap-pod-template.pod_template.yaml'],
'deploy_deactivated': False, 'dry_run': False, 'dry_run_format':
'colored_table', 'extra_launch_classpath': '', 'release_yaml_file':
'/home/saad/.heron/release.yaml', 'service_url':
'http://localhost:8001/api/v1/namespaces/default/services/heron-apiserver:9000/proxy',
'topology_main_jvm_property': [], 'verbose': 'True', 'verbose_gc': False,
'subcommand': 'submit', 'cluster': 'kubernetes', 'role': 'saad', 'environ':
'default', 'submit_user': 'saad', 'deploy_mode': 'server'}
[2021-10-04 17:54:40 -0400] [DEBUG]: Starting new HTTP connection (1):
localhost
[2021-10-04 17:54:43 -0400] [DEBUG]: http://localhost:8001 "POST
/api/v1/namespaces/default/services/heron-apiserver:9000/proxy/api/v1/topologies
HTTP/1.1" 201 78
[2021-10-04 17:54:43 -0400] [INFO]: Successfully launched topology 'acking'
[2021-10-04 17:54:43 -0400] [DEBUG]: Elapsed time: 2.732s.
```
</details>
<details>
<summary>Describe Pods</summary>
```bash
Name: acking-0
Namespace: default
Priority: 0
Node: <none>
Labels: app=heron
controller-revision-hash=acking-566bc767b6
statefulset.kubernetes.io/pod-name=acking-0
topology=acking
Annotations: prometheus.io/port: 8080
prometheus.io/scrape: true
Status: Pending
IP:
IPs: <none>
Controlled By: StatefulSet/acking
Containers:
executor:
Image: apache/heron:testbuild
Ports: 6002/TCP, 6003/TCP, 6001/TCP, 6009/TCP, 6008/TCP, 6004/TCP,
6006/TCP, 6007/TCP, 6005/TCP
Host Ports: 0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/TCP,
0/TCP
Command:
sh
-c
./heron-core/bin/heron-downloader-config kubernetes &&
./heron-core/bin/heron-downloader
distributedlog://zookeeper:2181/heronbkdl/acking-saad-tag-0--3038063642361095900.tar.gz
. && SHARD_ID=${POD_NAME##*-} && echo shardId=${SHARD_ID} &&
./heron-core/bin/heron-executor --topology-name=acking
--topology-id=acking56b40818-10c0-4cda-ac3a-2c27303fef23
--topology-defn-file=acking.defn --state-manager-connection=zookeeper:2181
--state-manager-root=/heron
--state-manager-config-file=./heron-conf/statemgr.yaml
--tmanager-binary=./heron-core/bin/heron-tmanager
--stmgr-binary=./heron-core/bin/heron-stmgr
--metrics-manager-classpath=./heron-core/lib/metricsmgr/*
--instance-jvm-opts="LVhYOitIZWFwRHVtcE9uT3V0T2ZNZW1vcnlFcnJvcg(61)(61)"
--classpath=heron-api-examples.jar
--heron-internals-config-file=./heron-conf/heron_internals.yaml
--override-config-file=./heron-conf/override.yaml
--component-ram-map=exclaim1:1073741824,word:1073741824 --component-jvm-opts=""
--pkg-type=jar --topology-b
inary-file=heron-api-examples.jar --heron-java-home=$JAVA_HOME
--heron-shell-binary=./heron-core/bin/heron-shell --cluster=kubernetes
--role=saad --environment=default
--instance-classpath=./heron-core/lib/instance/*
--metrics-sinks-config-file=./heron-conf/metrics_sinks.yaml
--scheduler-classpath=./heron-core/lib/scheduler/*:./heron-core/lib/packing/*:./heron-core/lib/statemgr/*
--python-instance-binary=./heron-core/bin/heron-python-instance
--cpp-instance-binary=./heron-core/bin/heron-cpp-instance
--metricscache-manager-classpath=./heron-core/lib/metricscachemgr/*
--metricscache-manager-mode=disabled --is-stateful=false
--checkpoint-manager-classpath=./heron-core/lib/ckptmgr/*:./heron-core/lib/statefulstorage/*:
--stateful-config-file=./heron-conf/stateful.yaml
--checkpoint-manager-ram=1073741824 --health-manager-mode=disabled
--health-manager-classpath=./heron-core/lib/healthmgr/* --shard=$SHARD_ID
--server-port=6001 --tmanager-controller-port=6002 --tmanager-stats-port=6003
--sh
ell-port=6004 --metrics-manager-port=6005 --scheduler-port=6006
--metricscache-manager-server-port=6007 --metricscache-manager-stats-port=6008
--checkpoint-manager-port=6009
Limits:
cpu: 3
memory: 4Gi
Requests:
cpu: 3
memory: 4Gi
Environment:
HOST: (v1:status.podIP)
POD_NAME: acking-0 (v1:metadata.name)
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from
kube-api-access-xfcf2 (ro)
Conditions:
Type Status
PodScheduled False
Volumes:
kube-api-access-xfcf2:
Type: Projected (a volume that contains injected data
from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
QoS Class: Guaranteed
Node-Selectors: <none>
Tolerations: node.alpha.kubernetes.io/notReady:NoExecute
op=Exists for 10s
node.alpha.kubernetes.io/unreachable:NoExecute
op=Exists for 10s
node.kubernetes.io/not-ready:NoExecute
op=Exists for 10s
node.kubernetes.io/unreachable:NoExecute
op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedScheduling 58s default-scheduler 0/1 nodes are
available: 1 Insufficient cpu.
Name: acking-1
Namespace: default
Priority: 0
Node: <none>
Labels: app=heron
controller-revision-hash=acking-566bc767b6
statefulset.kubernetes.io/pod-name=acking-1
topology=acking
Annotations: prometheus.io/port: 8080
prometheus.io/scrape: true
Status: Pending
IP:
IPs: <none>
Controlled By: StatefulSet/acking
Containers:
executor:
Image: apache/heron:testbuild
Ports: 6002/TCP, 6003/TCP, 6001/TCP, 6009/TCP, 6008/TCP, 6004/TCP,
6006/TCP, 6007/TCP, 6005/TCP
Host Ports: 0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/TCP,
0/TCP
Command:
sh
-c
./heron-core/bin/heron-downloader-config kubernetes &&
./heron-core/bin/heron-downloader
distributedlog://zookeeper:2181/heronbkdl/acking-saad-tag-0--3038063642361095900.tar.gz
. && SHARD_ID=${POD_NAME##*-} && echo shardId=${SHARD_ID} &&
./heron-core/bin/heron-executor --topology-name=acking
--topology-id=acking56b40818-10c0-4cda-ac3a-2c27303fef23
--topology-defn-file=acking.defn --state-manager-connection=zookeeper:2181
--state-manager-root=/heron
--state-manager-config-file=./heron-conf/statemgr.yaml
--tmanager-binary=./heron-core/bin/heron-tmanager
--stmgr-binary=./heron-core/bin/heron-stmgr
--metrics-manager-classpath=./heron-core/lib/metricsmgr/*
--instance-jvm-opts="LVhYOitIZWFwRHVtcE9uT3V0T2ZNZW1vcnlFcnJvcg(61)(61)"
--classpath=heron-api-examples.jar
--heron-internals-config-file=./heron-conf/heron_internals.yaml
--override-config-file=./heron-conf/override.yaml
--component-ram-map=exclaim1:1073741824,word:1073741824 --component-jvm-opts=""
--pkg-type=jar --topology-b
inary-file=heron-api-examples.jar --heron-java-home=$JAVA_HOME
--heron-shell-binary=./heron-core/bin/heron-shell --cluster=kubernetes
--role=saad --environment=default
--instance-classpath=./heron-core/lib/instance/*
--metrics-sinks-config-file=./heron-conf/metrics_sinks.yaml
--scheduler-classpath=./heron-core/lib/scheduler/*:./heron-core/lib/packing/*:./heron-core/lib/statemgr/*
--python-instance-binary=./heron-core/bin/heron-python-instance
--cpp-instance-binary=./heron-core/bin/heron-cpp-instance
--metricscache-manager-classpath=./heron-core/lib/metricscachemgr/*
--metricscache-manager-mode=disabled --is-stateful=false
--checkpoint-manager-classpath=./heron-core/lib/ckptmgr/*:./heron-core/lib/statefulstorage/*:
--stateful-config-file=./heron-conf/stateful.yaml
--checkpoint-manager-ram=1073741824 --health-manager-mode=disabled
--health-manager-classpath=./heron-core/lib/healthmgr/* --shard=$SHARD_ID
--server-port=6001 --tmanager-controller-port=6002 --tmanager-stats-port=6003
--sh
ell-port=6004 --metrics-manager-port=6005 --scheduler-port=6006
--metricscache-manager-server-port=6007 --metricscache-manager-stats-port=6008
--checkpoint-manager-port=6009
Limits:
cpu: 3
memory: 4Gi
Requests:
cpu: 3
memory: 4Gi
Environment:
HOST: (v1:status.podIP)
POD_NAME: acking-1 (v1:metadata.name)
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from
kube-api-access-c7vwm (ro)
Conditions:
Type Status
PodScheduled False
Volumes:
kube-api-access-c7vwm:
Type: Projected (a volume that contains injected data
from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
QoS Class: Guaranteed
Node-Selectors: <none>
Tolerations: node.alpha.kubernetes.io/notReady:NoExecute
op=Exists for 10s
node.alpha.kubernetes.io/unreachable:NoExecute
op=Exists for 10s
node.kubernetes.io/not-ready:NoExecute
op=Exists for 10s
node.kubernetes.io/unreachable:NoExecute
op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedScheduling 58s default-scheduler 0/1 nodes are
available: 1 Insufficient cpu.
Name: acking-2
Namespace: default
Priority: 0
Node: <none>
Labels: app=heron
controller-revision-hash=acking-566bc767b6
statefulset.kubernetes.io/pod-name=acking-2
topology=acking
Annotations: prometheus.io/port: 8080
prometheus.io/scrape: true
Status: Pending
IP:
IPs: <none>
Controlled By: StatefulSet/acking
Containers:
executor:
Image: apache/heron:testbuild
Ports: 6002/TCP, 6003/TCP, 6001/TCP, 6009/TCP, 6008/TCP, 6004/TCP,
6006/TCP, 6007/TCP, 6005/TCP
Host Ports: 0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/TCP,
0/TCP
Command:
sh
-c
./heron-core/bin/heron-downloader-config kubernetes &&
./heron-core/bin/heron-downloader
distributedlog://zookeeper:2181/heronbkdl/acking-saad-tag-0--3038063642361095900.tar.gz
. && SHARD_ID=${POD_NAME##*-} && echo shardId=${SHARD_ID} &&
./heron-core/bin/heron-executor --topology-name=acking
--topology-id=acking56b40818-10c0-4cda-ac3a-2c27303fef23
--topology-defn-file=acking.defn --state-manager-connection=zookeeper:2181
--state-manager-root=/heron
--state-manager-config-file=./heron-conf/statemgr.yaml
--tmanager-binary=./heron-core/bin/heron-tmanager
--stmgr-binary=./heron-core/bin/heron-stmgr
--metrics-manager-classpath=./heron-core/lib/metricsmgr/*
--instance-jvm-opts="LVhYOitIZWFwRHVtcE9uT3V0T2ZNZW1vcnlFcnJvcg(61)(61)"
--classpath=heron-api-examples.jar
--heron-internals-config-file=./heron-conf/heron_internals.yaml
--override-config-file=./heron-conf/override.yaml
--component-ram-map=exclaim1:1073741824,word:1073741824 --component-jvm-opts=""
--pkg-type=jar --topology-b
inary-file=heron-api-examples.jar --heron-java-home=$JAVA_HOME
--heron-shell-binary=./heron-core/bin/heron-shell --cluster=kubernetes
--role=saad --environment=default
--instance-classpath=./heron-core/lib/instance/*
--metrics-sinks-config-file=./heron-conf/metrics_sinks.yaml
--scheduler-classpath=./heron-core/lib/scheduler/*:./heron-core/lib/packing/*:./heron-core/lib/statemgr/*
--python-instance-binary=./heron-core/bin/heron-python-instance
--cpp-instance-binary=./heron-core/bin/heron-cpp-instance
--metricscache-manager-classpath=./heron-core/lib/metricscachemgr/*
--metricscache-manager-mode=disabled --is-stateful=false
--checkpoint-manager-classpath=./heron-core/lib/ckptmgr/*:./heron-core/lib/statefulstorage/*:
--stateful-config-file=./heron-conf/stateful.yaml
--checkpoint-manager-ram=1073741824 --health-manager-mode=disabled
--health-manager-classpath=./heron-core/lib/healthmgr/* --shard=$SHARD_ID
--server-port=6001 --tmanager-controller-port=6002 --tmanager-stats-port=6003
--sh
ell-port=6004 --metrics-manager-port=6005 --scheduler-port=6006
--metricscache-manager-server-port=6007 --metricscache-manager-stats-port=6008
--checkpoint-manager-port=6009
Limits:
cpu: 3
memory: 4Gi
Requests:
cpu: 3
memory: 4Gi
Environment:
HOST: (v1:status.podIP)
POD_NAME: acking-2 (v1:metadata.name)
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from
kube-api-access-snvvp (ro)
Conditions:
Type Status
PodScheduled False
Volumes:
kube-api-access-snvvp:
Type: Projected (a volume that contains injected data
from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
QoS Class: Guaranteed
Node-Selectors: <none>
Tolerations: node.alpha.kubernetes.io/notReady:NoExecute
op=Exists for 10s
node.alpha.kubernetes.io/unreachable:NoExecute
op=Exists for 10s
node.kubernetes.io/not-ready:NoExecute
op=Exists for 10s
node.kubernetes.io/unreachable:NoExecute
op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedScheduling 58s default-scheduler 0/1 nodes are
available: 1 Insufficient cpu.
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
