cdmikechen created SUBMARINE-1121:
-------------------------------------
Summary: Submarine-Operator Pod CrashLoopBackOff when create a
Submarine CRD
Key: SUBMARINE-1121
URL: https://issues.apache.org/jira/browse/SUBMARINE-1121
Project: Apache Submarine
Issue Type: Bug
Components: K8s Submitter
Reporter: cdmikechen
I use helm to create submarine in k8s, when I use Submarine CRD to create
server, submarine operator will crash.
Here is my CRD:
{code:yaml}
apiVersion: submarine.apache.org/v1alpha1
kind: Submarine
metadata:
name: example-submarine
spec:
version: "0.7.0-SNAPSHOT"
server:
image: "harbor.xxxx.com:5000/apache/submarine:server-0.7.0-SNAPSHOT" #
overwrite the image when development
replicas: 1
database:
image: "harbor.xxxx.com:5000/apache/submarine:database-0.7.0-SNAPSHOT" #
overwrite the image when development
replicas: 1
storageSize: "1Gi"
mysqlRootPasswordSecret: "root-pass-secret"
tensorboard:
enabled: true
storageSize: "10Gi"
mlflow:
enabled: true
storageSize: "10Gi"
minio:
enabled: true
storageSize: "10Gi"
{code}
Here is submarine operator error:
{code:go}
I1209 07:40:03.828576 1 controller_builder.go:98] Setting up event
handlers
I1209 07:40:03.828745 1 controller.go:138] Starting Submarine controller
I1209 07:40:03.828761 1 controller.go:141] Waiting for informer caches to
sync
W1209 07:40:03.849789 1 warnings.go:70] extensions/v1beta1 Ingress is
deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
W1209 07:40:03.875172 1 warnings.go:70] extensions/v1beta1 Ingress is
deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
I1209 07:40:03.929025 1 controller.go:146] Starting workers
I1209 07:40:03.929058 1 controller.go:152] Started workers
I1209 07:40:03.929097 1 controller.go:235] syncHandler:
submarine/example-submarine
I1209 07:40:03.929117 1 submarine_server.go:104] [createSubmarineServer]
I1209 07:40:03.965919 1 submarine_server.go:111] Create ServiceAccount:
E1209 07:40:03.966155 1 runtime.go:78] Observed a panic: "invalid memory
address or nil pointer dereference" (runtime error: invalid memory address or
nil pointer dereference)
goroutine 244 [running]:
k8s.io/apimachinery/pkg/util/runtime.logPanic(0x150ade0, 0x2185d30)
/go/pkg/mod/k8s.io/[email protected]/pkg/util/runtime/runtime.go:74
+0x95
k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0)
/go/pkg/mod/k8s.io/[email protected]/pkg/util/runtime/runtime.go:48
+0x86
panic(0x150ade0, 0x2185d30)
/usr/local/go/src/runtime/panic.go:965 +0x1b9
github.com/apache/submarine/submarine-cloud-v2/pkg/controller.(*Controller).checkSubmarineDependentsReady(0xc00027e400,
0xc000603200, 0x9d, 0xc0002a30e0, 0x11)
/usr/src/pkg/controller/controller.go:470 +0x159
github.com/apache/submarine/submarine-cloud-v2/pkg/controller.(*Controller).syncHandler(0xc00027e400,
0xc00068ab20, 0x1b, 0xc0004321b0, 0xc0004321b0)
/usr/src/pkg/controller/controller.go:274 +0x4a5
github.com/apache/submarine/submarine-cloud-v2/pkg/controller.(*Controller).processNextWorkItem.func1(0xc00027e400,
0x14aedc0, 0xc000768940, 0x0, 0x0)
/usr/src/pkg/controller/controller.go:189 +0xed
github.com/apache/submarine/submarine-cloud-v2/pkg/controller.(*Controller).processNextWorkItem(0xc00027e400,
0x203000)
/usr/src/pkg/controller/controller.go:199 +0x53
github.com/apache/submarine/submarine-cloud-v2/pkg/controller.(*Controller).runWorker(...)
/usr/src/pkg/controller/controller.go:163
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc000f4c8b0)
/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155 +0x5f
k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc000f4c8b0, 0x1870980,
0xc000f1f140, 0x1, 0xc00009ade0)
/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156 +0x9b
k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc000f4c8b0, 0x3b9aca00, 0x0,
0x175ba01, 0xc00009ade0)
/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133 +0x98
k8s.io/apimachinery/pkg/util/wait.Until(0xc000f4c8b0, 0x3b9aca00, 0xc00009ade0)
/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:90 +0x4d
created by
github.com/apache/submarine/submarine-cloud-v2/pkg/controller.(*Controller).Run
/usr/src/pkg/controller/controller.go:149 +0x2e5
panic: runtime error: invalid memory address or nil pointer dereference
[recovered]
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x460 pc=0x1355e39]
goroutine 244 [running]:
k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0)
/go/pkg/mod/k8s.io/[email protected]/pkg/util/runtime/runtime.go:55
+0x109
panic(0x150ade0, 0x2185d30)
/usr/local/go/src/runtime/panic.go:965 +0x1b9
github.com/apache/submarine/submarine-cloud-v2/pkg/controller.(*Controller).checkSubmarineDependentsReady(0xc00027e400,
0xc000603200, 0x9d, 0xc0002a30e0, 0x11)
/usr/src/pkg/controller/controller.go:470 +0x159
github.com/apache/submarine/submarine-cloud-v2/pkg/controller.(*Controller).syncHandler(0xc00027e400,
0xc00068ab20, 0x1b, 0xc0004321b0, 0xc0004321b0)
/usr/src/pkg/controller/controller.go:274 +0x4a5
github.com/apache/submarine/submarine-cloud-v2/pkg/controller.(*Controller).processNextWorkItem.func1(0xc00027e400,
0x14aedc0, 0xc000768940, 0x0, 0x0)
/usr/src/pkg/controller/controller.go:189 +0xed
github.com/apache/submarine/submarine-cloud-v2/pkg/controller.(*Controller).processNextWorkItem(0xc00027e400,
0x203000)
/usr/src/pkg/controller/controller.go:199 +0x53
github.com/apache/submarine/submarine-cloud-v2/pkg/controller.(*Controller).runWorker(...)
/usr/src/pkg/controller/controller.go:163
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc000f4c8b0)
/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155 +0x5f
k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc000f4c8b0, 0x1870980,
0xc000f1f140, 0x1, 0xc00009ade0)
/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156 +0x9b
k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc000f4c8b0, 0x3b9aca00, 0x0,
0x175ba01, 0xc00009ade0)
/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133 +0x98
k8s.io/apimachinery/pkg/util/wait.Until(0xc000f4c8b0, 0x3b9aca00, 0xc00009ade0)
/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:90 +0x4d
created by
github.com/apache/submarine/submarine-cloud-v2/pkg/controller.(*Controller).Run
/usr/src/pkg/controller/controller.go:149 +0x2e5
{code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
