[
https://issues.apache.org/jira/browse/SUBMARINE-1121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Che-Kuang Chu reassigned SUBMARINE-1121:
----------------------------------------
Assignee: Che-Kuang Chu
> Submarine-Operator Pod CrashLoopBackOff when create a Submarine CRD
> -------------------------------------------------------------------
>
> Key: SUBMARINE-1121
> URL: https://issues.apache.org/jira/browse/SUBMARINE-1121
> Project: Apache Submarine
> Issue Type: Bug
> Components: K8s Submitter
> Reporter: cdmikechen
> Assignee: Che-Kuang Chu
> Priority: Major
>
> I use helm to create submarine in k8s, when I use Submarine CRD to create
> server, submarine operator will crash.
> Here is my CRD:
> {code:yaml}
> apiVersion: submarine.apache.org/v1alpha1
> kind: Submarine
> metadata:
> name: example-submarine
> spec:
> version: "0.7.0-SNAPSHOT"
> server:
> image: "harbor.xxxx.com:5000/apache/submarine:server-0.7.0-SNAPSHOT" #
> overwrite the image when development
> replicas: 1
> database:
> image: "harbor.xxxx.com:5000/apache/submarine:database-0.7.0-SNAPSHOT" #
> overwrite the image when development
> replicas: 1
> storageSize: "1Gi"
> mysqlRootPasswordSecret: "root-pass-secret"
> tensorboard:
> enabled: true
> storageSize: "10Gi"
> mlflow:
> enabled: true
> storageSize: "10Gi"
> minio:
> enabled: true
> storageSize: "10Gi"
> {code}
> Here is submarine operator error:
> {code:go}
> I1209 07:40:03.828576 1 controller_builder.go:98] Setting up event
> handlers
> I1209 07:40:03.828745 1 controller.go:138] Starting Submarine controller
> I1209 07:40:03.828761 1 controller.go:141] Waiting for informer caches
> to sync
> W1209 07:40:03.849789 1 warnings.go:70] extensions/v1beta1 Ingress is
> deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
> W1209 07:40:03.875172 1 warnings.go:70] extensions/v1beta1 Ingress is
> deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
> I1209 07:40:03.929025 1 controller.go:146] Starting workers
> I1209 07:40:03.929058 1 controller.go:152] Started workers
> I1209 07:40:03.929097 1 controller.go:235] syncHandler:
> submarine/example-submarine
> I1209 07:40:03.929117 1 submarine_server.go:104] [createSubmarineServer]
> I1209 07:40:03.965919 1 submarine_server.go:111] Create
> ServiceAccount:
> E1209 07:40:03.966155 1 runtime.go:78] Observed a panic: "invalid
> memory address or nil pointer dereference" (runtime error: invalid memory
> address or nil pointer dereference)
> goroutine 244 [running]:
> k8s.io/apimachinery/pkg/util/runtime.logPanic(0x150ade0, 0x2185d30)
> /go/pkg/mod/k8s.io/[email protected]/pkg/util/runtime/runtime.go:74
> +0x95
> k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0)
> /go/pkg/mod/k8s.io/[email protected]/pkg/util/runtime/runtime.go:48
> +0x86
> panic(0x150ade0, 0x2185d30)
> /usr/local/go/src/runtime/panic.go:965 +0x1b9
> github.com/apache/submarine/submarine-cloud-v2/pkg/controller.(*Controller).checkSubmarineDependentsReady(0xc00027e400,
> 0xc000603200, 0x9d, 0xc0002a30e0, 0x11)
> /usr/src/pkg/controller/controller.go:470 +0x159
> github.com/apache/submarine/submarine-cloud-v2/pkg/controller.(*Controller).syncHandler(0xc00027e400,
> 0xc00068ab20, 0x1b, 0xc0004321b0, 0xc0004321b0)
> /usr/src/pkg/controller/controller.go:274 +0x4a5
> github.com/apache/submarine/submarine-cloud-v2/pkg/controller.(*Controller).processNextWorkItem.func1(0xc00027e400,
> 0x14aedc0, 0xc000768940, 0x0, 0x0)
> /usr/src/pkg/controller/controller.go:189 +0xed
> github.com/apache/submarine/submarine-cloud-v2/pkg/controller.(*Controller).processNextWorkItem(0xc00027e400,
> 0x203000)
> /usr/src/pkg/controller/controller.go:199 +0x53
> github.com/apache/submarine/submarine-cloud-v2/pkg/controller.(*Controller).runWorker(...)
> /usr/src/pkg/controller/controller.go:163
> k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc000f4c8b0)
> /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155 +0x5f
> k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc000f4c8b0, 0x1870980,
> 0xc000f1f140, 0x1, 0xc00009ade0)
> /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156 +0x9b
> k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc000f4c8b0, 0x3b9aca00, 0x0,
> 0x175ba01, 0xc00009ade0)
> /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133 +0x98
> k8s.io/apimachinery/pkg/util/wait.Until(0xc000f4c8b0, 0x3b9aca00,
> 0xc00009ade0)
> /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:90 +0x4d
> created by
> github.com/apache/submarine/submarine-cloud-v2/pkg/controller.(*Controller).Run
> /usr/src/pkg/controller/controller.go:149 +0x2e5
> panic: runtime error: invalid memory address or nil pointer dereference
> [recovered]
> panic: runtime error: invalid memory address or nil pointer dereference
> [signal SIGSEGV: segmentation violation code=0x1 addr=0x460 pc=0x1355e39]
> goroutine 244 [running]:
> k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0)
> /go/pkg/mod/k8s.io/[email protected]/pkg/util/runtime/runtime.go:55
> +0x109
> panic(0x150ade0, 0x2185d30)
> /usr/local/go/src/runtime/panic.go:965 +0x1b9
> github.com/apache/submarine/submarine-cloud-v2/pkg/controller.(*Controller).checkSubmarineDependentsReady(0xc00027e400,
> 0xc000603200, 0x9d, 0xc0002a30e0, 0x11)
> /usr/src/pkg/controller/controller.go:470 +0x159
> github.com/apache/submarine/submarine-cloud-v2/pkg/controller.(*Controller).syncHandler(0xc00027e400,
> 0xc00068ab20, 0x1b, 0xc0004321b0, 0xc0004321b0)
> /usr/src/pkg/controller/controller.go:274 +0x4a5
> github.com/apache/submarine/submarine-cloud-v2/pkg/controller.(*Controller).processNextWorkItem.func1(0xc00027e400,
> 0x14aedc0, 0xc000768940, 0x0, 0x0)
> /usr/src/pkg/controller/controller.go:189 +0xed
> github.com/apache/submarine/submarine-cloud-v2/pkg/controller.(*Controller).processNextWorkItem(0xc00027e400,
> 0x203000)
> /usr/src/pkg/controller/controller.go:199 +0x53
> github.com/apache/submarine/submarine-cloud-v2/pkg/controller.(*Controller).runWorker(...)
> /usr/src/pkg/controller/controller.go:163
> k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc000f4c8b0)
> /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155 +0x5f
> k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc000f4c8b0, 0x1870980,
> 0xc000f1f140, 0x1, 0xc00009ade0)
> /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156 +0x9b
> k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc000f4c8b0, 0x3b9aca00, 0x0,
> 0x175ba01, 0xc00009ade0)
> /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133 +0x98
> k8s.io/apimachinery/pkg/util/wait.Until(0xc000f4c8b0, 0x3b9aca00,
> 0xc00009ade0)
> /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:90 +0x4d
> created by
> github.com/apache/submarine/submarine-cloud-v2/pkg/controller.(*Controller).Run
> /usr/src/pkg/controller/controller.go:149 +0x2e5
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
