timebertt opened a new issue #6704:
URL: https://github.com/apache/incubator-pinot/issues/6704
When installing pinot using the helm chart, the minion immediately goes into
`CrashLoopBackOff`.
Error logs from the minion instance:
```text
$ k logs pinot-minion-0
ERROR StatusLogger File not found in file system or classpath:
/opt/pinot/conf/pinot-minion-log4j2.xml
ERROR StatusLogger Reconfiguration failed: No configuration found for
'7852e922' at 'null' in 'null'
14:14:01.433 [Start a Pinot [MINION]] ERROR
org.apache.pinot.tools.admin.command.StartServiceManagerCommand - Failed to
start a Pinot [MINION] at 0.241 since launch
java.io.IOException: Unable to delete directory /var/pinot/minion/data.
at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1541)
~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:2270)
~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
at org.apache.pinot.minion.MinionStarter.start(MinionStarter.java:146)
~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
at
org.apache.pinot.tools.service.PinotServiceManager.startMinion(PinotServiceManager.java:162)
~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
at
org.apache.pinot.tools.service.PinotServiceManager.startRole(PinotServiceManager.java:97)
~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
at
org.apache.pinot.tools.admin.command.StartServiceManagerCommand$1.lambda$run$0(StartServiceManagerCommand.java:260)
~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
at
org.apache.pinot.tools.admin.command.StartServiceManagerCommand.startPinotService(StartServiceManagerCommand.java:286)
[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
at
org.apache.pinot.tools.admin.command.StartServiceManagerCommand.access$000(StartServiceManagerCommand.java:57)
[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
at
org.apache.pinot.tools.admin.command.StartServiceManagerCommand$1.run(StartServiceManagerCommand.java:260)
[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
```
Might be related to
https://github.com/apache/incubator-pinot/pull/6430/files#diff-f2f45beb0be7ab7259835411627d8aed40dff1b4a92e53ab186cd07e1a57d7feR146-R148
cc @fx19880617
## Steps to reproduce
1. Add helm repo:
```bash
helm repo add pinot
https://raw.githubusercontent.com/apache/incubator-pinot/master/kubernetes/helm
```
2. Install pinot chart using the following command and values:
```bash
helm install pinot -f values.yaml pinot/pinot --version 0.2.3
```
```yaml
image:
repository: apachepinot/pinot
tag: 0.6.0
cluster:
name: pinot
```
3. Check on the cluster's state:
```text
$ k get po
NAME READY STATUS RESTARTS AGE
pinot-broker-0 1/1 Running 1 2m15s
pinot-controller-0 1/1 Running 1 2m14s
pinot-minion-0 0/1 CrashLoopBackOff 4 2m14s
pinot-server-0 1/1 Running 0 2m14s
pinot-zookeeper-0 1/1 Running 0 2m15s
```
4. Observe `pinot-minion-0` fail with the following error log from above.
## Workaround
Setting the chart value `minion.dataDir` to some subdirectory of
`minion.persistence.mountPath` fixes the problem:
```yaml
minion:
dataDir: /var/pinot/minion/data/data
```
```
$ k get po
NAME READY STATUS RESTARTS AGE
pinot-broker-0 1/1 Running 1 6m43s
pinot-controller-0 1/1 Running 1 6m42s
pinot-minion-0 1/1 Running 0 114s
pinot-server-0 1/1 Running 0 6m42s
pinot-zookeeper-0 1/1 Running 0 6m43s
```
## Additional information
My cluster is running k8s `v1.20.2` on GCP using CSI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
