Kengo Seki created BIGTOP-3578:
----------------------------------
Summary: Solr smoke test fails on ppc64le
Key: BIGTOP-3578
URL: https://issues.apache.org/jira/browse/BIGTOP-3578
Project: Bigtop
Issue Type: Bug
Components: deployment, solr, tests
Reporter: Kengo Seki
[https://ci.bigtop.apache.org/view/Test/job/Bigtop-trunk-smoke-tests/259/COMPONENTS=hdfs.zookeeper.solrcloud@solr,OS=fedora-33-ppc64le-deploy/]
Solr smoke test fails with the following error.
{code:java}
> Task :bigtop-tests:smoke-tests:solr:test
Downloading
https://repo.maven.apache.org/maven2/org/codehaus/woodstox/wstx-asl/3.2.7/wstx-asl-3.2.7.pom
to /tmp/gradle_download5228771710184587059bin
Downloading
https://repo.maven.apache.org/maven2/javax/servlet/jsp/jsp-api/2.1/jsp-api-2.1.pom
to /tmp/gradle_download7723517104372501069bin
Downloading
https://repo.maven.apache.org/maven2/org/slf4j/slf4j-log4j12/1.7.10/slf4j-log4j12-1.7.10.pom
to /tmp/gradle_download5040172534686132912bin
Downloading
https://repo.maven.apache.org/maven2/javax/servlet/jsp/jsp-api/2.1/jsp-api-2.1.jar
to /tmp/gradle_download8433979289897434251bin
Downloading
https://repo.maven.apache.org/maven2/org/codehaus/woodstox/wstx-asl/3.2.7/wstx-asl-3.2.7.jar
to /tmp/gradle_download7704021333708043047bin
Downloading
https://repo.maven.apache.org/maven2/org/slf4j/slf4j-log4j12/1.7.10/slf4j-log4j12-1.7.10.jar
to /tmp/gradle_download963266749008733834bin
Caching disabled for task ':bigtop-tests:smoke-tests:solr:test' because:
Build cache is disabled
Task ':bigtop-tests:smoke-tests:solr:test' is not up-to-date because:
No history is available.
Starting process 'command '/usr/bin/solrctl''. Working directory:
/bigtop-home/bigtop-tests/smoke-tests/solr Command: /usr/bin/solrctl
instancedir --generate /tmp/smoke_data
Successfully started process 'command '/usr/bin/solrctl''
Starting process 'command '/usr/bin/solrctl''. Working directory:
/bigtop-home/bigtop-tests/smoke-tests/solr Command: /usr/bin/solrctl
instancedir --create smoke /tmp/smoke_data
Successfully started process 'command '/usr/bin/solrctl''
Uploading configs from /tmp/smoke_data/conf to
f028d5337d92.bigtop.apache.org:2181/solr. This may take up to a minute.
Starting process 'command '/usr/bin/solrctl''. Working directory:
/bigtop-home/bigtop-tests/smoke-tests/solr Command: /usr/bin/solrctl collection
--create smoke
Successfully started process 'command '/usr/bin/solrctl''
Error: can't discover Solr URI. Please specify it explicitly via --solr.
> Task :bigtop-tests:smoke-tests:solr:test FAILED
:bigtop-tests:smoke-tests:solr:test (Thread[Execution worker for ':',5,main])
completed. Took 11.357 secs.
FAILURE: Build failed with an exception.
{code}
I could reproduce this failure by running the test on the ppc64le server
directly. It looks like that solr-server didn't start due to a memory issue.
{code:java}
jenkins@bigtop1slave:~/bigtop/provisioner/docker$ ./docker-hadoop.sh --exec 1
jps
1073 NameNode
1747 GradleDaemon
1257 DataNode
553 QuorumPeerMain
2031 Jps
jenkins@bigtop1slave:~/bigtop/provisioner/docker$ ./docker-hadoop.sh --exec 1
systemctl status solr-server
● solr-server.service - LSB: Apache Solr as a Service
Loaded: loaded (/etc/rc.d/init.d/solr-server; generated)
Active: active (exited) since Fri 2021-08-13 06:22:02 UTC; 1min 30s ago
Docs: man:systemd-sysv-generator(8)
Aug 13 06:21:51 31fbfd1187ec systemd[1]: Starting LSB: Apache Solr as a
Service...
Aug 13 06:21:51 31fbfd1187ec su[841]: (to solr) root on none
Aug 13 06:21:52 31fbfd1187ec su[841]: pam_unix(su:session): session opened for
user solr(uid=993) by (uid=0)
Aug 13 06:22:02 31fbfd1187ec solr-server[931]: Starting Apache Solr server in
cloud mode[ OK ]
Aug 13 06:22:02 31fbfd1187ec systemd[1]: Started LSB: Apache Solr as a Service.
jenkins@bigtop1slave:~/bigtop/provisioner/docker$ ./docker-hadoop.sh --exec 1
cat /var/log/solr/solr-8983-console.log
The stack size specified is too small, Specify at least 328k
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
{code}
That was not resolved by extending memory limit to 8GB.
{code:java}
jenkins@bigtop1slave:~/bigtop/provisioner/docker$ docker stats --no-stream
CONTAINER ID NAME CPU % MEM USAGE / LIMIT
MEM % NET I/O BLOCK I/O PIDS
31fbfd1187ec 20210813_061339_r31566_bigtop_1 0.03% 882.9MiB / 8GiB
10.78% 253MB / 1.27MB 655kB / 259MB 28
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
