lampnick opened a new issue #3750:
URL: https://github.com/apache/rocketmq/issues/3750
**BUG REPORT**
1. Please describe the issue you observed:
When I publish message to rocketmq ,get the flow error message:
```
service not available now. It may be caused by one of the following reasons:
the broker's disk is full [CL: 0.31 CQ: 0.31 INDEX: 0.31], messages are put
to the slave, message store has been shut down, etc.
```
- What did you do (The steps to reproduce)?
Normal send message.
2. Please tell us about your environment:
disk
```
文件系统 容量 已用 可用 已用% 挂载点
/dev/vda1 99G 29G 66G 31% /
devtmpfs 16G 0 16G 0% /dev
tmpfs 16G 392K 16G 1% /dev/shm
tmpfs 16G 680K 16G 1% /run
tmpfs 16G 0 16G 0% /sys/fs/cgroup
tmpfs 3.1G 0 3.1G 0% /run/user/0
```
mem
```
total used free shared buff/cache
available
Mem: 31490 15751 13509 1 2229
15356
Swap: 0 0 0
```
top
```
top - 17:12:28 up 1 day, 1:18, 2 users, load average: 0.10, 0.08, 0.10
Tasks: 106 total, 1 running, 105 sleeping, 0 stopped, 0 zombie
%Cpu(s): 1.3 us, 0.8 sy, 0.0 ni, 97.6 id, 0.0 wa, 0.0 hi, 0.3 si, 0.0
st
KiB Mem : 32246744 total, 13831264 free, 16130896 used, 2284584 buff/cache
KiB Swap: 0 total, 0 free, 0 used. 15723900 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
26176 root 20 0 22.3g 12.7g 31716 S 9.6 41.2 4:39.26 java
2612 root 20 0 7961572 1.8g 15500 S 1.0 5.8 7:38.43 java
4247 root 20 0 11.4g 604868 13820 S 0.3 1.9 10:48.68 java
```
3. Other information (e.g. detailed explanation, logs, related issues,
suggestions how to fix, etc):
And I catch the store.log file,get the flow error messages:
```
tmqlogs]# grep "put commit log position info" -rin ./*
./store.log:4881:2022-01-12 14:06:16 WARN ReputMessageService - [BUG]put
commit log position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 0
times
./store.log:4917:2022-01-12 14:06:17 WARN ReputMessageService - [BUG]put
commit log position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 1
times
./store.log:4953:2022-01-12 14:06:18 WARN ReputMessageService - [BUG]put
commit log position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 2
times
./store.log:4989:2022-01-12 14:06:19 WARN ReputMessageService - [BUG]put
commit log position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 3
times
./store.log:5025:2022-01-12 14:06:20 WARN ReputMessageService - [BUG]put
commit log position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 4
times
./store.log:5061:2022-01-12 14:06:21 WARN ReputMessageService - [BUG]put
commit log position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 5
times
./store.log:5097:2022-01-12 14:06:22 WARN ReputMessageService - [BUG]put
commit log position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 6
times
./store.log:5133:2022-01-12 14:06:23 WARN ReputMessageService - [BUG]put
commit log position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 7
times
./store.log:5169:2022-01-12 14:06:24 WARN ReputMessageService - [BUG]put
commit log position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 8
times
./store.log:5205:2022-01-12 14:06:25 WARN ReputMessageService - [BUG]put
commit log position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 9
times
./store.log:5241:2022-01-12 14:06:26 WARN ReputMessageService - [BUG]put
commit log position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry
10 times
./store.log:5279:2022-01-12 14:06:27 WARN ReputMessageService - [BUG]put
commit log position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry
11 times
./store.log:5315:2022-01-12 14:06:28 WARN ReputMessageService - [BUG]put
commit log position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry
12 times
./store.log:5351:2022-01-12 14:06:29 WARN ReputMessageService - [BUG]put
commit log position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry
13 times
./store.log:5387:2022-01-12 14:06:30 WARN ReputMessageService - [BUG]put
commit log position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry
14 times
./store.log:5423:2022-01-12 14:06:31 WARN ReputMessageService - [BUG]put
commit log position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry
15 times
./store.log:5459:2022-01-12 14:06:32 WARN ReputMessageService - [BUG]put
commit log position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry
16 times
./store.log:5495:2022-01-12 14:06:33 WARN ReputMessageService - [BUG]put
commit log position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry
17 times
./store.log:5531:2022-01-12 14:06:34 WARN ReputMessageService - [BUG]put
commit log position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry
18 times
./store.log:5567:2022-01-12 14:06:35 WARN ReputMessageService - [BUG]put
commit log position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry
19 times
./store.log:5603:2022-01-12 14:06:36 WARN ReputMessageService - [BUG]put
commit log position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry
20 times
./store.log:5639:2022-01-12 14:06:37 WARN ReputMessageService - [BUG]put
commit log position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry
21 times
./store.log:5675:2022-01-12 14:06:38 WARN ReputMessageService - [BUG]put
commit log position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry
22 times
./store.log:5711:2022-01-12 14:06:39 WARN ReputMessageService - [BUG]put
commit log position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry
23 times
./store.log:5747:2022-01-12 14:06:40 WARN ReputMessageService - [BUG]put
commit log position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry
24 times
./store.log:5783:2022-01-12 14:06:41 WARN ReputMessageService - [BUG]put
commit log position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry
25 times
./store.log:5819:2022-01-12 14:06:42 WARN ReputMessageService - [BUG]put
commit log position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry
26 times
./store.log:5855:2022-01-12 14:06:43 WARN ReputMessageService - [BUG]put
commit log position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry
27 times
./store.log:5891:2022-01-12 14:06:44 WARN ReputMessageService - [BUG]put
commit log position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry
28 times
./store.log:5927:2022-01-12 14:06:45 WARN ReputMessageService - [BUG]put
commit log position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry
29 times
./store.log:13233:2022-01-12 16:26:04 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 0 times
./store.log:13275:2022-01-12 16:26:05 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 1 times
./store.log:13317:2022-01-12 16:26:06 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 2 times
./store.log:13359:2022-01-12 16:26:07 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 3 times
./store.log:13401:2022-01-12 16:26:08 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 4 times
./store.log:13443:2022-01-12 16:26:09 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 5 times
./store.log:13485:2022-01-12 16:26:10 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 6 times
./store.log:13527:2022-01-12 16:26:11 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 7 times
./store.log:13569:2022-01-12 16:26:12 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 8 times
./store.log:13611:2022-01-12 16:26:13 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 9 times
./store.log:13653:2022-01-12 16:26:14 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 10 times
./store.log:13695:2022-01-12 16:26:15 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 11 times
./store.log:13737:2022-01-12 16:26:16 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 12 times
./store.log:13779:2022-01-12 16:26:17 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 13 times
./store.log:13821:2022-01-12 16:26:18 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 14 times
./store.log:13863:2022-01-12 16:26:19 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 15 times
./store.log:13905:2022-01-12 16:26:20 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 16 times
./store.log:13947:2022-01-12 16:26:21 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 17 times
./store.log:13989:2022-01-12 16:26:22 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 18 times
./store.log:14031:2022-01-12 16:26:23 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 19 times
./store.log:14073:2022-01-12 16:26:24 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 20 times
./store.log:14115:2022-01-12 16:26:25 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 21 times
./store.log:14157:2022-01-12 16:26:26 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 22 times
./store.log:14199:2022-01-12 16:26:27 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 23 times
./store.log:14241:2022-01-12 16:26:28 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 24 times
./store.log:14283:2022-01-12 16:26:29 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 25 times
./store.log:14325:2022-01-12 16:26:30 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 26 times
./store.log:14367:2022-01-12 16:26:31 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 27 times
./store.log:14409:2022-01-12 16:26:32 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 28 times
./store.log:14451:2022-01-12 16:26:33 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 29 times
./store.log:21568:2022-01-12 16:28:43 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 0 times
./store.log:21610:2022-01-12 16:28:44 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 1 times
./store.log:21652:2022-01-12 16:28:45 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 2 times
./store.log:21694:2022-01-12 16:28:46 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 3 times
./store.log:21736:2022-01-12 16:28:47 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 4 times
./store.log:21778:2022-01-12 16:28:48 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 5 times
./store.log:21820:2022-01-12 16:28:49 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 6 times
./store.log:21862:2022-01-12 16:28:50 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 7 times
./store.log:21904:2022-01-12 16:28:51 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 8 times
./store.log:21946:2022-01-12 16:28:52 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 9 times
./store.log:21988:2022-01-12 16:28:53 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 10 times
./store.log:22030:2022-01-12 16:28:54 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 11 times
./store.log:22072:2022-01-12 16:28:55 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 12 times
./store.log:22114:2022-01-12 16:28:56 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 13 times
./store.log:22156:2022-01-12 16:28:57 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 14 times
./store.log:22198:2022-01-12 16:28:58 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 15 times
./store.log:22240:2022-01-12 16:28:59 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 16 times
./store.log:22282:2022-01-12 16:29:00 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 17 times
./store.log:22324:2022-01-12 16:29:01 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 18 times
./store.log:22366:2022-01-12 16:29:02 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 19 times
./store.log:22408:2022-01-12 16:29:03 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 20 times
./store.log:22450:2022-01-12 16:29:04 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 21 times
./store.log:22492:2022-01-12 16:29:05 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 22 times
./store.log:22534:2022-01-12 16:29:06 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 23 times
./store.log:22576:2022-01-12 16:29:07 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 24 times
./store.log:22618:2022-01-12 16:29:08 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 25 times
./store.log:22660:2022-01-12 16:29:09 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 26 times
./store.log:22702:2022-01-12 16:29:10 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 27 times
./store.log:22744:2022-01-12 16:29:11 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 28 times
./store.log:22786:2022-01-12 16:29:12 WARN main - [BUG]put commit log
position info to %RETRY%GID_xxxxxxxx-local:0 5546976 failed, retry 29 times
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
