I recently setup geo-replication between two sites, and everything appears to be working, however, if I do a "gluster volume geo-replication SourceVol DestVol status detail", I see that the "failures" column seems to be incrementing for each file it transfers. I've looked at the logs in /var/log/glusterfs/geo-replication/SourceVol/* , but nothing stood out to me.
I've verified that the MD5 sum matches on the files at the destination after it's been replicated, so I'm unsure what is triggering the failure. Aside from the incrementing failure column, it all looks good to me. All boxes in both clusters are CentOS 6.7, running glusterfs 3.7.5 My questions: 1) What's causing the failures to be logged (please let me know what additional logs to provide)? 2) How do I reset the failures counter? It stays at the current value even once the files are deleted from the source 3) Is there documentation somewhere I missed that explains what Entry, Data, Meta, and Failures are in the status detail view for geo-replication? 4) Should "gluster volume geo-replication status" show me anything on the destination? Mine simply outputs "No active geo-replication sessions", though the command on the source works fine. Thanks, Kris ----------- Here's an example of what I'm seeing: [[email protected] temp]$ mount | grep "gfs-prd.mgmt:/prd_items" gfs-prd.mgmt:/prd_items on /mnt/temp type fuse.glusterfs (rw,default_permissions,allow_other,max_read=131072) [[email protected] temp]$ pwd /mnt/temp [[email protected] temp]$ gluster volume geo-replication prd_items [email protected]::dr_items status detail MASTER NODE MASTER VOL MASTER BRICK SLAVE USER SLAVE SLAVE NODE STATUS CRAWL STATUS LAST_SYNCED ENTRY DATA META FAILURES CHECKPOINT TIME CHECKPOINT COMPLETED CHECKPOINT COMPLETION TIME -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- gfs-prd02.mgmt prd_items /data/glusterfs/prd_items_brick1/brick1 georep [email protected]::dr_items gfs-slc03.mgmt Active Changelog Crawl 2016-02-03 01:28:26 0 0 0 2 N/A N/A N/A gfs-prd03.mgmt prd_items /data/glusterfs/prd_items_brick1/brick1 georep [email protected]::dr_items gfs-slc02.mgmt Passive N/A N/A N/A N/A N/A N/A N/A N/A N/A [[email protected] temp]$ dd if=/dev/urandom of=test01 bs=1M count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.179957 s, 5.8 MB/s [[email protected] temp]$ dd if=/dev/urandom of=test02 bs=1M count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.181704 s, 5.8 MB/s [[email protected] temp]$ dd if=/dev/urandom of=test03 bs=1M count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.181575 s, 5.8 MB/s [[email protected] temp]$ gluster volume geo-replication prd_items [email protected]::dr_items status detail MASTER NODE MASTER VOL MASTER BRICK SLAVE USER SLAVE SLAVE NODE STATUS CRAWL STATUS LAST_SYNCED ENTRY DATA META FAILURES CHECKPOINT TIME CHECKPOINT COMPLETED CHECKPOINT COMPLETION TIME -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- gfs-prd02.mgmt prd_items /data/glusterfs/prd_items_brick1/brick1 georep [email protected]::dr_items gfs-slc03.mgmt Active Changelog Crawl 2016-02-03 01:29:11 0 0 0 5 N/A N/A N/A gfs-prd03.mgmt prd_items /data/glusterfs/prd_items_brick1/brick1 georep [email protected]::dr_items gfs-slc02.mgmt Passive N/A N/A N/A N/A N/A N/A N/A N/A N/A Snippet from /var/log/glusterfs/geo-replication/prd_items/ssh%3A%2F%2Fgeorep%4010.124.0.82%3Agluster%3A%2F%2F127.0.0.1%3Adr_items.log ============== [2016-02-03 01:26:14.702147] I [master(/data/glusterfs/prd_items_brick1/brick1):552:crawlwrap] _GMaster: 20 crawls, 1 turns [2016-02-03 01:26:14.718751] I [master(/data/glusterfs/prd_items_brick1/brick1):1095:crawl] _GMaster: slave's time: (1454462711, 0) [2016-02-03 01:27:15.371404] I [master(/data/glusterfs/prd_items_brick1/brick1):552:crawlwrap] _GMaster: 20 crawls, 1 turns [2016-02-03 01:27:15.379936] I [master(/data/glusterfs/prd_items_brick1/brick1):1095:crawl] _GMaster: slave's time: (1454462771, 0) [2016-02-03 01:27:58.15177] I [master(/data/glusterfs/prd_items_brick1/brick1):1095:crawl] _GMaster: slave's time: (1454462831, 0) [2016-02-03 01:27:59.54204] W [master(/data/glusterfs/prd_items_brick1/brick1):1041:process] _GMaster: incomplete sync, retrying changelogs: CHANGELOG.1454462877 [2016-02-03 01:28:00.101433] W [master(/data/glusterfs/prd_items_brick1/brick1):1041:process] _GMaster: incomplete sync, retrying changelogs: CHANGELOG.1454462877 [2016-02-03 01:28:01.104391] W [master(/data/glusterfs/prd_items_brick1/brick1):1041:process] _GMaster: incomplete sync, retrying changelogs: CHANGELOG.1454462877 [2016-02-03 01:28:02.312911] W [master(/data/glusterfs/prd_items_brick1/brick1):1041:process] _GMaster: incomplete sync, retrying changelogs: CHANGELOG.1454462877 [2016-02-03 01:28:03.356967] W [master(/data/glusterfs/prd_items_brick1/brick1):1041:process] _GMaster: incomplete sync, retrying changelogs: CHANGELOG.1454462877 [2016-02-03 01:28:04.364518] W [master(/data/glusterfs/prd_items_brick1/brick1):1041:process] _GMaster: incomplete sync, retrying changelogs: CHANGELOG.1454462877 [2016-02-03 01:28:05.565407] W [master(/data/glusterfs/prd_items_brick1/brick1):1041:process] _GMaster: incomplete sync, retrying changelogs: CHANGELOG.1454462877 [2016-02-03 01:28:06.610732] W [master(/data/glusterfs/prd_items_brick1/brick1):1041:process] _GMaster: incomplete sync, retrying changelogs: CHANGELOG.1454462877 [2016-02-03 01:28:07.620142] W [master(/data/glusterfs/prd_items_brick1/brick1):1041:process] _GMaster: incomplete sync, retrying changelogs: CHANGELOG.1454462877 [2016-02-03 01:28:08.818785] W [master(/data/glusterfs/prd_items_brick1/brick1):1010:process] _GMaster: changelogs CHANGELOG.1454462877 could not be processed - moving on... [2016-02-03 01:28:08.826875] W [master(/data/glusterfs/prd_items_brick1/brick1):1014:process] _GMaster: SKIPPED GFID = f5a44b23-cdd8-4278-92d1-1b2a16e8c9d5 [2016-02-03 01:28:15.94074] I [master(/data/glusterfs/prd_items_brick1/brick1):1095:crawl] _GMaster: slave's time: (1454462876, 0) [2016-02-03 01:28:18.669955] I [master(/data/glusterfs/prd_items_brick1/brick1):552:crawlwrap] _GMaster: 17 crawls, 3 turns [2016-02-03 01:28:27.720731] I [master(/data/glusterfs/prd_items_brick1/brick1):1095:crawl] _GMaster: slave's time: (1454462891, 0) [2016-02-03 01:28:58.387859] I [master(/data/glusterfs/prd_items_brick1/brick1):1095:crawl] _GMaster: slave's time: (1454462906, 0) [2016-02-03 01:28:59.360420] W [master(/data/glusterfs/prd_items_brick1/brick1):1041:process] _GMaster: incomplete sync, retrying changelogs: CHANGELOG.1454462937 [2016-02-03 01:29:00.430566] W [master(/data/glusterfs/prd_items_brick1/brick1):1041:process] _GMaster: incomplete sync, retrying changelogs: CHANGELOG.1454462937 [2016-02-03 01:29:01.610398] W [master(/data/glusterfs/prd_items_brick1/brick1):1041:process] _GMaster: incomplete sync, retrying changelogs: CHANGELOG.1454462937 [2016-02-03 01:29:02.667408] W [master(/data/glusterfs/prd_items_brick1/brick1):1041:process] _GMaster: incomplete sync, retrying changelogs: CHANGELOG.1454462937 [2016-02-03 01:29:03.692369] W [master(/data/glusterfs/prd_items_brick1/brick1):1041:process] _GMaster: incomplete sync, retrying changelogs: CHANGELOG.1454462937 [2016-02-03 01:29:04.947613] W [master(/data/glusterfs/prd_items_brick1/brick1):1041:process] _GMaster: incomplete sync, retrying changelogs: CHANGELOG.1454462937 [2016-02-03 01:29:05.932200] W [master(/data/glusterfs/prd_items_brick1/brick1):1041:process] _GMaster: incomplete sync, retrying changelogs: CHANGELOG.1454462937 [2016-02-03 01:29:06.947702] W [master(/data/glusterfs/prd_items_brick1/brick1):1041:process] _GMaster: incomplete sync, retrying changelogs: CHANGELOG.1454462937 [2016-02-03 01:29:08.186101] W [master(/data/glusterfs/prd_items_brick1/brick1):1041:process] _GMaster: incomplete sync, retrying changelogs: CHANGELOG.1454462937 [2016-02-03 01:29:09.203042] W [master(/data/glusterfs/prd_items_brick1/brick1):1010:process] _GMaster: changelogs CHANGELOG.1454462937 could not be processed - moving on... [2016-02-03 01:29:09.205854] W [master(/data/glusterfs/prd_items_brick1/brick1):1014:process] _GMaster: SKIPPED GFID = c99a05c9-9e7d-41cf-b148-9bec91f527c8 [2016-02-03 01:29:15.432744] I [master(/data/glusterfs/prd_items_brick1/brick1):1095:crawl] _GMaster: slave's time: (1454462936, 0) [2016-02-03 01:29:16.743381] W [master(/data/glusterfs/prd_items_brick1/brick1):1041:process] _GMaster: incomplete sync, retrying changelogs: CHANGELOG.1454462952 [2016-02-03 01:29:17.944102] W [master(/data/glusterfs/prd_items_brick1/brick1):1041:process] _GMaster: incomplete sync, retrying changelogs: CHANGELOG.1454462952 [2016-02-03 01:29:18.970493] W [master(/data/glusterfs/prd_items_brick1/brick1):1041:process] _GMaster: incomplete sync, retrying changelogs: CHANGELOG.1454462952 [2016-02-03 01:29:20.77699] W [master(/data/glusterfs/prd_items_brick1/brick1):1041:process] _GMaster: incomplete sync, retrying changelogs: CHANGELOG.1454462952 [2016-02-03 01:29:21.197900] W [master(/data/glusterfs/prd_items_brick1/brick1):1041:process] _GMaster: incomplete sync, retrying changelogs: CHANGELOG.1454462952 [2016-02-03 01:29:22.259825] W [master(/data/glusterfs/prd_items_brick1/brick1):1041:process] _GMaster: incomplete sync, retrying changelogs: CHANGELOG.1454462952 [2016-02-03 01:29:23.349780] W [master(/data/glusterfs/prd_items_brick1/brick1):1041:process] _GMaster: incomplete sync, retrying changelogs: CHANGELOG.1454462952 [2016-02-03 01:29:24.469357] W [master(/data/glusterfs/prd_items_brick1/brick1):1041:process] _GMaster: incomplete sync, retrying changelogs: CHANGELOG.1454462952 [2016-02-03 01:29:25.526793] W [master(/data/glusterfs/prd_items_brick1/brick1):1041:process] _GMaster: incomplete sync, retrying changelogs: CHANGELOG.1454462952 [2016-02-03 01:29:26.612977] W [master(/data/glusterfs/prd_items_brick1/brick1):1010:process] _GMaster: changelogs CHANGELOG.1454462952 could not be processed - moving on... [2016-02-03 01:29:26.615757] W [master(/data/glusterfs/prd_items_brick1/brick1):1014:process] _GMaster: SKIPPED GFID = a7dc6c28-9239-44af-909f-a2938bb387b6,550ffdd3-4636-45b8-b278-fbfe120b9d85 [2016-02-03 01:29:29.828999] I [master(/data/glusterfs/prd_items_brick1/brick1):552:crawlwrap] _GMaster: 16 crawls, 3 turns [2016-02-03 01:30:15.98556] I [master(/data/glusterfs/prd_items_brick1/brick1):1095:crawl] _GMaster: slave's time: (1454462951, 0) [2016-02-03 01:30:30.709733] I [master(/data/glusterfs/prd_items_brick1/brick1):552:crawlwrap] _GMaster: 20 crawls, 1 turns [2016-02-03 01:31:12.918898] I [master(/data/glusterfs/prd_items_brick1/brick1):1095:crawl] _GMaster: slave's time: (1454463011, 0) [2016-02-03 01:31:31.423204] I [master(/data/glusterfs/prd_items_brick1/brick1):552:crawlwrap] _GMaster: 20 crawls, 1 turns Snippet from /var/log/glusterfs/geo-replication/prd_items/ssh%3A%2F%2Fgeorep%4010.124.0.82%3Agluster%3A%2F%2F127.0.0.1%3Adr_items.%2Fdata%2Fglusterfs%2Fprd_items_brick1%2Fbrick1-changes.log =============== [2016-02-03 01:26:12.472345] T [rpcsvc-auth.c:404:rpcsvc_auth_request_init] 0-rpc-service: Auth handler: AUTH_GLUSTERFS-v2 [2016-02-03 01:26:12.472396] T [rpcsvc.c:495:rpcsvc_request_create] 0-rpc-service: received rpc-message (XID: 0x1721, Ver: 2, Program: 1886350951, ProgVers: 1, Proc: 1) from rpc-transport (socket.gfchangelog) [2016-02-03 01:26:12.472415] T [auth-glusterfs.c:242:auth_glusterfs_v2_authenticate] 0-rpc-service: Auth Info: pid: 0, uid: 0, gid: 0, owner: 00000000 [2016-02-03 01:26:12.472426] T [rpcsvc.c:316:rpcsvc_program_actor] 0-rpc-service: Actor found: LIBGFCHANGELOG REBORP - CHANGELOG EVENT HANDLER for [2016-02-03 01:26:12.472440] D [gf-changelog-reborp.c:406:gf_changelog_event_handler] 0-gfchangelog: seq: 2478 [/data/glusterfs/prd_items_brick1/brick1] (time: 1454462772.472206), (vec: 1, len: 4104) [2016-02-03 01:26:12.472467] T [rpcsvc.c:1273:rpcsvc_submit_generic] 0-rpc-service: Tx message: 8 [2016-02-03 01:26:12.472481] T [rpcsvc.c:863:rpcsvc_record_build_header] 0-rpc-service: Reply fraglen 32, payload: 8, rpc hdr: 24 [2016-02-03 01:26:12.472502] T [rpcsvc.c:1321:rpcsvc_submit_generic] 0-rpc-service: submitted reply for rpc-message (XID: 0x6b9, Program: LIBGFCHANGELOG REBORP, ProgVers: 1, Proc: 1) to rpc-transport (socket.gfchangelog) [2016-02-03 01:26:14.952267] D [gf-changelog-api.c:53:gf_changelog_done] 0-gfchangelog: moving /var/lib/misc/glusterfsd/prd_items/ssh%3A%2F%2Fgeorep%4010.124.0.82%3Agluster%3A%2F%2F127.0.0.1%3Adr_items/ffdfc13dfaa3d74827a7b7cb855c6ce0/.processing/CHANGELOG.1454462772 to processed directory [2016-02-03 01:27:12.531138] T [rpcsvc-auth.c:404:rpcsvc_auth_request_init] 0-rpc-service: Auth handler: AUTH_GLUSTERFS-v2 [2016-02-03 01:27:12.531177] T [rpcsvc.c:495:rpcsvc_request_create] 0-rpc-service: received rpc-message (XID: 0x1722, Ver: 2, Program: 1886350951, ProgVers: 1, Proc: 1) from rpc-transport (socket.gfchangelog) [2016-02-03 01:27:12.531195] T [auth-glusterfs.c:242:auth_glusterfs_v2_authenticate] 0-rpc-service: Auth Info: pid: 0, uid: 0, gid: 0, owner: 00000000 [2016-02-03 01:27:12.531207] T [rpcsvc.c:316:rpcsvc_program_actor] 0-rpc-service: Actor found: LIBGFCHANGELOG REBORP - CHANGELOG EVENT HANDLER for [2016-02-03 01:27:12.531220] D [gf-changelog-reborp.c:406:gf_changelog_event_handler] 0-gfchangelog: seq: 2479 [/data/glusterfs/prd_items_brick1/brick1] (time: 1454462832.531035), (vec: 1, len: 4104) [2016-02-03 01:27:12.531241] T [rpcsvc.c:1273:rpcsvc_submit_generic] 0-rpc-service: Tx message: 8 [2016-02-03 01:27:12.531256] T [rpcsvc.c:863:rpcsvc_record_build_header] 0-rpc-service: Reply fraglen 32, payload: 8, rpc hdr: 24 [2016-02-03 01:27:12.531274] T [rpcsvc.c:1321:rpcsvc_submit_generic] 0-rpc-service: submitted reply for rpc-message (XID: 0x6ba, Program: LIBGFCHANGELOG REBORP, ProgVers: 1, Proc: 1) to rpc-transport (socket.gfchangelog) [2016-02-03 01:27:15.660249] D [gf-changelog-api.c:53:gf_changelog_done] 0-gfchangelog: moving /var/lib/misc/glusterfsd/prd_items/ssh%3A%2F%2Fgeorep%4010.124.0.82%3Agluster%3A%2F%2F127.0.0.1%3Adr_items/ffdfc13dfaa3d74827a7b7cb855c6ce0/.processing/CHANGELOG.1454462832 to processed directory [2016-02-03 01:27:57.576147] T [rpcsvc-auth.c:404:rpcsvc_auth_request_init] 0-rpc-service: Auth handler: AUTH_GLUSTERFS-v2 [2016-02-03 01:27:57.576198] T [rpcsvc.c:495:rpcsvc_request_create] 0-rpc-service: received rpc-message (XID: 0x1723, Ver: 2, Program: 1886350951, ProgVers: 1, Proc: 1) from rpc-transport (socket.gfchangelog) [2016-02-03 01:27:57.576217] T [auth-glusterfs.c:242:auth_glusterfs_v2_authenticate] 0-rpc-service: Auth Info: pid: 0, uid: 0, gid: 0, owner: 00000000 [2016-02-03 01:27:57.576230] T [rpcsvc.c:316:rpcsvc_program_actor] 0-rpc-service: Actor found: LIBGFCHANGELOG REBORP - CHANGELOG EVENT HANDLER for [2016-02-03 01:27:57.576246] D [gf-changelog-reborp.c:406:gf_changelog_event_handler] 0-gfchangelog: seq: 2480 [/data/glusterfs/prd_items_brick1/brick1] (time: 1454462877.576035), (vec: 1, len: 4104) [2016-02-03 01:27:57.576282] T [rpcsvc.c:1273:rpcsvc_submit_generic] 0-rpc-service: Tx message: 8 [2016-02-03 01:27:57.576297] T [rpcsvc.c:863:rpcsvc_record_build_header] 0-rpc-service: Reply fraglen 32, payload: 8, rpc hdr: 24 [2016-02-03 01:27:57.576327] T [rpcsvc.c:1321:rpcsvc_submit_generic] 0-rpc-service: submitted reply for rpc-message (XID: 0x6bb, Program: LIBGFCHANGELOG REBORP, ProgVers: 1, Proc: 1) to rpc-transport (socket.gfchangelog) [2016-02-03 01:28:08.833040] D [gf-changelog-api.c:53:gf_changelog_done] 0-gfchangelog: moving /var/lib/misc/glusterfsd/prd_items/ssh%3A%2F%2Fgeorep%4010.124.0.82%3Agluster%3A%2F%2F127.0.0.1%3Adr_items/ffdfc13dfaa3d74827a7b7cb855c6ce0/.processing/CHANGELOG.1454462877 to processed directory [2016-02-03 01:28:12.589377] T [rpcsvc-auth.c:404:rpcsvc_auth_request_init] 0-rpc-service: Auth handler: AUTH_GLUSTERFS-v2 [2016-02-03 01:28:12.589420] T [rpcsvc.c:495:rpcsvc_request_create] 0-rpc-service: received rpc-message (XID: 0x1724, Ver: 2, Program: 1886350951, ProgVers: 1, Proc: 1) from rpc-transport (socket.gfchangelog) [2016-02-03 01:28:12.589439] T [auth-glusterfs.c:242:auth_glusterfs_v2_authenticate] 0-rpc-service: Auth Info: pid: 0, uid: 0, gid: 0, owner: 00000000 [2016-02-03 01:28:12.589451] T [rpcsvc.c:316:rpcsvc_program_actor] 0-rpc-service: Actor found: LIBGFCHANGELOG REBORP - CHANGELOG EVENT HANDLER for [2016-02-03 01:28:12.589466] D [gf-changelog-reborp.c:406:gf_changelog_event_handler] 0-gfchangelog: seq: 2481 [/data/glusterfs/prd_items_brick1/brick1] (time: 1454462892.589270), (vec: 1, len: 4104) [2016-02-03 01:28:12.589487] T [rpcsvc.c:1273:rpcsvc_submit_generic] 0-rpc-service: Tx message: 8 [2016-02-03 01:28:12.589500] T [rpcsvc.c:863:rpcsvc_record_build_header] 0-rpc-service: Reply fraglen 32, payload: 8, rpc hdr: 24 [2016-02-03 01:28:12.589521] T [rpcsvc.c:1321:rpcsvc_submit_generic] 0-rpc-service: submitted reply for rpc-message (XID: 0x6bc, Program: LIBGFCHANGELOG REBORP, ProgVers: 1, Proc: 1) to rpc-transport (socket.gfchangelog) [2016-02-03 01:28:15.398144] D [gf-changelog-api.c:53:gf_changelog_done] 0-gfchangelog: moving /var/lib/misc/glusterfsd/prd_items/ssh%3A%2F%2Fgeorep%4010.124.0.82%3Agluster%3A%2F%2F127.0.0.1%3Adr_items/ffdfc13dfaa3d74827a7b7cb855c6ce0/.processing/CHANGELOG.1454462892 to processed directory [2016-02-03 01:28:27.604453] T [rpcsvc-auth.c:404:rpcsvc_auth_request_init] 0-rpc-service: Auth handler: AUTH_GLUSTERFS-v2 [2016-02-03 01:28:27.604510] T [rpcsvc.c:495:rpcsvc_request_create] 0-rpc-service: received rpc-message (XID: 0x1725, Ver: 2, Program: 1886350951, ProgVers: 1, Proc: 1) from rpc-transport (socket.gfchangelog) [2016-02-03 01:28:27.604530] T [auth-glusterfs.c:242:auth_glusterfs_v2_authenticate] 0-rpc-service: Auth Info: pid: 0, uid: 0, gid: 0, owner: 00000000 [2016-02-03 01:28:27.604542] T [rpcsvc.c:316:rpcsvc_program_actor] 0-rpc-service: Actor found: LIBGFCHANGELOG REBORP - CHANGELOG EVENT HANDLER for [2016-02-03 01:28:27.604557] D [gf-changelog-reborp.c:406:gf_changelog_event_handler] 0-gfchangelog: seq: 2482 [/data/glusterfs/prd_items_brick1/brick1] (time: 1454462907.604293), (vec: 1, len: 4104) [2016-02-03 01:28:27.604580] T [rpcsvc.c:1273:rpcsvc_submit_generic] 0-rpc-service: Tx message: 8 [2016-02-03 01:28:27.604594] T [rpcsvc.c:863:rpcsvc_record_build_header] 0-rpc-service: Reply fraglen 32, payload: 8, rpc hdr: 24 [2016-02-03 01:28:27.604615] T [rpcsvc.c:1321:rpcsvc_submit_generic] 0-rpc-service: submitted reply for rpc-message (XID: 0x6bd, Program: LIBGFCHANGELOG REBORP, ProgVers: 1, Proc: 1) to rpc-transport (socket.gfchangelog) [2016-02-03 01:28:28.066455] D [gf-changelog-api.c:53:gf_changelog_done] 0-gfchangelog: moving /var/lib/misc/glusterfsd/prd_items/ssh%3A%2F%2Fgeorep%4010.124.0.82%3Agluster%3A%2F%2F127.0.0.1%3Adr_items/ffdfc13dfaa3d74827a7b7cb855c6ce0/.processing/CHANGELOG.1454462907 to processed directory [2016-02-03 01:28:57.634864] T [rpcsvc-auth.c:404:rpcsvc_auth_request_init] 0-rpc-service: Auth handler: AUTH_GLUSTERFS-v2 [2016-02-03 01:28:57.634938] T [rpcsvc.c:495:rpcsvc_request_create] 0-rpc-service: received rpc-message (XID: 0x1726, Ver: 2, Program: 1886350951, ProgVers: 1, Proc: 1) from rpc-transport (socket.gfchangelog) [2016-02-03 01:28:57.635006] T [auth-glusterfs.c:242:auth_glusterfs_v2_authenticate] 0-rpc-service: Auth Info: pid: 0, uid: 0, gid: 0, owner: 00000000 [2016-02-03 01:28:57.635028] T [rpcsvc.c:316:rpcsvc_program_actor] 0-rpc-service: Actor found: LIBGFCHANGELOG REBORP - CHANGELOG EVENT HANDLER for [2016-02-03 01:28:57.635049] D [gf-changelog-reborp.c:406:gf_changelog_event_handler] 0-gfchangelog: seq: 2483 [/data/glusterfs/prd_items_brick1/brick1] (time: 1454462937.634652), (vec: 1, len: 4104) [2016-02-03 01:28:57.635079] T [rpcsvc.c:1273:rpcsvc_submit_generic] 0-rpc-service: Tx message: 8 [2016-02-03 01:28:57.635098] T [rpcsvc.c:863:rpcsvc_record_build_header] 0-rpc-service: Reply fraglen 32, payload: 8, rpc hdr: 24 [2016-02-03 01:28:57.635127] T [rpcsvc.c:1321:rpcsvc_submit_generic] 0-rpc-service: submitted reply for rpc-message (XID: 0x6be, Program: LIBGFCHANGELOG REBORP, ProgVers: 1, Proc: 1) to rpc-transport (socket.gfchangelog) [2016-02-03 01:29:09.211888] D [gf-changelog-api.c:53:gf_changelog_done] 0-gfchangelog: moving /var/lib/misc/glusterfsd/prd_items/ssh%3A%2F%2Fgeorep%4010.124.0.82%3Agluster%3A%2F%2F127.0.0.1%3Adr_items/ffdfc13dfaa3d74827a7b7cb855c6ce0/.processing/CHANGELOG.1454462937 to processed directory [2016-02-03 01:29:12.650651] T [rpcsvc-auth.c:404:rpcsvc_auth_request_init] 0-rpc-service: Auth handler: AUTH_GLUSTERFS-v2 [2016-02-03 01:29:12.650705] T [rpcsvc.c:495:rpcsvc_request_create] 0-rpc-service: received rpc-message (XID: 0x1727, Ver: 2, Program: 1886350951, ProgVers: 1, Proc: 1) from rpc-transport (socket.gfchangelog) [2016-02-03 01:29:12.650726] T [auth-glusterfs.c:242:auth_glusterfs_v2_authenticate] 0-rpc-service: Auth Info: pid: 0, uid: 0, gid: 0, owner: 00000000 [2016-02-03 01:29:12.650739] T [rpcsvc.c:316:rpcsvc_program_actor] 0-rpc-service: Actor found: LIBGFCHANGELOG REBORP - CHANGELOG EVENT HANDLER for [2016-02-03 01:29:12.650755] D [gf-changelog-reborp.c:406:gf_changelog_event_handler] 0-gfchangelog: seq: 2484 [/data/glusterfs/prd_items_brick1/brick1] (time: 1454462952.650534), (vec: 1, len: 4104) [2016-02-03 01:29:12.650787] T [rpcsvc.c:1273:rpcsvc_submit_generic] 0-rpc-service: Tx message: 8 [2016-02-03 01:29:12.650802] T [rpcsvc.c:863:rpcsvc_record_build_header] 0-rpc-service: Reply fraglen 32, payload: 8, rpc hdr: 24 [2016-02-03 01:29:12.650824] T [rpcsvc.c:1321:rpcsvc_submit_generic] 0-rpc-service: submitted reply for rpc-message (XID: 0x6bf, Program: LIBGFCHANGELOG REBORP, ProgVers: 1, Proc: 1) to rpc-transport (socket.gfchangelog) [2016-02-03 01:29:26.622432] D [gf-changelog-api.c:53:gf_changelog_done] 0-gfchangelog: moving /var/lib/misc/glusterfsd/prd_items/ssh%3A%2F%2Fgeorep%4010.124.0.82%3Agluster%3A%2F%2F127.0.0.1%3Adr_items/ffdfc13dfaa3d74827a7b7cb855c6ce0/.processing/CHANGELOG.1454462952 to processed directory [2016-02-03 01:30:12.693458] T [rpcsvc-auth.c:404:rpcsvc_auth_request_init] 0-rpc-service: Auth handler: AUTH_GLUSTERFS-v2 [2016-02-03 01:30:12.693499] T [rpcsvc.c:495:rpcsvc_request_create] 0-rpc-service: received rpc-message (XID: 0x1728, Ver: 2, Program: 1886350951, ProgVers: 1, Proc: 1) from rpc-transport (socket.gfchangelog) [2016-02-03 01:30:12.693528] T [auth-glusterfs.c:242:auth_glusterfs_v2_authenticate] 0-rpc-service: Auth Info: pid: 0, uid: 0, gid: 0, owner: 00000000 [2016-02-03 01:30:12.693544] T [rpcsvc.c:316:rpcsvc_program_actor] 0-rpc-service: Actor found: LIBGFCHANGELOG REBORP - CHANGELOG EVENT HANDLER for [2016-02-03 01:30:12.693562] D [gf-changelog-reborp.c:406:gf_changelog_event_handler] 0-gfchangelog: seq: 2485 [/data/glusterfs/prd_items_brick1/brick1] (time: 1454463012.693353), (vec: 1, len: 4104) [2016-02-03 01:30:12.693587] T [rpcsvc.c:1273:rpcsvc_submit_generic] 0-rpc-service: Tx message: 8 [2016-02-03 01:30:12.693610] T [rpcsvc.c:863:rpcsvc_record_build_header] 0-rpc-service: Reply fraglen 32, payload: 8, rpc hdr: 24 [2016-02-03 01:30:12.693638] T [rpcsvc.c:1321:rpcsvc_submit_generic] 0-rpc-service: submitted reply for rpc-message (XID: 0x6c0, Program: LIBGFCHANGELOG REBORP, ProgVers: 1, Proc: 1) to rpc-transport (socket.gfchangelog) [2016-02-03 01:30:15.432599] D [gf-changelog-api.c:53:gf_changelog_done] 0-gfchangelog: moving /var/lib/misc/glusterfsd/prd_items/ssh%3A%2F%2Fgeorep%4010.124.0.82%3Agluster%3A%2F%2F127.0.0.1%3Adr_items/ffdfc13dfaa3d74827a7b7cb855c6ce0/.processing/CHANGELOG.1454463012 to processed directory [2016-02-03 01:31:12.754211] T [rpcsvc-auth.c:404:rpcsvc_auth_request_init] 0-rpc-service: Auth handler: AUTH_GLUSTERFS-v2 [2016-02-03 01:31:12.754243] T [rpcsvc.c:495:rpcsvc_request_create] 0-rpc-service: received rpc-message (XID: 0x1729, Ver: 2, Program: 1886350951, ProgVers: 1, Proc: 1) from rpc-transport (socket.gfchangelog) [2016-02-03 01:31:12.754260] T [auth-glusterfs.c:242:auth_glusterfs_v2_authenticate] 0-rpc-service: Auth Info: pid: 0, uid: 0, gid: 0, owner: 00000000 [2016-02-03 01:31:12.754273] T [rpcsvc.c:316:rpcsvc_program_actor] 0-rpc-service: Actor found: LIBGFCHANGELOG REBORP - CHANGELOG EVENT HANDLER for [2016-02-03 01:31:12.754294] D [gf-changelog-reborp.c:406:gf_changelog_event_handler] 0-gfchangelog: seq: 2486 [/data/glusterfs/prd_items_brick1/brick1] (time: 1454463072.754125), (vec: 1, len: 4104) [2016-02-03 01:31:12.754323] T [rpcsvc.c:1273:rpcsvc_submit_generic] 0-rpc-service: Tx message: 8 [2016-02-03 01:31:12.754338] T [rpcsvc.c:863:rpcsvc_record_build_header] 0-rpc-service: Reply fraglen 32, payload: 8, rpc hdr: 24 [2016-02-03 01:31:12.754357] T [rpcsvc.c:1321:rpcsvc_submit_generic] 0-rpc-service: submitted reply for rpc-message (XID: 0x6c1, Program: LIBGFCHANGELOG REBORP, ProgVers: 1, Proc: 1) to rpc-transport (socket.gfchangelog) [2016-02-03 01:31:13.156812] D [gf-changelog-api.c:53:gf_changelog_done] 0-gfchangelog: moving /var/lib/misc/glusterfsd/prd_items/ssh%3A%2F%2Fgeorep%4010.124.0.82%3Agluster%3A%2F%2F127.0.0.1%3Adr_items/ffdfc13dfaa3d74827a7b7cb855c6ce0/.processing/CHANGELOG.1454463072 to processed directory [2016-02-03 01:32:12.816173] T [rpcsvc-auth.c:404:rpcsvc_auth_request_init] 0-rpc-service: Auth handler: AUTH_GLUSTERFS-v2 [2016-02-03 01:32:12.816219] T [rpcsvc.c:495:rpcsvc_request_create] 0-rpc-service: received rpc-message (XID: 0x1730, Ver: 2, Program: 1886350951, ProgVers: 1, Proc: 1) from rpc-transport (socket.gfchangelog) [2016-02-03 01:32:12.816236] T [auth-glusterfs.c:242:auth_glusterfs_v2_authenticate] 0-rpc-service: Auth Info: pid: 0, uid: 0, gid: 0, owner: 00000000 [2016-02-03 01:32:12.816248] T [rpcsvc.c:316:rpcsvc_program_actor] 0-rpc-service: Actor found: LIBGFCHANGELOG REBORP - CHANGELOG EVENT HANDLER for [2016-02-03 01:32:12.816261] D [gf-changelog-reborp.c:406:gf_changelog_event_handler] 0-gfchangelog: seq: 2487 [/data/glusterfs/prd_items_brick1/brick1] (time: 1454463132.816061), (vec: 1, len: 4104) [2016-02-03 01:32:12.816298] T [rpcsvc.c:1273:rpcsvc_submit_generic] 0-rpc-service: Tx message: 8 [2016-02-03 01:32:12.816319] T [rpcsvc.c:863:rpcsvc_record_build_header] 0-rpc-service: Reply fraglen 32, payload: 8, rpc hdr: 24 [2016-02-03 01:32:12.816361] T [rpcsvc.c:1321:rpcsvc_submit_generic] 0-rpc-service: submitted reply for rpc-message (XID: 0x6c2, Program: LIBGFCHANGELOG REBORP, ProgVers: 1, Proc: 1) to rpc-transport (socket.gfchangelog) [2016-02-03 01:32:13.864741] D [gf-changelog-api.c:53:gf_changelog_done] 0-gfchangelog: moving /var/lib/misc/glusterfsd/prd_items/ssh%3A%2F%2Fgeorep%4010.124.0.82%3Agluster%3A%2F%2F127.0.0.1%3Adr_items/ffdfc13dfaa3d74827a7b7cb855c6ce0/.processing/CHANGELOG.1454463132 to processed directory [2016-02-03 01:33:12.877209] T [rpcsvc-auth.c:404:rpcsvc_auth_request_init] 0-rpc-service: Auth handler: AUTH_GLUSTERFS-v2 [2016-02-03 01:33:12.877247] T [rpcsvc.c:495:rpcsvc_request_create] 0-rpc-service: received rpc-message (XID: 0x1731, Ver: 2, Program: 1886350951, ProgVers: 1, Proc: 1) from rpc-transport (socket.gfchangelog) [2016-02-03 01:33:12.877276] T [auth-glusterfs.c:242:auth_glusterfs_v2_authenticate] 0-rpc-service: Auth Info: pid: 0, uid: 0, gid: 0, owner: 00000000 [2016-02-03 01:33:12.877308] T [rpcsvc.c:316:rpcsvc_program_actor] 0-rpc-service: Actor found: LIBGFCHANGELOG REBORP - CHANGELOG EVENT HANDLER for [2016-02-03 01:33:12.877332] D [gf-changelog-reborp.c:406:gf_changelog_event_handler] 0-gfchangelog: seq: 2488 [/data/glusterfs/prd_items_brick1/brick1] (time: 1454463192.877131), (vec: 1, len: 4104) [2016-02-03 01:33:12.877360] T [rpcsvc.c:1273:rpcsvc_submit_generic] 0-rpc-service: Tx message: 8 [2016-02-03 01:33:12.877380] T [rpcsvc.c:863:rpcsvc_record_build_header] 0-rpc-service: Reply fraglen 32, payload: 8, rpc hdr: 24 [2016-02-03 01:33:12.877408] T [rpcsvc.c:1321:rpcsvc_submit_generic] 0-rpc-service: submitted reply for rpc-message (XID: 0x6c3, Program: LIBGFCHANGELOG REBORP, ProgVers: 1, Proc: 1) to rpc-transport (socket.gfchangelog) [2016-02-03 01:33:14.577963] D [gf-changelog-api.c:53:gf_changelog_done] 0-gfchangelog: moving /var/lib/misc/glusterfsd/prd_items/ssh%3A%2F%2Fgeorep%4010.124.0.82%3Agluster%3A%2F%2F127.0.0.1%3Adr_items/ffdfc13dfaa3d74827a7b7cb855c6ce0/.processing/CHANGELOG.1454463192 to processed directory Snippet from /var/log/glusterfs/geo-replication/prd_items/ssh%3A%2F%2Fgeorep%4010.124.0.82%3Agluster%3A%2F%2F127.0.0.1%3Adr_items.%2Fdata%2Fglusterfs%2Fprd_items_brick1%2Fbrick1.gluster.log ============= [2016-02-01 20:45:23.591390] I [MSGID: 108031] [afr-common.c:1783:afr_local_discovery_cbk] 0-prd_items-replicate-0: selecting local read_child prd_items-client-0 [2016-02-03 01:27:58.464278] E [fuse-bridge.c:3336:fuse_xattr_cbk] 0-glusterfs-fuse: extended attribute not supported by the backend storage [2016-02-03 01:29:03.558047] E [fuse-bridge.c:3336:fuse_xattr_cbk] 0-glusterfs-fuse: extended attribute not supported by the backend storage [2016-02-03 01:29:25.386117] E [fuse-bridge.c:3336:fuse_xattr_cbk] 0-glusterfs-fuse: extended attribute not supported by the backend storage
_______________________________________________ Gluster-users mailing list [email protected] http://www.gluster.org/mailman/listinfo/gluster-users
