We have an iOS application which is set up for one-way, continuous, pull
replication and are experiencing an issue whereby replication runs for a
while and then just stops.
The failure appears to happen when the replication is idle for more than 5
minutes. From examining the logs at both ends, it seems that CBL is not
reliably detecting that a WebSocket has closed and just sits there waiting
for more updates (which, without a WebSocket, never arrive). There are no
retries or errors logged by CBL – in fact nothing is logged at all after
the sync-gateway logs a “WebSocket closed” message – and changes on the
server are no longer replicated to the device.
By way of example, the sync-gateway logs typically show:
12:11:08.505349 HTTP: #134: GET
/courses/_local/1c07c124ecf0f2bcdd2f04aa2d01877a95f89fcb
12:11:08.505499 HTTP+: #134: --> 200 (0.0 ms)
12:11:08.505645 HTTP: #135: GET
/courses/_local/2ea8ff117644aa8087bc1d953fbb331b36f1904f
12:11:08.505749 HTTP+: #135: --> 200 (0.0 ms)
12:11:08.957366 HTTP: #136: GET /courses/_changes?feed=websocket
12:11:08.957464 HTTP+: #136: --> 101 Upgraded to WebSocket protocol (0.0
ms)
12:11:08.991339 Changes: MultiChangesFeed({*}, {Since:370 Limit:0
Conflicts:true IncludeDocs:false Wait:true Continuous:true
Terminator:0xc2108632a0}) ...
12:11:08.991380 Changes+: MultiChangesFeed: channels expand to *:1 ...
12:11:08.991391 Changes+: MultiChangesFeed waiting...
12:11:08.991398 Changes+: Waiting for "courses-sync"'s count to pass 327
12:16:08.991729 HTTP+: #136: --> WebSocket closed
At this point, CBL will *occasionally* detect the socket has gone away,
re-establish connection and log:
12:16:13.080‖ ChangeTracker: CBLWebSocketChangeTracker[0x8d44570 courses]:
disconnected with error 1006 / Socket closed by remote peer
12:16:13.081‖ CBLWebSocketChangeTracker[0x8d44570 courses]: Connection
error #1, retrying in 2.0 sec: The operation couldn’t be completed. Socket
closed by remote peer
12:16:15.083‖ ChangeTracker: CBLWebSocketChangeTracker[0x8d44570 courses]:
Starting...
12:16:15.083‖ SyncVerbose: CBLWebSocketChangeTracker[0x8d44570 courses]:
GET //xxxxx.xxxxx.co.uk:4984/courses/_changes?feed=websocket
12:16:15.085‖ ChangeTracker: CBLWebSocketChangeTracker[0x8d44570 courses]:
Started... <http://xxxxx.xxxxx.co.uk:4984/courses/_changes?feed=websocket>
12:16:15.174‖ ChangeTrackerVerbose: CBLWebSocketChangeTracker[0x8d44570
courses]: WebSocket opened
12:16:15.204‖ ChangeTrackerVerbose: CBLWebSocketChangeTracker[0x8d44570
courses]: Got a message: []
12:16:15.205‖ ChangeTrackerVerbose: CBLWebSocketChangeTracker[0x8d44570
courses]: read 2 bytes
12:16:15.205‖ ChangeTracker: CBLWebSocketChangeTracker[0x8d44570 courses]:
caught up!
Generally, though, CBL does nothing. No further messages appear in the log
– ever – and replication is silently dead until the app goes through a
background/foreground cycle, at which point replication is restarted.
Interestingly – and perhaps significantly – if we enable logging for “WS”
the problem is *much* less likely to occur (although it still does after 30
minutes or so). I can’t begin to imagine why this would be – unless there’s
some kind of subtle timing issue at work here.
Anyway, I guess there are two questions:
1. Is the Sync Gateway deliberately closing the WebSocket after 300 seconds
and is this normal behaviour when replication is idle? (From looking at the
code, I would say yes, this is the default timeout).
2. Why is CBL unreliably detecting the socket closure and allowing the
replication to silently fail with no retry? And why does switching on WS
logging appear to help?
It’s entirely possible that we don’t have something set up correctly but
I’m struggling to see what. I’ve tried the latest CBL build from Jenkins
(no difference), sync-gateway is at 1.0.0-23.
Any thoughts gratefully received!
--
You received this message because you are subscribed to the Google Groups
"Couchbase Mobile" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/mobile-couchbase/6d3186ea-bf11-433b-9695-990bacf51e9c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
