[GitHub] [couchdb] nickva commented on a diff in pull request #4394: Mango fields pushdown
nickva commented on code in PR #4394:
URL: https://github.com/apache/couchdb/pull/4394#discussion_r1082069624
##
src/mango/src/mango_cursor_view.erl:
##
@@ -441,7 +496,12 @@ doc_member(Cursor, RowProps) ->
case mango_util:defer(fabric, open_doc, [Db, Id, Opts]) of
{ok, #doc{} = DocProps} ->
Doc = couch_doc:to_json_obj(DocProps, []),
-match_doc(Selector, Doc, ExecutionStats1);
+case match_and_extract_doc(Doc, Selector,
Cursor#cursor.fields) of
+{match, FinalDoc} ->
+{ok, FinalDoc, {execution_stats, ExecutionStats1}};
+{no_match, _} ->
Review Comment:
Should we assert `{no_match, undefined}` since we know it should be only
that as the `no_match` choice? Not a big deal either way but it's nicer to have
stronger asserts when possible, unless there is a reason leave it `undefined`
(future compatibility?)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscr...@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [couchdb] nickva commented on a diff in pull request #4394: Mango fields pushdown
nickva commented on code in PR #4394:
URL: https://github.com/apache/couchdb/pull/4394#discussion_r1082069624
##
src/mango/src/mango_cursor_view.erl:
##
@@ -441,7 +496,12 @@ doc_member(Cursor, RowProps) ->
case mango_util:defer(fabric, open_doc, [Db, Id, Opts]) of
{ok, #doc{} = DocProps} ->
Doc = couch_doc:to_json_obj(DocProps, []),
-match_doc(Selector, Doc, ExecutionStats1);
+case match_and_extract_doc(Doc, Selector,
Cursor#cursor.fields) of
+{match, FinalDoc} ->
+{ok, FinalDoc, {execution_stats, ExecutionStats1}};
+{no_match, _} ->
Review Comment:
Should we assert `{no_match, undefined}` since we know it should be only
that as the `no_match` choice?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscr...@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [couchdb] nickva commented on a diff in pull request #4394: Mango fields pushdown
nickva commented on code in PR #4394:
URL: https://github.com/apache/couchdb/pull/4394#discussion_r1082067804
##
src/mango/src/mango_cursor_view.erl:
##
@@ -422,16 +472,21 @@ apply_opts([{_, _} | Rest], Args) ->
% Ignore unknown options
apply_opts(Rest, Args).
-doc_member(Cursor, RowProps) ->
+doc_member_and_extract(Cursor, RowProps) ->
Db = Cursor#cursor.db,
Opts = Cursor#cursor.opts,
ExecutionStats = Cursor#cursor.execution_stats,
Selector = Cursor#cursor.selector,
case couch_util:get_value(doc, RowProps) of
{DocProps} ->
-% only matching documents are returned; the selector
-% is evaluated at the shard level in view_cb({row, Row},
-{ok, {DocProps}, {execution_stats, ExecutionStats}};
+% If the query doesn't request quorum doc read via r>1,
+% match_and_extract_doc/3 is executed in in view_cb, ie, locally
Review Comment:
two `in`s in `is executed in in view_cb`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscr...@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[Jenkins] FAILURE: CouchDB » Full Platform Builds » jenkins-add-windows-pipeline #79
Boo, we failed. https://ci-couchdb.apache.org/job/jenkins-cm1/job/FullPlatformMatrix/job/jenkins-add-windows-pipeline/79/display/redirect
[GitHub] [couchdb] nickva commented on issue #4385: replication restart from start after server restart
nickva commented on issue #4385: URL: https://github.com/apache/couchdb/issues/4385#issuecomment-1397565505 In dock.zip I had noticed local.ini wasn't at text file but some kind of a binary. ``` % cat local.ini �E�c'K�cnO�cuxUT ``` One thing to pay attention is if the replication ID changes. Based on your start_log I see that the checkpoint was *not* found for the catalogues endpoint ``` notice] 2023-01-19T13:30:16.056403Z nonode@nohost <0.1150.0> 34af135719 couch1:5984 192.168.100.227 admin GET /catalogues/ 200 ok 2 [notice] 2023-01-19T13:30:16.131009Z nonode@nohost <0.1150.0> fb7b32c9cd couch1:5984 192.168.100.227 admin GET /catalogues/_local/475a01ff4762aae18390232479a85acd 404 ok 16 [notice] 2023-01-19T13:30:16.133231Z nonode@nohost <0.1150.0> ca52dcf06f couch1:5984 192.168.100.227 admin GET /catalogues/_local/21d33a0db3bd438bc2fe58ef3e64a1a7 404 ok 2 [notice] 2023-01-19T13:30:16.171877Z nonode@nohost <0.1150.0> 9adcacce61 couch1:5984 192.168.100.227 admin GET /catalogues/_local/9dea40c7f506f19799311d927b321449 404 ok 38 [notice] 2023-01-19T13:30:16.173819Z nonode@nohost <0.1150.0> 0ab11d84ae couch1:5984 192.168.100.227 admin GET /catalogues/_local/294224beca58e21bde9e0a5676df7d05 404 ok 1 ``` Notice the 404 on the `_local/$replicationid` docs. Not sure about lotimages_new as the logs start after "Starting replication..." already. So what may be happening is your replication IDs change inadvertently when you update docker configs. If replication IDs change, that means previous checkpoints won't be found, and replication will rewind from 0. Now, if your source, target and other replication parameters stay the same it's most like the the server uuid `[couchdb] uuid = ...` value that's no consistent. The setting is described [here](https://docs.couchdb.org/en/stable/config/couchdb.html#couchdb/uuid) Here is the description on the replication ID generation algorithm: https://docs.couchdb.org/en/stable/replication/protocol.html#generate-replication-id If that is not specified, a random value will be generated, and that would cause your replications IDs to be random every time if you spin up docker containers unless you persist your config or explicitly set `[couchdb] uuid ...`. That value doesn't have to be a proper UUID. You could use a hostname or some other identifier that uniquely identifies the same "cluster". In addition, make sure it's set to the same value on all the nodes in the cluster. If you have 3 nodes (couch1, couch2, couch3 ensure uuid is the same). Checkpoints are persisted on both source and target endpoints (database) in `_local/$base_replication_id` docs. Replications will only resume from a checkpoint if it can find the checkpoint on *both* source and target. So in your logs you could monitor for those 404s and hopefully finally it should find the last one is found (a 200 response). There are usually a few 404s expected as we try to load older versions of replication ID since the algorithm to generate has evolved at least 4 times. Then monitor if replication ID values stay the same or change. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@couchdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[Jenkins] SUCCESS: CouchDB » Full Platform Builds » main #466
Yay, we passed. https://ci-couchdb.apache.org/job/jenkins-cm1/job/FullPlatformMatrix/job/main/466/display/redirect
[Jenkins] SUCCESS: CouchDB » Full Platform Builds » 3.3.x #10
Yay, we passed. https://ci-couchdb.apache.org/job/jenkins-cm1/job/FullPlatformMatrix/job/3.3.x/10/display/redirect
[GitHub] [couchdb] nickva merged pull request #4387: ci(mac): re-enable mac CI for the full build
nickva merged PR #4387: URL: https://github.com/apache/couchdb/pull/4387 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@couchdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [couchdb] nickva closed pull request #4389: ci(mac): re-enable mac CI for the full build
nickva closed pull request #4389: ci(mac): re-enable mac CI for the full build URL: https://github.com/apache/couchdb/pull/4389 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@couchdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [couchdb] nickva commented on pull request #4389: ci(mac): re-enable mac CI for the full build
nickva commented on PR #4389: URL: https://github.com/apache/couchdb/pull/4389#issuecomment-1397474748 Original PR's full CI now passes. Closing this one. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@couchdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [couchdb] nickva merged pull request #4393: docs(mango): match description of `$mod` with reality (3.2.2)
nickva merged PR #4393: URL: https://github.com/apache/couchdb/pull/4393 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@couchdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [couchdb] nickva merged pull request #4392: docs(mango): match description of `$mod` with reality (3.3.x)
nickva merged PR #4392: URL: https://github.com/apache/couchdb/pull/4392 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@couchdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[Jenkins] SUCCESS: CouchDB » Full Platform Builds » 3.3.x #9
Yay, we passed. https://ci-couchdb.apache.org/job/jenkins-cm1/job/FullPlatformMatrix/job/3.3.x/9/display/redirect
[Jenkins] SUCCESS: CouchDB » Full Platform Builds » jenkins-ci-enable-mac #6
Yay, we passed. https://ci-couchdb.apache.org/job/jenkins-cm1/job/FullPlatformMatrix/job/jenkins-ci-enable-mac/6/display/redirect
[GitHub] [couchdb] mikerhodes commented on a diff in pull request #4394: Mango fields pushdown
mikerhodes commented on code in PR #4394:
URL: https://github.com/apache/couchdb/pull/4394#discussion_r1081599218
##
src/mango/src/mango_cursor_view.erl:
##
@@ -35,6 +35,19 @@
-define(HEARTBEAT_INTERVAL_IN_USEC, 400).
+% viewcbargs wraps up the arguments that view_cb uses into a single
+% entry in the mrargs.extra list. We use a Map to allow us to later
+% add fields without having old messages causing errors/crashes.
+viewcbargs_new(Selector, Fields) ->
+#{
+selector => Selector,
+fields => Fields
+}.
+viewcbargs_get(selector, Args) when is_map(Args) ->
Review Comment:
I'm going to leave this one as is for now. Let me know if you'd rather I
change it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscr...@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [couchdb] mikerhodes commented on a diff in pull request #4394: Mango fields pushdown
mikerhodes commented on code in PR #4394:
URL: https://github.com/apache/couchdb/pull/4394#discussion_r1081596322
##
src/mango/src/mango_cursor_view.erl:
##
@@ -248,6 +266,19 @@ view_cb({row, Row}, #mrargs{extra = Options} = Acc) ->
key = couch_util:get_value(key, Row),
doc = couch_util:get_value(doc, Row)
},
+% This supports receiving our "arguments" either as just the `selector`
+% or in the new record in `callback_args`. This is to support mid-upgrade
+% clusters where the non-upgraded coordinator nodes will send the older
style.
+% TODO remove this in a couple of couchdb versions.
+{Selector, Fields} =
+case couch_util:get_value(callback_args, Options) of
+% old style
+undefined ->
+{couch_util:get_value(selector, Options), undefined};
+% new style - assume a viewcbargs
+Args ->
Review Comment:
Added the pattern match to the Map as suggested. Squashed into the original
commit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscr...@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [couchdb] nickva commented on a diff in pull request #4394: Mango fields pushdown
nickva commented on code in PR #4394:
URL: https://github.com/apache/couchdb/pull/4394#discussion_r1081579998
##
src/mango/src/mango_cursor_view.erl:
##
@@ -248,6 +266,19 @@ view_cb({row, Row}, #mrargs{extra = Options} = Acc) ->
key = couch_util:get_value(key, Row),
doc = couch_util:get_value(doc, Row)
},
+% This supports receiving our "arguments" either as just the `selector`
+% or in the new record in `callback_args`. This is to support mid-upgrade
+% clusters where the non-upgraded coordinator nodes will send the older
style.
+% TODO remove this in a couple of couchdb versions.
+{Selector, Fields} =
+case couch_util:get_value(callback_args, Options) of
+% old style
+undefined ->
+{couch_util:get_value(selector, Options), undefined};
+% new style - assume a viewcbargs
+Args ->
Review Comment:
It's a balance, but I think it's slightly better to leak some of the
implementation for a stronger assert in the case. It also tells the reader what
the types are in that snippet right away ("we are dealing with a map or
undefined only") so serves as a shorthand reminder in a way.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscr...@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[Jenkins] FAILURE: CouchDB » Full Platform Builds » jenkins-ci-enable-mac #5
Boo, we failed. https://ci-couchdb.apache.org/job/jenkins-cm1/job/FullPlatformMatrix/job/jenkins-ci-enable-mac/5/display/redirect
[GitHub] [couchdb] mikerhodes commented on a diff in pull request #4394: Mango fields pushdown
mikerhodes commented on code in PR #4394:
URL: https://github.com/apache/couchdb/pull/4394#discussion_r1081539012
##
src/mango/src/mango_cursor_view.erl:
##
@@ -248,6 +266,19 @@ view_cb({row, Row}, #mrargs{extra = Options} = Acc) ->
key = couch_util:get_value(key, Row),
doc = couch_util:get_value(doc, Row)
},
+% This supports receiving our "arguments" either as just the `selector`
+% or in the new record in `callback_args`. This is to support mid-upgrade
+% clusters where the non-upgraded coordinator nodes will send the older
style.
+% TODO remove this in a couple of couchdb versions.
+{Selector, Fields} =
+case couch_util:get_value(callback_args, Options) of
+% old style
+undefined ->
+{couch_util:get_value(selector, Options), undefined};
+% new style - assume a viewcbargs
+Args ->
Review Comment:
I wanted to avoid exposing the "internals" of the viewcbargs here. I found
you couldn't use your own function in a guard only a BIF, ie, I tried `when
is_viewcbargs(Args)`. In the end, perhaps you're right that it's better to leak
the Map implementation such that there is a decent guard on the `case`
statement -- what do you think, I'm not sure what works well in Erlang here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscr...@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [couchdb] nickva commented on a diff in pull request #4394: Mango fields pushdown
nickva commented on code in PR #4394:
URL: https://github.com/apache/couchdb/pull/4394#discussion_r1081520232
##
src/mango/src/mango_cursor_view.erl:
##
@@ -248,6 +266,19 @@ view_cb({row, Row}, #mrargs{extra = Options} = Acc) ->
key = couch_util:get_value(key, Row),
doc = couch_util:get_value(doc, Row)
},
+% This supports receiving our "arguments" either as just the `selector`
+% or in the new record in `callback_args`. This is to support mid-upgrade
+% clusters where the non-upgraded coordinator nodes will send the older
style.
+% TODO remove this in a couple of couchdb versions.
+{Selector, Fields} =
+case couch_util:get_value(callback_args, Options) of
+% old style
+undefined ->
+{couch_util:get_value(selector, Options), undefined};
+% new style - assume a viewcbargs
+Args ->
Review Comment:
We will explicitly match `Args` to be a map in `viewcbargs_get(selector,
Args)` but it won't hurt to also match it here as `Args = #{}`. Just to make it
clear in this part of the code what we're dealing with.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscr...@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [couchdb] nickva commented on a diff in pull request #4394: Mango fields pushdown
nickva commented on code in PR #4394:
URL: https://github.com/apache/couchdb/pull/4394#discussion_r1081514712
##
src/mango/src/mango_cursor_view.erl:
##
@@ -35,6 +35,19 @@
-define(HEARTBEAT_INTERVAL_IN_USEC, 400).
+% viewcbargs wraps up the arguments that view_cb uses into a single
+% entry in the mrargs.extra list. We use a Map to allow us to later
+% add fields without having old messages causing errors/crashes.
+viewcbargs_new(Selector, Fields) ->
+#{
+selector => Selector,
+fields => Fields
+}.
+viewcbargs_get(selector, Args) when is_map(Args) ->
Review Comment:
Another alternative we typically use is `viewcbargs_get(selector, #{} =
Args)` but either way if fine.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscr...@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[Jenkins] FAILURE: CouchDB » Full Platform Builds » jenkins-ci-enable-mac #4
Boo, we failed. https://ci-couchdb.apache.org/job/jenkins-cm1/job/FullPlatformMatrix/job/jenkins-ci-enable-mac/4/display/redirect
[GitHub] [couchdb] janl commented on pull request #4387: ci(mac): re-enable mac CI for the full build
janl commented on PR #4387: URL: https://github.com/apache/couchdb/pull/4387#issuecomment-1397116512 PATH issue, thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@couchdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [couchdb] mikerhodes opened a new pull request, #4394: Mango fields pushdown
mikerhodes opened a new pull request, #4394: URL: https://github.com/apache/couchdb/pull/4394 ## Overview This PR aims to improve Mango by reducing the data transferred to the coordinator during query execution. It may reduce memory or CPU use at the coordinator but that isn't the primary goal. Currently, when documents are read at the shard level, they are compared locally at the shard with the selector to ensure they match before they are sent to the coordinator. This ensures we're not sending documents across the network that the coordinator immediately discards, saving bandwidth and coordinator processing. This PR further executes field projection (`fields` in the query) at the shard level. This should further save bandwidth, particularly for queries that project few fields from large documents. One item of complexity is that a query may request a quorum read of documents, meaning that we need to do the document read at the coordinator and not the shard, then perform the `selector` and `fields` processing there rather than at the shard. To ensure that documents are processed consistently whether at the shard or coordinator, match_and_extract_doc/3 is added. There is still one orphan call outside match_and_extract_doc/2 to extract/2 which supports cluster upgrade and should later be removed. Shard level processing is already performed in a callback, view_cb/2, that's passed to fabric's view processing to run for each row in the view result set. It's used for the shard local selector and fields processing. To make it clear what arguments are destined for this callback, the PR encapsulates the arguments, using viewcbargs_new/2 and viewcbargs_get/2. As we push down more functionality to the shard, the context this function needs to carry with it will increase, so having a record for it will be valuable. Supporting cluster upgrades: The PR supports shard pushdown for Mango `fields` processing for situations during rolling cluster upgrades. (Cloudant require this as they use rolling upgrades). In the state where the coordinator is speaking to an upgraded node, the view_cb/2 needs to support being passed just the `selector` outside of the new viewcbargs record. In this case, the shard will not process fields, but the coordinator will. In the situation where the coordinator is upgraded but the shard is not, we need to send the selector to the shard via `selector` and also execute the fields projection at the coordinator. Therefore we pass arguments to view_cb/2 via both `selector` and `callback_args` and have an apparently spurious field projection (mango_fields:extract/2) in the code that receives back values from the shard ( factored out into doc_member_and_extract). Both of these affordances should only need to exist through one minor version change and be removed thereafter -- if people are jumping several minor versions of CouchDB in one go, hopefully they are prepared for a bit of trouble. Testing upgrade states: As view_cb is completely separate from the rest of the cursor code, we can first try out the branch's code using view_cb from `main`, and then the other way -- the branch's view_cb with the rest of the file from main. I did both of these tests successfully. ## Testing recommendations This PR should not change anything from an end user perspective. Mango responses should remain the same as they currently are. I have run some basic performance locally tests using k6.io, which showed no meaningful change in the latency of requests. ## Related Issues or Pull Requests none. ## Checklist - [x] Code is written and works correctly - [x] Changes are covered by tests - [ ] Any new configurable parameters are documented in `rel/overlay/etc/default.ini` - [ ] Documentation changes were made in the `src/docs` folder - [ ] Documentation changes were backported (separated PR) to affected branches -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@couchdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[Jenkins] FAILURE: CouchDB » Full Platform Builds » jenkins-add-windows-pipeline #76
Boo, we failed. https://ci-couchdb.apache.org/job/jenkins-cm1/job/FullPlatformMatrix/job/jenkins-add-windows-pipeline/76/display/redirect
[GitHub] [couchdb] pgj opened a new pull request, #4393: docs(mango): match description of `$mod` with reality
pgj opened a new pull request, #4393: URL: https://github.com/apache/couchdb/pull/4393 Backport of aff7a6e7183e6c471a9675e37216baa0bad563a3. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@couchdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [couchdb] pgj opened a new pull request, #4392: docs(mango): match description of `$mod` with reality
pgj opened a new pull request, #4392: URL: https://github.com/apache/couchdb/pull/4392 Backport of aff7a6e7183e6c471a9675e37216baa0bad563a3. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@couchdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [couchdb] tophe commented on issue #4385: replication restart from start after server restart
tophe commented on issue #4385: URL: https://github.com/apache/couchdb/issues/4385#issuecomment-1397041792 I have just find the problem, it is docker dependent. when you do a docker stop/start everything is ok with replication. but when you update the container, running a docker run, with a new container, the replication restart from the beginning. (I have made change in local.ini) [start_log.txt](https://github.com/apache/couchdb/files/10457293/start_log.txt) [checkpoint_logs.txt](https://github.com/apache/couchdb/files/10457294/checkpoint_logs.txt) I have attache some logs to the post. in checkpoint_log.txt, you can see the last checkpoint for replication lotimages_new. at g1MLeJzLYWBg4MhgTmGQSs4vTc5ISXIoLskvStVLLCopKMpMTtVLyszJAaphSmRIkv in start log, you can see log asked from g1CReJzLYWBgYMpgTmHgzcvPy09JdcjLz8gvLskBCScyJNX in futon, I can see that replication restart from beginning. are the replication checkpoints stored in /opt/couchdb/data, that is a docker binded volume, or are they stored in the container ? and if so where are they stored ? I have attach some docker config in order to reproduce my setup. [dock.zip](https://github.com/apache/couchdb/files/10457471/dock.zip) you can download all the file and then run docker-compose build, docker-compose up to start couchdb container. set up a replication with some datas. after checkpointing, change local.ini , and docker-compose build, docker-compose up, to redeploy a new container. then compare starting point, with checkpoint. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@couchdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [couchdb] tophe closed issue #4385: replication restart from start after server restart
tophe closed issue #4385: replication restart from start after server restart URL: https://github.com/apache/couchdb/issues/4385 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@couchdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [couchdb] tophe commented on issue #4385: replication restart from start after server restart
tophe commented on issue #4385: URL: https://github.com/apache/couchdb/issues/4385#issuecomment-1396964894 hello, I have do many test stopping and starting all the replication nodes in different order and I can't reproduce the problem, it seem every things are working fine. After restart replication start from their last checkpoint, and start quickly, and run quite fast in 3.3.1. Perhaps this was caused by the upgrade from 3.2.1 to 3.3.1 which restart replication from beginning in the upgrade process ? I see that on many node. thank's for your help (and for that great peace of code). I close the issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@couchdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[Jenkins] SUCCESS: CouchDB » Full Platform Builds » jenkins-add-windows-pipeline #73
Yay, we passed. https://ci-couchdb.apache.org/job/jenkins-cm1/job/FullPlatformMatrix/job/jenkins-add-windows-pipeline/73/display/redirect
[GitHub] [couchdb-glazier] big-r81 merged pull request #20: docs: update installation instructions
big-r81 merged PR #20: URL: https://github.com/apache/couchdb-glazier/pull/20 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@couchdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [couchdb-glazier] pgj opened a new pull request, #20: docs: update installation instructions
pgj opened a new pull request, #20: URL: https://github.com/apache/couchdb-glazier/pull/20 - Unify spelling of SpiderMonkey. - Take a note about the missing SSL certificates that would be required for the SpiderMonkey build due to Rust. - Note that questions on `mach bootstrap` might be different. - Mention the concerns on clobbering. - Suggest the reader to verify the presence of the SpiderMonkey static library as probably the most important asset to have. - Start a list of currently known build problems. - Minor markdown nits. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@couchdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[Jenkins] SUCCESS: CouchDB » Full Platform Builds » jenkins-add-windows-pipeline #69
Yay, we passed. https://ci-couchdb.apache.org/job/jenkins-cm1/job/FullPlatformMatrix/job/jenkins-add-windows-pipeline/69/display/redirect
[GitHub] [couchdb] ishaiavrahami opened a new issue, #4391: Get reduce count for mango query
ishaiavrahami opened a new issue, #4391: URL: https://github.com/apache/couchdb/issues/4391 [NOTE]: # ( ^^ Provide a general summary of the request in the title above. ^^ ) ## Summary Hello i'm trying to get a count of the query like the reduce option in mango query just like the sort flag true [NOTE]: # ( Provide a brief overview of what the new feature is all about. ) ## Desired Behaviour return count [NOTE]: # ( Tell us how the new feature should work. Be specific. ) [TIP]: # ( Do NOT give us access or passwords to your actual CouchDB! ) ## Possible Solution [NOTE]: # ( Not required. Suggest how to implement the addition or change. ) ## Additional context [TIP]: # ( Why does this feature matter to you? What unique circumstances do you have? ) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@couchdb.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
