[GitHub] rnewson commented on a change in pull request #1176: Implement pluggable authentication and session support for replicator
rnewson commented on a change in pull request #1176: Implement pluggable
authentication and session support for replicator
URL: https://github.com/apache/couchdb/pull/1176#discussion_r170285825
##
File path: src/couch_replicator/src/couch_replicator_auth_session.erl
##
@@ -0,0 +1,545 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+
+% This is the replicator session auth plugin. It implements session based
+% authentication for the replicator. The only public API are the functions from
+% the couch_replicator_auth behaviour. Most of the logic and state is in the
+% gen_server. An instance of a gen_server could be spawned for the source and
+% target endpoints of each replication jobs.
+%
+% The workflow is roughly this:
+%
+% * On initialization, try to get a cookie in `refresh/1` If an error occurs,
+%the crash. If `_session` endpoint fails with a 404 (not found), return
+%`ignore` assuming session authentication is not support or we simply hit a
+%non-CouchDb server.
+%
+% * Before each request, auth framework calls `update_headers` API function.
+%Before updating the headers and returning, check if need to refresh again.
+%The check looks `next_refresh` time. If that time is set (not `infinity`)
+%and just expired, then obtain a new cookie, then update headers and
+%return.
+%
+% * After each request, auth framework calls `handle_response` function. If
+%request was successful check if a new cookie was sent by the server in the
+%`Set-Cookie` header. If it was then then that becomes the current cookie.
+%
+% * If last request has an auth failure, check if request used a stale cookie
+%In this case nothing is done, and the client is told to retry. Next time
+%it updates its headers befor the request it should pick up the latest
+%cookie.
+%
+% * If last request failed and cookie was the latest known cookie, schedule a
+%refresh and tell client to retry. However, if the cookie was just updated,
+%tell the client to continue such that it will handle the auth failure on
+%its own via a set of retries with exponential backoffs. This is it to
+%ensure if something goes wrong and one of the endpoints issues invalid
+%cookies, replicator won't be stuck in a busy loop refreshing them.
+
+
+-module(couch_replicator_auth_session).
+
+
+-behaviour(couch_replicator_auth).
+-behaviour(gen_server).
+
+
+-export([
+initialize/1,
+update_headers/2,
+handle_response/4,
+cleanup/1
+]).
+
+-export([
+init/1,
+terminate/2,
+handle_call/3,
+handle_cast/2,
+handle_info/2,
+code_change/3,
+format_status/2
+]).
+
+
+-include_lib("ibrowse/include/ibrowse.hrl").
+-include_lib("couch_replicator/include/couch_replicator_api_wrap.hrl").
+
+
+-type headers() :: [{string(), string()}].
+-type code() :: non_neg_integer().
+-type creds() :: {string() | undefined, string() | undefined}.
+
+
+% Behavior API callbacks
+
+
+-spec initialize(#httpdb{}) -> {ok, #httpdb{}, term()} | ignore.
+initialize(#httpdb{} = HttpDb) ->
+case remove_creds(HttpDb) of
+{ok, User, Pass, HttpDb1} ->
+case gen_server:start_link(?MODULE, [User, Pass, HttpDb1], []) of
+{ok, Pid} ->
+{ok, HttpDb1, {Pid, 0}};
+ignore ->
+ignore;
+{error, Error} ->
+{error, Error}
+end;
+{error, missing_credentials} ->
+ignore;
+{error, Error} ->
+{error, Error}
+end.
+
+
+-spec update_headers(term(), headers()) -> {headers(), term()}.
+update_headers({Pid, Epoch}, Headers) ->
+Args = {update_headers, Headers, Epoch},
+{Headers1, Epoch1} = gen_server:call(Pid, Args, infinity),
Review comment:
`infinity` timeout is a problem waiting to happen. I suggest a (small)
multiple of the request timeout to cover the time that might be spent in a
message queue.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] rnewson commented on a change in pull request #1176: Implement pluggable authentication and session support for replicator
rnewson commented on a change in pull request #1176: Implement pluggable
authentication and session support for replicator
URL: https://github.com/apache/couchdb/pull/1176#discussion_r170285923
##
File path: src/couch_replicator/src/couch_replicator_auth_session.erl
##
@@ -0,0 +1,545 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+
+% This is the replicator session auth plugin. It implements session based
+% authentication for the replicator. The only public API are the functions from
+% the couch_replicator_auth behaviour. Most of the logic and state is in the
+% gen_server. An instance of a gen_server could be spawned for the source and
+% target endpoints of each replication jobs.
+%
+% The workflow is roughly this:
+%
+% * On initialization, try to get a cookie in `refresh/1` If an error occurs,
+%the crash. If `_session` endpoint fails with a 404 (not found), return
+%`ignore` assuming session authentication is not support or we simply hit a
+%non-CouchDb server.
+%
+% * Before each request, auth framework calls `update_headers` API function.
+%Before updating the headers and returning, check if need to refresh again.
+%The check looks `next_refresh` time. If that time is set (not `infinity`)
+%and just expired, then obtain a new cookie, then update headers and
+%return.
+%
+% * After each request, auth framework calls `handle_response` function. If
+%request was successful check if a new cookie was sent by the server in the
+%`Set-Cookie` header. If it was then then that becomes the current cookie.
+%
+% * If last request has an auth failure, check if request used a stale cookie
+%In this case nothing is done, and the client is told to retry. Next time
+%it updates its headers befor the request it should pick up the latest
+%cookie.
+%
+% * If last request failed and cookie was the latest known cookie, schedule a
+%refresh and tell client to retry. However, if the cookie was just updated,
+%tell the client to continue such that it will handle the auth failure on
+%its own via a set of retries with exponential backoffs. This is it to
+%ensure if something goes wrong and one of the endpoints issues invalid
+%cookies, replicator won't be stuck in a busy loop refreshing them.
+
+
+-module(couch_replicator_auth_session).
+
+
+-behaviour(couch_replicator_auth).
+-behaviour(gen_server).
+
+
+-export([
+initialize/1,
+update_headers/2,
+handle_response/4,
+cleanup/1
+]).
+
+-export([
+init/1,
+terminate/2,
+handle_call/3,
+handle_cast/2,
+handle_info/2,
+code_change/3,
+format_status/2
+]).
+
+
+-include_lib("ibrowse/include/ibrowse.hrl").
+-include_lib("couch_replicator/include/couch_replicator_api_wrap.hrl").
+
+
+-type headers() :: [{string(), string()}].
+-type code() :: non_neg_integer().
+-type creds() :: {string() | undefined, string() | undefined}.
+
+
+% Behavior API callbacks
+
+
+-spec initialize(#httpdb{}) -> {ok, #httpdb{}, term()} | ignore.
+initialize(#httpdb{} = HttpDb) ->
+case remove_creds(HttpDb) of
+{ok, User, Pass, HttpDb1} ->
+case gen_server:start_link(?MODULE, [User, Pass, HttpDb1], []) of
+{ok, Pid} ->
+{ok, HttpDb1, {Pid, 0}};
+ignore ->
+ignore;
+{error, Error} ->
+{error, Error}
+end;
+{error, missing_credentials} ->
+ignore;
+{error, Error} ->
+{error, Error}
+end.
+
+
+-spec update_headers(term(), headers()) -> {headers(), term()}.
+update_headers({Pid, Epoch}, Headers) ->
+Args = {update_headers, Headers, Epoch},
+{Headers1, Epoch1} = gen_server:call(Pid, Args, infinity),
+{Headers1, {Pid, Epoch1}}.
+
+
+-spec handle_response(term(), code(), headers(), term()) ->
+{continue | retry, term()}.
+handle_response({Pid, Epoch}, Code, Headers, Body) ->
+Args = {handle_response, Code, Headers, Body, Epoch},
+{Retry, Epoch1} = gen_server:call(Pid, Args, infinity),
+{Retry, {Pid, Epoch1}}.
+
+
+-spec cleanup(term()) -> ok.
+cleanup({Pid, _Epoch}) ->
+gen_server:call(Pid, stop, infinity).
+
+
+%% Definitions
+
+-define(MIN_UPDATE_INTERVAL, 5).
+
+
+%% gen_server state
+
+-record(state, {
+epoch = 0 :: non_neg_integer(),
+cookie :: string() | undefined,
+user :: string() | undefined,
+pass :: string() | undefined,
[GitHub] rnewson commented on a change in pull request #1176: Implement pluggable authentication and session support for replicator
rnewson commented on a change in pull request #1176: Implement pluggable authentication and session support for replicator URL: https://github.com/apache/couchdb/pull/1176#discussion_r170285480 ## File path: src/couch_replicator/src/couch_replicator_auth_basic.erl ## @@ -0,0 +1,52 @@ +% Licensed under the Apache License, Version 2.0 (the "License"); you may not +% use this file except in compliance with the License. You may obtain a copy of +% the License at +% +% http://www.apache.org/licenses/LICENSE-2.0 +% +% Unless required by applicable law or agreed to in writing, software +% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +% License for the specific language governing permissions and limitations under +% the License. + +-module(couch_replicator_auth_basic). Review comment: the `http://user:pass@host/db` form is never what's sent in an http request, though. the user:pass part has to be converted to `Authentication: Basic `. That translation is happening elsewhere (in ibrowse), leaving this module empty. It's just a bit odd to call this module 'basic' as if it does basic auth, when in fact it does nothing (and basic auth happens independently). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rnewson commented on a change in pull request #1176: Implement pluggable authentication and session support for replicator
rnewson commented on a change in pull request #1176: Implement pluggable
authentication and session support for replicator
URL: https://github.com/apache/couchdb/pull/1176#discussion_r170086079
##
File path: src/couch_replicator/src/couch_replicator_auth.erl
##
@@ -0,0 +1,100 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_replicator_auth).
+
+
+-export([
+initialize/1,
+update_headers/2,
+handle_response/4,
+cleanup/1
+]).
+
+
+-include_lib("couch_replicator/include/couch_replicator_api_wrap.hrl").
+
+
+-type headers() :: [{string(), string()}].
+-type code() :: non_neg_integer().
+
+
+-define(DEFAULT_PLUGINS,
+"couch_replicator_auth_session,couch_replicator_auth_basic").
+
+
+% Behavior API
+
+-callback initialize(#httpdb{}) -> {ok, #httpdb{}, term()} | ignore.
+
+-callback update_headers(term(), headers()) -> {headers(), term()}.
Review comment:
this makes the (reasonable) assumption that all auth systems work via
headers, but it isn't an ideal abstraction. The intention behind allowing a
callback to update headers is to apply the authentication whatevers, so perhaps
that would be a better shape?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] rnewson commented on a change in pull request #1176: Implement pluggable authentication and session support for replicator
rnewson commented on a change in pull request #1176: Implement pluggable
authentication and session support for replicator
URL: https://github.com/apache/couchdb/pull/1176#discussion_r170089423
##
File path: src/couch_replicator/src/couch_replicator_auth_session.erl
##
@@ -0,0 +1,545 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+
+% This is the replicator session auth plugin. It implements session based
+% authentication for the replicator. The only public API are the functions from
+% the couch_replicator_auth behaviour. Most of the logic and state is in the
+% gen_server. An instance of a gen_server could be spawned for the source and
+% target endpoints of each replication jobs.
+%
+% The workflow is roughly this:
+%
+% * On initialization, try to get a cookie in `refresh/1` If an error occurs,
+%the crash. If `_session` endpoint fails with a 404 (not found), return
+%`ignore` assuming session authentication is not support or we simply hit a
+%non-CouchDb server.
+%
+% * Before each request, auth framework calls `update_headers` API function.
+%Before updating the headers and returning, check if need to refresh again.
+%The check looks `next_refresh` time. If that time is set (not `infinity`)
+%and just expired, then obtain a new cookie, then update headers and
+%return.
+%
+% * After each request, auth framework calls `handle_response` function. If
+%request was successful check if a new cookie was sent by the server in the
+%`Set-Cookie` header. If it was then then that becomes the current cookie.
+%
+% * If last request has an auth failure, check if request used a stale cookie
+%In this case nothing is done, and the client is told to retry. Next time
+%it updates its headers befor the request it should pick up the latest
+%cookie.
+%
+% * If last request failed and cookie was the latest known cookie, schedule a
+%refresh and tell client to retry. However, if the cookie was just updated,
+%tell the client to continue such that it will handle the auth failure on
+%its own via a set of retries with exponential backoffs. This is it to
+%ensure if something goes wrong and one of the endpoints issues invalid
+%cookies, replicator won't be stuck in a busy loop refreshing them.
+
+
+-module(couch_replicator_auth_session).
+
+
+-behaviour(couch_replicator_auth).
+-behaviour(gen_server).
+
+
+-export([
+initialize/1,
+update_headers/2,
+handle_response/4,
+cleanup/1
+]).
+
+-export([
+init/1,
+terminate/2,
+handle_call/3,
+handle_cast/2,
+handle_info/2,
+code_change/3,
+format_status/2
+]).
+
+
+-include_lib("ibrowse/include/ibrowse.hrl").
+-include_lib("couch_replicator/include/couch_replicator_api_wrap.hrl").
+
+
+-type headers() :: [{string(), string()}].
+-type code() :: non_neg_integer().
+-type creds() :: {string() | undefined, string() | undefined}.
+
+
+% Behavior API callbacks
+
+
+-spec initialize(#httpdb{}) -> {ok, #httpdb{}, term()} | ignore.
+initialize(#httpdb{} = HttpDb) ->
+case remove_creds(HttpDb) of
+{ok, User, Pass, HttpDb1} ->
+case gen_server:start_link(?MODULE, [User, Pass, HttpDb1], []) of
+{ok, Pid} ->
+{ok, HttpDb1, {Pid, 0}};
+ignore ->
+ignore;
+{error, Error} ->
+{error, Error}
+end;
+{error, missing_credentials} ->
+ignore;
+{error, Error} ->
+{error, Error}
+end.
+
+
+-spec update_headers(term(), headers()) -> {headers(), term()}.
+update_headers({Pid, Epoch}, Headers) ->
+Args = {update_headers, Headers, Epoch},
+{Headers1, Epoch1} = gen_server:call(Pid, Args, infinity),
+{Headers1, {Pid, Epoch1}}.
+
+
+-spec handle_response(term(), code(), headers(), term()) ->
+{continue | retry, term()}.
+handle_response({Pid, Epoch}, Code, Headers, Body) ->
+Args = {handle_response, Code, Headers, Body, Epoch},
+{Retry, Epoch1} = gen_server:call(Pid, Args, infinity),
+{Retry, {Pid, Epoch1}}.
+
+
+-spec cleanup(term()) -> ok.
+cleanup({Pid, _Epoch}) ->
+gen_server:call(Pid, stop, infinity).
+
+
+%% Definitions
+
+-define(MIN_UPDATE_INTERVAL, 5).
+
+
+%% gen_server state
+
+-record(state, {
+epoch = 0 :: non_neg_integer(),
+cookie :: string() | undefined,
+user :: string() | undefined,
+pass :: string() | undefined,
[GitHub] rnewson commented on a change in pull request #1176: Implement pluggable authentication and session support for replicator
rnewson commented on a change in pull request #1176: Implement pluggable authentication and session support for replicator URL: https://github.com/apache/couchdb/pull/1176#discussion_r170086460 ## File path: src/couch_replicator/src/couch_replicator_auth_basic.erl ## @@ -0,0 +1,52 @@ +% Licensed under the Apache License, Version 2.0 (the "License"); you may not +% use this file except in compliance with the License. You may obtain a copy of +% the License at +% +% http://www.apache.org/licenses/LICENSE-2.0 +% +% Unless required by applicable law or agreed to in writing, software +% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +% License for the specific language governing permissions and limitations under +% the License. + +-module(couch_replicator_auth_basic). Review comment: that this module has no code in it implies the abstraction is a bit off. There's no basic auth logic here. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rnewson commented on a change in pull request #1176: Implement pluggable authentication and session support for replicator
rnewson commented on a change in pull request #1176: Implement pluggable
authentication and session support for replicator
URL: https://github.com/apache/couchdb/pull/1176#discussion_r170088524
##
File path: src/couch_replicator/src/couch_replicator_auth_session.erl
##
@@ -0,0 +1,545 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+
+% This is the replicator session auth plugin. It implements session based
+% authentication for the replicator. The only public API are the functions from
+% the couch_replicator_auth behaviour. Most of the logic and state is in the
+% gen_server. An instance of a gen_server could be spawned for the source and
+% target endpoints of each replication jobs.
+%
+% The workflow is roughly this:
+%
+% * On initialization, try to get a cookie in `refresh/1` If an error occurs,
+%the crash. If `_session` endpoint fails with a 404 (not found), return
+%`ignore` assuming session authentication is not support or we simply hit a
+%non-CouchDb server.
+%
+% * Before each request, auth framework calls `update_headers` API function.
+%Before updating the headers and returning, check if need to refresh again.
+%The check looks `next_refresh` time. If that time is set (not `infinity`)
+%and just expired, then obtain a new cookie, then update headers and
+%return.
+%
+% * After each request, auth framework calls `handle_response` function. If
+%request was successful check if a new cookie was sent by the server in the
+%`Set-Cookie` header. If it was then then that becomes the current cookie.
+%
+% * If last request has an auth failure, check if request used a stale cookie
+%In this case nothing is done, and the client is told to retry. Next time
+%it updates its headers befor the request it should pick up the latest
+%cookie.
+%
+% * If last request failed and cookie was the latest known cookie, schedule a
+%refresh and tell client to retry. However, if the cookie was just updated,
+%tell the client to continue such that it will handle the auth failure on
+%its own via a set of retries with exponential backoffs. This is it to
+%ensure if something goes wrong and one of the endpoints issues invalid
+%cookies, replicator won't be stuck in a busy loop refreshing them.
+
+
+-module(couch_replicator_auth_session).
+
+
+-behaviour(couch_replicator_auth).
+-behaviour(gen_server).
+
+
+-export([
+initialize/1,
+update_headers/2,
+handle_response/4,
+cleanup/1
+]).
+
+-export([
+init/1,
+terminate/2,
+handle_call/3,
+handle_cast/2,
+handle_info/2,
+code_change/3,
+format_status/2
+]).
+
+
+-include_lib("ibrowse/include/ibrowse.hrl").
+-include_lib("couch_replicator/include/couch_replicator_api_wrap.hrl").
+
+
+-type headers() :: [{string(), string()}].
+-type code() :: non_neg_integer().
+-type creds() :: {string() | undefined, string() | undefined}.
+
+
+% Behavior API callbacks
+
+
+-spec initialize(#httpdb{}) -> {ok, #httpdb{}, term()} | ignore.
+initialize(#httpdb{} = HttpDb) ->
+case remove_creds(HttpDb) of
+{ok, User, Pass, HttpDb1} ->
+case gen_server:start_link(?MODULE, [User, Pass, HttpDb1], []) of
+{ok, Pid} ->
+{ok, HttpDb1, {Pid, 0}};
+ignore ->
+ignore;
+{error, Error} ->
+{error, Error}
+end;
+{error, missing_credentials} ->
+ignore;
+{error, Error} ->
+{error, Error}
+end.
+
+
+-spec update_headers(term(), headers()) -> {headers(), term()}.
+update_headers({Pid, Epoch}, Headers) ->
+Args = {update_headers, Headers, Epoch},
+{Headers1, Epoch1} = gen_server:call(Pid, Args, infinity),
+{Headers1, {Pid, Epoch1}}.
+
+
+-spec handle_response(term(), code(), headers(), term()) ->
+{continue | retry, term()}.
+handle_response({Pid, Epoch}, Code, Headers, Body) ->
+Args = {handle_response, Code, Headers, Body, Epoch},
+{Retry, Epoch1} = gen_server:call(Pid, Args, infinity),
+{Retry, {Pid, Epoch1}}.
+
+
+-spec cleanup(term()) -> ok.
+cleanup({Pid, _Epoch}) ->
+gen_server:call(Pid, stop, infinity).
+
+
+%% Definitions
+
+-define(MIN_UPDATE_INTERVAL, 5).
+
+
+%% gen_server state
+
+-record(state, {
+epoch = 0 :: non_neg_integer(),
+cookie :: string() | undefined,
+user :: string() | undefined,
+pass :: string() | undefined,
[GitHub] rnewson commented on a change in pull request #1176: Implement pluggable authentication and session support for replicator
rnewson commented on a change in pull request #1176: Implement pluggable
authentication and session support for replicator
URL: https://github.com/apache/couchdb/pull/1176#discussion_r170089748
##
File path: src/couch_replicator/src/couch_replicator_auth_session.erl
##
@@ -0,0 +1,545 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+
+% This is the replicator session auth plugin. It implements session based
+% authentication for the replicator. The only public API are the functions from
+% the couch_replicator_auth behaviour. Most of the logic and state is in the
+% gen_server. An instance of a gen_server could be spawned for the source and
+% target endpoints of each replication jobs.
+%
+% The workflow is roughly this:
+%
+% * On initialization, try to get a cookie in `refresh/1` If an error occurs,
+%the crash. If `_session` endpoint fails with a 404 (not found), return
+%`ignore` assuming session authentication is not support or we simply hit a
+%non-CouchDb server.
+%
+% * Before each request, auth framework calls `update_headers` API function.
+%Before updating the headers and returning, check if need to refresh again.
+%The check looks `next_refresh` time. If that time is set (not `infinity`)
+%and just expired, then obtain a new cookie, then update headers and
+%return.
+%
+% * After each request, auth framework calls `handle_response` function. If
+%request was successful check if a new cookie was sent by the server in the
+%`Set-Cookie` header. If it was then then that becomes the current cookie.
+%
+% * If last request has an auth failure, check if request used a stale cookie
+%In this case nothing is done, and the client is told to retry. Next time
+%it updates its headers befor the request it should pick up the latest
+%cookie.
+%
+% * If last request failed and cookie was the latest known cookie, schedule a
+%refresh and tell client to retry. However, if the cookie was just updated,
+%tell the client to continue such that it will handle the auth failure on
+%its own via a set of retries with exponential backoffs. This is it to
+%ensure if something goes wrong and one of the endpoints issues invalid
+%cookies, replicator won't be stuck in a busy loop refreshing them.
+
+
+-module(couch_replicator_auth_session).
+
+
+-behaviour(couch_replicator_auth).
+-behaviour(gen_server).
+
+
+-export([
+initialize/1,
+update_headers/2,
+handle_response/4,
+cleanup/1
+]).
+
+-export([
+init/1,
+terminate/2,
+handle_call/3,
+handle_cast/2,
+handle_info/2,
+code_change/3,
+format_status/2
+]).
+
+
+-include_lib("ibrowse/include/ibrowse.hrl").
+-include_lib("couch_replicator/include/couch_replicator_api_wrap.hrl").
+
+
+-type headers() :: [{string(), string()}].
+-type code() :: non_neg_integer().
+-type creds() :: {string() | undefined, string() | undefined}.
+
+
+% Behavior API callbacks
+
+
+-spec initialize(#httpdb{}) -> {ok, #httpdb{}, term()} | ignore.
+initialize(#httpdb{} = HttpDb) ->
+case remove_creds(HttpDb) of
+{ok, User, Pass, HttpDb1} ->
+case gen_server:start_link(?MODULE, [User, Pass, HttpDb1], []) of
+{ok, Pid} ->
+{ok, HttpDb1, {Pid, 0}};
+ignore ->
+ignore;
+{error, Error} ->
+{error, Error}
+end;
+{error, missing_credentials} ->
+ignore;
+{error, Error} ->
+{error, Error}
+end.
+
+
+-spec update_headers(term(), headers()) -> {headers(), term()}.
+update_headers({Pid, Epoch}, Headers) ->
+Args = {update_headers, Headers, Epoch},
+{Headers1, Epoch1} = gen_server:call(Pid, Args, infinity),
+{Headers1, {Pid, Epoch1}}.
+
+
+-spec handle_response(term(), code(), headers(), term()) ->
+{continue | retry, term()}.
+handle_response({Pid, Epoch}, Code, Headers, Body) ->
+Args = {handle_response, Code, Headers, Body, Epoch},
+{Retry, Epoch1} = gen_server:call(Pid, Args, infinity),
+{Retry, {Pid, Epoch1}}.
+
+
+-spec cleanup(term()) -> ok.
+cleanup({Pid, _Epoch}) ->
+gen_server:call(Pid, stop, infinity).
+
+
+%% Definitions
+
+-define(MIN_UPDATE_INTERVAL, 5).
+
+
+%% gen_server state
+
+-record(state, {
+epoch = 0 :: non_neg_integer(),
+cookie :: string() | undefined,
+user :: string() | undefined,
+pass :: string() | undefined,
[GitHub] rnewson commented on a change in pull request #1176: Implement pluggable authentication and session support for replicator
rnewson commented on a change in pull request #1176: Implement pluggable
authentication and session support for replicator
URL: https://github.com/apache/couchdb/pull/1176#discussion_r170086765
##
File path: src/couch_replicator/src/couch_replicator_auth_session.erl
##
@@ -0,0 +1,545 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+
+% This is the replicator session auth plugin. It implements session based
+% authentication for the replicator. The only public API are the functions from
+% the couch_replicator_auth behaviour. Most of the logic and state is in the
+% gen_server. An instance of a gen_server could be spawned for the source and
+% target endpoints of each replication jobs.
+%
+% The workflow is roughly this:
+%
+% * On initialization, try to get a cookie in `refresh/1` If an error occurs,
+%the crash. If `_session` endpoint fails with a 404 (not found), return
+%`ignore` assuming session authentication is not support or we simply hit a
+%non-CouchDb server.
+%
+% * Before each request, auth framework calls `update_headers` API function.
+%Before updating the headers and returning, check if need to refresh again.
+%The check looks `next_refresh` time. If that time is set (not `infinity`)
+%and just expired, then obtain a new cookie, then update headers and
+%return.
+%
+% * After each request, auth framework calls `handle_response` function. If
+%request was successful check if a new cookie was sent by the server in the
+%`Set-Cookie` header. If it was then then that becomes the current cookie.
+%
+% * If last request has an auth failure, check if request used a stale cookie
+%In this case nothing is done, and the client is told to retry. Next time
+%it updates its headers befor the request it should pick up the latest
+%cookie.
+%
+% * If last request failed and cookie was the latest known cookie, schedule a
+%refresh and tell client to retry. However, if the cookie was just updated,
+%tell the client to continue such that it will handle the auth failure on
+%its own via a set of retries with exponential backoffs. This is it to
+%ensure if something goes wrong and one of the endpoints issues invalid
+%cookies, replicator won't be stuck in a busy loop refreshing them.
+
+
+-module(couch_replicator_auth_session).
+
+
+-behaviour(couch_replicator_auth).
+-behaviour(gen_server).
+
+
+-export([
+initialize/1,
+update_headers/2,
+handle_response/4,
+cleanup/1
+]).
+
+-export([
+init/1,
+terminate/2,
+handle_call/3,
+handle_cast/2,
+handle_info/2,
+code_change/3,
+format_status/2
+]).
+
+
+-include_lib("ibrowse/include/ibrowse.hrl").
+-include_lib("couch_replicator/include/couch_replicator_api_wrap.hrl").
+
+
+-type headers() :: [{string(), string()}].
+-type code() :: non_neg_integer().
+-type creds() :: {string() | undefined, string() | undefined}.
+
+
+% Behavior API callbacks
+
+
+-spec initialize(#httpdb{}) -> {ok, #httpdb{}, term()} | ignore.
+initialize(#httpdb{} = HttpDb) ->
+case remove_creds(HttpDb) of
+{ok, User, Pass, HttpDb1} ->
+case gen_server:start_link(?MODULE, [User, Pass, HttpDb1], []) of
Review comment:
talk to @davisp, there's a defined pattern to follow for gen_servers that
might fail at startup.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] rnewson commented on a change in pull request #1176: Implement pluggable authentication and session support for replicator
rnewson commented on a change in pull request #1176: Implement pluggable
authentication and session support for replicator
URL: https://github.com/apache/couchdb/pull/1176#discussion_r170086911
##
File path: src/couch_replicator/src/couch_replicator_auth_session.erl
##
@@ -0,0 +1,545 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+
+% This is the replicator session auth plugin. It implements session based
+% authentication for the replicator. The only public API are the functions from
+% the couch_replicator_auth behaviour. Most of the logic and state is in the
+% gen_server. An instance of a gen_server could be spawned for the source and
+% target endpoints of each replication jobs.
+%
+% The workflow is roughly this:
+%
+% * On initialization, try to get a cookie in `refresh/1` If an error occurs,
+%the crash. If `_session` endpoint fails with a 404 (not found), return
+%`ignore` assuming session authentication is not support or we simply hit a
+%non-CouchDb server.
+%
+% * Before each request, auth framework calls `update_headers` API function.
+%Before updating the headers and returning, check if need to refresh again.
+%The check looks `next_refresh` time. If that time is set (not `infinity`)
+%and just expired, then obtain a new cookie, then update headers and
+%return.
+%
+% * After each request, auth framework calls `handle_response` function. If
+%request was successful check if a new cookie was sent by the server in the
+%`Set-Cookie` header. If it was then then that becomes the current cookie.
+%
+% * If last request has an auth failure, check if request used a stale cookie
+%In this case nothing is done, and the client is told to retry. Next time
+%it updates its headers befor the request it should pick up the latest
+%cookie.
+%
+% * If last request failed and cookie was the latest known cookie, schedule a
+%refresh and tell client to retry. However, if the cookie was just updated,
+%tell the client to continue such that it will handle the auth failure on
+%its own via a set of retries with exponential backoffs. This is it to
+%ensure if something goes wrong and one of the endpoints issues invalid
+%cookies, replicator won't be stuck in a busy loop refreshing them.
+
+
+-module(couch_replicator_auth_session).
+
+
+-behaviour(couch_replicator_auth).
+-behaviour(gen_server).
+
+
+-export([
+initialize/1,
+update_headers/2,
+handle_response/4,
+cleanup/1
+]).
+
+-export([
+init/1,
+terminate/2,
+handle_call/3,
+handle_cast/2,
+handle_info/2,
+code_change/3,
+format_status/2
+]).
+
+
+-include_lib("ibrowse/include/ibrowse.hrl").
+-include_lib("couch_replicator/include/couch_replicator_api_wrap.hrl").
+
+
+-type headers() :: [{string(), string()}].
+-type code() :: non_neg_integer().
+-type creds() :: {string() | undefined, string() | undefined}.
+
+
+% Behavior API callbacks
+
+
+-spec initialize(#httpdb{}) -> {ok, #httpdb{}, term()} | ignore.
+initialize(#httpdb{} = HttpDb) ->
+case remove_creds(HttpDb) of
Review comment:
`extract_creds` maybe better, given it not only removes them but returns
them.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] rnewson commented on a change in pull request #1176: Implement pluggable authentication and session support for replicator
rnewson commented on a change in pull request #1176: Implement pluggable
authentication and session support for replicator
URL: https://github.com/apache/couchdb/pull/1176#discussion_r170087462
##
File path: src/couch_replicator/src/couch_replicator_auth_session.erl
##
@@ -0,0 +1,545 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+
+% This is the replicator session auth plugin. It implements session based
+% authentication for the replicator. The only public API are the functions from
+% the couch_replicator_auth behaviour. Most of the logic and state is in the
+% gen_server. An instance of a gen_server could be spawned for the source and
+% target endpoints of each replication jobs.
+%
+% The workflow is roughly this:
+%
+% * On initialization, try to get a cookie in `refresh/1` If an error occurs,
+%the crash. If `_session` endpoint fails with a 404 (not found), return
+%`ignore` assuming session authentication is not support or we simply hit a
+%non-CouchDb server.
+%
+% * Before each request, auth framework calls `update_headers` API function.
+%Before updating the headers and returning, check if need to refresh again.
+%The check looks `next_refresh` time. If that time is set (not `infinity`)
+%and just expired, then obtain a new cookie, then update headers and
+%return.
+%
+% * After each request, auth framework calls `handle_response` function. If
+%request was successful check if a new cookie was sent by the server in the
+%`Set-Cookie` header. If it was then then that becomes the current cookie.
+%
+% * If last request has an auth failure, check if request used a stale cookie
+%In this case nothing is done, and the client is told to retry. Next time
+%it updates its headers befor the request it should pick up the latest
+%cookie.
+%
+% * If last request failed and cookie was the latest known cookie, schedule a
+%refresh and tell client to retry. However, if the cookie was just updated,
+%tell the client to continue such that it will handle the auth failure on
+%its own via a set of retries with exponential backoffs. This is it to
+%ensure if something goes wrong and one of the endpoints issues invalid
+%cookies, replicator won't be stuck in a busy loop refreshing them.
+
+
+-module(couch_replicator_auth_session).
+
+
+-behaviour(couch_replicator_auth).
+-behaviour(gen_server).
+
+
+-export([
+initialize/1,
+update_headers/2,
+handle_response/4,
+cleanup/1
+]).
+
+-export([
+init/1,
+terminate/2,
+handle_call/3,
+handle_cast/2,
+handle_info/2,
+code_change/3,
+format_status/2
+]).
+
+
+-include_lib("ibrowse/include/ibrowse.hrl").
+-include_lib("couch_replicator/include/couch_replicator_api_wrap.hrl").
+
+
+-type headers() :: [{string(), string()}].
+-type code() :: non_neg_integer().
+-type creds() :: {string() | undefined, string() | undefined}.
+
+
+% Behavior API callbacks
+
+
+-spec initialize(#httpdb{}) -> {ok, #httpdb{}, term()} | ignore.
+initialize(#httpdb{} = HttpDb) ->
+case remove_creds(HttpDb) of
+{ok, User, Pass, HttpDb1} ->
+case gen_server:start_link(?MODULE, [User, Pass, HttpDb1], []) of
+{ok, Pid} ->
+{ok, HttpDb1, {Pid, 0}};
+ignore ->
+ignore;
+{error, Error} ->
+{error, Error}
+end;
+{error, missing_credentials} ->
+ignore;
+{error, Error} ->
+{error, Error}
+end.
+
+
+-spec update_headers(term(), headers()) -> {headers(), term()}.
+update_headers({Pid, Epoch}, Headers) ->
+Args = {update_headers, Headers, Epoch},
+{Headers1, Epoch1} = gen_server:call(Pid, Args, infinity),
Review comment:
why infinity here when the only thing this call can do is an http request
with a defined timeout? infinity for :call is used in couchdb where we do disk
i/o as that's truly hard to predict, but best avoided if we can put a
reasonable finite bound instead.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] rnewson commented on a change in pull request #1176: Implement pluggable authentication and session support for replicator
rnewson commented on a change in pull request #1176: Implement pluggable
authentication and session support for replicator
URL: https://github.com/apache/couchdb/pull/1176#discussion_r170064207
##
File path: src/couch_replicator/include/couch_replicator_api_wrap.hrl
##
@@ -26,13 +26,6 @@
httpc_pool = nil,
http_connections,
first_error_timestamp = nil,
-proxy_url
-}).
-
--record(oauth, {
-consumer_key,
-token,
-token_secret,
-consumer_secret,
-signature_method
+proxy_url,
+auth_context = nil
Review comment:
before I go deeper, we're changing the default for one field and adding an
extra property. Are we sure there's no upgrade problem here? I think we never
pass httpdb's between nodes but this would be bad if we did.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] rnewson commented on a change in pull request #1176: Implement pluggable authentication and session support for replicator
rnewson commented on a change in pull request #1176: Implement pluggable
authentication and session support for replicator
URL: https://github.com/apache/couchdb/pull/1176#discussion_r170064207
##
File path: src/couch_replicator/include/couch_replicator_api_wrap.hrl
##
@@ -26,13 +26,6 @@
httpc_pool = nil,
http_connections,
first_error_timestamp = nil,
-proxy_url
-}).
-
--record(oauth, {
-consumer_key,
-token,
-token_secret,
-consumer_secret,
-signature_method
+proxy_url,
+auth_context = nil
Review comment:
before I go deeper, we're changing the default for one field and adding an
extra property. Are we sure there's no upgrade problem here? I think we never
pass httpdb's between nodes but this would be bad if we did.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
