Github user kxepal commented on the issue:
https://github.com/apache/couchdb-couch-replicator/pull/64
THATS. AN. EPIC.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
[
https://issues.apache.org/jira/browse/COUCHDB-3324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15925398#comment-15925398
]
Nick Vatamaniuc commented on COUCHDB-3324:
--
Fauxton PR https://github.com/apach
Github user rnewson commented on the issue:
https://github.com/apache/couchdb-couch/pull/237
tests fixed (sorry @eiri I got there first)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this f
Github user nickva commented on the issue:
https://github.com/apache/couchdb-couch/pull/237
Tried it locally. Looks good.
+1 After tests are fixed.
We'd want to performance tests this at some point.
---
If your project is set up for it, you can reply to this email
[
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15925154#comment-15925154
]
ASF GitHub Bot commented on COUCHDB-2992:
-
Github user tonysun83 closed the pull
Github user tonysun83 closed the pull request at:
https://github.com/apache/couchdb-chttpd/pull/114
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feat
Github user tonysun83 commented on the issue:
https://github.com/apache/couchdb-chttpd/pull/114
Closing this as new PRs have been created
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Robert Newson created COUCHDB-3325:
--
Summary: Track _replicator/_user shards with LRU
Key: COUCHDB-3325
URL: https://issues.apache.org/jira/browse/COUCHDB-3325
Project: CouchDB
Issue Type: B
[
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15924968#comment-15924968
]
ASF subversion and git services commented on COUCHDB-2992:
--
Com
[
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15924969#comment-15924969
]
ASF subversion and git services commented on COUCHDB-2992:
--
Com
[
https://issues.apache.org/jira/browse/COUCHDB-2964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15924970#comment-15924970
]
ASF subversion and git services commented on COUCHDB-2964:
--
Com
[
https://issues.apache.org/jira/browse/COUCHDB-3324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15924955#comment-15924955
]
ASF GitHub Bot commented on COUCHDB-3324:
-
GitHub user nickva opened a pull requ
GitHub user nickva opened a pull request:
https://github.com/apache/couchdb-chttpd/pull/158
63012 scheduler
This is part of a set of PRs to merge the new scheduling replicator
Main PR is this: apache/couchdb-couch-replicator#64
Top level PR to gather and help test a
[
https://issues.apache.org/jira/browse/COUCHDB-3324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15924947#comment-15924947
]
ASF GitHub Bot commented on COUCHDB-3324:
-
GitHub user nickva opened a pull requ
GitHub user nickva opened a pull request:
https://github.com/apache/couchdb-couch/pull/238
Add _replication_start_time to the doc field validation section.
This is part of a set of PRs to merge the new scheduling replicator
Main PR is this: https://github.com/apache/couchdb-
[
https://issues.apache.org/jira/browse/COUCHDB-3324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15924944#comment-15924944
]
ASF GitHub Bot commented on COUCHDB-3324:
-
GitHub user nickva opened a pull requ
GitHub user nickva opened a pull request:
https://github.com/apache/couchdb-couch-replicator/pull/64
63012 scheduler
Pull request to merge scheduling replicator work to ASF master.
This repository has most of the changes. The feature overall consists of
updates to 3 reposit
Github user rnewson commented on the issue:
https://github.com/apache/couchdb-couch/pull/237
ah, true, yes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so,
Github user sagelywizard commented on a diff in the pull request:
https://github.com/apache/couchdb-couch/pull/237#discussion_r106014184
--- Diff: src/couch_server.erl ---
@@ -542,17 +530,11 @@ handle_info(restart_config_listener, State) ->
handle_info(Info, Server) ->
Github user sagelywizard commented on a diff in the pull request:
https://github.com/apache/couchdb-couch/pull/237#discussion_r106014081
--- Diff: src/couch_server.erl ---
@@ -271,20 +268,18 @@ all_databases(Fun, Acc0) ->
{ok, FinalAcc}.
-make_room(Server, O
Github user sagelywizard commented on a diff in the pull request:
https://github.com/apache/couchdb-couch/pull/237#discussion_r106013787
--- Diff: src/couch_server.erl ---
@@ -91,11 +91,8 @@ open(DbName, Options0) ->
end
end.
-update_lru(DbName, Optio
[
https://issues.apache.org/jira/browse/COUCHDB-3324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15924899#comment-15924899
]
ASF GitHub Bot commented on COUCHDB-3324:
-
GitHub user nickva opened a pull requ
nickva opened a new pull request #454: Point to scheduling replicator
dependencies.
URL: https://github.com/apache/couchdb/pull/454
This updates chttpd, couch, and couch_replicator to point to current commits
on
63012-scheduler branches which contain code for the new scheduling
replica
Github user sagelywizard commented on the issue:
https://github.com/apache/couchdb-couch/pull/237
I think test/couchdb_view_tests.erl needs to be updated.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project do
Github user eiri commented on the issue:
https://github.com/apache/couchdb-couch/pull/237
Looks good to me.
For a reference [here is what I came up
with](https://github.com/apache/couchdb-couch/compare/master...cloudant:add-lru-excluded-option)
---
If your project is set up
Nick Vatamaniuc created COUCHDB-3324:
Summary: Scheduling Replicatorr
Key: COUCHDB-3324
URL: https://issues.apache.org/jira/browse/COUCHDB-3324
Project: CouchDB
Issue Type: New Feature
Github user rnewson commented on a diff in the pull request:
https://github.com/apache/couchdb-couch/pull/237#discussion_r106005626
--- Diff: src/couch_server.erl ---
@@ -91,11 +91,8 @@ open(DbName, Options0) ->
end
end.
-update_lru(DbName, Options) -
[
https://issues.apache.org/jira/browse/COUCHDB-3277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15924851#comment-15924851
]
ASF subversion and git services commented on COUCHDB-3277:
--
Com
Github user sagelywizard commented on a diff in the pull request:
https://github.com/apache/couchdb-couch/pull/237#discussion_r106000880
--- Diff: src/couch_server.erl ---
@@ -271,20 +268,18 @@ all_databases(Fun, Acc0) ->
{ok, FinalAcc}.
-make_room(Server, O
Github user sagelywizard commented on a diff in the pull request:
https://github.com/apache/couchdb-couch/pull/237#discussion_r106000729
--- Diff: src/couch_server.erl ---
@@ -91,11 +91,8 @@ open(DbName, Options0) ->
end
end.
-update_lru(DbName, Optio
Github user sagelywizard commented on a diff in the pull request:
https://github.com/apache/couchdb-couch/pull/237#discussion_r106001159
--- Diff: src/couch_server.erl ---
@@ -542,17 +530,11 @@ handle_info(restart_config_listener, State) ->
handle_info(Info, Server) ->
GitHub user rnewson opened a pull request:
https://github.com/apache/couchdb-couch/pull/237
Add sys_dbs to the LRU
_replicator and _users db's have been excluded from the LRU for a long
time. This is fine when you have one of each, it's not fine when you have tens
of thousands of e
Github user nickva commented on the issue:
https://github.com/apache/couchdb-couch/pull/236
And it would be non-portable (on windows at least)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user davisp commented on the issue:
https://github.com/apache/couchdb-couch/pull/236
I would do it in the Makefile before we run the test suite, not as part of
the code bits.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user nickva commented on the issue:
https://github.com/apache/couchdb-couch/pull/236
@davisp python has a way to check file descriptor limits, I am not sure
erlang has it:
```
In [9]: resource.getrlimit(resource.RLIMIT_NOFILE)
Out[9]: (65536, 65536)
In
Github user davisp commented on the issue:
https://github.com/apache/couchdb-couch/pull/236
@nickva @eiri Aha! Different conversation!
Removing the max_dbs_open hard limit and swapping to a soft limit has been
suggested a few times and then we'd end up relying on the nfile ker
Github user nickva commented on the issue:
https://github.com/apache/couchdb-couch/pull/236
Updated with using hibernate `wakeup` trick and not having the silly timer
in there as well
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitH
Github user nickva commented on the issue:
https://github.com/apache/couchdb-couch/pull/236
@eiri the eviction policy would be used when we are bumping against the
limit. This could happen under a minute time-frame. A minute could be for
example 1 hour (maybe it should be?). Then if w
Github user eiri commented on the issue:
https://github.com/apache/couchdb-couch/pull/236
@davisp ok, I'm not talking about the issue we had, I'm talking about this
whole change - we effectively not using lru as eviction policy anymore, so
there are no point of tracking shards access
Github user davisp commented on a diff in the pull request:
https://github.com/apache/couchdb-couch-mrview/pull/67#discussion_r105942223
--- Diff: src/couch_mrview_index.erl ---
@@ -230,20 +230,23 @@ update_local_purge_doc(Db, State) ->
verify_index_exists(Options) ->
Github user davisp commented on the issue:
https://github.com/apache/couchdb-couch/pull/236
@eiri Am I mistaken that not calling db_closed was the issue? I'm not
seeing the subtlety of your clarification.
---
If your project is set up for it, you can reply to this email and have your
Github user eiri commented on the issue:
https://github.com/apache/couchdb-couch/pull/236
@davisp @nickva I think I haven't made myself clear. If we are closing
_everything_ idle, sys and none-sys, then that means when we are hitting
max_dbs_open we have max_dbs_open number of none-id
Github user davisp commented on the issue:
https://github.com/apache/couchdb-couch/pull/236
@nickva I don't think its worth having two settings when we're looking at
minutes as anything under load ends up reverting to the current behavior
(assuming we don't introduce bugs ;).
Github user davisp commented on the issue:
https://github.com/apache/couchdb-couch/pull/236
@eiri The subtlety here is that anything marked as a sys_db is outside the
LRU and never gets closed (ie, they exist outside the max_dbs_open setting).
The change here is to close anything that
Github user nickva commented on the issue:
https://github.com/apache/couchdb-couch/pull/236
@eiri lru would be used to remove and makes room immediately if we are
bumping against the limit, specifically it would remove non sys db shards
first.
idle-based remove is to clean u
Github user eiri commented on the issue:
https://github.com/apache/couchdb-couch/pull/236
@davisp, @nickva I think I'm a bit confused here: if we don't use `lru` to
decide what to evict, what the point of having it? I mean, with this change we
indiscriminately removing anything that
Github user nickva commented on the issue:
https://github.com/apache/couchdb-couch/pull/236
Hibernate thing can be simplified then. I'll give that a try.
On _users I see it with a few replications running. And I see replication
sources and _users shard appear in the ets table.
Github user davisp commented on the issue:
https://github.com/apache/couchdb-couch/pull/236
I'm not seeing the users thing as being an issue. Unless you mean its being
actively read when it closes in which case our is_idle check is wrong?
---
If your project is set up for it, you can
Github user davisp commented on the issue:
https://github.com/apache/couchdb-couch/pull/236
For GC, that may be enough. Granted hibernate is more aggressive and just
calling erlang:garbage_collect() can run into issues that hibernate fixes (ie,
lots of ram and not much activity, it'll
Github user davisp commented on a diff in the pull request:
https://github.com/apache/couchdb-couch-mrview/pull/67#discussion_r105929893
--- Diff: src/couch_mrview_index.erl ---
@@ -204,3 +208,56 @@ index_file_exists(State) ->
} = State,
IndexFName = couch_mrview_u
Github user davisp commented on a diff in the pull request:
https://github.com/apache/couchdb-couch-mrview/pull/67#discussion_r105926830
--- Diff: test/couch_mrview_purge_docs_tests.erl ---
@@ -0,0 +1,126 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you
Github user davisp commented on a diff in the pull request:
https://github.com/apache/couchdb-couch-mrview/pull/67#discussion_r105926127
--- Diff: src/couch_mrview_index.erl ---
@@ -204,3 +208,56 @@ index_file_exists(State) ->
} = State,
IndexFName = couch_mrview_u
Github user davisp commented on a diff in the pull request:
https://github.com/apache/couchdb-couch-mrview/pull/67#discussion_r105926680
--- Diff: src/couch_mrview_util.erl ---
@@ -39,6 +41,26 @@
-include_lib("couch_mrview/include/couch_mrview.hrl").
+get_local_
Github user davisp commented on a diff in the pull request:
https://github.com/apache/couchdb-couch-mrview/pull/67#discussion_r105926330
--- Diff: src/couch_mrview_index.erl ---
@@ -204,3 +208,56 @@ index_file_exists(State) ->
} = State,
IndexFName = couch_mrview_u
Github user nickva commented on the issue:
https://github.com/apache/couchdb-couch/pull/236
Ah makes sense we don't do that on delete so was wondering about that.
You're right will update code.
Also what do you think of not using hibernate and instead just a gc to
avoid manag
Github user davisp commented on a diff in the pull request:
https://github.com/apache/couchdb-couch/pull/236#discussion_r105919410
--- Diff: src/couch_db_updater.erl ---
@@ -1454,3 +1468,44 @@ default_security_object(_DbName) ->
"everyone" ->
[]
Github user davisp commented on the issue:
https://github.com/apache/couchdb-couch/pull/236
@eiri @nickva Ah! Though slightly different. The bug is that we weren't
calling db_closed to decrement the number of open databases. Nothing to do with
system dbs being in ets. If you read the
Github user nickva commented on a diff in the pull request:
https://github.com/apache/couchdb-couch/pull/236#discussion_r105918717
--- Diff: src/couch_db_updater.erl ---
@@ -1454,3 +1468,44 @@ default_security_object(_DbName) ->
"everyone" ->
[]
Github user davisp commented on a diff in the pull request:
https://github.com/apache/couchdb-couch/pull/236#discussion_r105918204
--- Diff: src/couch_db_updater.erl ---
@@ -1454,3 +1468,28 @@ default_security_object(_DbName) ->
"everyone" ->
[]
Github user rnewson commented on a diff in the pull request:
https://github.com/apache/couchdb-couch/pull/236#discussion_r105881200
--- Diff: src/couch_db_updater.erl ---
@@ -1454,3 +1468,44 @@ default_security_object(_DbName) ->
"everyone" ->
[]
Github user nickva commented on the issue:
https://github.com/apache/couchdb-couch/pull/235
@sagelywizard should be a bit better now. But ended up with another pr for
fabric as well.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHu
Github user nickva commented on the issue:
https://github.com/apache/couchdb-couch/pull/235
Related pr:
* https://github.com/apache/couchdb-fabric/pull/91
* https://github.com/apache/couchdb-chttpd/pull/157
---
If your project is set up for it, you can reply to thi
[
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15923697#comment-15923697
]
ASF GitHub Bot commented on COUCHDB-2992:
-
GitHub user nickva opened a pull requ
GitHub user nickva opened a pull request:
https://github.com/apache/couchdb-fabric/pull/91
Allow limiting maximum document body size
Update doc function to check and validate document body sizes
Main implementation is in PR:
https://github.com/apache/couchdb-couch/
Github user nickva commented on the issue:
https://github.com/apache/couchdb-fabric/pull/91
Related to: https://github.com/apache/couchdb-couch/pull/235
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
65 matches
Mail list logo
