Re: [2/4] git commit: restore couch_util:verify call in oauth.

2011-10-13 Thread Randall Leeds
I'm not sure why this change was made unless it's to prevent timing attacks?
couch_util:verify seems to do something equivalent and it's strange to me
that we should be making calls into the couchdb sources from erlang-oauth,
which should be independent. If it's for the timing reason, can we put a
verify inside erlang-oauth and get it upstream or something for the future?

-Randall

On Thu, Oct 13, 2011 at 07:14, rnew...@apache.org wrote:

 restore couch_util:verify call in oauth.


 Project: http://git-wip-us.apache.org/repos/asf/couchdb/repo
 Commit: http://git-wip-us.apache.org/repos/asf/couchdb/commit/ed9b6663
 Tree: http://git-wip-us.apache.org/repos/asf/couchdb/tree/ed9b6663
 Diff: http://git-wip-us.apache.org/repos/asf/couchdb/diff/ed9b6663

 Branch: refs/heads/1.2.x
 Commit: ed9b6663f4a232e0728c509a0cf582fd27cc0ffa
 Parents: 94313f3
 Author: Robert Newson rnew...@apache.org
 Authored: Thu Oct 13 15:09:02 2011 +0100
 Committer: Robert Newson rnew...@apache.org
 Committed: Thu Oct 13 15:09:18 2011 +0100

 --
  src/erlang-oauth/oauth_hmac_sha1.erl |2 +-
  src/erlang-oauth/oauth_plaintext.erl |2 +-
  2 files changed, 2 insertions(+), 2 deletions(-)
 --



 http://git-wip-us.apache.org/repos/asf/couchdb/blob/ed9b6663/src/erlang-oauth/oauth_hmac_sha1.erl
 --
 diff --git a/src/erlang-oauth/oauth_hmac_sha1.erl
 b/src/erlang-oauth/oauth_hmac_sha1.erl
 index 69064ed..35549cf 100644
 --- a/src/erlang-oauth/oauth_hmac_sha1.erl
 +++ b/src/erlang-oauth/oauth_hmac_sha1.erl
 @@ -8,4 +8,4 @@ signature(BaseString, CS, TS) -
   base64:encode_to_string(crypto:sha_mac(Key, BaseString)).

  verify(Signature, BaseString, CS, TS) -
 -  Signature =:= signature(BaseString, CS, TS).
 +  couch_util:verify(Signature, signature(BaseString, CS, TS)).


 http://git-wip-us.apache.org/repos/asf/couchdb/blob/ed9b6663/src/erlang-oauth/oauth_plaintext.erl
 --
 diff --git a/src/erlang-oauth/oauth_plaintext.erl
 b/src/erlang-oauth/oauth_plaintext.erl
 index d8085e0..9544a0a 100644
 --- a/src/erlang-oauth/oauth_plaintext.erl
 +++ b/src/erlang-oauth/oauth_plaintext.erl
 @@ -7,4 +7,4 @@ signature(CS, TS) -
   oauth_uri:calate(, [CS, TS]).

  verify(Signature, CS, TS) -
 -  Signature =:= signature(CS, TS).
 +  couch_util:verify(Signature, signature(CS, TS)).




[jira] [Commented] (COUCHDB-257) HTTP caching headers don't provide expected behaviour

2011-10-13 Thread carsten madsen (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/COUCHDB-257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13126700#comment-13126700
 ] 

carsten madsen commented on COUCHDB-257:


This bug also affects me on ie9. Running on cloudant (couchdb 1.0.2) ./_session 
in IE9 with jquery.couch always gives 304 as result.

 HTTP caching headers don't provide expected behaviour
 -

 Key: COUCHDB-257
 URL: https://issues.apache.org/jira/browse/COUCHDB-257
 Project: CouchDB
  Issue Type: Bug
  Components: HTTP Interface
Affects Versions: 0.8.1, 0.9
 Environment: Server: Ubuntu Hardy on x86. Client: Windows XP (32-bit).
Reporter: Vinay Sajip
Priority: Minor
 Attachments: caching-header-patch.diff, expires.patch


 The HTTP caching headers currently put out cause IE (for example) to not 
 display information correctly in Futon. It's easy to reproduce: I open 
 windows in Firefox and IE simultaneously, do an update using Firefox (e.g. 
 add a new document) and refresh the IE window. The updated document count is 
 not shown. If I clear the browser cache and try again, the updated 
 information is displayed.  The HTTP header put out is
 Cache-Control: must-revalidate
 which seems to me insufficient - for IE, at least. Is there way of 
 configuring these headers, to for example 
 Cache-Control: no-cache
 Pragma: no-cache
 Expires: some date in the past, or the same value as the Date: header
 Christopher Lenz has said about this that This is due to extra-aggressive 
 (and against the HTTP spec) caching   that IE does on XMLHTTPRequests. A 
 patch would need to do user agent sniffing to conditionally add the  cache: 
 false  parameter to the jQuery ajax() invocations in   jquery.couch.js (and 
 maybe elsewhere). I wouldn't want to add this for all user agents, as it 
 basically circumvents any caching for AJAX  requests (even for 
 not-craptastically-broken implementations), and  thus would add quite a bit 
 of unnecessary overhead.
 To this, I would comment that I don't believe a patch to the client-side code 
 in Futon would be sufficient. There are other clients out there, some of 
 which will be on Windows and so by default use the (acknowledgely broken) 
 Microsoft stack. In my view it is more important to err on the side of 
 correctness than performance - so I believe the headers generated server-side 
 need to change, as well as perhaps Futon client-side changes.
 I note that handle_uuids_req in couch_httpd_misc_handlers.erl uses the 
 no-cache/Expires scheme I mention.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




Re: status of universal binary json in couchdb

2011-10-13 Thread Riyad Kalla
I'd be happy to sponsor getting this work into CouchDB proper if anyone was
interested.

On Tue, Oct 11, 2011 at 10:59 PM, Dominic Tarr dominic.t...@gmail.comwrote:

 okay, interesting.

 that sounds a bit more difficult than the parser! but nevertheless the
 right
 approach.

 I'll admit now that I do not know erlang yet. I've been wanting to get into
 it though, and implementing ubj seemed both interesting and useful.
 refactoring couchbd, prehaps more challenging.

 good to know the lie of the land, thanks for your answer!

 cheers, Dominic



 On Wed, Oct 12, 2011 at 3:43 PM, Paul Davis paul.joseph.da...@gmail.com
 wrote:

  On Tue, Oct 11, 2011 at 11:17 PM, Dominic Tarr dominic.t...@gmail.com
  wrote:
   I read here
  https://plus.google.com/107397941677313236670/posts/LFBB233PKQ1
   that
   UBJ was under consideration?
  
   The spec for ubj looks quite good, but there are not many
 implementations
   available yet.
   I am considering writing some in (node)js and erlang.
  
   would support for ubj be added if there where supporting libraries
   available.
  
   I have searched the archives and not found any discussion on this yet.
  
   cheers, Dominic.
  
 
  Dominic,
 
  We've discussed the possibility (possibility mind you) of adding it.
  As you note it's quite young and hasn't reached wide spread adoption
  and its always hard to guess about the future.
 
  The current status is basically that it'd be interesting to see a
  patch to CouchDB that abstracts the content generation for the JSON
  responses such that alternative content types could be implemented
  along with a patch that uses that API for JSON and UBJSON. Then we'd
  need to look into the performance characteristics and end-user support
  (ie, perhaps it could be a plugin).
 
  I won't sugar coat it. It'll be a tough sell to get something other
  than JSON into CouchDB as an officially supported Content-Type. It'll
  take quite a bit of work marshaling the patch through the technical
  and community aspects of such a change.
 
  That said if you or anyone else wants to move forward with it, I would
  propose a rough outline. First, a patch that abstracts the JSON
  generation into a generic API as well as the JSON reference
  implementation of that API. Second, a robust system for choosing the
  correct content-type based on HTTP Accept headers. And third, an
  alternative implementation using UBJSON.
 
  I'll restate that this would be a significant amount of work and
  there's always the possibility that the end result doesn't include
  alternative content-types in trunk. (Assuming the abstract API was
  solid I don't see us not acception that patch, but I make no promises
  :)
 
  Other notes:
 
  There was a discussion here:
 
 
 
 http://mail-archives.apache.org/mod_mbox/couchdb-dev/201109.mbox/%3CCABn9xAGK0WbPhpkbxyKQo9N1KEsm=rx81v7fvdargpjxjnc...@mail.gmail.com%3E
 
  Also, if you're interested in writing an Erlang version, I'd be
  interested in a patch to Jiffy [1]. I've been meaning to sit down and
  write it but its not a priority.
 
  [1] https://github.com/davisp/jiffy
 



[jira] [Closed] (COUCHDB-1295) couch_util:collate/3 leaks ports

2011-10-13 Thread Jan Lehnardt (Closed) (JIRA)

 [ 
https://issues.apache.org/jira/browse/COUCHDB-1295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Lehnardt closed COUCHDB-1295.
-

Resolution: Not A Problem

 couch_util:collate/3 leaks ports
 

 Key: COUCHDB-1295
 URL: https://issues.apache.org/jira/browse/COUCHDB-1295
 Project: CouchDB
  Issue Type: Bug
  Components: Database Core
Reporter: Yurii Rashkovskii

 couch_util:collate/3 opens a port but never closes nor links to it. Unless I 
 am missing something, it leaks ports. The most trivial solution that comes to 
 mind is to link to the said port but I had problems with it as those 
 processes didn't seem to die therefore port leakage continued.
 Explicitly closing port after every operation seems to do the trick so far, 
 but I admit it is an ugly solution.
 Any ideas?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Closed] (COUCHDB-1267) Bignums in views aren't sorted correctly.

2011-10-13 Thread Jan Lehnardt (Closed) (JIRA)

 [ 
https://issues.apache.org/jira/browse/COUCHDB-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Lehnardt closed COUCHDB-1267.
-

Resolution: Fixed

This has been fixed in Erlang R14B04.

 Bignums in views aren't sorted correctly.
 -

 Key: COUCHDB-1267
 URL: https://issues.apache.org/jira/browse/COUCHDB-1267
 Project: CouchDB
  Issue Type: Bug
  Components: JavaScript View Server
Affects Versions: 1.1.1, 1.2
Reporter: Paul Joseph Davis
Priority: Blocker

 Easily tested by creating a doc and hitting this as a temp view:
 function(doc) {
   emit([bylatest,-1303085266000], null);
   emit([bylatest,-1298817134000], null);
   emit([bylatest,-1294536544000], null);
   emit([bylatest,-1294505612000], null);
   emit([bylatest,-117870480], null);
 }

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Closed] (COUCHDB-1182) Crash after compaction is completed

2011-10-13 Thread Jan Lehnardt (Closed) (JIRA)

 [ 
https://issues.apache.org/jira/browse/COUCHDB-1182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Lehnardt closed COUCHDB-1182.
-

Resolution: Fixed

Closed as link issue is closed. Feel free to reopen with more info if the 
problem persists.

 Crash after compaction is completed
 ---

 Key: COUCHDB-1182
 URL: https://issues.apache.org/jira/browse/COUCHDB-1182
 Project: CouchDB
  Issue Type: Bug
Affects Versions: 1.0.2
 Environment: Centos 5.6
Reporter: Clare Walsh
  Labels: compaction
 Attachments: compaction fail.txt, post patch friday - noproc.txt


 When compaction is run, it seems to complete successfully (disappears from 
 active tasks), but then at the end or a minute or so afterwards couch crashes 
 [EDIT: that's a bit dramatic, it logs a crash, it goes up a few levels of 
 crash, but doesn't get all the way to crashed before it starts working again].
 This has happened twice so far, I am attaching logs for both occasions. The 
 'huge chunks of numbers' are all in a single binary array (ie no structure 
 has been removed) and are hundreds of pages long (hence removed). I have 
 truncated the logs as the same error appears three times as the supervisor is 
 restarting child, and then the rest of the chain upwards as it gradually 
 folds into a heap was very long. If any of this chain (or any other 
 information) is needed please let me know.
 Thanks

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Closed] (COUCHDB-1150) URL-Encoded DELETE Request returns 500 ( vs. %22 )

2011-10-13 Thread Jan Lehnardt (Closed) (JIRA)

 [ 
https://issues.apache.org/jira/browse/COUCHDB-1150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Lehnardt closed COUCHDB-1150.
-

Resolution: Not A Problem

DELETE doesn't require neither  nor %22.

 URL-Encoded DELETE Request returns 500 (  vs. %22 )
 

 Key: COUCHDB-1150
 URL: https://issues.apache.org/jira/browse/COUCHDB-1150
 Project: CouchDB
  Issue Type: Bug
  Components: HTTP Interface
Affects Versions: 1.0.2
 Environment: Windows 7 Pro, 64 bit
 HttpBuilder 0.5.1
 curl 7.21.5 (i386-pc-win32) libcurl/7.21.5 OpenSSL/0.9.8r zlib/1.2.5
Reporter: Armin Weisser
  Labels: Encoding

 I'm facing an issue with URL-Encoded DELETE Requests where double quotes () 
 are translated to %22.
 These GETS are equivalent
 curl -X GET http://localhost:5984/mydb/_design/views/_view/accounts?key=4711;
 curl -X GET 
 http://localhost:5984/mydb/_design/views/_view/accounts?key=%224711%22
 This DELETE is OK
 curl -X DELETE 
 http://localhost:5984/mydb/4711?rev=1-7511a58b76f9a9dec6a666ecc74302f6;
 But this one returns a 500
 curl -X DELETE 
 http://localhost:5984/mydb/4711?rev=%221-7511a58b76f9a9dec6a666ecc74302f6%22
 The problem accurs in Java (using HttpBuilder 0.5.1) and on cmd shell (using 
 curl).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Closed] (COUCHDB-701) Replication failed then couchdb crashes

2011-10-13 Thread Jan Lehnardt (Closed) (JIRA)

 [ 
https://issues.apache.org/jira/browse/COUCHDB-701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Lehnardt closed COUCHDB-701.


Resolution: Cannot Reproduce

 Replication failed then couchdb crashes
 ---

 Key: COUCHDB-701
 URL: https://issues.apache.org/jira/browse/COUCHDB-701
 Project: CouchDB
  Issue Type: Bug
  Components: Replication
Affects Versions: 0.10.1
 Environment: host A : 2.6.32.2 Debian 5 lenny
 host B : 2.6.30.7 Debian 5 lenny
Reporter: linkfluence
Priority: Critical
  Labels: couchdb, replication
 Attachments: couchdb.log.gz


 We have two hosts, host A and host B.
 We created an empty database named my_couch_db on each host.
 We configured a continuous replication on host A where source is 
 my_couch_db and target is http://host.B/my_couch_db;.
 When adding documents on host A, they are replicated to database on host B, 
 but after a few hours the replication crashed and then couchdb too.
 The replication stopped after replicating 30231 documents on my_couch_db of 
 host B.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Resolved] (COUCHDB-997) Prevent multiple keys from entering a btree.

2011-10-13 Thread Damien Katz (Resolved) (JIRA)

 [ 
https://issues.apache.org/jira/browse/COUCHDB-997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Katz resolved COUCHDB-997.
-

Resolution: Won't Fix
  Assignee: Damien Katz

We should rely on callers to the btree to make sure they don't introduce 
duplicates. If they don't, the bug should be fixed in the caller. Adding extra 
logic to elimnate duplicates silently may cause more problems, and isn't as 
correct as the callers doing the right things.

 Prevent multiple keys from entering a btree.
 

 Key: COUCHDB-997
 URL: https://issues.apache.org/jira/browse/COUCHDB-997
 Project: CouchDB
  Issue Type: Bug
Reporter: Paul Joseph Davis
Assignee: Damien Katz

 s/sort/usort/ at 
 https://github.com/apache/couchdb/blob/trunk/src/couchdb/couch_btree.erl#L181
 This should be completely transparent and incur minimal overhead. This hasn't 
 bitten us quite yet, but it would've prevented some of the crazier behavior 
 from COUCHDB-968

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




Re: CouchDB Replication lacking resilience for many database

2011-10-13 Thread Mark Hahn
cool.  Thanks.

On Tue, Oct 11, 2011 at 7:03 AM, Jan Lehnardt j...@apache.org wrote:


 On Oct 11, 2011, at 14:20 , Mark Hahn wrote:

  It would be nice to have a control panel that displays things like this
  message queue depth, connection counts, memory consumed, cpu consumed,
  reads/writes per second, view rebuilds/sec, avg response times, etc.  I'm
  sure someone could come up with many more pertinent vars.
 
  For extra credit the values could be plotted against time.  When someone
 has
  a problem they could post the log here.

 See /_stats :)

 It doesn't have all the things you ask for, but adding new stats isn't
 hard:

  http://wiki.apache.org/couchdb/Adding_Runtime_Statistics

 Cheers
 Jan
 --



 
  On Mon, Oct 10, 2011 at 10:15 PM, Paul Davis 
 paul.joseph.da...@gmail.comwrote:
 
  On Mon, Oct 10, 2011 at 11:03 PM, Chris Stockton
  chrisstockto...@gmail.com wrote:
  Hello,
 
  On Mon, Oct 10, 2011 at 5:18 PM, Adam Kocoloski kocol...@apache.org
  wrote:
  On Oct 10, 2011, at 8:02 PM, Chris Stockton wrote:
 
  Hello,
 
  On Mon, Oct 10, 2011 at 4:19 PM, Filipe David Manana
  fdman...@apache.org wrote:
  On Tue, Oct 11, 2011 at 12:03 AM, Chris Stockton
  chrisstockto...@gmail.com wrote:
  Chris,
 
  That said work is in the'1.2.x' branch (and master).
  CouchDB recently migrated from SVN to GIT, see:
  http://couchdb.apache.org/community/code.html
 
 
  Thank you very much for the response Filipe, do you possibly have any
  documentation or more detailed summary on what these changes include
  and possible benefits of them? I would love to hear about any
 tweaking
  or replication tips you may have for our growth issues, perhaps you
  could answer a basic question if nothing else: Do the changes in this
  branch minimize the performance impact of continuous replication on
  many databases?
 
  Regardless I plan on getting a build of that branch and doing some
  testing of my own very soon.
 
  Thank you!
 
  -Chris
 
  I'm pretty sure that even in 1.2.x and master each replication with a
  remote source still requires one dedicated TCP connection to consume the
  _changes feed.  Replications with a local source have always been able
 to
  use a connection pool per host:port combination.  That's not to downplay
 the
  significance of the rewrite of the replicator in 1.2.x; Filipe put quite
 a
  lot of time into it.
 
  The link to those darn errors just pointed to the mbox browser for
  September 2011.  Do you have a more specific link?  Regards,
 
  Adam
 
  Well I will remain optimistic that the rewrite could hopefully have
  solved several of my issues regardless I hope. I guess the idle TCP
  connections by themselves are not too bad, when they all start to work
  simultaneously I think is what becomes the issue =)
 
  Sorry Adam, here is a better link
 
 
 http://mail-archives.apache.org/mod_mbox/couchdb-user/201109.mbox/%3ccalkfbxuugljjy-nh46u0u584l+xdqm3ngspenxsjyrxospe...@mail.gmail.com%3E
  ,
  the actual text was:
 
  ---
 
  It seems that randomly I am getting errors about crashes as our
  replicator runs, all this replicator does is make sure that all
  databases on the master server replicate to our failover by checking
  status.
 
  Details:
  - I notice the below error in the logs, anywhere from 0 to 30 at a
 time.
  - It seems that a database might start replicating okay then stop.
  - These errors [1] are on the failover pulling from master
  - No errors are displayed on the master server
  - The databases inside the URL in the db_not_found portion of the
  error, are always available from curl from the failover machine, which
  makes the error strange, somehow it thinks it can't find the database
  - Master seems healthy at all times, all database are available, no
  errors in log
 
  [1] --
  [Mon, 12 Sep 2011 18:34:14 GMT] [error] [0.22466.5305]
  {error_report,0.30.0,
  {0.22466.5305,crash_report,
 
  [[{initial_call,{couch_rep,init,['Argument__1']}},
 {pid,0.22466.5305},
 {registered_name,[]},
 {error_info,
  {exit,
   {db_not_found,
http://user:pass@server
  :5984/db_10944/},
   [{gen_server,init_it,6},
{proc_lib,init_p_do_apply,3}]}},
 {ancestors,
  [couch_rep_sup,couch_primary_services,
   couch_server_sup,0.31.0]},
 {messages,[]},
 {links,[0.81.0]},
 {dictionary,[]},
 {trap_exit,true},
 {status,running},
 {heap_size,2584},
 {stack_size,24},
 {reductions,794}],
[]]}}
 
 
  One 

Re: CouchDB Replication lacking resilience for many database

2011-10-13 Thread Mark Hahn
It would be nice to have a control panel that displays things like this
message queue depth, connection counts, memory consumed, cpu consumed,
reads/writes per second, view rebuilds/sec, avg response times, etc.  I'm
sure someone could come up with many more pertinent vars.

For extra credit the values could be plotted against time.  When someone has
a problem they could post the log here.

On Mon, Oct 10, 2011 at 10:15 PM, Paul Davis paul.joseph.da...@gmail.comwrote:

 On Mon, Oct 10, 2011 at 11:03 PM, Chris Stockton
 chrisstockto...@gmail.com wrote:
  Hello,
 
  On Mon, Oct 10, 2011 at 5:18 PM, Adam Kocoloski kocol...@apache.org
 wrote:
  On Oct 10, 2011, at 8:02 PM, Chris Stockton wrote:
 
  Hello,
 
  On Mon, Oct 10, 2011 at 4:19 PM, Filipe David Manana
  fdman...@apache.org wrote:
  On Tue, Oct 11, 2011 at 12:03 AM, Chris Stockton
  chrisstockto...@gmail.com wrote:
  Chris,
 
  That said work is in the'1.2.x' branch (and master).
  CouchDB recently migrated from SVN to GIT, see:
  http://couchdb.apache.org/community/code.html
 
 
  Thank you very much for the response Filipe, do you possibly have any
  documentation or more detailed summary on what these changes include
  and possible benefits of them? I would love to hear about any tweaking
  or replication tips you may have for our growth issues, perhaps you
  could answer a basic question if nothing else: Do the changes in this
  branch minimize the performance impact of continuous replication on
  many databases?
 
  Regardless I plan on getting a build of that branch and doing some
  testing of my own very soon.
 
  Thank you!
 
  -Chris
 
  I'm pretty sure that even in 1.2.x and master each replication with a
 remote source still requires one dedicated TCP connection to consume the
 _changes feed.  Replications with a local source have always been able to
 use a connection pool per host:port combination.  That's not to downplay the
 significance of the rewrite of the replicator in 1.2.x; Filipe put quite a
 lot of time into it.
 
  The link to those darn errors just pointed to the mbox browser for
 September 2011.  Do you have a more specific link?  Regards,
 
  Adam
 
  Well I will remain optimistic that the rewrite could hopefully have
  solved several of my issues regardless I hope. I guess the idle TCP
  connections by themselves are not too bad, when they all start to work
  simultaneously I think is what becomes the issue =)
 
  Sorry Adam, here is a better link
 
 http://mail-archives.apache.org/mod_mbox/couchdb-user/201109.mbox/%3ccalkfbxuugljjy-nh46u0u584l+xdqm3ngspenxsjyrxospe...@mail.gmail.com%3E
 ,
  the actual text was:
 
  ---
 
  It seems that randomly I am getting errors about crashes as our
  replicator runs, all this replicator does is make sure that all
  databases on the master server replicate to our failover by checking
  status.
 
  Details:
   - I notice the below error in the logs, anywhere from 0 to 30 at a time.
   - It seems that a database might start replicating okay then stop.
   - These errors [1] are on the failover pulling from master
   - No errors are displayed on the master server
   - The databases inside the URL in the db_not_found portion of the
  error, are always available from curl from the failover machine, which
  makes the error strange, somehow it thinks it can't find the database
   - Master seems healthy at all times, all database are available, no
  errors in log
 
  [1] --
   [Mon, 12 Sep 2011 18:34:14 GMT] [error] [0.22466.5305]
  {error_report,0.30.0,
   {0.22466.5305,crash_report,
 
 [[{initial_call,{couch_rep,init,['Argument__1']}},
  {pid,0.22466.5305},
  {registered_name,[]},
  {error_info,
   {exit,
{db_not_found,
 http://user:pass@server
 :5984/db_10944/},
[{gen_server,init_it,6},
 {proc_lib,init_p_do_apply,3}]}},
  {ancestors,
   [couch_rep_sup,couch_primary_services,
couch_server_sup,0.31.0]},
  {messages,[]},
  {links,[0.81.0]},
  {dictionary,[]},
  {trap_exit,true},
  {status,running},
  {heap_size,2584},
  {stack_size,24},
  {reductions,794}],
 []]}}
 

 One place I've seen this error pop up when it looks like it shouldn't
 is if couch_server gets backed up. If you remsh into one of those db's
 you could try the following:

 process_info(whereis(couch_server), message_queue_len).

 And if that number keeps growing, that could be the issue.



Mouch is Make Couchapps with GNU Make

2011-10-13 Thread Johannes J. Schmidt
Dear developers,

I just released Mouch [1] (Make Couch) on github.
Mouch is a little helper to build Couchapps. I am using it since half a
year now almost every day and it does quiet well.
I hope someone else find it useful.

Mouch follows a radical minimalistic approach (125 LOC). It consists
only of three files, which you simply copy inside your project. Mouch
has only a few requirements: Ruby, a Ruby JSON lib, Make and cURL. You
might already have most of them installed.
With Mouch you build a JSON file and push it to the server. You talk to
Mouch via GNU Make.

I would love to hear discussion about Mouchs approach.

Johannes

[1] https://github.com/jo/mouch



[jira] [Created] (COUCHDB-1305) keep db crashes from bubbling through couch_server when possible

2011-10-13 Thread Randall Leeds (Created) (JIRA)
keep db crashes from bubbling through couch_server when possible


 Key: COUCHDB-1305
 URL: https://issues.apache.org/jira/browse/COUCHDB-1305
 Project: CouchDB
  Issue Type: Improvement
  Components: Database Core
Reporter: Randall Leeds
Assignee: Randall Leeds
Priority: Minor
 Fix For: 1.3


COUCHDB-1050 shows how an error in a db process can cause unexpected 'EXIT' 
messages to reach couch_server. These messages cause couch_server to restart, 
forcing all open databases to close, which has nasty effects like killing 
replications and index generation in unrelated databases. I suggest we use the 
couch_dbs_by_pid table to swallow the error, cleanup the ETS tables, log a 
message and continue. If a match isn't found in the table then the current 
unexpected message crash should be preserved.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (COUCHDB-1305) keep db crashes from bubbling through couch_server when possible

2011-10-13 Thread Paul Joseph Davis (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/COUCHDB-1305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13127064#comment-13127064
 ] 

Paul Joseph Davis commented on COUCHDB-1305:


Sounds like a good idea to me.

 keep db crashes from bubbling through couch_server when possible
 

 Key: COUCHDB-1305
 URL: https://issues.apache.org/jira/browse/COUCHDB-1305
 Project: CouchDB
  Issue Type: Improvement
  Components: Database Core
Reporter: Randall Leeds
Assignee: Randall Leeds
Priority: Minor
 Fix For: 1.3


 COUCHDB-1050 shows how an error in a db process can cause unexpected 'EXIT' 
 messages to reach couch_server. These messages cause couch_server to restart, 
 forcing all open databases to close, which has nasty effects like killing 
 replications and index generation in unrelated databases. I suggest we use 
 the couch_dbs_by_pid table to swallow the error, cleanup the ETS tables, log 
 a message and continue. If a match isn't found in the table then the current 
 unexpected message crash should be preserved.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (COUCHDB-1074) trouble URL rewriting for key/startkey/endkey qs params

2011-10-13 Thread Benjamin Young (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/COUCHDB-1074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13127232#comment-13127232
 ] 

Benjamin Young commented on COUCHDB-1074:
-

As mentioned in the comments on COUCHDB-1290 the fix for this issue forces 
Boolean values to be quoted, and therefore means past rewrite rules will break 
when someone upgrades to 1.1+

Rewrites need to be expressible in JSON, and requiring to quote Boolean values 
breaks that and adds a requirement for additional knowledge (write JSON docs 
everywhere but here...) to get rewrites to work in 1.1+.

Perhaps this is what Jan was referring to in his earlier comments.

 trouble URL rewriting for key/startkey/endkey qs params
 ---

 Key: COUCHDB-1074
 URL: https://issues.apache.org/jira/browse/COUCHDB-1074
 Project: CouchDB
  Issue Type: Bug
  Components: HTTP Interface
Reporter: Benoit Chesneau
Assignee: Benoit Chesneau
Priority: Blocker
  Labels: rewriter
 Fix For: 1.1, 1.2

 Attachments: 
 0001-fix-issue-COUCHDB-1074.-fix-variable-substitution.patch, 
 0001-fix-issue-COUCHDB-1074.-fix-variable-substitution.patch, 
 0001-fix-issue-COUCHDB-1074.-fix-variable-substitution.patch, 
 0001-fix-variable-substitutions.-handle-cases.patch, 
 0002-add-formating-in-variables-substitution.patch


 the variable substution not to happen, e.g.
 
  {
   query: { key: \:arg\ }
  }
 
 relative to this ml thread:
 http://mail-archives.apache.org/mod_mbox/couchdb-dev/201102.mbox/browser

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Created] (COUCHDB-1306) Allow

2011-10-13 Thread Benjamin Young (Created) (JIRA)
Allow 
--

 Key: COUCHDB-1306
 URL: https://issues.apache.org/jira/browse/COUCHDB-1306
 Project: CouchDB
  Issue Type: Bug
Reporter: Benjamin Young




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (COUCHDB-1306) Allow Booleans to be quoted in Rewrite rules

2011-10-13 Thread Benjamin Young (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/COUCHDB-1306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Young updated COUCHDB-1306:


  Skill Level: Regular Contributors Level (Easy to Medium)
   Labels: couchapp rewriter  (was: )
  Description: COUCHDB-1074 breaks the JSON format of Rewrite rules 
requiring that Boolean values always be quoted. This requires additional 
knowledge for a user to even understand why their JSON doesn't work with 
CouchDB.
Affects Version/s: 1.2
   1.1
  Summary: Allow Booleans to be quoted in Rewrite rules  (was: 
Allow )

 Allow Booleans to be quoted in Rewrite rules
 

 Key: COUCHDB-1306
 URL: https://issues.apache.org/jira/browse/COUCHDB-1306
 Project: CouchDB
  Issue Type: Bug
Affects Versions: 1.1, 1.2
Reporter: Benjamin Young
  Labels: couchapp, rewriter

 COUCHDB-1074 breaks the JSON format of Rewrite rules requiring that Boolean 
 values always be quoted. This requires additional knowledge for a user to 
 even understand why their JSON doesn't work with CouchDB.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (COUCHDB-1129) file descriptors sometimes not closed after compaction

2011-10-13 Thread Dan Everton (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/COUCHDB-1129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13127279#comment-13127279
 ] 

Dan Everton commented on COUCHDB-1129:
--

We're still seeing this issue with CouchDB 1.1.0. The database isn't under 
particularly heavy write load but every single compaction leaks file handles. 
This is on R14B with the default CouchDB command line settings.

beam.smp 5283 couchdb   13u   REG 253,9   80646251770050 
/apps/couchdb/.inventory_design/cee4a2835bdfb63165f1a2271260fef4.view
beam.smp 5283 couchdb   15u   REG 253,9 4025593965 49158 
/apps/couchdb/inventory.couch
beam.smp 5283 couchdb   16u   REG 253,9 7186174063 49157 
/apps/couchdb/.delete/e3ed87587b162839186b76c8c073cc2f (deleted)
beam.smp 5283 couchdb   17u   REG 253,9 4025593965 49158 
/apps/couchdb/inventory.couch
beam.smp 5283 couchdb   18u   REG 253,9  77913 49154 
/apps/couchdb/_replicator.couch
beam.smp 5283 couchdb   23u   REG 253,9 7124119663 49155 
/apps/couchdb/.delete/307d7d9ee4cb3593eb72d374e622fca4 (deleted)
beam.smp 5283 couchdb   58u   REG 253,9   4185 49153 
/apps/couchdb/_users.couch

 file descriptors sometimes not closed after compaction
 --

 Key: COUCHDB-1129
 URL: https://issues.apache.org/jira/browse/COUCHDB-1129
 Project: CouchDB
  Issue Type: Bug
  Components: Database Core
Affects Versions: 1.0.2
Reporter: Randall Leeds
 Fix For: 1.2


 It seems there are still cases where file descriptors are not released upon 
 compaction finishing.
 When I asked on IRC rnewson confirmed he'd seen the behavior also and the 
 last comment on 926 also suggests there are still times where this occurs.
 Someone needs to take a careful eye to any race conditions we might have 
 between opening the file and subscribing to the compaction notification.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira