Jason, made some more tests with larger documents (template is
https://gist.github.com/1930804) and a different map function:
function(doc) {
emit([doc.type, doc.category], doc.nested.coords);
}
(patch http://friendpaste.com/5C99aqXocN6N6H1BAYIigs)
Here's the results I got ( https://gist.github.com/1930807 )
Before COUCHDB-1186
fdmanana 23:21:05 ~/git/hub/slow_couchdb (master)> docs=500000
batch=5000 ./bench.sh wow.tpl
Server: CouchDB/1.2.0a-a68a792-git (Erlang OTP/R14B03)
{"couchdb":"Welcome","version":"1.2.0a-a68a792-git"}
[INFO] Created DB named `db1'
[INFO] Uploaded 5000 documents via _bulk_docs
(....)
[INFO] Uploaded 5000 documents via _bulk_docs
Building view.
{"total_rows":500000,"offset":0,"rows":[
{"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"value":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.11,"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]}
]}
real 5m6.676s
user 0m0.009s
sys 0m0.010s
After COUCHDB-1186
fdmanana 23:50:07 ~/git/hub/slow_couchdb (master)> docs=500000
batch=5000 ./bench.sh wow.tpl
Server: CouchDB/1.2.0a-f023052-git (Erlang OTP/R14B03)
{"couchdb":"Welcome","version":"1.2.0a-f023052-git"}
[INFO] Created DB named `db1'
[INFO] Uploaded 5000 documents via _bulk_docs
(....)
[INFO] Uploaded 5000 documents via _bulk_docs
Building view.
{"total_rows":500000,"offset":0,"rows":[
{"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"value":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.11,"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]}
]}
real 5m1.395s
user 0m0.008s
sys 0m0.010s
After COUCHDB-1186 + better queueing patch (
http://friendpaste.com/178nPFgfyyeGf2vtNRpL0w )
fdmanana 00:14:25 ~/git/hub/slow_couchdb (master)> docs=500000
batch=5000 ./bench.sh wow.tpl
Server: CouchDB/1.2.0a-f023052-git (Erlang OTP/R14B03)
{"couchdb":"Welcome","version":"1.2.0a-f023052-git"}
[INFO] Created DB named `db1'
[INFO] Uploaded 5000 documents via _bulk_docs
(....)
[INFO] Uploaded 5000 documents via _bulk_docs
Building view.
{"total_rows":500000,"offset":0,"rows":[
{"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"value":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.11,"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]}
]}
real 4m48.175s
user 0m0.008s
sys 0m0.009s
Disk model is APPLE SSD TS128C, quad core machine, 8Gb of ram.
Unfortunately I don't have access to the machine I used for the tests
in COUCHDB-1186 (spinning disk, Linux) before next week.
On Mon, Feb 27, 2012 at 7:49 PM, Paul Davis <paul.joseph.da...@gmail.com> wrote:
> On Mon, Feb 27, 2012 at 7:18 PM, Filipe David Manana
> <fdman...@apache.org> wrote:
>> Jason, can't reproduce those results, not even close:
>>
>> http://friendpaste.com/1L4pHH8WQchaLIMCWhKX9Z
>>
>> Before COUCHDB-1186
>>
>> fdmanana 16:58:02 ~/git/hub/slow_couchdb (master)> docs=500000
>> batch=50000 ./bench.sh small_doc.tpl
>> Server: CouchDB/1.2.0a-a68a792-git (Erlang OTP/R14B03)
>> {"couchdb":"Welcome","version":"1.2.0a-a68a792-git"}
>>
>> [INFO] Created DB named `db1'
>> [INFO] Uploaded 50000 documents via _bulk_docs
>> [INFO] Uploaded 50000 documents via _bulk_docs
>> [INFO] Uploaded 50000 documents via _bulk_docs
>> [INFO] Uploaded 50000 documents via _bulk_docs
>> [INFO] Uploaded 50000 documents via _bulk_docs
>> [INFO] Uploaded 50000 documents via _bulk_docs
>> [INFO] Uploaded 50000 documents via _bulk_docs
>> [INFO] Uploaded 50000 documents via _bulk_docs
>> [INFO] Uploaded 50000 documents via _bulk_docs
>> [INFO] Uploaded 50000 documents via _bulk_docs
>> Building view.
>> {"total_rows":500000,"offset":0,"rows":[
>> {"id":"doc1","key":1,"value":1}
>> ]}
>>
>> real 0m56.241s
>> user 0m0.006s
>> sys 0m0.005s
>>
>>
>> After COUCHDB-1186
>>
>> fdmanana 17:02:02 ~/git/hub/slow_couchdb (master)> docs=500000
>> batch=50000 ./bench.sh small_doc.tpl
>> Server: CouchDB/1.2.0a-f023052-git (Erlang OTP/R14B03)
>> {"couchdb":"Welcome","version":"1.2.0a-f023052-git"}
>>
>> [INFO] Created DB named `db1'
>> [INFO] Uploaded 50000 documents via _bulk_docs
>> [INFO] Uploaded 50000 documents via _bulk_docs
>> [INFO] Uploaded 50000 documents via _bulk_docs
>> [INFO] Uploaded 50000 documents via _bulk_docs
>> [INFO] Uploaded 50000 documents via _bulk_docs
>> [INFO] Uploaded 50000 documents via _bulk_docs
>> [INFO] Uploaded 50000 documents via _bulk_docs
>> [INFO] Uploaded 50000 documents via _bulk_docs
>> [INFO] Uploaded 50000 documents via _bulk_docs
>> [INFO] Uploaded 50000 documents via _bulk_docs
>> Building view.
>> {"total_rows":500000,"offset":0,"rows":[
>> {"id":"doc1","key":1,"value":1}
>> ]}
>>
>> real 1m11.694s
>> user 0m0.006s
>> sys 0m0.005s
>> fdmanana 17:06:01 ~/git/hub/slow_couchdb (master)>
>>
>>
>> 1.2.0a-f023052-git with patch
>> http://friendpaste.com/178nPFgfyyeGf2vtNRpL0w applied on top
>>
>> fdmanana 17:06:53 ~/git/hub/slow_couchdb (master)> docs=500000
>> batch=50000 ./bench.sh small_doc.tpl
>> Server: CouchDB/1.2.0a-f023052-git (Erlang OTP/R14B03)
>> {"couchdb":"Welcome","version":"1.2.0a-f023052-git"}
>>
>> [INFO] Created DB named `db1'
>> [INFO] Uploaded 50000 documents via _bulk_docs
>> [INFO] Uploaded 50000 documents via _bulk_docs
>> [INFO] Uploaded 50000 documents via _bulk_docs
>> [INFO] Uploaded 50000 documents via _bulk_docs
>> [INFO] Uploaded 50000 documents via _bulk_docs
>> [INFO] Uploaded 50000 documents via _bulk_docs
>> [INFO] Uploaded 50000 documents via _bulk_docs
>> [INFO] Uploaded 50000 documents via _bulk_docs
>> [INFO] Uploaded 50000 documents via _bulk_docs
>> [INFO] Uploaded 50000 documents via _bulk_docs
>> Building view.
>> {"total_rows":500000,"offset":0,"rows":[
>> {"id":"doc1","key":1,"value":1}
>> ]}
>>
>> real 0m51.089s
>> user 0m0.006s
>> sys 0m0.004s
>> fdmanana 17:10:29 ~/git/hub/slow_couchdb (master)>
>>
>>
>> Can you try with R14B0x and also with the patch
>> http://friendpaste.com/178nPFgfyyeGf2vtNRpL0w ?
>>
>> Back then I made all testing on a machine with a spinning disk, so the
>> writer process was slower and likely dequeing more KV pairs from the
>> work queue on each dequeue operation. The tests I did just now are on
>> a machine with a ssd disk.
>>
>
> Yeah, I've seen the btree behave quite differently on SSD's vs HDD's
> (same code had drastically different runtime characteristics).
>
> In other words, can we get a report of what type of disk everyone is running
> on?
>
>>
>> On Mon, Feb 27, 2012 at 4:40 PM, Jason Smith <j...@apache.org> wrote:
>>> Hi, Filipe. Most people seem to be holding their OTP build constant
>>> for these tests.
>>>
>>> If you have the time, would you please check out
>>> https://github.com/jhs/slow_couchdb
>>>
>>> It uses seatoncouch mixed with Bob's script to run a basic benchmark.
>>> I expect more template types to grow to help create different data
>>> profiles.
>>>
>>> Anyway, here are my results with 500k documents. Note that I built
>>> from your optimization commit, then its parent.
>>>
>>> https://gist.github.com/1928169
>>>
>>> tl;dr = 2:50 before your commit; 4:13 after.
>>>
>>> On Mon, Feb 27, 2012 at 11:33 PM, Filipe David Manana
>>> <fdman...@apache.org> wrote:
>>>> I just tried Jason's script (modified it to use 500 000 docs instead
>>>> of 50 000) against 1.2.x and 1.1.1, using OTP R14B03. Here's my
>>>> results:
>>>>
>>>> 1.2.x:
>>>>
>>>> $ port=5984 ./test.sh
>>>> "none"
>>>> Filling db.
>>>> done
>>>> HTTP/1.1 200 OK
>>>> Server: CouchDB/1.2.0 (Erlang OTP/R14B03)
>>>> Date: Mon, 27 Feb 2012 16:08:43 GMT
>>>> Content-Type: text/plain; charset=utf-8
>>>> Content-Length: 252
>>>> Cache-Control: must-revalidate
>>>>
>>>> {"db_name":"db1","doc_count":500001,"doc_del_count":0,"update_seq":500001,"purge_seq":0,"compact_running":false,"disk_size":130494577,"data_size":130490673,"instance_start_time":"1330358830830086","disk_format_version":6,"committed_update_seq":500001}
>>>> Building view.
>>>>
>>>> real 1m5.725s
>>>> user 0m0.006s
>>>> sys 0m0.005s
>>>> done
>>>>
>>>>
>>>> 1.1.1:
>>>>
>>>> $ port=5984 ./test.sh
>>>> ""
>>>> Filling db.
>>>> done
>>>> HTTP/1.1 200 OK
>>>> Server: CouchDB/1.1.2a785d32f-git (Erlang OTP/R14B03)
>>>> Date: Mon, 27 Feb 2012 16:15:33 GMT
>>>> Content-Type: text/plain;charset=utf-8
>>>> Content-Length: 230
>>>> Cache-Control: must-revalidate
>>>>
>>>> {"db_name":"db1","doc_count":500001,"doc_del_count":0,"update_seq":500001,"purge_seq":0,"compact_running":false,"disk_size":122142818,"instance_start_time":"1330359233327316","disk_format_version":5,"committed_update_seq":500001}
>>>> Building view.
>>>>
>>>> real 1m4.249s
>>>> user 0m0.006s
>>>> sys 0m0.005s
>>>> done
>>>>
>>>>
>>>> I don't see any significant difference there.
>>>>
>>>> Regarding COUCHDB-1186, the only thing that might cause some non
>>>> determinism and affect performance is the queing/dequeing. Depending
>>>> on timings, it's possible the writer is dequeing less items per
>>>> dequeue operation and therefore inserting smaller batches into the
>>>> btree. The following small change ensures larger batches (while still
>>>> respecting the queue max size/item count):
>>>>
>>>> http://friendpaste.com/178nPFgfyyeGf2vtNRpL0w
>>>>
>>>> Running the test with this change:
>>>>
>>>> $ port=5984 ./test.sh
>>>> "none"
>>>> Filling db.
>>>> done
>>>> HTTP/1.1 200 OK
>>>> Server: CouchDB/1.2.0 (Erlang OTP/R14B03)
>>>> Date: Mon, 27 Feb 2012 16:23:20 GMT
>>>> Content-Type: text/plain; charset=utf-8
>>>> Content-Length: 252
>>>> Cache-Control: must-revalidate
>>>>
>>>> {"db_name":"db1","doc_count":500001,"doc_del_count":0,"update_seq":500001,"purge_seq":0,"compact_running":false,"disk_size":130494577,"data_size":130490673,"instance_start_time":"1330359706846104","disk_format_version":6,"committed_update_seq":500001}
>>>> Building view.
>>>>
>>>> real 0m49.762s
>>>> user 0m0.006s
>>>> sys 0m0.005s
>>>> done
>>>>
>>>>
>>>> If there's no objection, I'll push that patch.
>>>>
>>>> Also, another note, I noticed sometime ago that with master, using OTP
>>>> R15B I got a performance drop of 10% to 15% compared to using master
>>>> with OTP R14B04. Maybe it applies to 1.2.x as well.
>>>>
>>>>
>>>> On Mon, Feb 27, 2012 at 5:33 AM, Robert Newson <rnew...@apache.org> wrote:
>>>>> Bob D, can you give more details on the data set you're testing?
>>>>> Number of docs, size/complexity of docs, etc? Basically, enough info
>>>>> that I could write a script to automate building an equivalent
>>>>> database.
>>>>>
>>>>> I wrote a quick bash script to make a database and time a view build
>>>>> here: http://friendpaste.com/7kBiKJn3uX1KiGJAFPv4nK
>>>>>
>>>>> B.
>>>>>
>>>>> On 27 February 2012 13:15, Jan Lehnardt <j...@apache.org> wrote:
>>>>>>
>>>>>> On Feb 27, 2012, at 12:58 , Bob Dionne wrote:
>>>>>>
>>>>>>> Thanks for the clarification. I hope I'm not conflating things by
>>>>>>> continuing the discussion here, I thought that's what you requested?
>>>>>>
>>>>>> The discussion we had on IRC was regarding collecting more data items
>>>>>> for the performance regression before we start to draw conclusions.
>>>>>>
>>>>>> My intention here is to understand what needs doing before we can
>>>>>> release 1.2.0.
>>>>>>
>>>>>> I'll reply inline for the other issues.
>>>>>>
>>>>>>> I just downloaded the release candidate again to start fresh. "make
>>>>>>> distcheck" hangs on this step:
>>>>>>>
>>>>>>> /Users/bitdiddle/Downloads/apache-couchdb-1.2.0/apache-couchdb-1.2.0/_build/../test/etap/150-invalid-view-seq.t
>>>>>>> ......... 6/?
>>>>>>>
>>>>>>> Just stops completely. This is on R15B which has been rebuilt to use
>>>>>>> the recommended older SSL version. I haven't looked into this crashing
>>>>>>> too closely but I'm suspicious that I only see it with couchdb and
>>>>>>> never with bigcouch and never using the 1.2.x branch from source or any
>>>>>>> branch for that matter
>>>>>>
>>>>>> From the release you should run `make check`, not make distcheck. But I
>>>>>> assume you see a hang there too, as I have and others (yet not
>>>>>> everybody), too. I can't comment on BigCouch and what is different
>>>>>> there. It is interesting that 1.2.x won't hang. For me, `make check` in
>>>>>> 1.2.x on R15B hangs sometimes, in different places. I'm currently trying
>>>>>> to gather more information about this.
>>>>>>
>>>>>> The question here is whether `make check` passing in R15B is a release
>>>>>> requirement. In my vote I considered no, but I am happy to go with a
>>>>>> community decision if it emerges. What is your take here?
>>>>>>
>>>>>> In addition, this just shouldn't be a question, so we should investigate
>>>>>> why this happens at all and address the issue, hence COUCHDB-1424. Any
>>>>>> insight here would be appreciated as well.
>>>>>>
>>>>>>
>>>>>>> In the command line tests, 2,7, 27, and 32 fail. but it differs from
>>>>>>> run to run.
>>>>>>
>>>>>> I assume you mean the JS tests. Again, this isn't supposed to work in
>>>>>> 1.2.x. I'm happy to backport my changes from master to 1.2.x to make
>>>>>> that work, but I refrained from that because I didn't want to bring too
>>>>>> much change to a release branch. I'm happy to reconsider, but I don't
>>>>>> think a release vote is a good place to discuss feature backports.
>>>>>>
>>>>>>
>>>>>>> On Chrome attachment_ranges fails and it hangs on replicator_db
>>>>>>
>>>>>> This one is an "explaining away", but I think it is warranted. Chrome is
>>>>>> broken for attachment_ranges. I don't know if we reported this upstream
>>>>>> (Robert N?), but this isn't a release blocker. For the replicator_db
>>>>>> test, can you try running that in other browsers. I understand it is not
>>>>>> the best of situation (hence the move to the cli test suite for master),
>>>>>> but if you get this test to pass in at least one other browsers, this
>>>>>> isn't a problem that holds 1.2.x.
>>>>>>
>>>>>>
>>>>>>> With respect to performance I think comparisons with 1.1.x are
>>>>>>> important. I think almost any use case, contrived or otherwise should
>>>>>>> not be dismissed as a pathological or edge case. Bob's script is as
>>>>>>> simple as it gets and to me is a great smoke test. We need to figure
>>>>>>> out the reason 1.2 is clearly slower in this case. If there are
>>>>>>> specific scenarios that 1.2.x is optimized for then we should document
>>>>>>> that and provide reasons for the trade-offs
>>>>>>
>>>>>> I want to make absolutely clear that I take any report of performance
>>>>>> regression very seriously. But I'm rather annoyed that no information
>>>>>> about this ends up on dev@. I understand that on IRC there's some shared
>>>>>> understanding of a few scenarios where performance regressions can be
>>>>>> shown. I asked three times now that these be posted to this mailing
>>>>>> list. I'm not asking for a comprehensive report, but anything really. I
>>>>>> found Robert Newson's simple test script on IRC and ran that to test a
>>>>>> suspicion of mine which I posted in an earlier mail (tiny docs ->
>>>>>> slower, bigger docs -> faster). Nobody else bothered to post this here.
>>>>>> I see no discussion about what is observed, what is expected, what would
>>>>>> be acceptable for a release of 1.2.0 as is and what not.
>>>>>>
>>>>>> As far as this list is concerned, we know that a few people claimed that
>>>>>> things are slower and it's very real and that we should hold the 1.2.0
>>>>>> release for it. I'm more than happy to hold the release until we figured
>>>>>> out the things I asked for above and help out figuring it all out. But
>>>>>> we need something to work with here.
>>>>>>
>>>>>> I also understand that this is a voluntary project and people don't have
>>>>>> infinite time to spend, but at least a message of "we're collecting
>>>>>> things, will report when done", would be *great* to start. So far we
>>>>>> only have a "hold the horses, there might be a something going on".
>>>>>>
>>>>>> Please let me know if this request is unreasonable or whether I am
>>>>>> overreacting.
>>>>>>
>>>>>> Sorry for the rant.
>>>>>>
>>>>>> To anyone who has been looking into performance regression, can you
>>>>>> please send to this list any info you have? If you have a comprehensive
>>>>>> analysis, awesome, if you just ran some script on a machine, just send
>>>>>> us that, let's collect all the data to get this situation solved! We
>>>>>> need your help.
>>>>>>
>>>>>>
>>>>>> tl;dr:
>>>>>>
>>>>>> There's three issues at hand:
>>>>>>
>>>>>> - Robert D -1'd a release artefact. We want to understand what needs to
>>>>>> happen to make a release. This includes assessing the issues he raises
>>>>>> and squaring them against the release vote.
>>>>>>
>>>>>> - There's a vague (as far as dev@ is concerned) report about a
>>>>>> performance regression. We need to get behind that.
>>>>>>
>>>>>> - There's been a non-dev@ discussion about the performance regression
>>>>>> and that is referenced to influence a dev@ decision. We need that
>>>>>> discussion's information on dev@ to proceed.
>>>>>>
>>>>>>
>>>>>> And to make it absolutely clear again. The performance regression *is*
>>>>>> an issue and I am very grateful for the people, including Robert Newson,
>>>>>> Robert Dionne and Jason Smith, who look into it. It's just that we need
>>>>>> to treat this as an issue and get all this info onto dev@ or into JRIA.
>>>>>>
>>>>>>
>>>>>> Cheers
>>>>>> Jan
>>>>>> --
>>>>>>
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> Cheers,
>>>>>>>
>>>>>>> Bob
>>>>>>>
>>>>>>>
>>>>>>> On Feb 26, 2012, at 4:07 PM, Jan Lehnardt wrote:
>>>>>>>
>>>>>>>> Bob,
>>>>>>>>
>>>>>>>> thanks for your reply
>>>>>>>>
>>>>>>>> I wasn't implying we should try to explain anything away. All of these
>>>>>>>> are valid concerns, I just wanted to get a better understanding on
>>>>>>>> where the bit flips from +0 to -1 and subsequently, how to address
>>>>>>>> that boundary.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> Ideally we can just fix all of the things you mention, but I think it
>>>>>>>> is important to understand them in detail, that's why I was going into
>>>>>>>> them. Ultimately, I want to understand what we need to do to ship
>>>>>>>> 1.2.0.
>>>>>>>>
>>>>>>>> On Feb 26, 2012, at 21:22 , Bob Dionne wrote:
>>>>>>>>
>>>>>>>>> Jan,
>>>>>>>>>
>>>>>>>>> I'm -1 based on all of my evaluation. I've spent a few hours on this
>>>>>>>>> release now yesterday and today. It doesn't really pass what I would
>>>>>>>>> call the "smoke test". Almost everything I've run into has an
>>>>>>>>> explanation:
>>>>>>>>>
>>>>>>>>> 1. crashes out of the box - that's R15B, you need to recompile SSL
>>>>>>>>> and Erlang (we'll note on release notes)
>>>>>>>>
>>>>>>>> Have we spent any time on figuring out what the trouble here is?
>>>>>>>>
>>>>>>>>
>>>>>>>>> 2. etaps hang running make check. Known issue. Our etap code is out
>>>>>>>>> of date, recent versions of etap don't even run their own unit tests
>>>>>>>>
>>>>>>>> I have seen the etap hang as well, and I wasn't diligent enough to
>>>>>>>> report it in JIRA, I have done so now (COUCHDB-1424).
>>>>>>>>
>>>>>>>>
>>>>>>>>> 3. Futon tests fail. Some are known bugs (attachment ranges in
>>>>>>>>> Chrome) . Both Chrome and Safari also hang
>>>>>>>>
>>>>>>>> Do you have more details on where Chrome and Safari hang? Can you try
>>>>>>>> their private browsing features, double/triple check that caches are
>>>>>>>> empty? Can you get to a situation where you get all tests succeeding
>>>>>>>> across all browsers, even if individual ones fail on one or two others?
>>>>>>>>
>>>>>>>>
>>>>>>>>> 4. standalone JS tests fail. Again most of these run when run by
>>>>>>>>> themselves
>>>>>>>>
>>>>>>>> Which ones?
>>>>>>>>
>>>>>>>>
>>>>>>>>> 5. performance. I used real production data *because* Stefan on user
>>>>>>>>> reported performance degradation on his data set. Any numbers are
>>>>>>>>> meaningless for a single test. I also ran scripts that BobN and Jason
>>>>>>>>> Smith posted that show a difference between 1.1.x and 1.2.x
>>>>>>>>
>>>>>>>> You are conflating an IRC discussion we've had into this thread. The
>>>>>>>> performance regression reported is a good reason to look into other
>>>>>>>> scenarios where we can show slowdowns. But we need to understand
>>>>>>>> what's happening. Just from looking at dev@ all I see is some
>>>>>>>> handwaving about some reports some people have done (Not to discourage
>>>>>>>> any work that has been done on IRC and user@, but for the sake of a
>>>>>>>> release vote thread, this related information needs to be on this
>>>>>>>> mailing list).
>>>>>>>>
>>>>>>>> As I said on IRC, I'm happy to get my hands dirty to understand the
>>>>>>>> regression at hand. But we need to know where we'd draw a line and say
>>>>>>>> this isn't acceptable for a 1.2.0.
>>>>>>>>
>>>>>>>>
>>>>>>>>> 6. Reviewed patch pointed to by Jason that may be the cause but it's
>>>>>>>>> hard to say without knowing the code analysis that went into the
>>>>>>>>> changes. You can see obvious local optimizations that make good sense
>>>>>>>>> but those are often the ones that get you, without knowing the call
>>>>>>>>> counts.
>>>>>>>>
>>>>>>>> That is a point that wasn't included in your previous mail. It's great
>>>>>>>> that there is progress, thanks for looking into this!
>>>>>>>>
>>>>>>>>
>>>>>>>>> Many of these issues can be explained away, but I think end users
>>>>>>>>> will be less forgiving. I think we already struggle with view
>>>>>>>>> performance. I'm interested to see how others evaluate this
>>>>>>>>> regression.
>>>>>>>>> I'll try this seatoncouch tool you mention later to see if I can
>>>>>>>>> construct some more definitive tests.
>>>>>>>>
>>>>>>>> Again, I'm not trying to explain anything away. I want to get a shared
>>>>>>>> understanding of the issues you raised and where we stand on solving
>>>>>>>> them squared against the ongoing 1.2.0 release.
>>>>>>>>
>>>>>>>> And again: Thanks for doing this thorough review and looking into
>>>>>>>> performance issue. I hope with your help we can understand all these
>>>>>>>> things a lot better very soon :)
>>>>>>>>
>>>>>>>> Cheers
>>>>>>>> Jan
>>>>>>>> --
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Best,
>>>>>>>>>
>>>>>>>>> Bob
>>>>>>>>> On Feb 26, 2012, at 2:29 PM, Jan Lehnardt wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Feb 26, 2012, at 13:58 , Bob Dionne wrote:
>>>>>>>>>>
>>>>>>>>>>> -1
>>>>>>>>>>>
>>>>>>>>>>> R15B on OS X Lion
>>>>>>>>>>>
>>>>>>>>>>> I rebuilt OTP with an older SSL and that gets past all the crashes
>>>>>>>>>>> (thanks Filipe). I still see hangs when running make check, though
>>>>>>>>>>> any particular etap that hangs will run ok by itself. The Futon
>>>>>>>>>>> tests never run to completion in Chrome without hanging and the
>>>>>>>>>>> standalone JS tests also have fails.
>>>>>>>>>>
>>>>>>>>>> What part of this do you consider the -1? Can you try running the JS
>>>>>>>>>> tests in Firefox and or Safari? Can you get all tests pass at least
>>>>>>>>>> once across all browsers? The cli JS suite isn't supposed to work,
>>>>>>>>>> so that isn't a criterion. I've seen the hang in make check for R15B
>>>>>>>>>> while individual tests run as well, but I don't consider this
>>>>>>>>>> blocking. While I understand and support the notion that tests
>>>>>>>>>> shouldn't fail, period, we gotta work with what we have and master
>>>>>>>>>> already has significant improvements. What would you like to see
>>>>>>>>>> changed to not -1 this release?
>>>>>>>>>>
>>>>>>>>>>> I tested the performance of view indexing, using a modest 200K doc
>>>>>>>>>>> db with a large complex view and there's a clear regression between
>>>>>>>>>>> 1.1.x and 1.2.x Others report similar results
>>>>>>>>>>
>>>>>>>>>> What is a large complex view? The complexity of the map/reduce
>>>>>>>>>> functions is rarely an indicator of performance, it's usually input
>>>>>>>>>> doc size and output/emit()/reduce data size. How big are the docs in
>>>>>>>>>> your test and how big is the returned data? I understand the changes
>>>>>>>>>> for 1.2.x will improve larger-data scenarios more significantly.
>>>>>>>>>>
>>>>>>>>>> Cheers
>>>>>>>>>> Jan
>>>>>>>>>> --
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Feb 23, 2012, at 5:25 PM, Bob Dionne wrote:
>>>>>>>>>>>
>>>>>>>>>>>> sorry Noah, I'm in debug mode today so I don't care to start
>>>>>>>>>>>> mucking with my stack, recompiling erlang, etc...
>>>>>>>>>>>>
>>>>>>>>>>>> I did try using that build repeatedly and it crashes all the time.
>>>>>>>>>>>> I find it very odd and I had seen those before as I said on my
>>>>>>>>>>>> older macbook.
>>>>>>>>>>>>
>>>>>>>>>>>> I do see the hangs Jan describes in the etaps, they have been
>>>>>>>>>>>> there right along, so I'm confident this just the SSL issue. Why
>>>>>>>>>>>> it only happens on the build is puzzling, any source build of any
>>>>>>>>>>>> branch works just peachy.
>>>>>>>>>>>>
>>>>>>>>>>>> So I'd say I'm +1 based on my use of the 1.2.x branch but I'd like
>>>>>>>>>>>> to hear from Stefan, who reported the severe performance
>>>>>>>>>>>> regression. BobN seems to think we can ignore that, it's something
>>>>>>>>>>>> flaky in that fellow's environment. I tend to agree but I'm
>>>>>>>>>>>> conservative
>>>>>>>>>>>>
>>>>>>>>>>>> On Feb 23, 2012, at 1:23 PM, Noah Slater wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Can someone convince me this bus error stuff and segfaults is not
>>>>>>>>>>>>> a
>>>>>>>>>>>>> blocking issue.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Bob tells me that he's followed the steps above and he's still
>>>>>>>>>>>>> experiencing
>>>>>>>>>>>>> the issues.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Bob, you did follow the steps to install your own SSL right?
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Thu, Feb 23, 2012 at 5:09 PM, Jan Lehnardt <j...@apache.org>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Feb 23, 2012, at 00:28 , Noah Slater wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I would like call a vote for the Apache CouchDB 1.2.0 release,
>>>>>>>>>>>>>>> second
>>>>>>>>>>>>>> round.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> We encourage the whole community to download and test these
>>>>>>>>>>>>>>> release artifacts so that any critical issues can be resolved
>>>>>>>>>>>>>>> before the
>>>>>>>>>>>>>>> release is made. Everyone is free to vote on this release, so
>>>>>>>>>>>>>>> get stuck
>>>>>>>>>>>>>> in!
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> We are voting on the following release artifacts:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> http://people.apache.org/~nslater/dist/1.2.0/
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> These artifacts have been built from the following tree-ish in
>>>>>>>>>>>>>>> Git:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 4cd60f3d1683a3445c3248f48ae064fb573db2a1
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Please follow the test procedure before voting:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> http://wiki.apache.org/couchdb/Test_procedure
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thank you.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Happy voting,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Signature and hashes check out.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Mac OS X 10.7.3, 64bit, SpiderMonkey 1.8.0, Erlang R14B04: make
>>>>>>>>>>>>>> check
>>>>>>>>>>>>>> works fine, browser tests in Safari work fine.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Mac OS X 10.7.3, 64bit, SpiderMonkey 1.8.5, Erlang R14B04: make
>>>>>>>>>>>>>> check
>>>>>>>>>>>>>> works fine, browser tests in Safari work fine.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> FreeBSD 9.0, 64bit, SpiderMonkey 1.7.0, Erlang R14B04: make
>>>>>>>>>>>>>> check works
>>>>>>>>>>>>>> fine, browser tests in Safari work fine.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> CentOS 6.2, 64bit, SpiderMonkey 1.8.5, Erlang R14B04: make check
>>>>>>>>>>>>>> works
>>>>>>>>>>>>>> fine, browser tests in Firefox work fine.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Ubuntu 11.4, 64bit, SpiderMonkey 1.8.5, Erlang R14B02: make
>>>>>>>>>>>>>> check works
>>>>>>>>>>>>>> fine, browser tests in Firefox work fine.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Ubuntu 10.4, 32bit, SpiderMonkey 1.8.0, Erlang R13B03: make
>>>>>>>>>>>>>> check fails in
>>>>>>>>>>>>>> - 076-file-compression.t: https://gist.github.com/1893373
>>>>>>>>>>>>>> - 220-compaction-daemon.t: https://gist.github.com/1893387
>>>>>>>>>>>>>> This on runs in a VM and is 32bit, so I don't know if there's
>>>>>>>>>>>>>> anything in
>>>>>>>>>>>>>> the tests that rely on 64bittyness or the R14B03. Filipe, I
>>>>>>>>>>>>>> think you
>>>>>>>>>>>>>> worked on both features, do you have an idea?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I tried running it all through Erlang R15B on Mac OS X 1.7.3,
>>>>>>>>>>>>>> but a good
>>>>>>>>>>>>>> way into `make check` the tests would just stop and hang. The
>>>>>>>>>>>>>> last time,
>>>>>>>>>>>>>> repeatedly in 160-vhosts.t, but when run alone, that test
>>>>>>>>>>>>>> finished in under
>>>>>>>>>>>>>> five seconds. I'm not sure what the issue is here.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Despite the things above, I'm happy to give this a +1 if we put
>>>>>>>>>>>>>> a warning
>>>>>>>>>>>>>> about R15B on the download page.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Great work all!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Cheers
>>>>>>>>>>>>>> Jan
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Filipe David Manana,
>>>>
>>>> "Reasonable men adapt themselves to the world.
>>>> Unreasonable men adapt the world to themselves.
>>>> That's why all progress depends on unreasonable men."
>>
>>
>>
>> --
>> Filipe David Manana,
>>
>> "Reasonable men adapt themselves to the world.
>> Unreasonable men adapt the world to themselves.
>> That's why all progress depends on unreasonable men."
--
Filipe David Manana,
"Reasonable men adapt themselves to the world.
Unreasonable men adapt the world to themselves.
That's why all progress depends on unreasonable men."
