jaydoane commented on code in PR #4888: URL: https://github.com/apache/couchdb/pull/4888#discussion_r1614723074
########## src/docs/src/whatsnew/3.4.rst: ########## @@ -0,0 +1,360 @@ +.. Licensed under the Apache License, Version 2.0 (the "License"); you may not +.. use this file except in compliance with the License. You may obtain a copy of +.. the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, software +.. distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +.. WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +.. License for the specific language governing permissions and limitations under +.. the License. + +.. _release/3.4.x: + +============ +3.4.x Branch +============ + +.. contents:: + :depth: 1 + :local: + +.. _release/3.4.0: + +Version 3.4.0 +============= + +Breaking Changes +---------------- + +* :ghissue:`5046`: JWT: require valid ``exp`` claim by default + + Users of JWT rightly expect tokens to be considered invalid once they expire. It + is a surprise to some that this requires a change to the default + configuration. In the interest of security we will now require a valid ``exp`` + claim in tokens. Administrators can disable the check by changing + ``required_claims`` back to the empty string. + + We recommend adding ``nbf`` as a required claim if you know your tokens will + include it. + +Highlights +---------- + +.. code-block:: text + + .-. + / | + /\ | .-._.) ( ) .-..-. .-. ) ( + / \ |( )( )( / ./.-'_( | ( ) + .-' / \| `-' `--': \_/ (__.' `-'-'`--': + (__.' `. + +* :ghissue:`4291`: Introducing Nouveau (beta) a modern, from-the-ground-up + implementation of Lucene-based full-text search for CouchDB. Please test + this thoroughly and report back any issues you might find. + + - :ref:`Setup instructions <nouveauconfig>` + - :ref:`Usage <ddoc/nouveau>` + - `Report a bug`_. + +.. _Report a bug: https://github.com/apache/couchdb/issues/new/choose + +* :ghissue:`4627`: Add `QuickJS`_ as a JavaScript engine option. + +.. _QuickJS: https://bellard.org/quickjs/` + + Advantages over SpiderMonkey: + + - significantly smaller and easier to integrate codebase. We're using ~6 + C files vs 700+ SM91 C++ files. + - Built with Apache CouchDB as opposed having to maintain a separate + SpiderMonkey package for OSs that don’t support it + (\*cough*RedHat9\*cough\*). + - Better sandboxing support. + - Preliminary test results show multiple performance improvements. + + - 4x faster than SpiderMonkey 1.8.5. + - 5x faster than SpiderMonkey 91. + - 6x reduced memory usage per ``couchjs`` process (5MB vs 30MB). + + - Allows compiling JavaScript bytecode ahead of time. + - QuickJS can be built alongside SpiderMonkey and toggled on/off at runtime:: + + ./configure --dev --js-engine=quickjs + + - This makes it the default engine. But SpiderMonkey can still be set in the + config option:: + + [couchdb] + js_engine = spidermonkey | quickjs + +* :ghissue:`4570`, :ghissue:`4578`, :ghissue:`4576`: Adopt xxHash in favour + of md5 for ``couch_file`` checksums and ETag calculation. 30% performance + increase for large (128K) docs. No difference for smaller docs. + +* :ghissue:`4814`: Introduce PBKDF2-SHA256 for password hashing. The existing + PBKDF2-SHA1 variant is now deprecated. Increases the default iteration count + to ``500000``. Also introduce a password hash in-memory cache with a low + iteration number, to keep interactive requests fast for a fixed time. + + Entries in the password hash cache are time-limited, unused entries are + automatically deleted, and there is a capacity bound. + +* :ghissue:`4512`: Mango: add keys-only covering indexes. Improves query + response times for certain queries up to 10x at p(95). + +* :ghissue:`4681`: Introduce optional countermeasures as we run out of + disk space. + +* :ghissue:`4847`: Require auth for ``_replicate`` endpoint. This continues + the 3.x closed-by-default design goal. + +* Many small small performance improvements, see :ref:`the Performance + section <performance430>`. + +Features and Enhancements +------------------------- + +* :ghissue:`5069`: Update Fauxton to v1.3.1. +* :ghissue:`5067`: Support Erlang 27. +* :ghissue:`5053`: Use the built-in ``crypto:pbkdf2_hmac`` function. +* :ghissue:`5036`: Remove ``replication_job_supervisor``. +* :ghissue:`5035`: Modernise ``couch_replicator_supervisor``. +* :ghissue:`5019`: Remove unused build files. +* :ghissue:`5017`: Remove unused ``boot_dev_cluster.sh``. +* :ghissue:`5014`: Add Couch Scanner module. +* :ghissue:`5013`: Improve dist diagnostics. +* :ghissue:`4990`: Add ``dbname`` to mango exec stats. +* :ghissue:`4987`: Replace ``khash`` with maps in ``ddoc_cache_lru``. +* :ghissue:`4984`: Fabric: switch to maps for view rows. +* :ghissue:`4979`: Git ignore top level ``clouseau`` directory. +* :ghissue:`4977`: Replace ``khash`` with maps in ``couch_event_server``. +* :ghissue:`4976`: Add metrics for fast vs slow password hashing. +* :ghissue:`4965`: Handle multiple response copies for ``_purged_infos`` API. +* :ghissue:`4878`: Add an option to scrub some sensitive headers from external + json. +* :ghissue:`4834`: Wait for newly set admin creds to be hashed in setup. +* :ghissue:`4821`: Do not fail compactions if the last step is delayed + by ``ioq``. +* :ghissue:`4810`: Mango: add ``$beginsWith`` operator . +* :ghissue:`4769`: Improve replicator error handling. +* :ghissue:`4766`: Add new HTTP endpoint ``/_node/_local/_smoosh/status``. +* :ghissue:`4736`: Stop client process and clean up if client disconnects. +* :ghissue:`4703`: Add ``_purged_infos`` endpoint. +* :ghissue:`4685`: Add ``"CouchDB-Replicator/..."`` user agent to + replicator ``/_session`` requests +* :ghissue:`4680`: Shard splitting: allow resumption of failed jobs and + make timeout configurable. +* :ghissue:`4677`: Crash replication jobs on unexpected ``4xx`` errors. +* :ghissue:`4670`: Allow setting of additional ``ibrowse`` options + like ``prefer_ipv6``. +* :ghissue:`4662`: Mango: extend ``_explain`` with candidate indexes and + selector hints. +* :ghissue:`4625`: Add optional logging of security issues when + replicating. +* :ghissue:`4623`: Better upgrade handling of ``instance_start_time`` + in replicator. +* :ghissue:`4613`: Add option to suppress version info via HTTP header. +* :ghissue:`4601`: Add simple ``fabric`` benchmark. +* :ghissue:`4581`: Support Erlang/OTP 26. +* :ghissue:`4575`: Add ``{verify, verify_peer}`` for TLS validation. +* :ghissue:`4569`: Mango: add ``keys_examined`` for ``execution_stats``. +* :ghissue:`4558`: Make Erlang/OTP 24 the minimum supported Erlang version. +* :ghissue:`4503`: Make timeouts for ``_view`` and ``_search`` configurable. +* :ghissue:`4483`: Add RFC5424 compliant report logging. +* :ghissue:`4475`: Add type and descriptions to prometheus output. +* :ghissue:`4443`: Automatically enable FIPS mode at runtime. +* :ghissue:`4438`: Upgrade hash algorithm for proxy auth. +* :ghissue:`4432`: Hide shard-sync and purge documents from ``_local_docs``. +* :ghissue:`4431`: Allow definition of JWT roles claim as comma-separated + list. +* :ghissue:`4404`: Respond with 503 immediately if search not available. +* :ghissue:`4347`: Remove failed ``couch_plugins`` experiment. +* :ghissue:`5046`: JWT: require valid `exp` claim by default +* :ghissue:`5065`: Update Fauxton UI to version v1.3.1 + +.. _performance430: + +Performance +----------- + +* :ghissue:`5037`: Optimise ``fabric:all_dbs()``. +* :ghissue:`4911`: Optimise and clean up ``couch_multidb_changes``. +* :ghissue:`4852`: Optimise ``_active_tasks``. +* :ghissue:`4786`, :ghissue:`4789`: Add extra timing stats for ``couch_js`` + engine commands. +* :ghissue:`4679`: Fix multipart parse ``attachment longer than expected`` + error. +* :ghissue:`4672`: Remove ``folsom`` and reimplement required functionality + with new Erlang/OTP primitives resulting in up to 19x faster histogram + operations. +* :ghissue:`4617`: Use a faster sets implementation available since OTP 24. +* :ghissue:`4608`: Add metrics for fsync calls and query engine operations. +* :ghissue:`4604`: 6x speedup for common ``mem3:dbname/1`` function. +* :ghissue:`4603`: Update ``vm.args`` settings, increased Erlang distribution + buffer size to 32MB. +* :ghissue:`4598`: Speed up internal replicator. +* :ghissue:`4507`, :ghissue:`4525`: Add more prometheus metrics. +* :ghissue:`4505`: Treat JavaScript internal errors as fatal. +* :ghissue:`4494`: Treat single-element keys as key. +* :ghissue:`4473`: Avoid re-compiling filter view functions. +* :ghissue:`4401`: Enforce doc ids ``_changes`` filter optimisation limit and + raise it from ``100`` to ``1000``. +* :ghissue:`4394`: Mango: push ``fields`` selection down to data nodes. + +Bugfixes +-------- + +* :ghissue:`5068`: Fix flakiness in ``fabric_bench``. +* :ghissue:`5066`: Handle multiple ``Set-Cookie`` headers in replicator session + plugin. +* :ghissue:`5060`: Cleanup a few clauses in ``fabric_view_changes``. +* :ghissue:`5030`: Always commit if we upgrade 2.x view files. Fixes + misleading ``wrong signature`` error. +* :ghissue:`5025`: Fix ``seedlist`` to not return duplicate json keys. +* :ghissue:`5008`: Fix case clause error in replicator ``_scheduler/docs`` + response. +* :ghissue:`5000`: Remove repetitive word in source commends (5000!). +* :ghissue:`4962`: Make multidb changes shard map aware. +* :ghissue:`4958`: Mango: use rolling execution statistics. +* :ghissue:`4921`: Make sure to reply to ``couch_index_server`` clients. +* :ghissue:`4910`: ``couch_passwords:verify`` should always return false for + bad inputs. +* :ghissue:`4908`: Mango: communicate rows read for global stats collection. +* :ghissue:`4906`: Flush ``chttpd_db`` monitor refs on demonitor. +* :ghissue:`4904`: Git ignore all .hypothesis directories. +* :ghissue:`4887`: Look up search node name in config for weatherreport. +* :ghissue:`4837`: Fix update bug in ``ets_lru``. +* :ghissue:`4811`: Prevent delayed opener error from crashing index servers. +* :ghissue:`4794`: Fix incorrect raising of ``database_does_not_exist`` error. +* :ghissue:`4784`: Fix parsing of node name from ``ERL_FLAGS1`` + in ``remsh``. +* :ghissue:`4782`, :ghissue:`4891`: Mango: prevent occasional + duplication of paginated text results. +* :ghissue:`4761`: Fix badrecord error when replicator is logging http usage. +* :ghissue:`4759`: TLS: use HTTP rules for hostname verification. +* :ghissue:`4758`: Remove sensitive headers from the ``mochiweb`` request + in ``pdict``. +* :ghissue:`4751`: Mango: correct behaviour of ``fields`` on ``_explain``. +* :ghissue:`4722`: Fix badmatch error when purge requests time out. +* :ghissue:`4716`: Fix pending count for reverse changes feed. +* :ghissue:`4709`: Mango: improve handling of invalid fields. +* :ghissue:`4704`, :ghissue:`4707`: Fix empty facet search results. Review Comment: Duplicate :ghissue:`4707` ########## src/docs/src/whatsnew/3.4.rst: ########## @@ -0,0 +1,360 @@ +.. Licensed under the Apache License, Version 2.0 (the "License"); you may not +.. use this file except in compliance with the License. You may obtain a copy of +.. the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, software +.. distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +.. WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +.. License for the specific language governing permissions and limitations under +.. the License. + +.. _release/3.4.x: + +============ +3.4.x Branch +============ + +.. contents:: + :depth: 1 + :local: + +.. _release/3.4.0: + +Version 3.4.0 +============= + +Breaking Changes +---------------- + +* :ghissue:`5046`: JWT: require valid ``exp`` claim by default + + Users of JWT rightly expect tokens to be considered invalid once they expire. It + is a surprise to some that this requires a change to the default + configuration. In the interest of security we will now require a valid ``exp`` + claim in tokens. Administrators can disable the check by changing + ``required_claims`` back to the empty string. + + We recommend adding ``nbf`` as a required claim if you know your tokens will + include it. + +Highlights +---------- + +.. code-block:: text + + .-. + / | + /\ | .-._.) ( ) .-..-. .-. ) ( + / \ |( )( )( / ./.-'_( | ( ) + .-' / \| `-' `--': \_/ (__.' `-'-'`--': + (__.' `. + +* :ghissue:`4291`: Introducing Nouveau (beta) a modern, from-the-ground-up + implementation of Lucene-based full-text search for CouchDB. Please test + this thoroughly and report back any issues you might find. + + - :ref:`Setup instructions <nouveauconfig>` + - :ref:`Usage <ddoc/nouveau>` + - `Report a bug`_. + +.. _Report a bug: https://github.com/apache/couchdb/issues/new/choose + +* :ghissue:`4627`: Add `QuickJS`_ as a JavaScript engine option. + +.. _QuickJS: https://bellard.org/quickjs/` + + Advantages over SpiderMonkey: + + - significantly smaller and easier to integrate codebase. We're using ~6 + C files vs 700+ SM91 C++ files. + - Built with Apache CouchDB as opposed having to maintain a separate + SpiderMonkey package for OSs that don’t support it + (\*cough*RedHat9\*cough\*). + - Better sandboxing support. + - Preliminary test results show multiple performance improvements. + + - 4x faster than SpiderMonkey 1.8.5. + - 5x faster than SpiderMonkey 91. + - 6x reduced memory usage per ``couchjs`` process (5MB vs 30MB). + + - Allows compiling JavaScript bytecode ahead of time. + - QuickJS can be built alongside SpiderMonkey and toggled on/off at runtime:: + + ./configure --dev --js-engine=quickjs + + - This makes it the default engine. But SpiderMonkey can still be set in the + config option:: + + [couchdb] + js_engine = spidermonkey | quickjs + +* :ghissue:`4570`, :ghissue:`4578`, :ghissue:`4576`: Adopt xxHash in favour + of md5 for ``couch_file`` checksums and ETag calculation. 30% performance + increase for large (128K) docs. No difference for smaller docs. + +* :ghissue:`4814`: Introduce PBKDF2-SHA256 for password hashing. The existing + PBKDF2-SHA1 variant is now deprecated. Increases the default iteration count + to ``500000``. Also introduce a password hash in-memory cache with a low + iteration number, to keep interactive requests fast for a fixed time. + + Entries in the password hash cache are time-limited, unused entries are + automatically deleted, and there is a capacity bound. + +* :ghissue:`4512`: Mango: add keys-only covering indexes. Improves query + response times for certain queries up to 10x at p(95). + +* :ghissue:`4681`: Introduce optional countermeasures as we run out of + disk space. + +* :ghissue:`4847`: Require auth for ``_replicate`` endpoint. This continues + the 3.x closed-by-default design goal. + +* Many small small performance improvements, see :ref:`the Performance + section <performance430>`. + +Features and Enhancements +------------------------- + +* :ghissue:`5069`: Update Fauxton to v1.3.1. +* :ghissue:`5067`: Support Erlang 27. Review Comment: Maybe change to "Erlang/OTP 27" for consistency with other references below? ########## src/docs/src/whatsnew/3.4.rst: ########## @@ -0,0 +1,360 @@ +.. Licensed under the Apache License, Version 2.0 (the "License"); you may not +.. use this file except in compliance with the License. You may obtain a copy of +.. the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, software +.. distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +.. WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +.. License for the specific language governing permissions and limitations under +.. the License. + +.. _release/3.4.x: + +============ +3.4.x Branch +============ + +.. contents:: + :depth: 1 + :local: + +.. _release/3.4.0: + +Version 3.4.0 +============= + +Breaking Changes +---------------- + +* :ghissue:`5046`: JWT: require valid ``exp`` claim by default + + Users of JWT rightly expect tokens to be considered invalid once they expire. It + is a surprise to some that this requires a change to the default + configuration. In the interest of security we will now require a valid ``exp`` + claim in tokens. Administrators can disable the check by changing + ``required_claims`` back to the empty string. + + We recommend adding ``nbf`` as a required claim if you know your tokens will + include it. + +Highlights +---------- + +.. code-block:: text + + .-. + / | + /\ | .-._.) ( ) .-..-. .-. ) ( + / \ |( )( )( / ./.-'_( | ( ) + .-' / \| `-' `--': \_/ (__.' `-'-'`--': + (__.' `. + +* :ghissue:`4291`: Introducing Nouveau (beta) a modern, from-the-ground-up + implementation of Lucene-based full-text search for CouchDB. Please test + this thoroughly and report back any issues you might find. + + - :ref:`Setup instructions <nouveauconfig>` + - :ref:`Usage <ddoc/nouveau>` + - `Report a bug`_. + +.. _Report a bug: https://github.com/apache/couchdb/issues/new/choose + +* :ghissue:`4627`: Add `QuickJS`_ as a JavaScript engine option. + +.. _QuickJS: https://bellard.org/quickjs/` + + Advantages over SpiderMonkey: + + - significantly smaller and easier to integrate codebase. We're using ~6 + C files vs 700+ SM91 C++ files. + - Built with Apache CouchDB as opposed having to maintain a separate + SpiderMonkey package for OSs that don’t support it + (\*cough*RedHat9\*cough\*). + - Better sandboxing support. + - Preliminary test results show multiple performance improvements. + + - 4x faster than SpiderMonkey 1.8.5. + - 5x faster than SpiderMonkey 91. + - 6x reduced memory usage per ``couchjs`` process (5MB vs 30MB). + + - Allows compiling JavaScript bytecode ahead of time. + - QuickJS can be built alongside SpiderMonkey and toggled on/off at runtime:: + + ./configure --dev --js-engine=quickjs + + - This makes it the default engine. But SpiderMonkey can still be set in the + config option:: + + [couchdb] + js_engine = spidermonkey | quickjs + +* :ghissue:`4570`, :ghissue:`4578`, :ghissue:`4576`: Adopt xxHash in favour + of md5 for ``couch_file`` checksums and ETag calculation. 30% performance + increase for large (128K) docs. No difference for smaller docs. + +* :ghissue:`4814`: Introduce PBKDF2-SHA256 for password hashing. The existing + PBKDF2-SHA1 variant is now deprecated. Increases the default iteration count + to ``500000``. Also introduce a password hash in-memory cache with a low + iteration number, to keep interactive requests fast for a fixed time. + + Entries in the password hash cache are time-limited, unused entries are + automatically deleted, and there is a capacity bound. + +* :ghissue:`4512`: Mango: add keys-only covering indexes. Improves query + response times for certain queries up to 10x at p(95). + +* :ghissue:`4681`: Introduce optional countermeasures as we run out of + disk space. + +* :ghissue:`4847`: Require auth for ``_replicate`` endpoint. This continues + the 3.x closed-by-default design goal. + +* Many small small performance improvements, see :ref:`the Performance + section <performance430>`. + +Features and Enhancements +------------------------- + +* :ghissue:`5069`: Update Fauxton to v1.3.1. +* :ghissue:`5067`: Support Erlang 27. +* :ghissue:`5053`: Use the built-in ``crypto:pbkdf2_hmac`` function. +* :ghissue:`5036`: Remove ``replication_job_supervisor``. +* :ghissue:`5035`: Modernise ``couch_replicator_supervisor``. +* :ghissue:`5019`: Remove unused build files. +* :ghissue:`5017`: Remove unused ``boot_dev_cluster.sh``. +* :ghissue:`5014`: Add Couch Scanner module. +* :ghissue:`5013`: Improve dist diagnostics. +* :ghissue:`4990`: Add ``dbname`` to mango exec stats. +* :ghissue:`4987`: Replace ``khash`` with maps in ``ddoc_cache_lru``. +* :ghissue:`4984`: Fabric: switch to maps for view rows. +* :ghissue:`4979`: Git ignore top level ``clouseau`` directory. +* :ghissue:`4977`: Replace ``khash`` with maps in ``couch_event_server``. +* :ghissue:`4976`: Add metrics for fast vs slow password hashing. +* :ghissue:`4965`: Handle multiple response copies for ``_purged_infos`` API. +* :ghissue:`4878`: Add an option to scrub some sensitive headers from external + json. +* :ghissue:`4834`: Wait for newly set admin creds to be hashed in setup. +* :ghissue:`4821`: Do not fail compactions if the last step is delayed + by ``ioq``. +* :ghissue:`4810`: Mango: add ``$beginsWith`` operator . +* :ghissue:`4769`: Improve replicator error handling. +* :ghissue:`4766`: Add new HTTP endpoint ``/_node/_local/_smoosh/status``. +* :ghissue:`4736`: Stop client process and clean up if client disconnects. +* :ghissue:`4703`: Add ``_purged_infos`` endpoint. +* :ghissue:`4685`: Add ``"CouchDB-Replicator/..."`` user agent to + replicator ``/_session`` requests +* :ghissue:`4680`: Shard splitting: allow resumption of failed jobs and + make timeout configurable. +* :ghissue:`4677`: Crash replication jobs on unexpected ``4xx`` errors. +* :ghissue:`4670`: Allow setting of additional ``ibrowse`` options + like ``prefer_ipv6``. +* :ghissue:`4662`: Mango: extend ``_explain`` with candidate indexes and + selector hints. +* :ghissue:`4625`: Add optional logging of security issues when + replicating. +* :ghissue:`4623`: Better upgrade handling of ``instance_start_time`` + in replicator. +* :ghissue:`4613`: Add option to suppress version info via HTTP header. +* :ghissue:`4601`: Add simple ``fabric`` benchmark. +* :ghissue:`4581`: Support Erlang/OTP 26. +* :ghissue:`4575`: Add ``{verify, verify_peer}`` for TLS validation. +* :ghissue:`4569`: Mango: add ``keys_examined`` for ``execution_stats``. +* :ghissue:`4558`: Make Erlang/OTP 24 the minimum supported Erlang version. +* :ghissue:`4503`: Make timeouts for ``_view`` and ``_search`` configurable. +* :ghissue:`4483`: Add RFC5424 compliant report logging. +* :ghissue:`4475`: Add type and descriptions to prometheus output. +* :ghissue:`4443`: Automatically enable FIPS mode at runtime. +* :ghissue:`4438`: Upgrade hash algorithm for proxy auth. +* :ghissue:`4432`: Hide shard-sync and purge documents from ``_local_docs``. +* :ghissue:`4431`: Allow definition of JWT roles claim as comma-separated + list. +* :ghissue:`4404`: Respond with 503 immediately if search not available. +* :ghissue:`4347`: Remove failed ``couch_plugins`` experiment. +* :ghissue:`5046`: JWT: require valid `exp` claim by default +* :ghissue:`5065`: Update Fauxton UI to version v1.3.1 + +.. _performance430: + +Performance +----------- + +* :ghissue:`5037`: Optimise ``fabric:all_dbs()``. +* :ghissue:`4911`: Optimise and clean up ``couch_multidb_changes``. +* :ghissue:`4852`: Optimise ``_active_tasks``. +* :ghissue:`4786`, :ghissue:`4789`: Add extra timing stats for ``couch_js`` + engine commands. +* :ghissue:`4679`: Fix multipart parse ``attachment longer than expected`` + error. +* :ghissue:`4672`: Remove ``folsom`` and reimplement required functionality + with new Erlang/OTP primitives resulting in up to 19x faster histogram + operations. +* :ghissue:`4617`: Use a faster sets implementation available since OTP 24. +* :ghissue:`4608`: Add metrics for fsync calls and query engine operations. +* :ghissue:`4604`: 6x speedup for common ``mem3:dbname/1`` function. +* :ghissue:`4603`: Update ``vm.args`` settings, increased Erlang distribution + buffer size to 32MB. +* :ghissue:`4598`: Speed up internal replicator. +* :ghissue:`4507`, :ghissue:`4525`: Add more prometheus metrics. +* :ghissue:`4505`: Treat JavaScript internal errors as fatal. +* :ghissue:`4494`: Treat single-element keys as key. +* :ghissue:`4473`: Avoid re-compiling filter view functions. +* :ghissue:`4401`: Enforce doc ids ``_changes`` filter optimisation limit and + raise it from ``100`` to ``1000``. +* :ghissue:`4394`: Mango: push ``fields`` selection down to data nodes. + +Bugfixes +-------- + +* :ghissue:`5068`: Fix flakiness in ``fabric_bench``. +* :ghissue:`5066`: Handle multiple ``Set-Cookie`` headers in replicator session + plugin. +* :ghissue:`5060`: Cleanup a few clauses in ``fabric_view_changes``. +* :ghissue:`5030`: Always commit if we upgrade 2.x view files. Fixes + misleading ``wrong signature`` error. +* :ghissue:`5025`: Fix ``seedlist`` to not return duplicate json keys. +* :ghissue:`5008`: Fix case clause error in replicator ``_scheduler/docs`` + response. +* :ghissue:`5000`: Remove repetitive word in source commends (5000!). +* :ghissue:`4962`: Make multidb changes shard map aware. +* :ghissue:`4958`: Mango: use rolling execution statistics. +* :ghissue:`4921`: Make sure to reply to ``couch_index_server`` clients. +* :ghissue:`4910`: ``couch_passwords:verify`` should always return false for + bad inputs. +* :ghissue:`4908`: Mango: communicate rows read for global stats collection. +* :ghissue:`4906`: Flush ``chttpd_db`` monitor refs on demonitor. +* :ghissue:`4904`: Git ignore all .hypothesis directories. +* :ghissue:`4887`: Look up search node name in config for weatherreport. +* :ghissue:`4837`: Fix update bug in ``ets_lru``. +* :ghissue:`4811`: Prevent delayed opener error from crashing index servers. +* :ghissue:`4794`: Fix incorrect raising of ``database_does_not_exist`` error. +* :ghissue:`4784`: Fix parsing of node name from ``ERL_FLAGS1`` + in ``remsh``. +* :ghissue:`4782`, :ghissue:`4891`: Mango: prevent occasional + duplication of paginated text results. +* :ghissue:`4761`: Fix badrecord error when replicator is logging http usage. Review Comment: Should "http" be all caps? ########## src/docs/src/whatsnew/3.4.rst: ########## @@ -0,0 +1,360 @@ +.. Licensed under the Apache License, Version 2.0 (the "License"); you may not +.. use this file except in compliance with the License. You may obtain a copy of +.. the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, software +.. distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +.. WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +.. License for the specific language governing permissions and limitations under +.. the License. + +.. _release/3.4.x: + +============ +3.4.x Branch +============ + +.. contents:: + :depth: 1 + :local: + +.. _release/3.4.0: + +Version 3.4.0 +============= + +Breaking Changes +---------------- + +* :ghissue:`5046`: JWT: require valid ``exp`` claim by default + + Users of JWT rightly expect tokens to be considered invalid once they expire. It + is a surprise to some that this requires a change to the default + configuration. In the interest of security we will now require a valid ``exp`` + claim in tokens. Administrators can disable the check by changing + ``required_claims`` back to the empty string. + + We recommend adding ``nbf`` as a required claim if you know your tokens will + include it. + +Highlights +---------- + +.. code-block:: text + + .-. + / | + /\ | .-._.) ( ) .-..-. .-. ) ( + / \ |( )( )( / ./.-'_( | ( ) + .-' / \| `-' `--': \_/ (__.' `-'-'`--': + (__.' `. + +* :ghissue:`4291`: Introducing Nouveau (beta) a modern, from-the-ground-up + implementation of Lucene-based full-text search for CouchDB. Please test + this thoroughly and report back any issues you might find. + + - :ref:`Setup instructions <nouveauconfig>` + - :ref:`Usage <ddoc/nouveau>` + - `Report a bug`_. + +.. _Report a bug: https://github.com/apache/couchdb/issues/new/choose + +* :ghissue:`4627`: Add `QuickJS`_ as a JavaScript engine option. + +.. _QuickJS: https://bellard.org/quickjs/` + + Advantages over SpiderMonkey: + + - significantly smaller and easier to integrate codebase. We're using ~6 + C files vs 700+ SM91 C++ files. + - Built with Apache CouchDB as opposed having to maintain a separate + SpiderMonkey package for OSs that don’t support it + (\*cough*RedHat9\*cough\*). + - Better sandboxing support. + - Preliminary test results show multiple performance improvements. + + - 4x faster than SpiderMonkey 1.8.5. + - 5x faster than SpiderMonkey 91. + - 6x reduced memory usage per ``couchjs`` process (5MB vs 30MB). + + - Allows compiling JavaScript bytecode ahead of time. + - QuickJS can be built alongside SpiderMonkey and toggled on/off at runtime:: + + ./configure --dev --js-engine=quickjs + + - This makes it the default engine. But SpiderMonkey can still be set in the + config option:: + + [couchdb] + js_engine = spidermonkey | quickjs + +* :ghissue:`4570`, :ghissue:`4578`, :ghissue:`4576`: Adopt xxHash in favour + of md5 for ``couch_file`` checksums and ETag calculation. 30% performance + increase for large (128K) docs. No difference for smaller docs. + +* :ghissue:`4814`: Introduce PBKDF2-SHA256 for password hashing. The existing + PBKDF2-SHA1 variant is now deprecated. Increases the default iteration count + to ``500000``. Also introduce a password hash in-memory cache with a low + iteration number, to keep interactive requests fast for a fixed time. + + Entries in the password hash cache are time-limited, unused entries are + automatically deleted, and there is a capacity bound. + +* :ghissue:`4512`: Mango: add keys-only covering indexes. Improves query + response times for certain queries up to 10x at p(95). + +* :ghissue:`4681`: Introduce optional countermeasures as we run out of + disk space. + +* :ghissue:`4847`: Require auth for ``_replicate`` endpoint. This continues + the 3.x closed-by-default design goal. + +* Many small small performance improvements, see :ref:`the Performance + section <performance430>`. + +Features and Enhancements +------------------------- + +* :ghissue:`5069`: Update Fauxton to v1.3.1. +* :ghissue:`5067`: Support Erlang 27. +* :ghissue:`5053`: Use the built-in ``crypto:pbkdf2_hmac`` function. +* :ghissue:`5036`: Remove ``replication_job_supervisor``. +* :ghissue:`5035`: Modernise ``couch_replicator_supervisor``. +* :ghissue:`5019`: Remove unused build files. +* :ghissue:`5017`: Remove unused ``boot_dev_cluster.sh``. +* :ghissue:`5014`: Add Couch Scanner module. +* :ghissue:`5013`: Improve dist diagnostics. +* :ghissue:`4990`: Add ``dbname`` to mango exec stats. +* :ghissue:`4987`: Replace ``khash`` with maps in ``ddoc_cache_lru``. +* :ghissue:`4984`: Fabric: switch to maps for view rows. +* :ghissue:`4979`: Git ignore top level ``clouseau`` directory. +* :ghissue:`4977`: Replace ``khash`` with maps in ``couch_event_server``. +* :ghissue:`4976`: Add metrics for fast vs slow password hashing. +* :ghissue:`4965`: Handle multiple response copies for ``_purged_infos`` API. +* :ghissue:`4878`: Add an option to scrub some sensitive headers from external + json. +* :ghissue:`4834`: Wait for newly set admin creds to be hashed in setup. +* :ghissue:`4821`: Do not fail compactions if the last step is delayed + by ``ioq``. +* :ghissue:`4810`: Mango: add ``$beginsWith`` operator . +* :ghissue:`4769`: Improve replicator error handling. +* :ghissue:`4766`: Add new HTTP endpoint ``/_node/_local/_smoosh/status``. +* :ghissue:`4736`: Stop client process and clean up if client disconnects. +* :ghissue:`4703`: Add ``_purged_infos`` endpoint. +* :ghissue:`4685`: Add ``"CouchDB-Replicator/..."`` user agent to + replicator ``/_session`` requests +* :ghissue:`4680`: Shard splitting: allow resumption of failed jobs and + make timeout configurable. +* :ghissue:`4677`: Crash replication jobs on unexpected ``4xx`` errors. +* :ghissue:`4670`: Allow setting of additional ``ibrowse`` options + like ``prefer_ipv6``. +* :ghissue:`4662`: Mango: extend ``_explain`` with candidate indexes and + selector hints. +* :ghissue:`4625`: Add optional logging of security issues when + replicating. +* :ghissue:`4623`: Better upgrade handling of ``instance_start_time`` + in replicator. +* :ghissue:`4613`: Add option to suppress version info via HTTP header. +* :ghissue:`4601`: Add simple ``fabric`` benchmark. +* :ghissue:`4581`: Support Erlang/OTP 26. +* :ghissue:`4575`: Add ``{verify, verify_peer}`` for TLS validation. +* :ghissue:`4569`: Mango: add ``keys_examined`` for ``execution_stats``. +* :ghissue:`4558`: Make Erlang/OTP 24 the minimum supported Erlang version. +* :ghissue:`4503`: Make timeouts for ``_view`` and ``_search`` configurable. +* :ghissue:`4483`: Add RFC5424 compliant report logging. +* :ghissue:`4475`: Add type and descriptions to prometheus output. +* :ghissue:`4443`: Automatically enable FIPS mode at runtime. +* :ghissue:`4438`: Upgrade hash algorithm for proxy auth. +* :ghissue:`4432`: Hide shard-sync and purge documents from ``_local_docs``. +* :ghissue:`4431`: Allow definition of JWT roles claim as comma-separated + list. +* :ghissue:`4404`: Respond with 503 immediately if search not available. +* :ghissue:`4347`: Remove failed ``couch_plugins`` experiment. +* :ghissue:`5046`: JWT: require valid `exp` claim by default +* :ghissue:`5065`: Update Fauxton UI to version v1.3.1 + +.. _performance430: + +Performance +----------- + +* :ghissue:`5037`: Optimise ``fabric:all_dbs()``. +* :ghissue:`4911`: Optimise and clean up ``couch_multidb_changes``. +* :ghissue:`4852`: Optimise ``_active_tasks``. +* :ghissue:`4786`, :ghissue:`4789`: Add extra timing stats for ``couch_js`` + engine commands. +* :ghissue:`4679`: Fix multipart parse ``attachment longer than expected`` + error. +* :ghissue:`4672`: Remove ``folsom`` and reimplement required functionality + with new Erlang/OTP primitives resulting in up to 19x faster histogram + operations. +* :ghissue:`4617`: Use a faster sets implementation available since OTP 24. +* :ghissue:`4608`: Add metrics for fsync calls and query engine operations. +* :ghissue:`4604`: 6x speedup for common ``mem3:dbname/1`` function. +* :ghissue:`4603`: Update ``vm.args`` settings, increased Erlang distribution + buffer size to 32MB. +* :ghissue:`4598`: Speed up internal replicator. +* :ghissue:`4507`, :ghissue:`4525`: Add more prometheus metrics. +* :ghissue:`4505`: Treat JavaScript internal errors as fatal. +* :ghissue:`4494`: Treat single-element keys as key. +* :ghissue:`4473`: Avoid re-compiling filter view functions. +* :ghissue:`4401`: Enforce doc ids ``_changes`` filter optimisation limit and + raise it from ``100`` to ``1000``. +* :ghissue:`4394`: Mango: push ``fields`` selection down to data nodes. + +Bugfixes +-------- + +* :ghissue:`5068`: Fix flakiness in ``fabric_bench``. Review Comment: This could maybe go under `Tests` section? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
