[trafficserver] branch 9.0.x updated: Fix volume/stripe calcs when using forced volumes (#6995)
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
The following commit(s) were added to refs/heads/9.0.x by this push:
new c428edd Fix volume/stripe calcs when using forced volumes (#6995)
c428edd is described below
commit c428eddb344e3c64cb17a51b07f1a9c6b45b3fe2
Author: Gancho Tenev <10522628+gte...@users.noreply.github.com>
AuthorDate: Tue Jul 14 12:40:12 2020 -0700
Fix volume/stripe calcs when using forced volumes (#6995)
Fixed problems with initialization of cache volumes when at least
one volume is being forced to a specific "exclusive" span.
Problem description:
Disks are cleared in the following configuration where volume sizes are
specified using percentages and also one of the volumes is forced to a
specific span (disk):
storage.config:
/dev/disk1
/dev/disk2 volume=3 # <- exclusive span forced to a specific volume
volume.config:
volume=1 scheme=http size=50%
volume=2 scheme=http size=50%
volume=3 scheme=http size=512 # <- volume forced to an exclusive span
During the first start ATS identifies the clears disks and does the
following:
1. creates and spreads new volume 1 and 2 blocks across disk1 and disk2
2. deletes all volume 1 and 2 blocks from disk2 to make space for volume 3
3. creates new volume 3 that takes over the whole disk2.
In step (1) volumes are caclulated larger and spread to disk2 only to be
deleted in step (2) to make space for the forced volume 3.
During the initial start the global volume list cp_list would end up
containing "zombie" CacheVol instances which corespond to the volume 1
and 2 blocks deleted from disk2 to make space for the volume 3 and the
mapping of domains to volumes (hosting.config) could end up mapping
to any of the deleted volume blocks.
This problem disappears after restart since cp_list will be initialized
from the disks and cp_list will contain only valid CacheVol instances.
The fix:
This fix prevents this from happening by making sure all volumes meant
to have "exclusive" disks are created first to make sure span free
spaces are updated correctly and by excluding the size of
the "exclusive" disks from the total cache size used for volume size
calculations when sizes are specified in percentages (volume.config).
(cherry picked from commit 17ee97aa90191767c25a485958965cd76f0e5013)
---
doc/admin-guide/files/volume.config.en.rst | 27 +++
iocore/cache/Cache.cc | 77 +++---
2 files changed, 87 insertions(+), 17 deletions(-)
diff --git a/doc/admin-guide/files/volume.config.en.rst
b/doc/admin-guide/files/volume.config.en.rst
index 46395f1..c96d2fb 100644
--- a/doc/admin-guide/files/volume.config.en.rst
+++ b/doc/admin-guide/files/volume.config.en.rst
@@ -68,6 +68,33 @@ sits in front of a volume. This may be desirable if you are
using something lik
ramdisks, to avoid wasting RAM and cpu time on double caching objects.
+Exclusive spans and volume sizes
+
+
+In the following sample configuration 2 spans `/dev/disk1` and `/dev/disk2`
are defined
+in :file:`storage.config`, where span `/dev/disk2` is assigned to `volume 3`
exclusively
+(`volume 3` is forced to an "exclusive" span `/dev/disk2`).
+In :file:`volume.config` there are 3 volumes defined, where `volume 1` and
`volume 2`
+occupy span `/dev/disk1` taking each 50% of its space and `volume 3` takes
100% of span
+`/dev/disk2` exclusively.
+
+storage.config::
+
+ /dev/disk1
+ /dev/disk2 volume=3 # <- exclusinve span
+
+volume.config::
+
+ volume=1 scheme=http size=50%
+ volume=2 scheme=http size=50%
+ volume=3 scheme=http size=512 # <- volume forced to a specific exclusive
span
+
+It is important to note that when percentages are used to specify volume sizes
+and "exclusive" spans are assigned (forced) to a particular volume (in this
case `volume 3`),
+the "exclusive" spans (in this case `/dev/disk2`) are excluded from the total
cache
+space when the "non-forced" volumes sizes are calculated (in this case `volume
1` and `volume 2`).
+
+
Examples
diff --git a/iocore/cache/Cache.cc b/iocore/cache/Cache.cc
index 0192220..5650359 100644
--- a/iocore/cache/Cache.cc
+++ b/iocore/cache/Cache.cc
@@ -2532,6 +2532,8 @@ cplist_init()
}
}
+static int fillExclusiveDisks(CacheVol *cp);
+
void
cplist_update()
{
@@ -2588,6 +2590,37 @@ cplist_update()
cp = cp->link.next;
}
}
+
+ // Look for (exclusive) spans forced to a specific volume but not yet
referenced by any volumes in cp_list,
+ // if found then create a new volume. This also makes sure new exclusive
disk volumes are created first
+ // before
[trafficserver] branch master updated: Fix volume/stripe calcs when using forced volumes (#6995)
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
The following commit(s) were added to refs/heads/master by this push:
new 17ee97a Fix volume/stripe calcs when using forced volumes (#6995)
17ee97a is described below
commit 17ee97aa90191767c25a485958965cd76f0e5013
Author: Gancho Tenev <10522628+gte...@users.noreply.github.com>
AuthorDate: Tue Jul 14 12:40:12 2020 -0700
Fix volume/stripe calcs when using forced volumes (#6995)
Fixed problems with initialization of cache volumes when at least
one volume is being forced to a specific "exclusive" span.
Problem description:
Disks are cleared in the following configuration where volume sizes are
specified using percentages and also one of the volumes is forced to a
specific span (disk):
storage.config:
/dev/disk1
/dev/disk2 volume=3 # <- exclusive span forced to a specific volume
volume.config:
volume=1 scheme=http size=50%
volume=2 scheme=http size=50%
volume=3 scheme=http size=512 # <- volume forced to an exclusive span
During the first start ATS identifies the clears disks and does the
following:
1. creates and spreads new volume 1 and 2 blocks across disk1 and disk2
2. deletes all volume 1 and 2 blocks from disk2 to make space for volume 3
3. creates new volume 3 that takes over the whole disk2.
In step (1) volumes are caclulated larger and spread to disk2 only to be
deleted in step (2) to make space for the forced volume 3.
During the initial start the global volume list cp_list would end up
containing "zombie" CacheVol instances which corespond to the volume 1
and 2 blocks deleted from disk2 to make space for the volume 3 and the
mapping of domains to volumes (hosting.config) could end up mapping
to any of the deleted volume blocks.
This problem disappears after restart since cp_list will be initialized
from the disks and cp_list will contain only valid CacheVol instances.
The fix:
This fix prevents this from happening by making sure all volumes meant
to have "exclusive" disks are created first to make sure span free
spaces are updated correctly and by excluding the size of
the "exclusive" disks from the total cache size used for volume size
calculations when sizes are specified in percentages (volume.config).
---
doc/admin-guide/files/volume.config.en.rst | 27 +++
iocore/cache/Cache.cc | 77 +++---
2 files changed, 87 insertions(+), 17 deletions(-)
diff --git a/doc/admin-guide/files/volume.config.en.rst
b/doc/admin-guide/files/volume.config.en.rst
index 46395f1..c96d2fb 100644
--- a/doc/admin-guide/files/volume.config.en.rst
+++ b/doc/admin-guide/files/volume.config.en.rst
@@ -68,6 +68,33 @@ sits in front of a volume. This may be desirable if you are
using something lik
ramdisks, to avoid wasting RAM and cpu time on double caching objects.
+Exclusive spans and volume sizes
+
+
+In the following sample configuration 2 spans `/dev/disk1` and `/dev/disk2`
are defined
+in :file:`storage.config`, where span `/dev/disk2` is assigned to `volume 3`
exclusively
+(`volume 3` is forced to an "exclusive" span `/dev/disk2`).
+In :file:`volume.config` there are 3 volumes defined, where `volume 1` and
`volume 2`
+occupy span `/dev/disk1` taking each 50% of its space and `volume 3` takes
100% of span
+`/dev/disk2` exclusively.
+
+storage.config::
+
+ /dev/disk1
+ /dev/disk2 volume=3 # <- exclusinve span
+
+volume.config::
+
+ volume=1 scheme=http size=50%
+ volume=2 scheme=http size=50%
+ volume=3 scheme=http size=512 # <- volume forced to a specific exclusive
span
+
+It is important to note that when percentages are used to specify volume sizes
+and "exclusive" spans are assigned (forced) to a particular volume (in this
case `volume 3`),
+the "exclusive" spans (in this case `/dev/disk2`) are excluded from the total
cache
+space when the "non-forced" volumes sizes are calculated (in this case `volume
1` and `volume 2`).
+
+
Examples
diff --git a/iocore/cache/Cache.cc b/iocore/cache/Cache.cc
index 11156c2..f0131d8 100644
--- a/iocore/cache/Cache.cc
+++ b/iocore/cache/Cache.cc
@@ -2530,6 +2530,8 @@ cplist_init()
}
}
+static int fillExclusiveDisks(CacheVol *cp);
+
void
cplist_update()
{
@@ -2586,6 +2588,37 @@ cplist_update()
cp = cp->link.next;
}
}
+
+ // Look for (exclusive) spans forced to a specific volume but not yet
referenced by any volumes in cp_list,
+ // if found then create a new volume. This also makes sure new exclusive
disk volumes are created first
+ // before any other new volumes to assure proper span free space calculation
and
[trafficserver] branch 8.1.x updated: Updated ChangeLog
This is an automated email from the ASF dual-hosted git repository. zwoop pushed a commit to branch 8.1.x in repository https://gitbox.apache.org/repos/asf/trafficserver.git The following commit(s) were added to refs/heads/8.1.x by this push: new 78dbdc5 Updated ChangeLog 78dbdc5 is described below commit 78dbdc56dacd1637758fef1dc25f7bd3df5b2006 Author: Leif Hedstrom AuthorDate: Tue Jul 14 10:08:06 2020 -0600 Updated ChangeLog --- CHANGELOG-8.1.0 | 2 ++ 1 file changed, 2 insertions(+) diff --git a/CHANGELOG-8.1.0 b/CHANGELOG-8.1.0 index 0ed846b..3fa1af0 100644 --- a/CHANGELOG-8.1.0 +++ b/CHANGELOG-8.1.0 @@ -205,5 +205,7 @@ Changes with Apache Traffic Server 8.1.0 #6876 - Make Http2ClientSession inactive on EOS event #6885 - Revert 4028 for 8.1.x branch #6955 - Do not create Host Status stats for Origin servers. + #6985 - Fixes use after free when boringssl is used #6986 - Fix memory leak in header_rewrite #6987 - Fixes use of TS_USE_TLS_OCSP + #6990 - [8.1.x] Disable openclose_h2 AuTest on 8.1.x
[trafficserver] branch 9.0.x updated: Updated ChangeLog
This is an automated email from the ASF dual-hosted git repository. zwoop pushed a commit to branch 9.0.x in repository https://gitbox.apache.org/repos/asf/trafficserver.git The following commit(s) were added to refs/heads/9.0.x by this push: new 7fe8d6f Updated ChangeLog 7fe8d6f is described below commit 7fe8d6f5e9aa973ba65b1e9428b1332a7f7c3ada Author: Leif Hedstrom AuthorDate: Tue Jul 14 10:05:46 2020 -0600 Updated ChangeLog --- CHANGELOG-9.0.0 | 1 - 1 file changed, 1 deletion(-) diff --git a/CHANGELOG-9.0.0 b/CHANGELOG-9.0.0 index 729df82..42ad20c 100644 --- a/CHANGELOG-9.0.0 +++ b/CHANGELOG-9.0.0 @@ -1003,5 +1003,4 @@ Changes with Apache Traffic Server 9.0.0 #6969 - Update docs for some DNS config settings #6977 - Preserve cert name through ssl vc migration #6984 - Fix out of source tree builds for QUIC - #6985 - Fixes use after free when boringssl is used #6994 - Adds null check
[trafficserver] branch 8.1.x updated: Disable openclose_h2 AuTest on 8.1.x (#6990)
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 8.1.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
The following commit(s) were added to refs/heads/8.1.x by this push:
new 31fe963 Disable openclose_h2 AuTest on 8.1.x (#6990)
31fe963 is described below
commit 31fe963c2f29231bf762f8c76951c6fcffe6346a
Author: Masaori Koshiba
AuthorDate: Wed Jul 15 00:57:26 2020 +0900
Disable openclose_h2 AuTest on 8.1.x (#6990)
---
tests/gold_tests/continuations/openclose_h2.test.py | 5 +
1 file changed, 5 insertions(+)
diff --git a/tests/gold_tests/continuations/openclose_h2.test.py
b/tests/gold_tests/continuations/openclose_h2.test.py
index f3bba8f..06c10be 100644
--- a/tests/gold_tests/continuations/openclose_h2.test.py
+++ b/tests/gold_tests/continuations/openclose_h2.test.py
@@ -20,6 +20,11 @@ import os
Test.Summary = '''
Test transactions and sessions for http2, making sure they open and close in
the proper order.
'''
+
+# Disable this test on the 8.1.x branch because this test is sensitive to
timing issues of SSN/TXN close hook.
+# 9.0.x+ should fix issues and enabled this test. Details in #6983.
+Test.SkipIf(Condition.true("This test is sensitive to timing issues of SSN/TXN
close hook which makes it flaky."))
+
Test.SkipUnless(
Condition.HasProgram("curl", "Curl needs to be installed on system for
this test to work"),
Condition.HasCurlFeature('http2')
[trafficserver] branch 8.1.x updated: Fixes use after free when boringssl is used (#6998)
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 8.1.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
The following commit(s) were added to refs/heads/8.1.x by this push:
new 9fdb4f6 Fixes use after free when boringssl is used (#6998)
9fdb4f6 is described below
commit 9fdb4f6ff8a78bf2af5498e328a4ae3d35122901
Author: Randall Meyer
AuthorDate: Tue Jul 14 08:51:21 2020 -0700
Fixes use after free when boringssl is used (#6998)
Ownership of the ca_list is transferred when SSL_CTX_set_client_CA_list
is called. This change delays that transfer to after the elements are
hashed.
(cherry picked from commit be234547bde4bb50e7b05a0cae37a1efaa45eac6)
Conflicts:
iocore/net/SSLUtils.cc
---
iocore/net/SSLUtils.cc | 8 +++-
1 file changed, 3 insertions(+), 5 deletions(-)
diff --git a/iocore/net/SSLUtils.cc b/iocore/net/SSLUtils.cc
index 2787f3c..7ffcb41 100644
--- a/iocore/net/SSLUtils.cc
+++ b/iocore/net/SSLUtils.cc
@@ -1879,13 +1879,8 @@ SSLInitServerContext(const SSLConfigParams *params,
const ssl_user_config *sslMu
SSL_CTX_set_verify_depth(ctx, params->verify_depth); // might want to make
configurable at some point.
}
- // Set the list of CA's to send to client if we ask for a client
- // certificate
if (params->serverCACertFilename) {
ca_list = SSL_load_client_CA_file(params->serverCACertFilename);
-if (ca_list) {
- SSL_CTX_set_client_CA_list(ctx, ca_list);
-}
}
if (EVP_DigestInit_ex(digest, evp_md_func, nullptr) == 0) {
@@ -1912,6 +1907,9 @@ SSLInitServerContext(const SSLConfigParams *params, const
ssl_user_config *sslMu
goto fail;
}
}
+
+// Set the list of CA's to send to client if we ask for a client
certificate
+SSL_CTX_set_client_CA_list(ctx, ca_list);
}
if (EVP_DigestFinal_ex(digest, hash_buf, _len) == 0) {
