[trafficserver] branch 9.0.x updated: Add support for a simple_server_retry_responses list (#6605)
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
The following commit(s) were added to refs/heads/9.0.x by this push:
new e95503f Add support for a simple_server_retry_responses list (#6605)
e95503f is described below
commit e95503f1c6c7311d5f7ba9dda5466512b1bb54b4
Author: Evan Zelkowitz
AuthorDate: Thu Apr 2 08:15:35 2020 -0700
Add support for a simple_server_retry_responses list (#6605)
This adds support for simple_server_retry_responses to allow the user to
set a list of 4xx reponse codes which trigger a simple retry, instead of the
current functionality which will only trigger on a 404. This mimics the
unavailable responses list in the same manner except for the 4xx simple retry
case
(cherry picked from commit 0d22ab057a42cf3fbc0b782479e964df2c1c8047)
---
doc/admin-guide/files/parent.config.en.rst | 10 ++-
proxy/ParentSelection.cc | 42 ++
proxy/ParentSelection.h| 29 -
proxy/http/HttpTransact.cc | 29 +
4 files changed, 103 insertions(+), 7 deletions(-)
diff --git a/doc/admin-guide/files/parent.config.en.rst
b/doc/admin-guide/files/parent.config.en.rst
index 83584ea..c7fc123 100644
--- a/doc/admin-guide/files/parent.config.en.rst
+++ b/doc/admin-guide/files/parent.config.en.rst
@@ -207,7 +207,8 @@ The following list shows the possible actions and their
allowed values.
.. _parent-config-format-parent-retry:
``parent_retry``
-- ``simple_retry`` - If the parent origin server returns a 404 response on
a request
+- ``simple_retry`` - If the parent returns a 404 response or if the
response matches
+ a list of http 4xx responses defined in
``simple_server_retry_responses`` on a request
a new parent is selected and the request is retried. The number of
retries is controlled
by ``max_simple_retries`` which is set to 1 by default.
- ``unavailable_server_retry`` - If the parent returns a 503 response or
if the response matches
@@ -216,6 +217,13 @@ The following list shows the possible actions and their
allowed values.
retries is controlled by ``max_unavailable_server_retries`` which is set
to 1 by default.
- ``both`` - This enables both ``simple_retry`` and
``unavailable_server_retry`` as described above.
+.. _parent-config-format-simple-server-retry-responses:
+
+``simple_server_retry_responses``
+ If ``parent_retry`` is set to either ``simple_retry`` or ``both``, this
parameter is a comma separated list of
+ http 4xx response codes that will invoke the ``simple_retry`` described in
the ``parent_retry`` section. By
+ default, ``simple_server_retry_responses`` is set to 404.
+
.. _parent-config-format-unavailable-server-retry-responses:
``unavailable_server_retry_responses``
diff --git a/proxy/ParentSelection.cc b/proxy/ParentSelection.cc
index 2589ce9..1b6a861 100644
--- a/proxy/ParentSelection.cc
+++ b/proxy/ParentSelection.cc
@@ -350,6 +350,33 @@
UnavailableServerResponseCodes::UnavailableServerResponseCodes(char *val)
std::sort(codes.begin(), codes.end());
}
+SimpleRetryResponseCodes::SimpleRetryResponseCodes(char *val)
+{
+ Tokenizer pTok(", \t\r");
+ int numTok = 0, c;
+
+ if (val == nullptr) {
+Warning("SimpleRetryResponseCodes - simple_server_retry_responses is null
loading default 404 code.");
+codes.push_back(HTTP_STATUS_NOT_FOUND);
+return;
+ }
+ numTok = pTok.Initialize(val, SHARE_TOKS);
+ if (numTok == 0) {
+c = atoi(val);
+if (c > 399 && c < 500) {
+ codes.push_back(HTTP_STATUS_NOT_FOUND);
+}
+ }
+ for (int i = 0; i < numTok; i++) {
+c = atoi(pTok[i]);
+if (c > 399 && c < 500) {
+ Debug("parent_select", "loading simple response code: %d", c);
+ codes.push_back(c);
+}
+ }
+ std::sort(codes.begin(), codes.end());
+}
+
void
ParentRecord::PreProcessParents(const char *val, const int line_num, char
*buf, size_t len)
{
@@ -694,6 +721,9 @@ ParentRecord::Init(matcher_line *line_info)
} else if (strcasecmp(label, "unavailable_server_retry_responses") == 0 &&
unavailable_server_retry_responses == nullptr) {
unavailable_server_retry_responses = new
UnavailableServerResponseCodes(val);
used = true;
+} else if (strcasecmp(label, "simple_server_retry_responses") == 0 &&
simple_server_retry_responses == nullptr) {
+ simple_server_retry_responses = new SimpleRetryResponseCodes(val);
+ used = true;
} else if (strcasecmp(label, "max_simple_retries") == 0) {
int v = atoi(val);
if (v >= 1 && v < MAX_SIMPLE_RETRIES) {
@@ -750,6 +780,18 @@ ParentRecord::Init(matcher_line *line_info)
unavailable_server_retry_responses = new
UnavailableServerResponseCodes(nullptr);
}
+
[trafficserver] branch 9.0.x updated: Fixes to hostDB to avoid event and memory leaks (#6686)
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
The following commit(s) were added to refs/heads/9.0.x by this push:
new e05d9e1 Fixes to hostDB to avoid event and memory leaks (#6686)
e05d9e1 is described below
commit e05d9e1ee3f9065fc1621e42b9bfd050a2acf3b9
Author: Susan Hinrichs
AuthorDate: Wed May 20 14:50:51 2020 -0500
Fixes to hostDB to avoid event and memory leaks (#6686)
Co-authored-by: Susan Hinrichs
(cherry picked from commit 6094492e3efd178298d87629a54d0383d575c9d9)
---
iocore/hostdb/HostDB.cc | 60 ---
iocore/hostdb/P_HostDBProcessor.h | 1 -
2 files changed, 31 insertions(+), 30 deletions(-)
diff --git a/iocore/hostdb/HostDB.cc b/iocore/hostdb/HostDB.cc
index 3e6aaad..555ea95 100644
--- a/iocore/hostdb/HostDB.cc
+++ b/iocore/hostdb/HostDB.cc
@@ -43,7 +43,6 @@ HostDBContinuation::Options const
HostDBContinuation::DEFAULT_OPTIONS;
int hostdb_enable = true;
int hostdb_migrate_on_demand = true;
int hostdb_lookup_timeout = 30;
-int hostdb_insert_timeout = 160;
int hostdb_re_dns_on_reload= false;
int hostdb_ttl_mode= TTL_OBEY;
unsigned int hostdb_round_robin_max_count = 16;
@@ -1175,12 +1174,13 @@ HostDBContinuation::dnsEvent(int event, HostEnt *e)
timeout = nullptr;
}
EThread *thread = mutex->thread_holding;
- if (event == EVENT_INTERVAL) {
+ if (event != DNS_EVENT_LOOKUP) {
+// This was an event_interval or an event_immediate
+// Either we timed out, or remove_trigger_pending gave up on us
if (!action.continuation) {
// give up on insert, it has been too long
- // remove_trigger_pending_dns will notify and clean up all requests
- // including this one.
- remove_trigger_pending_dns();
+ hostDB.pending_dns_for_hash(hash.hash).remove(this);
+ hostdb_cont_free(this);
return EVENT_DONE;
}
MUTEX_TRY_LOCK(lock, action.mutex, thread);
@@ -1196,8 +1196,6 @@ HostDBContinuation::dnsEvent(int event, HostEnt *e)
action.continuation->handleEvent(EVENT_HOST_DB_LOOKUP, nullptr);
}
action = nullptr;
-// do not exit yet, wait to see if we can insert into DB
-timeout = thread->schedule_in(this, HRTIME_SECONDS(hostdb_insert_timeout));
return EVENT_DONE;
} else {
bool failed = !e || !e->good;
@@ -1420,37 +1418,38 @@ HostDBContinuation::dnsEvent(int event, HostEnt *e)
return EVENT_CONT;
}
- // We have seen cases were the action.mutex != action.continuation.mutex.
+ // We have seen cases were the action.mutex !=
action.continuation.mutex. However, it seems that case
+ // is likely a memory corruption... Thus the introduction of the assert.
// Since reply_to_cont will call the handler on the action.continuation,
it is important that we hold
// that mutex.
bool need_to_reschedule = true;
MUTEX_TRY_LOCK(lock, action.mutex, thread);
if (lock.is_locked()) {
-need_to_reschedule = !action.cancelled;
if (!action.cancelled) {
if (action.continuation->mutex) {
-MUTEX_TRY_LOCK(lock2, action.continuation->mutex, thread);
-if (lock2.is_locked()) {
- reply_to_cont(action.continuation, r, is_srv());
- need_to_reschedule = false;
-}
- } else {
-reply_to_cont(action.continuation, r, is_srv());
-need_to_reschedule = false;
+ink_release_assert(action.continuation->mutex == action.mutex);
}
+ reply_to_cont(action.continuation, r, is_srv());
}
+need_to_reschedule = false;
}
+
if (need_to_reschedule) {
SET_HANDLER((HostDBContHandler)::probeEvent);
-// remove_trigger_pending_dns should kick off the current hostDB too
-// No need to explicitly reschedule
-remove_trigger_pending_dns();
+// Will reschedule on affinity thread or current thread
+timeout = eventProcessor.schedule_in(this, HOST_DB_RETRY_PERIOD);
return EVENT_CONT;
}
}
+
+// Clean ourselves up
+hostDB.pending_dns_for_hash(hash.hash).remove(this);
+
// wake up everyone else who is waiting
remove_trigger_pending_dns();
+hostdb_cont_free(this);
+
// all done, or at least scheduled to be all done
//
return EVENT_DONE;
@@ -1523,12 +1522,17 @@ HostDBContinuation::probeEvent(int /* event ATS_UNUSED
*/, Event *e)
ink_assert(!link.prev && !link.next);
EThread *t = e ? e->ethread : this_ethread();
+ if (timeout) {
+timeout->cancel(this);
+timeout = nullptr;
+ }
+
MUTEX_TRY_LOCK(lock, action.mutex, t);
// Separating lock checks here to make sure
[trafficserver] branch 9.0.x updated: Add an optional ramcache setting to volume.config to be able to disable it (#6746)
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
The following commit(s) were added to refs/heads/9.0.x by this push:
new af1f699 Add an optional ramcache setting to volume.config to be able
to disable it (#6746)
af1f699 is described below
commit af1f699024a4a7c0bebf70de62d69b906591c23d
Author: Evan Zelkowitz
AuthorDate: Wed May 20 12:57:37 2020 -0700
Add an optional ramcache setting to volume.config to be able to disable it
(#6746)
* Add an optional ramcache setting to volume.config to be able to disable it
This adds an additional option of `ramcache=true/false` to the
volume.config.
The default is true, so enabled, to leave current functionality as is and
for
the general use case. If set to false then this will stop the cache from
initializing
a ramcache for that specific volume and avoid the ramcache all together.
This may
be desirable for people running with volumes composed of ram disks to avoid
double
ram caching and reclaim memory+time
(cherry picked from commit 91a9db1b172648b05d7c4b56c6190b270792899e)
---
doc/admin-guide/files/volume.config.en.rst | 17 +--
doc/admin-guide/storage/index.en.rst | 9
iocore/cache/Cache.cc | 22 +++
iocore/cache/CacheHosting.cc | 35 +-
iocore/cache/P_CacheHosting.h | 1 +
iocore/cache/P_CacheVol.h | 13 ++-
6 files changed, 70 insertions(+), 27 deletions(-)
diff --git a/doc/admin-guide/files/volume.config.en.rst
b/doc/admin-guide/files/volume.config.en.rst
index 7fdfc8e..46395f1 100644
--- a/doc/admin-guide/files/volume.config.en.rst
+++ b/doc/admin-guide/files/volume.config.en.rst
@@ -57,14 +57,27 @@ volumes without deleting and clearing the existing volumes.
Changing this file to add, remove or modify volumes effectively invalidates
the cache.
+
+Optional ramcache setting
+-
+
+You can also add an option ``ramcache=true/false`` to the volume configuration
+line. True is the default setting and so not needed unless you want to
explicitly
+set it. Setting ``ramcache=false`` will disable the ramcache that normally
+sits in front of a volume. This may be desirable if you are using something
like
+ramdisks, to avoid wasting RAM and cpu time on double caching objects.
+
+
Examples
The following example partitions the cache across 5 volumes to decreasing
-single-lock pressure for a machine with few drives.::
+single-lock pressure for a machine with few drives. The last volume being
+an example of one that might be composed of purely ramdisks so that the
+ramcache has been disabled.::
volume=1 scheme=http size=20%
volume=2 scheme=http size=20%
volume=3 scheme=http size=20%
volume=4 scheme=http size=20%
-volume=5 scheme=http size=20%
+volume=5 scheme=http size=20% ramcache=false
diff --git a/doc/admin-guide/storage/index.en.rst
b/doc/admin-guide/storage/index.en.rst
index cdd3cc9..68bcd81 100644
--- a/doc/admin-guide/storage/index.en.rst
+++ b/doc/admin-guide/storage/index.en.rst
@@ -137,6 +137,15 @@ To change the RAM cache size:
1GB or more, then restart with the :program:`trafficserver` command
(refer to :ref:`start-traffic-server`).
+Disabling the RAM Cache
+---
+
+It is possible to disable the RAM cache. If you have configured your
+storage using the :file:`volume.config` you can add an optional directive
+of ``ramcache=false`` to whichever volumes you wish to have it disabled on.
+This may be desirable for volumes composed of storage like RAM disks where
+you may want to avoid double RAM caching.
+
Changing Cache Capacity
===
diff --git a/iocore/cache/Cache.cc b/iocore/cache/Cache.cc
index 841fa06..0192220 100644
--- a/iocore/cache/Cache.cc
+++ b/iocore/cache/Cache.cc
@@ -911,12 +911,14 @@ CacheProcessor::cacheInitialized()
Debug("cache_init", "CacheProcessor::cacheInitialized -
cache_config_ram_cache_size == AUTO_SIZE_RAM_CACHE");
for (i = 0; i < gnvol; i++) {
vol = gvol[i];
- gvol[i]->ram_cache->init(vol->dirlen() *
DEFAULT_RAM_CACHE_MULTIPLIER, vol);
- ram_cache_bytes += gvol[i]->dirlen();
- Debug("cache_init", "CacheProcessor::cacheInitialized -
ram_cache_bytes = %" PRId64 " = %" PRId64 "Mb", ram_cache_bytes,
-ram_cache_bytes / (1024 * 1024));
- CACHE_VOL_SUM_DYN_STAT(cache_ram_cache_bytes_total_stat,
(int64_t)gvol[i]->dirlen());
+ if (gvol[i]->cache_vol->ramcache_enabled) {
+gvol[i]->ram_cache->init(vol->dirlen() *
DEFAULT_RAM_CACHE_MULTIPLIER, vol);
+ram_cache_bytes += gvol[i]->dirlen();
+Debug("cache_init", "CacheProcessor::cacheInitialized -
[trafficserver] branch master updated (6094492 -> 91a9db1)
This is an automated email from the ASF dual-hosted git repository. eze pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git. from 6094492 Fixes to hostDB to avoid event and memory leaks (#6686) add 91a9db1 Add an optional ramcache setting to volume.config to be able to disable it (#6746) No new revisions were added by this update. Summary of changes: doc/admin-guide/files/volume.config.en.rst | 17 +-- doc/admin-guide/storage/index.en.rst | 9 iocore/cache/Cache.cc | 22 +++ iocore/cache/CacheHosting.cc | 35 +- iocore/cache/P_CacheHosting.h | 1 + iocore/cache/P_CacheVol.h | 13 ++- 6 files changed, 70 insertions(+), 27 deletions(-)
[trafficserver] branch master updated (34b57fc -> 6094492)
This is an automated email from the ASF dual-hosted git repository. shinrich pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git. from 34b57fc Add TXN_CLOSE hook to CPPAPI TransactionPlugin (#6800) add 6094492 Fixes to hostDB to avoid event and memory leaks (#6686) No new revisions were added by this update. Summary of changes: iocore/hostdb/HostDB.cc | 60 --- iocore/hostdb/P_HostDBProcessor.h | 1 - 2 files changed, 31 insertions(+), 30 deletions(-)
[trafficserver] branch 8.1.x updated: Updated ChangeLog
This is an automated email from the ASF dual-hosted git repository. zwoop pushed a commit to branch 8.1.x in repository https://gitbox.apache.org/repos/asf/trafficserver.git The following commit(s) were added to refs/heads/8.1.x by this push: new 59b18e2 Updated ChangeLog 59b18e2 is described below commit 59b18e2f5501a562a03edef4a564b59872d383d9 Author: Leif Hedstrom AuthorDate: Wed May 20 12:00:13 2020 -0600 Updated ChangeLog --- CHANGELOG-8.1.0 | 3 +++ 1 file changed, 3 insertions(+) diff --git a/CHANGELOG-8.1.0 b/CHANGELOG-8.1.0 index c756095..ab90213 100644 --- a/CHANGELOG-8.1.0 +++ b/CHANGELOG-8.1.0 @@ -187,5 +187,8 @@ Changes with Apache Traffic Server 8.1.0 #6708 - Adding logging fields for collapsed forwarding metrics #6729 - Fix a bug that current_active_client_connections doesn't decrease #6750 - backports parent.config ignore_self_detect flag to 8.1.x + #6753 - Fix HPACK Dynamic Table Cleanup #6756 - Correct `schedule_every_local` to schedule locally #6779 - 8.1.x backport: header_rewrite: this fixes parsing where the [ ] section gets merged into values + #6785 - clang-analyzer: Fix dead nested assignment issues [8.1.x] + #6786 - HPACK: send back an error to the client when the index is invalid
[trafficserver] branch 9.0.x updated: Updated ChangeLog
This is an automated email from the ASF dual-hosted git repository. zwoop pushed a commit to branch 9.0.x in repository https://gitbox.apache.org/repos/asf/trafficserver.git The following commit(s) were added to refs/heads/9.0.x by this push: new 06d4938 Updated ChangeLog 06d4938 is described below commit 06d4938d72eaea0ab174a7528e59cf638deef80b Author: Leif Hedstrom AuthorDate: Wed May 20 11:59:15 2020 -0600 Updated ChangeLog --- CHANGELOG-9.0.0 | 17 + 1 file changed, 17 insertions(+) diff --git a/CHANGELOG-9.0.0 b/CHANGELOG-9.0.0 index 7737cd6..db421ae 100644 --- a/CHANGELOG-9.0.0 +++ b/CHANGELOG-9.0.0 @@ -709,6 +709,8 @@ Changes with Apache Traffic Server 9.0.0 #6095 - For remap_stats, removes configure time dependency on search.h #6096 - Fixing log cleanup candidate selection and adding a test for it. #6099 - doc + unittest TSRemap(Init|NewInstance) failures + #6103 - Use enqueue_local when scheduling on the same thread + #6104 - Cleanup the eventloop #6106 - Removes proxy.config.cache.storage_filename #6116 - Updated to clang-format v9.0.0 #6120 - Fix null pointer dereference reported by clang-analyzer @@ -730,6 +732,7 @@ Changes with Apache Traffic Server 9.0.0 #6159 - Detect bogus jemalloc version #6160 - Fixes misc. spelling and whitespace #6163 - Update yaml-cpp to 0.6.3 + #6166 - Make sure time is consistent between calculations #6167 - Fixed gcc7 issue with yaml-cpp 0.6.3 #6169 - Fixes cppcheck issues for cookie_remap plugin #6170 - Add test to catch regressions in sni and override configs @@ -839,6 +842,8 @@ Changes with Apache Traffic Server 9.0.0 #6475 - Remove noisy mutex warning #6477 - traffic_dump: Fixing content:size collection. #6480 - Remove some outdated files. + #6483 - Rework server side SSL_CTX creation to better handle dual_cert mismatches + #6486 - Issue 3546: Add "overridable" to the configuration variable description. #6487 - Fix session reuse plugin shutdown crashes and cleanups #6488 - Bikeshedding some code structures for reloadable plugins config #6492 - Add null check to fix error tunnel crash @@ -847,6 +852,7 @@ Changes with Apache Traffic Server 9.0.0 #6501 - Remove method that does nothing. #6508 - Include start line of HTTP messages in xdebug probe output. #6513 - Moves hosting.config finished loading message outside of parsing loop + #6516 - Fix SDK_API_TSSslServerContextCreate #6517 - Make traffic_ctl limp along for now with large records.snap #6519 - Adds support for configure option --enable-yaml-headers #6523 - Removes noisy log statement from xdebug @@ -860,6 +866,7 @@ Changes with Apache Traffic Server 9.0.0 #6566 - Add more options to session_sharing.match #6567 - Moved printing the incoming headers for debug before remapping #6569 - AuTest for server_push_preload plugin + #6571 - Lua plugin number of states configuration and stats printing #6573 - SSL: Always renew TLS Session Tickets iff TLSv1.3 is being used #6576 - Ensure TSContSchedule API family are called from an EThread. #6577 - When using TSContSchedule() and TSContScheduleAPI() set the calling thread as the thread affinity when not already set @@ -875,10 +882,12 @@ Changes with Apache Traffic Server 9.0.0 #6618 - Fix missing virtual destructor for PluginUserArgsMixin. #6628 - Use default rwlock attributes on initialize #6632 - Fixes a bug where the nexthop markNextHop method to mark a host down is not called. + #6642 - Fixes crash loading combined(cert+key) certs #6645 - Traffic Dump: Adding an SNI filtering option. #6655 - Fix origin scheme selection with partial-blind addition #6656 - Check sni against SSL object #6658 - Update TSStatFindName to check that sync callback is set on the stat + #6662 - Fixes memory leak loading certs #6663 - Fixes memory leak during log configuration #6664 - cache_range_requests: remove unnecessary Last-Modified header from tests #6677 - Format to match perferred if/else formatting for sh scripts @@ -893,15 +902,23 @@ Changes with Apache Traffic Server 9.0.0 #6717 - Fixup .gitignores to match repo reality #6718 - gcc10: fixed warning about returning local variable in int64_to_str() #6723 - Document ip_allow in sni.yaml + #6727 - traffic_dump: add tls information to dump. #6730 - Add HttpTransact::get_max_age and TSHttpTxnGetMaxAge #6731 - Fix g++ 10 compile errors. #6734 - Update expired test certificates for cert_update + #6735 - Remove unused index for SSL application specific data #6740 - Cleans up doubled words in documentation #6742 - gcc10: fixed clearing an object of non-trivial type for wccp + #6743 - traffic_dump: refactor to make transactions written atomically #6754 - Enforce Active Connection limits #6755 - Add metrics to track default inactivity timed out connections #6757 - ASAN: Fixed one definition rule violation + #6760 - Do not fail multicert load
[trafficserver] branch 9.0.x updated (62e1dcb -> 96eaa3aa)
This is an automated email from the ASF dual-hosted git repository. zwoop pushed a change to branch 9.0.x in repository https://gitbox.apache.org/repos/asf/trafficserver.git. from 62e1dcb Issue 3546: Add "overridable" to the configuration variable description. new d15c454 cleanup the eventloop new 96eaa3aa put events into local queue when scheduling on the same thread as the scheduler The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: iocore/aio/AIO.cc | 4 +-- iocore/cache/CacheWrite.cc | 6 ++-- iocore/dns/DNS.cc | 2 +- iocore/eventsystem/I_EThread.h | 6 +--- iocore/eventsystem/I_EventProcessor.h | 8 ++--- iocore/eventsystem/I_ProtectedQueue.h | 4 +-- iocore/eventsystem/P_UnixEThread.h | 19 +--- iocore/eventsystem/P_UnixEventProcessor.h | 36 +- iocore/eventsystem/ProtectedQueue.cc | 23 ++ iocore/eventsystem/UnixEThread.cc | 16 +- iocore/net/UnixNetAccept.cc| 2 +- .../null_transform/gold/null_transform-tag.gold| 2 +- 12 files changed, 37 insertions(+), 91 deletions(-)
[trafficserver] 01/02: cleanup the eventloop
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
commit d15c45484e15ec7c141c6af66116c57dd2ce62e2
Author: Fei Deng
AuthorDate: Wed Oct 30 15:55:15 2019 -0500
cleanup the eventloop
(cherry picked from commit c179620382dc37eb33f39357f458175800756f14)
Conflicts:
iocore/net/UnixNetAccept.cc
---
iocore/aio/AIO.cc | 4 ++--
iocore/cache/CacheWrite.cc| 6 +++---
iocore/dns/DNS.cc | 2 +-
iocore/eventsystem/I_EThread.h| 6 +-
iocore/eventsystem/I_EventProcessor.h | 8 ++--
iocore/eventsystem/I_ProtectedQueue.h | 4 +---
iocore/eventsystem/P_UnixEThread.h| 13 ++---
iocore/eventsystem/P_UnixEventProcessor.h | 18 ++
iocore/eventsystem/ProtectedQueue.cc | 21 +
iocore/eventsystem/UnixEThread.cc | 16 +---
iocore/net/UnixNetAccept.cc | 2 +-
11 files changed, 17 insertions(+), 83 deletions(-)
diff --git a/iocore/aio/AIO.cc b/iocore/aio/AIO.cc
index 5543919..df8e27b 100644
--- a/iocore/aio/AIO.cc
+++ b/iocore/aio/AIO.cc
@@ -480,9 +480,9 @@ aio_thread_main(void *arg)
SCOPED_MUTEX_LOCK(lock, op->mutex, thr_info->mutex->thread_holding);
op->handleEvent(EVENT_NONE, nullptr);
} else if (op->thread == AIO_CALLBACK_THREAD_ANY) {
-eventProcessor.schedule_imm_signal(op);
+eventProcessor.schedule_imm(op);
} else {
-op->thread->schedule_imm_signal(op);
+op->thread->schedule_imm(op);
}
ink_mutex_acquire(_aio_req->aio_mutex);
} while (true);
diff --git a/iocore/cache/CacheWrite.cc b/iocore/cache/CacheWrite.cc
index 944d527..cff1818 100644
--- a/iocore/cache/CacheWrite.cc
+++ b/iocore/cache/CacheWrite.cc
@@ -360,7 +360,7 @@ Vol::aggWriteDone(int event, Event *e)
CacheVC *c = nullptr;
while ((c = sync.dequeue())) {
if (UINT_WRAP_LTE(c->write_serial + 2, header->write_serial)) {
- eventProcessor.schedule_imm_signal(c, ET_CALL, AIO_EVENT_DONE);
+ eventProcessor.schedule_imm(c, ET_CALL, AIO_EVENT_DONE);
} else {
sync.push(c); // put it back on the front
break;
@@ -1028,7 +1028,7 @@ Lagain:
ink_assert(false);
while ((c = agg.dequeue())) {
agg_todo_size -= c->agg_len;
-eventProcessor.schedule_imm_signal(c, ET_CALL, AIO_EVENT_DONE);
+eventProcessor.schedule_imm(c, ET_CALL, AIO_EVENT_DONE);
}
return EVENT_CONT;
}
@@ -1092,7 +1092,7 @@ Lwait:
if (event == EVENT_CALL && c->mutex->thread_holding ==
mutex->thread_holding) {
ret = EVENT_RETURN;
} else {
- eventProcessor.schedule_imm_signal(c, ET_CALL, AIO_EVENT_DONE);
+ eventProcessor.schedule_imm(c, ET_CALL, AIO_EVENT_DONE);
}
}
return ret;
diff --git a/iocore/dns/DNS.cc b/iocore/dns/DNS.cc
index 6713003..d33c4dc 100644
--- a/iocore/dns/DNS.cc
+++ b/iocore/dns/DNS.cc
@@ -1437,7 +1437,7 @@ DNSEntry::post(DNSHandler *h, HostEnt *ent)
} else {
mutex = action.mutex;
SET_HANDLER(::postOneEvent);
-submit_thread->schedule_imm_signal(this);
+submit_thread->schedule_imm(this);
}
return 0;
}
diff --git a/iocore/eventsystem/I_EThread.h b/iocore/eventsystem/I_EThread.h
index c70fb04..45afc73 100644
--- a/iocore/eventsystem/I_EThread.h
+++ b/iocore/eventsystem/I_EThread.h
@@ -126,7 +126,6 @@ public:
*/
Event *schedule_imm(Continuation *c, int callback_event = EVENT_IMMEDIATE,
void *cookie = nullptr);
- Event *schedule_imm_signal(Continuation *c, int callback_event =
EVENT_IMMEDIATE, void *cookie = nullptr);
/**
Schedules the continuation on this EThread to receive an event
@@ -300,7 +299,7 @@ public:
EThread =(const EThread &) = delete;
~EThread() override;
- Event *schedule(Event *e, bool fast_signal = false);
+ Event *schedule(Event *e);
/** Block of memory to allocate thread specific data e.g. stat system
arrays. */
char thread_private[PER_THREAD_DATA];
@@ -314,9 +313,6 @@ public:
ProtectedQueue EventQueueExternal;
PriorityEventQueue EventQueue;
- EThread **ethreads_to_be_signalled = nullptr;
- int n_ethreads_to_be_signalled = 0;
-
static constexpr int NO_ETHREAD_ID = -1;
int id = NO_ETHREAD_ID;
unsigned int event_types = 0;
diff --git a/iocore/eventsystem/I_EventProcessor.h
b/iocore/eventsystem/I_EventProcessor.h
index 227ea19..782137a 100644
--- a/iocore/eventsystem/I_EventProcessor.h
+++ b/iocore/eventsystem/I_EventProcessor.h
@@ -151,11 +151,7 @@ public:
*/
Event *schedule_imm(Continuation *c, EventType event_type = ET_CALL, int
callback_event = EVENT_IMMEDIATE,
void *cookie = nullptr);
- /*
-provides the same functionality as schedule_imm and also signals the
thread
[trafficserver] 02/02: put events into local queue when scheduling on the same thread as the scheduler
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
commit 96eaa3aa44129eb06c46e1dec945e63c47bead6a
Author: Fei Deng
AuthorDate: Thu Oct 31 14:16:10 2019 -0500
put events into local queue when scheduling on the same thread as the
scheduler
update docs
(cherry picked from commit 6a1c5f7397af98dd45d1e6f227219ab463c6363a)
---
iocore/eventsystem/P_UnixEThread.h | 8 +++-
iocore/eventsystem/P_UnixEventProcessor.h | 22 ++
iocore/eventsystem/ProtectedQueue.cc | 2 +-
.../null_transform/gold/null_transform-tag.gold| 2 +-
4 files changed, 23 insertions(+), 11 deletions(-)
diff --git a/iocore/eventsystem/P_UnixEThread.h
b/iocore/eventsystem/P_UnixEThread.h
index 55f093a..520ffdf 100644
--- a/iocore/eventsystem/P_UnixEThread.h
+++ b/iocore/eventsystem/P_UnixEThread.h
@@ -91,7 +91,13 @@ EThread::schedule(Event *e)
// The continuation that gets scheduled later is not always the
// client VC, it can be HttpCacheSM etc. so save the flags
e->continuation->control_flags.set_flags(get_cont_flags().get_flags());
- EventQueueExternal.enqueue(e);
+
+ if (e->ethread == this_ethread()) {
+EventQueueExternal.enqueue_local(e);
+ } else {
+EventQueueExternal.enqueue(e);
+ }
+
return e;
}
diff --git a/iocore/eventsystem/P_UnixEventProcessor.h
b/iocore/eventsystem/P_UnixEventProcessor.h
index fe89945..a8ba4f4 100644
--- a/iocore/eventsystem/P_UnixEventProcessor.h
+++ b/iocore/eventsystem/P_UnixEventProcessor.h
@@ -97,18 +97,18 @@ EventProcessor::schedule(Event *e, EventType etype)
return nullptr;
}
- EThread *ethread = e->continuation->getThreadAffinity();
- if (ethread != nullptr && ethread->is_event_type(etype)) {
-e->ethread = ethread;
+ EThread *affinity_thread = e->continuation->getThreadAffinity();
+ EThread *curr_thread = this_ethread();
+ if (affinity_thread != nullptr && affinity_thread->is_event_type(etype)) {
+e->ethread = affinity_thread;
} else {
-ethread = this_ethread();
// Is the current thread eligible?
-if (ethread != nullptr && ethread->is_event_type(etype)) {
- e->ethread = ethread;
+if (curr_thread != nullptr && curr_thread->is_event_type(etype)) {
+ e->ethread = curr_thread;
} else {
e->ethread = assign_thread(etype);
}
-if (e->continuation->getThreadAffinity() == nullptr) {
+if (affinity_thread == nullptr) {
e->continuation->setThreadAffinity(e->ethread);
}
}
@@ -116,7 +116,13 @@ EventProcessor::schedule(Event *e, EventType etype)
if (e->continuation->mutex) {
e->mutex = e->continuation->mutex;
}
- e->ethread->EventQueueExternal.enqueue(e);
+
+ if (curr_thread != nullptr && e->ethread == curr_thread) {
+e->ethread->EventQueueExternal.enqueue_local(e);
+ } else {
+e->ethread->EventQueueExternal.enqueue(e);
+ }
+
return e;
}
diff --git a/iocore/eventsystem/ProtectedQueue.cc
b/iocore/eventsystem/ProtectedQueue.cc
index d8a14da..d7936c6 100644
--- a/iocore/eventsystem/ProtectedQueue.cc
+++ b/iocore/eventsystem/ProtectedQueue.cc
@@ -101,7 +101,7 @@ ProtectedQueue::wait(ink_hrtime timeout)
* - And then the Event Thread goes to sleep and waits for the wakeup
signal of `EThread::might_have_data`,
* - The `EThread::lock` will be locked again when the Event Thread wakes
up.
*/
- if (INK_ATOMICLIST_EMPTY(al)) {
+ if (INK_ATOMICLIST_EMPTY(al) && localQueue.empty()) {
timespec ts = ink_hrtime_to_timespec(timeout);
ink_cond_timedwait(_have_data, , );
}
diff --git a/tests/gold_tests/null_transform/gold/null_transform-tag.gold
b/tests/gold_tests/null_transform/gold/null_transform-tag.gold
index 7f60846..733c4d1 100644
--- a/tests/gold_tests/null_transform/gold/null_transform-tag.gold
+++ b/tests/gold_tests/null_transform/gold/null_transform-tag.gold
@@ -1 +1 @@
-``DIAG: (null_transform)``
\ No newline at end of file
+``DIAG: (null_transform)``
[trafficserver] branch 9.0.x updated: Issue 3546: Add "overridable" to the configuration variable description.
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
The following commit(s) were added to refs/heads/9.0.x by this push:
new 62e1dcb Issue 3546: Add "overridable" to the configuration variable
description.
62e1dcb is described below
commit 62e1dcb621991819bebb49464afb095dc8d0123c
Author: Alan M. Carroll
AuthorDate: Thu Mar 5 13:15:45 2020 -0600
Issue 3546: Add "overridable" to the configuration variable description.
(cherry picked from commit 5efcddef263ec213aea5ede76f5ac95dd9c3b343)
---
CMakeLists.txt | 3 +-
doc/developer-guide/config-vars.en.rst | 17 ++--
include/shared/overridable_txn_vars.h | 33 +++
src/shared/overridable_txn_vars.cc | 162 +
src/traffic_ctl/Makefile.inc | 3 +
src/traffic_ctl/config.cc | 33 +++
src/traffic_server/InkAPI.cc | 142 +
src/traffic_server/Makefile.inc| 1 +
8 files changed, 245 insertions(+), 149 deletions(-)
diff --git a/CMakeLists.txt b/CMakeLists.txt
index b464fcb..918ff24 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -63,7 +63,7 @@ endmacro(CPP_LIB)
macro(CPP_ADD_SOURCES target path)
file(GLOB cpp_add_src_files ${path}/*.h ${path}/*.cc)
-target_sources(${target} PUBLIC ${cpp_add_src_files})
+target_sources(${target} PRIVATE ${cpp_add_src_files})
endmacro(CPP_ADD_SOURCES)
CC_EXEC(traffic_cache_tool src/traffic_cache_tool)
@@ -74,6 +74,7 @@ CC_EXEC(traffic_logcat src/traffic_logcat)
CC_EXEC(traffic_logstats src/traffic_logstats)
CC_EXEC(traffic_manager src/traffic_manager)
CC_EXEC(traffic_server src/traffic_server)
+target_sources(traffic_server PRIVATE src/shared/overridable_txn_vars.cc)
CC_EXEC(traffic_top src/traffic_top)
CC_EXEC(traffic_via src/traffic_via)
CC_EXEC(traffic_wccp src/traffic_wccp)
diff --git a/doc/developer-guide/config-vars.en.rst
b/doc/developer-guide/config-vars.en.rst
index 11fd9c8..a234a0e 100644
--- a/doc/developer-guide/config-vars.en.rst
+++ b/doc/developer-guide/config-vars.en.rst
@@ -33,11 +33,15 @@
.. |InkAPI.cc| replace:: ``InkAPI.cc``
-.. _InkAPI.cc:
https://github.com/apache/trafficserver/blob/master/proxy/api/InkAPI.cc
+.. _InkAPI.cc:
https://github.com/apache/trafficserver/blob/master/src/traffic_server/InkAPI.cc
.. |InkAPITest.cc| replace:: ``InkAPITest.cc``
-.. _InkAPITest.cc:
https://github.com/apache/trafficserver/blob/master/proxy/api/InkAPITest.cc
+.. _InkAPITest.cc:
https://github.com/apache/trafficserver/blob/master/src/traffic_server/InkAPITest.cc
+
+.. |overridable_txn_vars.cc| replace:: ``overridable_txn_vars.cc``
+
+.. _overridable_txn_vars.cc:
https://github.com/apache/trafficserver/blob/master/src/shared/overridable_txn_vars.cc
.. |ts_lua_http_config.c| replace:: ``ts_lua_http_config.c``
@@ -308,13 +312,10 @@ required for generic access:
#. Add a value to the ``TSOverridableConfigKey`` enumeration in
|apidefs.h.in|_.
-#. Augment the ``TSHttpTxnConfigFind`` function to return this enumeration
value
- when given the name of the configuration variable. Be sure to count the
- characters very carefully.
+#. Augment ``Overridable_Map`` in |overridable_txn_vars.cc|_ to include
configuration variable.
-#. Augment the ``_conf_to_memberp`` function in |InkAPI.cc|_ to return a
pointer
- to the appropriate member of ``OverridableHttpConfigParams`` and set the
type
- if not a byte value.
+#. Update the function ``_conf_to_memberp`` in |InkAPI.cc|_ to have a case for
the enumeration value
+ in ``TSOverridableConfigKey``.
#. Update the testing logic in |InkAPITest.cc|_ by adding the string name of
the
configuration variable to the ``SDK_Overridable_Configs`` array.
diff --git a/include/shared/overridable_txn_vars.h
b/include/shared/overridable_txn_vars.h
new file mode 100644
index 000..31fcb1c
--- /dev/null
+++ b/include/shared/overridable_txn_vars.h
@@ -0,0 +1,33 @@
+/** @file
+
+ Map of transaction overridable configuration variables and names.
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the
[trafficserver] branch 9.0.x updated: make sure time is consistent between calculations
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
The following commit(s) were added to refs/heads/9.0.x by this push:
new 5c74416 make sure time is consistent between calculations
5c74416 is described below
commit 5c7441690514e4f692a2ec8d9eb89fdbae055782
Author: Fei Deng
AuthorDate: Tue Nov 12 15:40:37 2019 -0600
make sure time is consistent between calculations
(cherry picked from commit f5efd7652bc924b4b5515e4116da9d6b5f6eea3a)
---
iocore/eventsystem/UnixEThread.cc | 10 +++---
1 file changed, 3 insertions(+), 7 deletions(-)
diff --git a/iocore/eventsystem/UnixEThread.cc
b/iocore/eventsystem/UnixEThread.cc
index be7a6be..df44c71 100644
--- a/iocore/eventsystem/UnixEThread.cc
+++ b/iocore/eventsystem/UnixEThread.cc
@@ -143,11 +143,7 @@ EThread::process_event(Event *e, int calling_code)
if (e->period < 0) {
e->timeout_at = e->period;
} else {
- this->get_hrtime_updated();
- e->timeout_at = cur_time + e->period;
- if (e->timeout_at < cur_time) {
-e->timeout_at = cur_time;
- }
+ e->timeout_at = Thread::get_hrtime_updated() + e->period;
}
EventQueueExternal.enqueue_local(e);
}
@@ -239,7 +235,7 @@ EThread::execute_regular()
do {
done_one = false;
// execute all the eligible internal events
- EventQueue.check_ready(cur_time, this);
+ EventQueue.check_ready(loop_start_time, this);
while ((e = EventQueue.dequeue_ready(cur_time))) {
ink_assert(e);
ink_assert(e->timeout_at > 0);
@@ -284,7 +280,7 @@ EThread::execute_regular()
tail_cb->waitForActivity(sleep_time);
// loop cleanup
-loop_finish_time = this->get_hrtime_updated();
+loop_finish_time = Thread::get_hrtime_updated();
delta= loop_finish_time - loop_start_time;
// This can happen due to time of day adjustments (which apparently happen
quite frequently). I
[trafficserver] 05/06: Fixes crash loading combined(cert+key) certs
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
commit 5be94b6f7e672abf4b52eb9818a15d598e565f12
Author: Randall Meyer
AuthorDate: Fri Apr 10 09:58:58 2020 -0700
Fixes crash loading combined(cert+key) certs
This crash was introduced by f729c9dc41ff1635132f4bdc6331ce826f3bc2fe
(cherry picked from commit 96e1f4613316bda260debe0578cb626b0443f6a8)
---
iocore/net/SSLUtils.cc | 13 +++--
1 file changed, 11 insertions(+), 2 deletions(-)
diff --git a/iocore/net/SSLUtils.cc b/iocore/net/SSLUtils.cc
index 5d297a3..f204aed 100644
--- a/iocore/net/SSLUtils.cc
+++ b/iocore/net/SSLUtils.cc
@@ -1400,7 +1400,9 @@ SSLMultiCertConfigLoader::_store_ssl_ctx(SSLCertLookup
*lookup, const shared_SSL
std::set common_names;
std::unordered_map> unique_names;
SSLMultiCertConfigLoader::CertLoadData data;
+
const SSLConfigParams *params = this->_params;
+
this->load_certs_and_cross_reference_names(cert_list, data, params,
sslMultCertSettings.get(), common_names, unique_names);
int i = 0;
@@ -1923,8 +1925,15 @@
SSLMultiCertConfigLoader::load_certs_and_cross_reference_names(std::vectorcert ?
(const char *)sslMultCertSettings->cert : "",
SSL_CERT_SEPARATE_DELIM);
- SimpleTokenizer key_tok((sslMultCertSettings && sslMultCertSettings->key ?
(const char *)sslMultCertSettings->key : ""),
- SSL_CERT_SEPARATE_DELIM);
+
+ SimpleTokenizer key_tok(SSL_CERT_SEPARATE_DELIM);
+ if (sslMultCertSettings && sslMultCertSettings->key) {
+key_tok.setString((const char *)sslMultCertSettings->key);
+ } else if (sslMultCertSettings && sslMultCertSettings->cert) {
+key_tok.setString((const char *)sslMultCertSettings->cert);
+ } else {
+key_tok.setString("");
+ }
if (sslMultCertSettings && sslMultCertSettings->key &&
cert_tok.getNumTokensRemaining() != key_tok.getNumTokensRemaining()) {
Error("the number of certificates in ssl_cert_name and ssl_key_name
doesn't match");
[trafficserver] 04/06: Do not fail multicert load if line does not create entry (#6760)
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
commit 31d49587d48e113dc1c378d3510ee76cbd19911e
Author: Susan Hinrichs
AuthorDate: Wed May 13 10:03:47 2020 -0500
Do not fail multicert load if line does not create entry (#6760)
Co-authored-by: Susan Hinrichs
(cherry picked from commit 0265ac7f8a9f0c4773fc6f7d00f6f212bdc08558)
---
iocore/net/QUICMultiCertConfigLoader.cc | 12
iocore/net/SSLUtils.cc | 12
2 files changed, 16 insertions(+), 8 deletions(-)
diff --git a/iocore/net/QUICMultiCertConfigLoader.cc
b/iocore/net/QUICMultiCertConfigLoader.cc
index 288c0f0..cf9c74f 100644
--- a/iocore/net/QUICMultiCertConfigLoader.cc
+++ b/iocore/net/QUICMultiCertConfigLoader.cc
@@ -190,8 +190,13 @@ QUICMultiCertConfigLoader::_store_ssl_ctx(SSLCertLookup
*lookup, const shared_SS
shared_ssl_ticket_key_block keyblock = nullptr;
if (!ctx || !multi_cert_params || !this->_store_single_ssl_ctx(lookup,
multi_cert_params, ctx, common_names)) {
-lookup->is_valid = false;
-retval = false;
+retval = false;
+std::string names;
+for (auto name : data.cert_names_list) {
+ names.append(name);
+ names.append(" ");
+}
+Warning("QUIC: Failed to insert SSL_CTX for certificate %s entries for
names already made", names.c_str());
}
for (auto iter = unique_names.begin(); retval && iter != unique_names.end();
++iter) {
@@ -205,8 +210,7 @@ QUICMultiCertConfigLoader::_store_ssl_ctx(SSLCertLookup
*lookup, const shared_SS
shared_SSL_CTX unique_ctx(this->init_server_ssl_ctx(single_data,
multi_cert_params.get(), iter->second), SSL_CTX_free);
if (!unique_ctx || !this->_store_single_ssl_ctx(lookup, multi_cert_params,
unique_ctx, iter->second)) {
- lookup->is_valid = false;
- retval = false;
+ retval = false;
}
}
diff --git a/iocore/net/SSLUtils.cc b/iocore/net/SSLUtils.cc
index ffc6e8a..5d297a3 100644
--- a/iocore/net/SSLUtils.cc
+++ b/iocore/net/SSLUtils.cc
@@ -1418,8 +1418,13 @@ SSLMultiCertConfigLoader::_store_ssl_ctx(SSLCertLookup
*lookup, const shared_SSL
shared_SSL_CTX ctx(this->init_server_ssl_ctx(data,
sslMultCertSettings.get(), common_names), SSL_CTX_free);
if (!ctx || !sslMultCertSettings || !this->_store_single_ssl_ctx(lookup,
sslMultCertSettings, ctx, common_names)) {
-lookup->is_valid = false;
-retval = false;
+retval = false;
+std::string names;
+for (auto name : data.cert_names_list) {
+ names.append(name);
+ names.append(" ");
+}
+Warning("Failed to insert SSL_CTX for certificate %s entries for names
already made", names.c_str());
}
for (auto iter = unique_names.begin(); retval && iter != unique_names.end();
++iter) {
@@ -1433,8 +1438,7 @@ SSLMultiCertConfigLoader::_store_ssl_ctx(SSLCertLookup
*lookup, const shared_SSL
shared_SSL_CTX unique_ctx(this->init_server_ssl_ctx(single_data,
sslMultCertSettings.get(), iter->second), SSL_CTX_free);
if (!unique_ctx || !this->_store_single_ssl_ctx(lookup,
sslMultCertSettings, unique_ctx, iter->second)) {
- lookup->is_valid = false;
- retval = false;
+ retval = false;
}
}
[trafficserver] 06/06: Add TXN_CLOSE hook to CPPAPI TransactionPlugin (#6800)
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
commit 1264746ba577158217a452a4295a22e2610a34d6
Author: Sudheer Vinukonda
AuthorDate: Wed May 20 09:13:58 2020 -0700
Add TXN_CLOSE hook to CPPAPI TransactionPlugin (#6800)
* Add TXN_CLOSE hook to CPPAPI TransactionPlugin
* Clean up TransactionPlugin object and associated Continuation in txn_close
* Address review comments
* More review comments
(cherry picked from commit 34b57fccb40ef711ce2e6b31042c96efc74c0ecc)
---
include/tscpp/api/Plugin.h| 12 ++-
include/tscpp/api/TransactionPlugin.h | 4
src/tscpp/api/GlobalPlugin.cc | 1 +
src/tscpp/api/utils_internal.cc | 39 ---
4 files changed, 48 insertions(+), 8 deletions(-)
diff --git a/include/tscpp/api/Plugin.h b/include/tscpp/api/Plugin.h
index 2f57352..f37b805 100644
--- a/include/tscpp/api/Plugin.h
+++ b/include/tscpp/api/Plugin.h
@@ -58,7 +58,8 @@ public:
HOOK_READ_REQUEST_HEADERS, /**< This hook will be fired after the request
is read. */
HOOK_READ_CACHE_HEADERS,/**< This hook will be fired after the CACHE
hdrs. */
HOOK_CACHE_LOOKUP_COMPLETE, /**< This hook will be fired after cache
lookup complete. */
-HOOK_SELECT_ALT /**< This hook will be fired after select alt.
*/
+HOOK_TXN_CLOSE, /**< This hook will be fired after send response headers,
only for TransactionPlugins::registerHook()!. */
+HOOK_SELECT_ALT /**< This hook will be fired after select alt. */
};
/**
@@ -143,6 +144,15 @@ public:
};
/**
+ * This method must be implemented when you hook HOOK_TXN_CLOSE
+ */
+ virtual void
+ handleTxnClose(Transaction )
+ {
+transaction.resume();
+ };
+
+ /**
* This method must be implemented when you hook HOOK_SELECT_ALT
*/
virtual void handleSelectAlt(const Request , const Request
, const Response ){};
diff --git a/include/tscpp/api/TransactionPlugin.h
b/include/tscpp/api/TransactionPlugin.h
index b34fba0..ce3f1ca 100644
--- a/include/tscpp/api/TransactionPlugin.h
+++ b/include/tscpp/api/TransactionPlugin.h
@@ -93,6 +93,10 @@ public:
* see HookType and Plugin for the correspond HookTypes and callback
methods. If you fail to implement the
* callback, a default implementation will be used that will only resume
the Transaction.
*
+ * \note For automatic destruction, you must either register dynamically
allocated instances of
+ * classes derived from this class with the the corresponding Transaction
object (using
+ * Transaction::addPlugin()), or register HOOK_TXN_CLOSE (but not both).
+ *
* @param HookType the type of hook you wish to register
* @see HookType
* @see Plugin
diff --git a/src/tscpp/api/GlobalPlugin.cc b/src/tscpp/api/GlobalPlugin.cc
index b1be230..8e5f05c 100644
--- a/src/tscpp/api/GlobalPlugin.cc
+++ b/src/tscpp/api/GlobalPlugin.cc
@@ -87,6 +87,7 @@ GlobalPlugin::~GlobalPlugin()
void
GlobalPlugin::registerHook(Plugin::HookType hook_type)
{
+ assert(hook_type != Plugin::HOOK_TXN_CLOSE);
TSHttpHookID hook_id =
utils::internal::convertInternalHookToTsHook(hook_type);
TSHttpHookAdd(hook_id, state_->cont_);
LOG_DEBUG("Registered global plugin %p for hook %s", this,
HOOK_TYPE_STRINGS[hook_type].c_str());
diff --git a/src/tscpp/api/utils_internal.cc b/src/tscpp/api/utils_internal.cc
index 7cb86e0..61f9044 100644
--- a/src/tscpp/api/utils_internal.cc
+++ b/src/tscpp/api/utils_internal.cc
@@ -49,6 +49,25 @@ resetTransactionHandles(Transaction , TSEvent
event)
return;
}
+void
+cleanupTransaction(Transaction , TSHttpTxn ats_txn_handle)
+{
+ delete
+ // reset the txn arg to prevent use-after-free
+ TSUserArgSet(ats_txn_handle, TRANSACTION_STORAGE_INDEX, nullptr);
+}
+
+void
+cleanupTransactionPlugin(Plugin *plugin)
+{
+ TransactionPlugin *transaction_plugin = static_cast(plugin);
+ std::shared_ptr trans_mutex=
utils::internal::getTransactionPluginMutex(*transaction_plugin);
+ LOG_DEBUG("Locking TransactionPlugin mutex to delete transaction plugin at
%p", transaction_plugin);
+ trans_mutex->lock();
+ delete transaction_plugin;
+ trans_mutex->unlock();
+}
+
int
handleTransactionEvents(TSCont cont, TSEvent event, void *edata)
{
@@ -77,14 +96,9 @@ handleTransactionEvents(TSCont cont, TSEvent event, void
*edata)
resetTransactionHandles(transaction, event);
const std::list =
utils::internal::getTransactionPlugins(transaction);
for (auto plugin : plugins) {
- std::shared_ptr trans_mutex =
utils::internal::getTransactionPluginMutex(*plugin);
- LOG_DEBUG("Locking TransactionPlugin mutex to delete transaction plugin
at %p", plugin);
- trans_mutex->lock();
- LOG_DEBUG("Locked Mutex...Deleting transaction plugin at %p", plugin);
- delete plugin;
-
[trafficserver] 01/06: Rework server side SSL_CTX creation to better handle dual_cert mismatches (#6483)
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
commit c986f64b8c67f38993d208b11666b4bae070cc8d
Author: Susan Hinrichs
AuthorDate: Thu Mar 12 08:41:38 2020 -0500
Rework server side SSL_CTX creation to better handle dual_cert mismatches
(#6483)
Rework server side SSL_CTX creation to better handle dual_cert name
mismatches
(cherry picked from commit f729c9dc41ff1635132f4bdc6331ce826f3bc2fe)
---
iocore/net/P_SSLCertLookup.h | 2 +-
iocore/net/P_SSLUtils.h| 26 +-
iocore/net/QUICMultiCertConfigLoader.cc| 112 ++-
iocore/net/QUICMultiCertConfigLoader.h | 6 +-
iocore/net/SSLUtils.cc | 348 ++---
tests/gold_tests/tls/ssl/signed-foo-ec.key | 8 +
tests/gold_tests/tls/ssl/signed-foo-ec.pem | 14 +
tests/gold_tests/tls/ssl/signed-san-ec.key | 5 +
tests/gold_tests/tls/ssl/signed-san-ec.pem | 15 +
tests/gold_tests/tls/ssl/signed-san.key| 28 ++
tests/gold_tests/tls/ssl/signed-san.pem| 19 ++
tests/gold_tests/tls/ssl/signer.pem| 15 -
.../tls/tls_check_dual_cert_selection.test.py | 127
13 files changed, 505 insertions(+), 220 deletions(-)
diff --git a/iocore/net/P_SSLCertLookup.h b/iocore/net/P_SSLCertLookup.h
index 1e89361..7ee0f2a 100644
--- a/iocore/net/P_SSLCertLookup.h
+++ b/iocore/net/P_SSLCertLookup.h
@@ -100,7 +100,7 @@ public:
{
}
SSLCertContext(shared_SSL_CTX sc, shared_SSLMultiCertConfigParams u)
-: ctx_mutex(), ctx(sc), opt(u->opt), userconfig(nullptr), keyblock(nullptr)
+: ctx_mutex(), ctx(sc), opt(u->opt), userconfig(u), keyblock(nullptr)
{
}
SSLCertContext(shared_SSL_CTX sc, shared_SSLMultiCertConfigParams u,
shared_ssl_ticket_key_block kb)
diff --git a/iocore/net/P_SSLUtils.h b/iocore/net/P_SSLUtils.h
index da85e1a..ac43cf2 100644
--- a/iocore/net/P_SSLUtils.h
+++ b/iocore/net/P_SSLUtils.h
@@ -34,6 +34,9 @@
#include "records/I_RecCore.h"
#include "P_SSLCertLookup.h"
+#include
+#include
+
struct SSLConfigParams;
class SSLNetVConnection;
@@ -54,28 +57,39 @@ ssl_curve_id SSLGetCurveNID(SSL *ssl);
class SSLMultiCertConfigLoader
{
public:
+ struct CertLoadData {
+std::vector cert_names_list, key_list, ca_list, ocsp_list;
+ };
SSLMultiCertConfigLoader(const SSLConfigParams *p) : _params(p) {}
virtual ~SSLMultiCertConfigLoader(){};
bool load(SSLCertLookup *lookup);
virtual SSL_CTX *default_server_ssl_ctx();
- virtual SSL_CTX *init_server_ssl_ctx(std::vector , const
SSLMultiCertConfigParams *sslMultCertSettings);
-
- static bool load_certs(SSL_CTX *ctx, std::vector , const
SSLConfigParams *params,
- const SSLMultiCertConfigParams
*ssl_multi_cert_params);
+ virtual SSL_CTX *init_server_ssl_ctx(CertLoadData const , const
SSLMultiCertConfigParams *sslMultCertSettings,
+ std::set );
+
+ static bool load_certs(SSL_CTX *ctx, CertLoadData const , const
SSLConfigParams *params,
+ const SSLMultiCertConfigParams *sslMultCertSettings);
+ bool load_certs_and_cross_reference_names(std::vector _list,
CertLoadData , const SSLConfigParams *params,
+const SSLMultiCertConfigParams
*sslMultCertSettings,
+std::set
_names,
+std::unordered_map> _names);
static bool set_session_id_context(SSL_CTX *ctx, const SSLConfigParams
*params,
const SSLMultiCertConfigParams
*sslMultCertSettings);
- static bool index_certificate(SSLCertLookup *lookup, SSLCertContext const
, X509 *cert, const char *certname);
+ static bool index_certificate(SSLCertLookup *lookup, SSLCertContext const
, const char *sni_name);
static int check_server_cert_now(X509 *cert, const char *certname);
static void clear_pw_references(SSL_CTX *ssl_ctx);
protected:
const SSLConfigParams *_params;
+ bool _store_single_ssl_ctx(SSLCertLookup *lookup,
shared_SSLMultiCertConfigParams sslMultCertSettings, shared_SSL_CTX ctx,
+ std::set );
+
private:
- virtual SSL_CTX *_store_ssl_ctx(SSLCertLookup *lookup, const
shared_SSLMultiCertConfigParams ssl_multi_cert_params);
+ virtual bool _store_ssl_ctx(SSLCertLookup *lookup,
shared_SSLMultiCertConfigParams ssl_multi_cert_params);
virtual void _set_handshake_callbacks(SSL_CTX *ctx);
};
diff --git a/iocore/net/QUICMultiCertConfigLoader.cc
b/iocore/net/QUICMultiCertConfigLoader.cc
index 2b1aecf..288c0f0 100644
--- a/iocore/net/QUICMultiCertConfigLoader.cc
+++ b/iocore/net/QUICMultiCertConfigLoader.cc
@@ -80,7 +80,8 @@ QUICMultiCertConfigLoader::default_server_ssl_ctx()
}
[trafficserver] 03/06: Fix SDK_API_TSSslServerContextCreate
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
commit d18538b232131e83066b46c830ce707ad78ce6c8
Author: Susan Hinrichs
AuthorDate: Fri Mar 13 18:45:32 2020 +
Fix SDK_API_TSSslServerContextCreate
(cherry picked from commit 3808b31d3c679d4490d144383ae6df70a75fd150)
---
iocore/net/SSLUtils.cc | 12 +++-
1 file changed, 7 insertions(+), 5 deletions(-)
diff --git a/iocore/net/SSLUtils.cc b/iocore/net/SSLUtils.cc
index 6b3774d..ffc6e8a 100644
--- a/iocore/net/SSLUtils.cc
+++ b/iocore/net/SSLUtils.cc
@@ -1917,16 +1917,18 @@
SSLMultiCertConfigLoader::load_certs_and_cross_reference_names(std::vector _names,
std::unordered_map> _names)
{
- SimpleTokenizer cert_tok(sslMultCertSettings->cert ? (const char
*)sslMultCertSettings->cert : "", SSL_CERT_SEPARATE_DELIM);
- SimpleTokenizer key_tok((sslMultCertSettings->key ? (const char
*)sslMultCertSettings->key : ""), SSL_CERT_SEPARATE_DELIM);
+ SimpleTokenizer cert_tok(sslMultCertSettings && sslMultCertSettings->cert ?
(const char *)sslMultCertSettings->cert : "",
+ SSL_CERT_SEPARATE_DELIM);
+ SimpleTokenizer key_tok((sslMultCertSettings && sslMultCertSettings->key ?
(const char *)sslMultCertSettings->key : ""),
+ SSL_CERT_SEPARATE_DELIM);
- if (sslMultCertSettings->key && cert_tok.getNumTokensRemaining() !=
key_tok.getNumTokensRemaining()) {
+ if (sslMultCertSettings && sslMultCertSettings->key &&
cert_tok.getNumTokensRemaining() != key_tok.getNumTokensRemaining()) {
Error("the number of certificates in ssl_cert_name and ssl_key_name
doesn't match");
return false;
}
SimpleTokenizer ca_tok("", SSL_CERT_SEPARATE_DELIM);
- if (sslMultCertSettings->ca) {
+ if (sslMultCertSettings && sslMultCertSettings->ca) {
ca_tok.setString(sslMultCertSettings->ca);
if (cert_tok.getNumTokensRemaining() != ca_tok.getNumTokensRemaining()) {
Error("the number of certificates in ssl_cert_name and ssl_ca_name
doesn't match");
@@ -1935,7 +1937,7 @@
SSLMultiCertConfigLoader::load_certs_and_cross_reference_names(std::vectorocsp_response) {
+ if (sslMultCertSettings && sslMultCertSettings->ocsp_response) {
ocsp_tok.setString(sslMultCertSettings->ocsp_response);
if (cert_tok.getNumTokensRemaining() != ocsp_tok.getNumTokensRemaining()) {
Error("the number of certificates in ssl_cert_name and ssl_ocsp_name
doesn't match");
[trafficserver] branch 9.0.x updated (f0ba454 -> 1264746)
This is an automated email from the ASF dual-hosted git repository. zwoop pushed a change to branch 9.0.x in repository https://gitbox.apache.org/repos/asf/trafficserver.git. from f0ba454 Remove unused index for SSL application specific data new c986f64 Rework server side SSL_CTX creation to better handle dual_cert mismatches (#6483) new 7c08b05 Fixes memory leak loading certs new d18538b Fix SDK_API_TSSslServerContextCreate new 31d4958 Do not fail multicert load if line does not create entry (#6760) new 5be94b6 Fixes crash loading combined(cert+key) certs new 1264746 Add TXN_CLOSE hook to CPPAPI TransactionPlugin (#6800) The 6 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: include/tscpp/api/Plugin.h | 12 +- include/tscpp/api/TransactionPlugin.h | 4 + iocore/net/P_SSLCertLookup.h | 2 +- iocore/net/P_SSLUtils.h| 26 +- iocore/net/QUICMultiCertConfigLoader.cc| 114 ++- iocore/net/QUICMultiCertConfigLoader.h | 6 +- iocore/net/SSLUtils.cc | 371 ++--- src/tscpp/api/GlobalPlugin.cc | 1 + src/tscpp/api/utils_internal.cc| 39 ++- tests/gold_tests/tls/ssl/signed-foo-ec.key | 8 + tests/gold_tests/tls/ssl/signed-foo-ec.pem | 14 + tests/gold_tests/tls/ssl/signed-san-ec.key | 5 + tests/gold_tests/tls/ssl/signed-san-ec.pem | 15 + tests/gold_tests/tls/ssl/signed-san.key| 28 ++ tests/gold_tests/tls/ssl/signed-san.pem| 19 ++ tests/gold_tests/tls/ssl/signer.pem| 15 - .../tls/tls_check_dual_cert_selection.test.py | 127 +++ 17 files changed, 576 insertions(+), 230 deletions(-) create mode 100644 tests/gold_tests/tls/ssl/signed-foo-ec.key create mode 100644 tests/gold_tests/tls/ssl/signed-foo-ec.pem create mode 100644 tests/gold_tests/tls/ssl/signed-san-ec.key create mode 100644 tests/gold_tests/tls/ssl/signed-san-ec.pem create mode 100644 tests/gold_tests/tls/ssl/signed-san.key create mode 100644 tests/gold_tests/tls/ssl/signed-san.pem create mode 100644 tests/gold_tests/tls/tls_check_dual_cert_selection.test.py
[trafficserver] 02/06: Fixes memory leak loading certs
This is an automated email from the ASF dual-hosted git repository. zwoop pushed a commit to branch 9.0.x in repository https://gitbox.apache.org/repos/asf/trafficserver.git commit 7c08b051cd8a70a0e271c29b8f4d55dffa78529e Author: Randall Meyer AuthorDate: Wed Apr 15 08:44:21 2020 -0700 Fixes memory leak loading certs This leak was introduced by f729c9dc41ff1635132f4bdc6331ce826f3bc2fe (cherry picked from commit 9fb6f961baf31383b106d734f524e89406e2fa1e) --- iocore/net/SSLUtils.cc | 2 ++ 1 file changed, 2 insertions(+) diff --git a/iocore/net/SSLUtils.cc b/iocore/net/SSLUtils.cc index 2c32232..6b3774d 100644 --- a/iocore/net/SSLUtils.cc +++ b/iocore/net/SSLUtils.cc @@ -2012,7 +2012,9 @@ SSLMultiCertConfigLoader::load_certs_and_cross_reference_names(std::vector
[trafficserver] branch master updated: Add TXN_CLOSE hook to CPPAPI TransactionPlugin (#6800)
This is an automated email from the ASF dual-hosted git repository.
sudheerv pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
The following commit(s) were added to refs/heads/master by this push:
new 34b57fc Add TXN_CLOSE hook to CPPAPI TransactionPlugin (#6800)
34b57fc is described below
commit 34b57fccb40ef711ce2e6b31042c96efc74c0ecc
Author: Sudheer Vinukonda
AuthorDate: Wed May 20 09:13:58 2020 -0700
Add TXN_CLOSE hook to CPPAPI TransactionPlugin (#6800)
* Add TXN_CLOSE hook to CPPAPI TransactionPlugin
* Clean up TransactionPlugin object and associated Continuation in txn_close
* Address review comments
* More review comments
---
include/tscpp/api/Plugin.h| 12 ++-
include/tscpp/api/TransactionPlugin.h | 4
src/tscpp/api/GlobalPlugin.cc | 1 +
src/tscpp/api/utils_internal.cc | 39 ---
4 files changed, 48 insertions(+), 8 deletions(-)
diff --git a/include/tscpp/api/Plugin.h b/include/tscpp/api/Plugin.h
index 2f57352..f37b805 100644
--- a/include/tscpp/api/Plugin.h
+++ b/include/tscpp/api/Plugin.h
@@ -58,7 +58,8 @@ public:
HOOK_READ_REQUEST_HEADERS, /**< This hook will be fired after the request
is read. */
HOOK_READ_CACHE_HEADERS,/**< This hook will be fired after the CACHE
hdrs. */
HOOK_CACHE_LOOKUP_COMPLETE, /**< This hook will be fired after cache
lookup complete. */
-HOOK_SELECT_ALT /**< This hook will be fired after select alt.
*/
+HOOK_TXN_CLOSE, /**< This hook will be fired after send response headers,
only for TransactionPlugins::registerHook()!. */
+HOOK_SELECT_ALT /**< This hook will be fired after select alt. */
};
/**
@@ -143,6 +144,15 @@ public:
};
/**
+ * This method must be implemented when you hook HOOK_TXN_CLOSE
+ */
+ virtual void
+ handleTxnClose(Transaction )
+ {
+transaction.resume();
+ };
+
+ /**
* This method must be implemented when you hook HOOK_SELECT_ALT
*/
virtual void handleSelectAlt(const Request , const Request
, const Response ){};
diff --git a/include/tscpp/api/TransactionPlugin.h
b/include/tscpp/api/TransactionPlugin.h
index b34fba0..ce3f1ca 100644
--- a/include/tscpp/api/TransactionPlugin.h
+++ b/include/tscpp/api/TransactionPlugin.h
@@ -93,6 +93,10 @@ public:
* see HookType and Plugin for the correspond HookTypes and callback
methods. If you fail to implement the
* callback, a default implementation will be used that will only resume
the Transaction.
*
+ * \note For automatic destruction, you must either register dynamically
allocated instances of
+ * classes derived from this class with the the corresponding Transaction
object (using
+ * Transaction::addPlugin()), or register HOOK_TXN_CLOSE (but not both).
+ *
* @param HookType the type of hook you wish to register
* @see HookType
* @see Plugin
diff --git a/src/tscpp/api/GlobalPlugin.cc b/src/tscpp/api/GlobalPlugin.cc
index b1be230..8e5f05c 100644
--- a/src/tscpp/api/GlobalPlugin.cc
+++ b/src/tscpp/api/GlobalPlugin.cc
@@ -87,6 +87,7 @@ GlobalPlugin::~GlobalPlugin()
void
GlobalPlugin::registerHook(Plugin::HookType hook_type)
{
+ assert(hook_type != Plugin::HOOK_TXN_CLOSE);
TSHttpHookID hook_id =
utils::internal::convertInternalHookToTsHook(hook_type);
TSHttpHookAdd(hook_id, state_->cont_);
LOG_DEBUG("Registered global plugin %p for hook %s", this,
HOOK_TYPE_STRINGS[hook_type].c_str());
diff --git a/src/tscpp/api/utils_internal.cc b/src/tscpp/api/utils_internal.cc
index 7cb86e0..61f9044 100644
--- a/src/tscpp/api/utils_internal.cc
+++ b/src/tscpp/api/utils_internal.cc
@@ -49,6 +49,25 @@ resetTransactionHandles(Transaction , TSEvent
event)
return;
}
+void
+cleanupTransaction(Transaction , TSHttpTxn ats_txn_handle)
+{
+ delete
+ // reset the txn arg to prevent use-after-free
+ TSUserArgSet(ats_txn_handle, TRANSACTION_STORAGE_INDEX, nullptr);
+}
+
+void
+cleanupTransactionPlugin(Plugin *plugin)
+{
+ TransactionPlugin *transaction_plugin = static_cast(plugin);
+ std::shared_ptr trans_mutex=
utils::internal::getTransactionPluginMutex(*transaction_plugin);
+ LOG_DEBUG("Locking TransactionPlugin mutex to delete transaction plugin at
%p", transaction_plugin);
+ trans_mutex->lock();
+ delete transaction_plugin;
+ trans_mutex->unlock();
+}
+
int
handleTransactionEvents(TSCont cont, TSEvent event, void *edata)
{
@@ -77,14 +96,9 @@ handleTransactionEvents(TSCont cont, TSEvent event, void
*edata)
resetTransactionHandles(transaction, event);
const std::list =
utils::internal::getTransactionPlugins(transaction);
for (auto plugin : plugins) {
- std::shared_ptr trans_mutex =
utils::internal::getTransactionPluginMutex(*plugin);
- LOG_DEBUG("Locking TransactionPlugin mutex to delete transaction plugin
at %p", plugin);
- trans_mutex->lock();
-
[trafficserver] branch 9.0.x updated: Remove unused index for SSL application specific data
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
The following commit(s) were added to refs/heads/9.0.x by this push:
new f0ba454 Remove unused index for SSL application specific data
f0ba454 is described below
commit f0ba454f10d85dd8e5341cf95aa82224b88073f7
Author: Masakazu Kitajo
AuthorDate: Tue May 5 12:03:49 2020 +0900
Remove unused index for SSL application specific data
ssl_session_ticket_index is registerd but unused.
(cherry picked from commit e5d51e478ee94fa77b5ebf91f71effdc25839a8d)
---
iocore/net/SSLUtils.cc | 11 ---
1 file changed, 11 deletions(-)
diff --git a/iocore/net/SSLUtils.cc b/iocore/net/SSLUtils.cc
index 95dc636..f9f8ec4 100644
--- a/iocore/net/SSLUtils.cc
+++ b/iocore/net/SSLUtils.cc
@@ -91,10 +91,6 @@ static constexpr char SSL_CERT_SEPARATE_DELIM = ',';
SSLSessionCache *session_cache; // declared extern in P_SSLConfig.h
-#if TS_HAVE_OPENSSL_SESSION_TICKETS
-static int ssl_session_ticket_index = -1;
-#endif
-
static int ssl_vc_index = -1;
static ink_mutex *mutex_buf = nullptr;
@@ -911,13 +907,6 @@ SSLInitializeLibrary()
CRYPTO_set_dynlock_destroy_callback(ssl_dyn_destroy_callback);
}
-#ifdef TS_HAVE_OPENSSL_SESSION_TICKETS
- ssl_session_ticket_index = SSL_CTX_get_ex_new_index(0, nullptr, nullptr,
nullptr, ssl_session_ticket_free);
- if (ssl_session_ticket_index == -1) {
-SSLError("failed to create session ticket index");
- }
-#endif
-
#if TS_USE_TLS_OCSP
ssl_stapling_ex_init();
#endif /* TS_USE_TLS_OCSP */
[trafficserver] branch 9.0.x updated: traffic_dump: refactor to make transactions atomically written
This is an automated email from the ASF dual-hosted git repository. zwoop pushed a commit to branch 9.0.x in repository https://gitbox.apache.org/repos/asf/trafficserver.git The following commit(s) were added to refs/heads/9.0.x by this push: new 6743ecd traffic_dump: refactor to make transactions atomically written 6743ecd is described below commit 6743ecd37c876e97c83f93a47342ef6a4beae962 Author: bneradt AuthorDate: Fri May 1 18:55:18 2020 + traffic_dump: refactor to make transactions atomically written This refactor is occasioned by the need to make transaction writes atomic. Mainly, this encapsulates session and transaction handling in SessionData and TransactionData classes. (cherry picked from commit 59ebfd82a67bfdbe323c21cf304d08345540e461) --- .gitignore | 1 + plugins/experimental/traffic_dump/Makefile.inc | 28 +- .../experimental/traffic_dump/global_variables.h | 25 + plugins/experimental/traffic_dump/json_utils.cc| 178 plugins/experimental/traffic_dump/json_utils.h | 58 ++ .../experimental/traffic_dump/sensitive_fields.h | 54 ++ plugins/experimental/traffic_dump/session_data.cc | 486 +++ plugins/experimental/traffic_dump/session_data.h | 182 plugins/experimental/traffic_dump/traffic_dump.cc | 929 ++--- .../experimental/traffic_dump/transaction_data.cc | 358 .../experimental/traffic_dump/transaction_data.h | 110 +++ .../traffic_dump/unit_tests/test_json_utils.cc | 59 ++ .../unit_tests/test_sensitive_fields.cc| 37 + .../traffic_dump/unit_tests/unit_test_main.cc | 25 + .../pluginTest/traffic_dump/gold/200.gold | 2 +- .../traffic_dump/gold/4_byte_response_body.gold| 7 + .../traffic_dump/gold/two_transactions.gold| 11 + .../pluginTest/traffic_dump/traffic_dump.test.py | 125 ++- .../pluginTest/traffic_dump/verify_replay.py | 6 +- tests/tools/lib/replay_schema.json | 2 +- 20 files changed, 1796 insertions(+), 887 deletions(-) diff --git a/.gitignore b/.gitignore index 305b05d..20c2bfa 100644 --- a/.gitignore +++ b/.gitignore @@ -130,6 +130,7 @@ plugins/experimental/esi/*_test plugins/experimental/slice/test_* plugins/experimental/sslheaders/test_sslheaders plugins/s3_auth/test_s3auth +plugins/experimental/traffic_dump/test_* plugins/esi/docnode_test plugins/esi/gzip_test diff --git a/plugins/experimental/traffic_dump/Makefile.inc b/plugins/experimental/traffic_dump/Makefile.inc index ac87596..411a9f7 100644 --- a/plugins/experimental/traffic_dump/Makefile.inc +++ b/plugins/experimental/traffic_dump/Makefile.inc @@ -16,4 +16,30 @@ pkglib_LTLIBRARIES += experimental/traffic_dump/traffic_dump.la -experimental_traffic_dump_traffic_dump_la_SOURCES = experimental/traffic_dump/traffic_dump.cc +experimental_traffic_dump_traffic_dump_la_SOURCES = \ +experimental/traffic_dump/global_variables.h \ +experimental/traffic_dump/json_utils.cc \ +experimental/traffic_dump/json_utils.h \ +experimental/traffic_dump/sensitive_fields.h \ +experimental/traffic_dump/session_data.cc \ +experimental/traffic_dump/session_data.h \ +experimental/traffic_dump/traffic_dump.cc \ +experimental/traffic_dump/transaction_data.cc \ +experimental/traffic_dump/transaction_data.h + +check_PROGRAMS += \ +experimental/traffic_dump/test_traffic_dump + +experimental_traffic_dump_test_traffic_dump_CPPFLAGS = \ +$(AM_CPPFLAGS) \ +-I$(abs_top_srcdir)/plugins/experimental/traffic_dump \ +-I$(abs_top_srcdir)/tests/include + +experimental_traffic_dump_test_traffic_dump_SOURCES = \ + experimental/traffic_dump/unit_tests/unit_test_main.cc \ +experimental/traffic_dump/unit_tests/test_json_utils.cc \ +experimental/traffic_dump/unit_tests/test_sensitive_fields.cc \ +experimental/traffic_dump/json_utils.cc \ +experimental/traffic_dump/sensitive_fields.h + +# vim: ft=make ts=8 sw=8 et: diff --git a/plugins/experimental/traffic_dump/global_variables.h b/plugins/experimental/traffic_dump/global_variables.h new file mode 100644 index 000..204ce20 --- /dev/null +++ b/plugins/experimental/traffic_dump/global_variables.h @@ -0,0 +1,25 @@ +/** @sensitive_fields.h + The set of fields considered user-sensitive. + @section license License + Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + http://www.apache.org/licenses/LICENSE-2.0 + Unless required by applicable law or agreed to in
[trafficserver] branch 9.0.x updated: traffic_dump: add tls information to dump. (#6727)
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
The following commit(s) were added to refs/heads/9.0.x by this push:
new 4f3ac6b traffic_dump: add tls information to dump. (#6727)
4f3ac6b is described below
commit 4f3ac6b4504d9b6fcbef35d931145e9dba153e79
Author: Brian Neradt
AuthorDate: Fri May 1 09:27:40 2020 -0500
traffic_dump: add tls information to dump. (#6727)
This change adds tls information nodes like the following:
"tls": {
"sni": "",
"verify_mode": ""
},
Co-authored-by: bneradt
(cherry picked from commit bfafd9187d4c766bfd30fa4146109777d1f2)
---
plugins/experimental/traffic_dump/traffic_dump.cc | 163 ++---
.../pluginTest/traffic_dump/traffic_dump.test.py | 8 +-
.../traffic_dump/traffic_dump_sni_filter.test.py | 10 +-
.../pluginTest/traffic_dump/verify_replay.py | 44 ++
4 files changed, 202 insertions(+), 23 deletions(-)
diff --git a/plugins/experimental/traffic_dump/traffic_dump.cc
b/plugins/experimental/traffic_dump/traffic_dump.cc
index 938d640..f37bb7b 100644
--- a/plugins/experimental/traffic_dump/traffic_dump.cc
+++ b/plugins/experimental/traffic_dump/traffic_dump.cc
@@ -75,9 +75,9 @@ public:
};
/// Fields considered sensitive because they may contain user-private
-/// information. These fields are replaced with auto-generated generic content
-/// by default. To turn off this behavior, the user should add the
-/// --promiscuous-mode flag as a commandline argument.
+/// information. These fields are replaced with auto-generated generic content
by
+/// default. To override this behavior, the user should specify their own
fields
+/// they consider sensitive with --sensitive-fields.
///
/// While these are specified with case, they are matched case-insensitively.
std::unordered_set
default_sensitive_fields = {
@@ -568,6 +568,143 @@ session_txn_handler(TSCont contp, TSEvent event, void
*edata)
return TS_SUCCESS;
}
+/** Create a TLS characteristics node.
+ *
+ * This function encapsulates the logic common between the client-side and
+ * server-side logic for populating the TLS node.
+ *
+ * @param[in] ssnp The pointer for this session.
+ *
+ * @return The node describing the TLS properties of this session.
+ */
+std::string
+get_tls_description_helper(TSVConn ssn_vc)
+{
+ TSSslConnection ssl_conn = TSVConnSslConnectionGet(ssn_vc);
+ SSL *ssl_obj = (SSL *)ssl_conn;
+ if (ssl_obj == nullptr) {
+return "";
+ }
+ std::ostringstream tls_description;
+ tls_description << R"("tls":{)";
+ const char *sni_ptr = SSL_get_servername(ssl_obj, TLSEXT_NAMETYPE_host_name);
+ if (sni_ptr != nullptr) {
+std::string_view sni{sni_ptr};
+if (!sni.empty()) {
+ tls_description << R"("sni":")" << sni << R"(")";
+}
+ }
+ tls_description << R"(,"verify_mode":")" <<
std::to_string(SSL_get_verify_mode(ssl_obj)) << R"(")";
+ tls_description << "}";
+ return tls_description.str();
+}
+
+/** Create a server-side TLS characteristics node.
+ *
+ * @param[in] ssnp The pointer for this session.
+ *
+ * @return The node describing the TLS properties of this session.
+ */
+std::string
+get_server_tls_description(TSHttpSsn ssnp)
+{
+ TSVConn ssn_vc = TSHttpSsnServerVConnGet(ssnp);
+ return get_tls_description_helper(ssn_vc);
+}
+
+/** Create a client-side TLS characteristics node.
+ *
+ * @param[in] ssnp The pointer for this session.
+ *
+ * @return The node describing the TLS properties of this session.
+ */
+std::string
+get_client_tls_description(TSHttpSsn ssnp)
+{
+ TSVConn ssn_vc = TSHttpSsnClientVConnGet(ssnp);
+ return get_tls_description_helper(ssn_vc);
+}
+
+/// A named boolean for callers who pass the is_client parameter.
+constexpr bool IS_CLIENT = true;
+
+/** Create the nodes that describe the session's sub-HTTP protocols.
+ *
+ * This function encapsulates the logic common between the client-side and
+ * server-side logic for describing the session's characteristics.
+ *
+ * This will create the string representing the "protocol" and "tls" nodes. The
+ * "tls" node will only be present if the connection is over SSL/TLS.
+ *
+ * @param[in] ssnp The pointer for this session.
+ *
+ * @return The description of the protocol stack and certain TLS attributes.
+ */
+std::string
+get_protocol_description_helper(TSHttpSsn ssnp, bool is_client)
+{
+ std::ostringstream protocol_description;
+ protocol_description << R"("protocol":[)";
+
+ const char *protocol[10];
+ int count = -1;
+ if (is_client) {
+TSAssert(TS_SUCCESS == TSHttpSsnClientProtocolStackGet(ssnp, 10, protocol,
));
+ } else {
+// See the TODO below in the commented out defintion of
get_server_protocol_description.
+// TSAssert(TS_SUCCESS == TSHttpSsnServerProtocolStackGet(ssnp, 10,
protocol, ));
+ }
+ for (int i = 0; i < count; i++) {
[trafficserver] branch 8.1.x updated: Fix HPACK Dynamic Table Cleanup
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 8.1.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
The following commit(s) were added to refs/heads/8.1.x by this push:
new df3a981 Fix HPACK Dynamic Table Cleanup
df3a981 is described below
commit df3a981a662669004569bfb7a267c24f064ed8c7
Author: Masaori Koshiba
AuthorDate: Mon May 11 08:56:54 2020 +0900
Fix HPACK Dynamic Table Cleanup
(cherry picked from commit 3376d438b4a6410187e1ddedd87d2e89279ec196)
Conflicts:
proxy/http2/unit_tests/test_HpackIndexingTable.cc
---
proxy/http2/HPACK.cc | 22 +++---
proxy/http2/HPACK.h | 6 +++---
2 files changed, 10 insertions(+), 18 deletions(-)
diff --git a/proxy/http2/HPACK.cc b/proxy/http2/HPACK.cc
index eefdef2..97a2626 100644
--- a/proxy/http2/HPACK.cc
+++ b/proxy/http2/HPACK.cc
@@ -327,10 +327,10 @@ HpackIndexingTable::size() const
return _dynamic_table->size();
}
-bool
+void
HpackIndexingTable::update_maximum_size(uint32_t new_size)
{
- return _dynamic_table->update_maximum_size(new_size);
+ _dynamic_table->update_maximum_size(new_size);
}
//
@@ -403,11 +403,11 @@ HpackDynamicTable::size() const
// are evicted from the end of the header table until the size of the
// header table is less than or equal to the maximum size.
//
-bool
+void
HpackDynamicTable::update_maximum_size(uint32_t new_size)
{
this->_maximum_size = new_size;
- return this->_evict_overflowed_entries();
+ this->_evict_overflowed_entries();
}
uint32_t
@@ -416,12 +416,12 @@ HpackDynamicTable::length() const
return this->_headers.size();
}
-bool
+void
HpackDynamicTable::_evict_overflowed_entries()
{
if (this->_current_size <= this->_maximum_size) {
// Do nothing
-return true;
+return;
}
for (auto h = this->_headers.rbegin(); h != this->_headers.rend(); ++h) {
@@ -438,13 +438,7 @@ HpackDynamicTable::_evict_overflowed_entries()
}
}
- if (this->_headers.size() == 0) {
-return false;
- }
-
this->_mime_hdr_gc();
-
- return true;
}
/**
@@ -774,9 +768,7 @@ update_dynamic_table_size(const uint8_t *buf_start, const
uint8_t *buf_end, Hpac
return HPACK_ERROR_COMPRESSION_ERROR;
}
- if (indexing_table.update_maximum_size(size) == false) {
-return HPACK_ERROR_COMPRESSION_ERROR;
- }
+ indexing_table.update_maximum_size(size);
return len;
}
diff --git a/proxy/http2/HPACK.h b/proxy/http2/HPACK.h
index 34d1f6d..e38ccb7 100644
--- a/proxy/http2/HPACK.h
+++ b/proxy/http2/HPACK.h
@@ -119,12 +119,12 @@ public:
uint32_t maximum_size() const;
uint32_t size() const;
- bool update_maximum_size(uint32_t new_size);
+ void update_maximum_size(uint32_t new_size);
uint32_t length() const;
private:
- bool _evict_overflowed_entries();
+ void _evict_overflowed_entries();
void _mime_hdr_gc();
uint32_t _current_size = 0;
@@ -148,7 +148,7 @@ public:
void add_header_field(const MIMEField *field);
uint32_t maximum_size() const;
uint32_t size() const;
- bool update_maximum_size(uint32_t new_size);
+ void update_maximum_size(uint32_t new_size);
private:
HpackDynamicTable *_dynamic_table;
[trafficserver] branch 9.0.x updated: Fix HPACK Dynamic Table Cleanup
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
The following commit(s) were added to refs/heads/9.0.x by this push:
new 0ee6834 Fix HPACK Dynamic Table Cleanup
0ee6834 is described below
commit 0ee6834cede8a7671e9682e41472828e22effdac
Author: Masaori Koshiba
AuthorDate: Mon May 11 08:56:54 2020 +0900
Fix HPACK Dynamic Table Cleanup
(cherry picked from commit 3376d438b4a6410187e1ddedd87d2e89279ec196)
Conflicts:
proxy/http2/unit_tests/test_HpackIndexingTable.cc
---
proxy/http2/HPACK.cc | 22 +++---
proxy/http2/HPACK.h | 6 +++---
2 files changed, 10 insertions(+), 18 deletions(-)
diff --git a/proxy/http2/HPACK.cc b/proxy/http2/HPACK.cc
index eefdef2..97a2626 100644
--- a/proxy/http2/HPACK.cc
+++ b/proxy/http2/HPACK.cc
@@ -327,10 +327,10 @@ HpackIndexingTable::size() const
return _dynamic_table->size();
}
-bool
+void
HpackIndexingTable::update_maximum_size(uint32_t new_size)
{
- return _dynamic_table->update_maximum_size(new_size);
+ _dynamic_table->update_maximum_size(new_size);
}
//
@@ -403,11 +403,11 @@ HpackDynamicTable::size() const
// are evicted from the end of the header table until the size of the
// header table is less than or equal to the maximum size.
//
-bool
+void
HpackDynamicTable::update_maximum_size(uint32_t new_size)
{
this->_maximum_size = new_size;
- return this->_evict_overflowed_entries();
+ this->_evict_overflowed_entries();
}
uint32_t
@@ -416,12 +416,12 @@ HpackDynamicTable::length() const
return this->_headers.size();
}
-bool
+void
HpackDynamicTable::_evict_overflowed_entries()
{
if (this->_current_size <= this->_maximum_size) {
// Do nothing
-return true;
+return;
}
for (auto h = this->_headers.rbegin(); h != this->_headers.rend(); ++h) {
@@ -438,13 +438,7 @@ HpackDynamicTable::_evict_overflowed_entries()
}
}
- if (this->_headers.size() == 0) {
-return false;
- }
-
this->_mime_hdr_gc();
-
- return true;
}
/**
@@ -774,9 +768,7 @@ update_dynamic_table_size(const uint8_t *buf_start, const
uint8_t *buf_end, Hpac
return HPACK_ERROR_COMPRESSION_ERROR;
}
- if (indexing_table.update_maximum_size(size) == false) {
-return HPACK_ERROR_COMPRESSION_ERROR;
- }
+ indexing_table.update_maximum_size(size);
return len;
}
diff --git a/proxy/http2/HPACK.h b/proxy/http2/HPACK.h
index dc0b517..70f95ed 100644
--- a/proxy/http2/HPACK.h
+++ b/proxy/http2/HPACK.h
@@ -123,12 +123,12 @@ public:
uint32_t maximum_size() const;
uint32_t size() const;
- bool update_maximum_size(uint32_t new_size);
+ void update_maximum_size(uint32_t new_size);
uint32_t length() const;
private:
- bool _evict_overflowed_entries();
+ void _evict_overflowed_entries();
void _mime_hdr_gc();
uint32_t _current_size = 0;
@@ -157,7 +157,7 @@ public:
void add_header_field(const MIMEField *field);
uint32_t maximum_size() const;
uint32_t size() const;
- bool update_maximum_size(uint32_t new_size);
+ void update_maximum_size(uint32_t new_size);
private:
HpackDynamicTable *_dynamic_table;
[trafficserver] branch 9.0.x updated: clang-analyzer: uninitialized va_list (#6798)
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
The following commit(s) were added to refs/heads/9.0.x by this push:
new c2cc786 clang-analyzer: uninitialized va_list (#6798)
c2cc786 is described below
commit c2cc7863f170a4d1ed9139db34b074554717a8a3
Author: Leif Hedstrom
AuthorDate: Wed May 20 09:23:48 2020 -0600
clang-analyzer: uninitialized va_list (#6798)
(cherry picked from commit 0c9fab53bd820c24b3f02cbb01d5a981053ca990)
---
src/tscore/Diags.cc | 12
1 file changed, 4 insertions(+), 8 deletions(-)
diff --git a/src/tscore/Diags.cc b/src/tscore/Diags.cc
index 5f42bc9..bc143f0 100644
--- a/src/tscore/Diags.cc
+++ b/src/tscore/Diags.cc
@@ -460,15 +460,12 @@ Diags::dump(FILE *fp) const
void
Diags::error_va(DiagsLevel level, const SourceLocation *loc, const char
*format_string, va_list ap) const
{
- va_list ap2;
-
- if (DiagsLevel_IsTerminal(level)) {
-va_copy(ap2, ap);
- }
-
print_va(nullptr, level, loc, format_string, ap);
if (DiagsLevel_IsTerminal(level)) {
+va_list ap2;
+
+va_copy(ap2, ap);
if (cleanup_func) {
cleanup_func();
}
@@ -479,9 +476,8 @@ Diags::error_va(DiagsLevel level, const SourceLocation
*loc, const char *format_
} else {
ink_fatal_va(format_string, ap2);
}
+va_end(ap2);
}
-
- va_end(ap2);
}
/*
[trafficserver] branch master updated (16fb809 -> 0c9fab5)
This is an automated email from the ASF dual-hosted git repository. zwoop pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git. from 16fb809 Generalize callbacks for TLS session resumption add 0c9fab5 clang-analyzer: uninitialized va_list (#6798) No new revisions were added by this update. Summary of changes: src/tscore/Diags.cc | 12 1 file changed, 4 insertions(+), 8 deletions(-)
