[trafficserver] branch master updated (793d95b -> 7ff627f)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git. from 793d95b Make request/response body as an option for AuTest microserver add 7ff627f Ensure TSContSchedule API family are called from an EThread. No new revisions were added by this update. Summary of changes: doc/developer-guide/api/functions/TSContSchedule.en.rst | 3 +++ .../api/functions/TSContScheduleOnPool.en.rst| 3 +++ .../api/functions/TSContScheduleOnThread.en.rst | 3 +++ src/traffic_server/InkAPI.cc | 12 4 files changed, 21 insertions(+)
[trafficserver] branch 8.1.x updated: Updated ChangeLog
This is an automated email from the ASF dual-hosted git repository. zwoop pushed a commit to branch 8.1.x in repository https://gitbox.apache.org/repos/asf/trafficserver.git The following commit(s) were added to refs/heads/8.1.x by this push: new dfd2d2e Updated ChangeLog dfd2d2e is described below commit dfd2d2e6105db85688c8dbed671b0caacf21b0bf Author: Leif Hedstrom AuthorDate: Thu Mar 26 20:01:31 2020 -0600 Updated ChangeLog --- CHANGELOG-8.1.0 | 14 +- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/CHANGELOG-8.1.0 b/CHANGELOG-8.1.0 index d2b1bcf..8f4f126 100644 --- a/CHANGELOG-8.1.0 +++ b/CHANGELOG-8.1.0 @@ -1,6 +1,5 @@ Changes with Apache Traffic Server 8.1.0 #3899 - Add a --reason tag option to the host subcommand - #4028 - proxy.config.http.connect_attempts_timeout tracks TTBF instead of connect #4203 - Allow enabling MPTCP on a listening socket through the records.config #4217 - Fix a regression in the traffic_ctl host status subcommand #4242 - header_rewrite: Adds basic string concatenation @@ -11,6 +10,7 @@ Changes with Apache Traffic Server 8.1.0 #4441 - Corrects the wording of the usage messages for traffic_ctl host subcommand #4461 - Fixes memory leaks in header_rewrite #4568 - Adjusts the format for header_rewrite string concatenation + #4586 - Turn this Debug back into an Error #4664 - Use hash string when it is provided for hashing #4762 - Fix OCSP stapling with dual cert #4869 - Cleanup: remove lib/cppapi from clang-format, it's been moved. @@ -22,35 +22,47 @@ Changes with Apache Traffic Server 8.1.0 #5143 - Fix a self detection issue where parents are not marked down during ATS startup. #5149 - Add nullptr check on ConfigProcessor::release() #5163 - Fix memleak in ProcessManager::stop() + #5176 - Adds a new log tag for the MPTCP state of a connection #5209 - Allows logging of negative values + #5219 - url_sig debug fix for when url is missing the signature query string #5227 - Normalize on negative value for milestone metrics #5234 - Correct interpretation of proxy.config.ssl.client.verify.server #5236 - Update documentation for SSL VERIFY hooks. #5242 - Adds a missing header field for the tcpinfo log #5252 - Make host status persistent across restarts + #5268 - fix If-None-Match and If-Modified-Since priority problem #5329 - Fixes clang-analyzer error dereferencing nullptr in parent #5348 - Don't assign if this and other are the same object #5354 - Password can not be nullptr here, so don't check #5417 - Change HostStatus to use only one stat per host. + #5571 - Removes the abort() from header_rewrite, and try to deal with errors #5585 - Do not change stream state from HALF_CLOSED_REMOTE with CONTINUATION frame + #5589 - Adds support for set-destination SCHEME/URL to take string expansions + #5697 - Coalesce empty objects in HdrHeap + #5701 - fix If-Match and If-Unmodified-Since priority problem #5811 - Fix no_activity timeout for server session reuse. #5865 - cachekey: added --canonical-prefix parameter #5872 - cachekey: added --key-type to allow parent selection URL to be modified #5921 - Cleanup AuTest for HTTP/2 #5945 - cachekey: allow multiple values for `--key-type` + #5969 - Change the ready checks for ats to use the actual ats port #6154 - tslua: Exposes set/get method for server request objects + #6168 - Add cache_range_requests autest, add to x-parentselectiion-key to xdebug plugin. #6182 - tslua: Exposes set/get version for server request objects #6191 - Limit this check to Linux, where ldd always works #6234 - Don't sleep if ProtectedQueue.localQueue is not empty. (#6234) + #6247 - Add some stats collections to cache_promote. #6264 - Change HTTP/2 error rate log to warning #6294 - Enhance Connection Collapse in ATS core - back port to 8.x #6328 - Document the order of the X-Cache header #6343 - Fixes an assertion failure when proxy.config.http.no_dns_just_forward_to_parent is enabled #6392 - Free TSMgmtString after using it. #6442 - Disable the chunked_encoding test + #6464 - cache_range_requests plugin: detect and handle TSCacheUrlSet failures… #6518 - Fixes a bug where HostStatus::getHostStatus() creates too many stats. #6534 - Ignore MgmtMessageHdr if ProcessManager is not running #6535 - Add MIMEHdr Garbage Collection to HPACK Dynamic Table + #6540 - Disables "virtual host not used with AWS auth v4" error in s3_auth #6541 - Add cachekey latest features to 8.1.x #6553 - Options to roll empty logs and log trimming (8.1.x) #6557 - free(map); -> delete map;
[trafficserver] branch 8.1.x updated: Coalesce empty objects in HdrHeap
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 8.1.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
The following commit(s) were added to refs/heads/8.1.x by this push:
new ec0f5f9 Coalesce empty objects in HdrHeap
ec0f5f9 is described below
commit ec0f5f93947eb8d515a42f74b143b027868def4a
Author: Masaori Koshiba
AuthorDate: Tue Jun 25 09:25:15 2019 +0900
Coalesce empty objects in HdrHeap
HdrHeap could be filled by `HDR_HEAP_OBJ_EMPTY` type objects in some cases
especially HPACK.
In this situation, `HdrHeap::required_space_for_evacuation()` &
`HdrHeap::evacuate_from_str_heaps()`
could be slow because these functions always walk through `HdrHeapObjImpl`
one by one.
Coalescing empty objects next to each other can reduce number of objects
and mitigate this issue.
(cherry picked from commit 8461afa927471988753f10a286cfac4cef3dbd88)
Conflicts:
proxy/hdrs/HdrHeap.cc
---
proxy/hdrs/HdrHeap.cc | 19 +--
1 file changed, 17 insertions(+), 2 deletions(-)
diff --git a/proxy/hdrs/HdrHeap.cc b/proxy/hdrs/HdrHeap.cc
index 0a8831f..9268104 100644
--- a/proxy/hdrs/HdrHeap.cc
+++ b/proxy/hdrs/HdrHeap.cc
@@ -37,7 +37,8 @@
#include "HTTP.h"
#include "I_EventSystem.h"
-#define MAX_LOST_STR_SPACE 1024
+static constexpr size_t MAX_LOST_STR_SPACE= 1024;
+static constexpr uint32_t MAX_HDR_HEAP_OBJ_LENGTH = (1 << 20) - 1; ///<
m_length is 20 bit
Allocator hdrHeapAllocator("hdrHeap", HDR_HEAP_DEFAULT_SIZE);
static HdrHeap proto_heap;
@@ -440,7 +441,8 @@ HdrHeap::required_space_for_evacuation()
size_t ret = 0;
HdrHeap *h = this;
while (h) {
-char *data = h->m_data_start;
+char *data = h->m_data_start;
+HdrHeapObjImpl *prev_obj = nullptr;
while (data < h->m_free_start) {
HdrHeapObjImpl *obj = (HdrHeapObjImpl *)data;
@@ -465,6 +467,19 @@ HdrHeap::required_space_for_evacuation()
default:
ink_release_assert(0);
}
+
+ // coalesce empty objects next to each other
+ if (obj->m_type == HDR_HEAP_OBJ_EMPTY) {
+if (prev_obj != nullptr && prev_obj->m_length <
(MAX_HDR_HEAP_OBJ_LENGTH - obj->m_length)) {
+ prev_obj->m_length += obj->m_length;
+ ink_release_assert(prev_obj->m_length > 0);
+} else {
+ prev_obj = obj;
+}
+ } else {
+prev_obj = nullptr;
+ }
+
data = data + obj->m_length;
}
h = h->m_next;
[trafficserver] branch master updated (adc0248 -> 793d95b)
This is an automated email from the ASF dual-hosted git repository. masaori pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git. from adc0248 Moved printing the incoming headers for debug before remapping add 793d95b Make request/response body as an option for AuTest microserver No new revisions were added by this update. Summary of changes: tests/gold_tests/autest-site/microserver.test.ext | 11 +-- 1 file changed, 9 insertions(+), 2 deletions(-)
[trafficserver] branch 8.1.x updated: Add some stats collections to cache_promote.
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 8.1.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
The following commit(s) were added to refs/heads/8.1.x by this push:
new 3db3549 Add some stats collections to cache_promote.
3db3549 is described below
commit 3db354933800d137254a43caaa06aac577248f2c
Author: John Rushford
AuthorDate: Thu Dec 5 22:21:48 2019 +
Add some stats collections to cache_promote.
(cherry picked from commit 0da369c80e2a846e5e8fdbdc3b27c57f6af2e3dd)
Conflicts:
plugins/cache_promote/cache_promote.cc
---
doc/admin-guide/plugins/cache_promote.en.rst | 15 +++
plugins/cache_promote/cache_promote.cc | 152 +--
2 files changed, 161 insertions(+), 6 deletions(-)
diff --git a/doc/admin-guide/plugins/cache_promote.en.rst
b/doc/admin-guide/plugins/cache_promote.en.rst
index aa831f1..7e3cbc4 100644
--- a/doc/admin-guide/plugins/cache_promote.en.rst
+++ b/doc/admin-guide/plugins/cache_promote.en.rst
@@ -49,6 +49,21 @@ If :option:`--policy` is set to ``lru`` the following
options are also available
The size (number of entries) of the LRU.
+.. option:: --stats-enable-with-id
+
+ Enables collecting statistics. The option requires an argument, the
+ remap-identifier. The remap-identifier is a string that is concatenated
+ to the stat name. The following stats are collected.
+
+* **plugin.cache_promote.${remap-identifier}.cache_hits** - Cache hit total,
available for all policies.
+* **plugin.cache_promote.${remap-identifier}.freelist_size** - Size of the
freelist when using the LRU policy.
+* **plugin.cache_promote.${remap-identifier}.lru_size** - Size of the LRU
when using the LRU policy.
+* **plugin.cache_promote.${remap-identifier}.lru_hit** - LRU hit count when
using the LRU policy.
+* **plugin.cache_promote.${remap-identifier}.lru_miss** - LRU miss count when
using the LRU policy.
+* **plugin.cache_promote.${remap-identifier}.lru_vacated** - count of LRU
entries removed to make room for a new request.
+* **plugin.cache_promote.${remap-identifier}.promoted** - count requests
promoted, available in all policies.
+* **plugin.cache_promote.${remap-identifier}.total_requests** - count of all
requests.
+
These two options combined with your usage patterns will control how likely a
URL is to become promoted to enter the cache.
diff --git a/plugins/cache_promote/cache_promote.cc
b/plugins/cache_promote/cache_promote.cc
index d05127a..316ba6b 100644
--- a/plugins/cache_promote/cache_promote.cc
+++ b/plugins/cache_promote/cache_promote.cc
@@ -28,11 +28,16 @@
#include
#include
+#include "tscore/ink_config.h"
+#include "tscore/ink_defs.h"
+#include "tscore/BufferWriter.h"
+
#include "ts/ts.h"
#include "ts/remap.h"
#include "tscore/ink_config.h"
#define MINIMUM_BUCKET_SIZE 10
+#define MAX_STAT_LENGTH (1 << 8)
static const char *PLUGIN_NAME = "cache_promote";
@@ -45,6 +50,7 @@ static const struct option longopt[] = {
{const_cast("sample"), required_argument, nullptr, 's'},
// For the LRU policy
{const_cast("buckets"), required_argument, nullptr, 'b'},
+ {const_cast("stats-enable-with-id"), required_argument, nullptr,
'e'},
{const_cast("hits"), required_argument, nullptr, 'h'},
// EOF
{nullptr, no_argument, nullptr, '\0'},
@@ -92,7 +98,43 @@ public:
return true;
}
- virtual ~PromotionPolicy(){};
+ virtual ~PromotionPolicy() = default;
+
+ int
+ create_stat(std::string_view name, std::string_view remap_identifier)
+ {
+int stat_id = -1;
+ts::LocalBufferWriter stat_name;
+stat_name.reset().clip(1);
+stat_name.print("plugin.{}.{}.{}", PLUGIN_NAME, remap_identifier, name);
+stat_name.extend(1).write('\0');
+
+if (TS_ERROR == TSStatFindName(stat_name.data(), _id)) {
+ stat_id = TSStatCreate(stat_name.data(), TS_RECORDDATATYPE_INT,
TS_STAT_NON_PERSISTENT, TS_STAT_SYNC_SUM);
+ if (TS_ERROR == stat_id) {
+TSDebug(PLUGIN_NAME, "Error creating stat_name: %s", stat_name.data());
+ } else {
+TSDebug(PLUGIN_NAME, "Created stat_name: %s, stat_id: %d",
stat_name.data(), stat_id);
+ }
+}
+
+return stat_id;
+ }
+ void
+ decrementStat(const int stat, const int amount)
+ {
+if (stats_enabled) {
+ TSStatIntDecrement(stat, amount);
+}
+ }
+
+ void
+ incrementStat(const int stat, const int amount)
+ {
+if (stats_enabled) {
+ TSStatIntIncrement(stat, amount);
+}
+ }
virtual bool
parseOption(int opt, char *optarg)
@@ -101,9 +143,16 @@ public:
}
// These are pure virtual
- virtual bool doPromote(TSHttpTxn txnp) = 0;
- virtual const char *policyName() const = 0;
- virtual void usage() const = 0;
+ virtual bool doPromote(TSHttpTxn txnp) = 0;
+ virtual const char *policyName() const = 0;
+ virtual void usage() const = 0;
+
[trafficserver] branch 8.1.x updated: Adds support for set-destination SCHEME/URL to take string expansions
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 8.1.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
The following commit(s) were added to refs/heads/8.1.x by this push:
new 60038ae Adds support for set-destination SCHEME/URL to take string
expansions
60038ae is described below
commit 60038ae7d40003b57721a2f88e5eed3119bfd857
Author: Leif Hedstrom
AuthorDate: Thu May 30 15:19:44 2019 -0600
Adds support for set-destination SCHEME/URL to take string expansions
Closes #4143
(cherry picked from commit a98a1e35fbd1e79dfb976edadf390382fab9adab)
---
plugins/header_rewrite/operators.cc | 18 ++
1 file changed, 10 insertions(+), 8 deletions(-)
diff --git a/plugins/header_rewrite/operators.cc
b/plugins/header_rewrite/operators.cc
index 9c2e5c0..f8c39a7 100644
--- a/plugins/header_rewrite/operators.cc
+++ b/plugins/header_rewrite/operators.cc
@@ -240,26 +240,28 @@ OperatorSetDestination::exec(const Resources ) const
}
break;
case URL_QUAL_URL:
- if (_value.empty()) {
+ _value.append_value(value, res);
+ if (value.empty()) {
TSDebug(PLUGIN_NAME, "Would set destination URL to an empty value,
skipping");
} else {
-const char *start = _value.get_value().c_str();
-const char *end = _value.get_value().size() + start;
+const char *start = value.c_str();
+const char *end = start + value.size();
TSMLoc new_url_loc;
if (TSUrlCreate(bufp, _url_loc) == TS_SUCCESS && TSUrlParse(bufp,
new_url_loc, , end) == TS_PARSE_DONE &&
TSHttpHdrUrlSet(bufp, res.hdr_loc, new_url_loc) == TS_SUCCESS) {
- TSDebug(PLUGIN_NAME, "Set destination URL to %s",
_value.get_value().c_str());
+ TSDebug(PLUGIN_NAME, "Set destination URL to %s", value.c_str());
} else {
- TSDebug(PLUGIN_NAME, "Failed to set URL %s",
_value.get_value().c_str());
+ TSDebug(PLUGIN_NAME, "Failed to set URL %s", value.c_str());
}
}
break;
case URL_QUAL_SCHEME:
- if (_value.empty()) {
+ _value.append_value(value, res);
+ if (value.empty()) {
TSDebug(PLUGIN_NAME, "Would set destination SCHEME to an empty value,
skipping");
} else {
-TSUrlSchemeSet(bufp, url_m_loc, _value.get_value().c_str(),
_value.get_value().length());
-TSDebug(PLUGIN_NAME, "OperatorSetDestination::exec() invoked with
SCHEME: %s", _value.get_value().c_str());
+TSUrlSchemeSet(bufp, url_m_loc, value.c_str(), value.length());
+TSDebug(PLUGIN_NAME, "OperatorSetDestination::exec() invoked with
SCHEME: %s", value.c_str());
}
break;
default:
[trafficserver] branch 8.1.x updated: Removes the abort() from header_rewrite, and try to deal with errors
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 8.1.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
The following commit(s) were added to refs/heads/8.1.x by this push:
new 4566854 Removes the abort() from header_rewrite, and try to deal with
errors
4566854 is described below
commit 4566854918bcebf051e299180b6f46d3af42bcb2
Author: Leif Hedstrom
AuthorDate: Thu May 23 16:31:55 2019 -0600
Removes the abort() from header_rewrite, and try to deal with errors
Bryan suggested using exceptions here, rather than just producing errors
in the log files. As ugly as it is, it does seem to work ... But I blame
him for all future issues! Long term, I think it'd be better to eliminate
the exceptions here, and fix all the various layers of returning and
perculating errors all the way back up through the stack.
(cherry picked from commit 244ecc65e237e9b219357aacbee0c5f961765756)
---
plugins/header_rewrite/header_rewrite.cc | 25 -
plugins/header_rewrite/matcher.h | 7 +--
2 files changed, 21 insertions(+), 11 deletions(-)
diff --git a/plugins/header_rewrite/header_rewrite.cc
b/plugins/header_rewrite/header_rewrite.cc
index 60caecd..99b1961 100644
--- a/plugins/header_rewrite/header_rewrite.cc
+++ b/plugins/header_rewrite/header_rewrite.cc
@@ -17,6 +17,7 @@
*/
#include
#include
+#include
#include "ts/ts.h"
#include "ts/remap.h"
@@ -220,16 +221,22 @@ RulesConfig::parse_config(const std::string ,
TSHttpHookID default_hook)
}
}
-if (p.is_cond()) {
- if (!rule->add_condition(p, filename.c_str(), lineno)) {
-delete rule;
-return false;
- }
-} else {
- if (!rule->add_operator(p, filename.c_str(), lineno)) {
-delete rule;
-return false;
+// This is pretty ugly, but it turns out, some conditions / operators can
fail (regexes), which didn't use to be the case.
+// Long term, maybe we need to percolate all this up through
add_condition() / add_operator() rather than this big ugly try.
+try {
+ if (p.is_cond()) {
+if (!rule->add_condition(p, filename.c_str(), lineno)) {
+ throw std::runtime_error("add_condition() failed");
+}
+ } else {
+if (!rule->add_operator(p, filename.c_str(), lineno)) {
+ throw std::runtime_error("add_operator() failed");
+}
}
+} catch (std::runtime_error ) {
+ TSError("[%s] header_rewrite configuration exception: %s", PLUGIN_NAME,
e.what());
+ delete rule;
+ return false;
}
}
diff --git a/plugins/header_rewrite/matcher.h b/plugins/header_rewrite/matcher.h
index 0e285e5..88a2399 100644
--- a/plugins/header_rewrite/matcher.h
+++ b/plugins/header_rewrite/matcher.h
@@ -23,6 +23,7 @@
#include
#include
+#include
#include "ts/ts.h"
@@ -94,9 +95,11 @@ public:
std::stringstream ss;
ss << _data;
TSError("[%s] Invalid regex: failed to precompile: %s", PLUGIN_NAME,
ss.str().c_str());
- abort();
+ TSDebug(PLUGIN_NAME, "Invalid regex: failed to precompile: %s",
ss.str().c_str());
+ throw std::runtime_error("Malformed regex");
+} else {
+ TSDebug(PLUGIN_NAME, "Regex precompiled successfully");
}
-TSDebug(PLUGIN_NAME, "Regex precompiled successfully");
}
void
[trafficserver] branch 8.1.x updated: Turn this Debug back into an Error
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 8.1.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
The following commit(s) were added to refs/heads/8.1.x by this push:
new 44830f2 Turn this Debug back into an Error
44830f2 is described below
commit 44830f2e8da53007da9dcbf1aac92f4e049aa045
Author: Leif Hedstrom
AuthorDate: Fri Nov 9 16:05:42 2018 -0700
Turn this Debug back into an Error
(cherry picked from commit 9db7acfe0996cb86a93d4b32ac2cbc5215b92af4)
---
plugins/header_rewrite/factory.cc | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/plugins/header_rewrite/factory.cc
b/plugins/header_rewrite/factory.cc
index 41e29fe..028fc62 100644
--- a/plugins/header_rewrite/factory.cc
+++ b/plugins/header_rewrite/factory.cc
@@ -144,7 +144,7 @@ condition_factory(const std::string )
} else if (c_name == "INBOUND") {
c = new ConditionInbound();
} else {
-TSDebug(PLUGIN_NAME, "Unknown condition: %s", c_name.c_str());
+TSError("[%s] Unknown condition %s", PLUGIN_NAME, c_name.c_str());
return nullptr;
}
[trafficserver] branch 8.1.x updated: url_sig debug fix for when url is missing the signature query string
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 8.1.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
The following commit(s) were added to refs/heads/8.1.x by this push:
new eefaea7 url_sig debug fix for when url is missing the signature query
string
eefaea7 is described below
commit eefaea7916c2b71b8fe5a60b1c963f99eb19d2bc
Author: Brian Olsen
AuthorDate: Tue Apr 2 13:25:48 2019 +
url_sig debug fix for when url is missing the signature query string
(cherry picked from commit 3d6e63d764bb8741d4f6a30227f651f0e7c6118a)
---
plugins/experimental/url_sig/README| 2 +-
plugins/experimental/url_sig/url_sig.c | 14 --
2 files changed, 9 insertions(+), 7 deletions(-)
diff --git a/plugins/experimental/url_sig/README
b/plugins/experimental/url_sig/README
index a8b4360..9dc4dee 100644
--- a/plugins/experimental/url_sig/README
+++ b/plugins/experimental/url_sig/README
@@ -133,7 +133,7 @@ Example
Add the remap rule like
- map http://test-remap.domain.com http://google.com
@plugin=url_sig.so @pparam=sign_test.config
+ map http://test-remap.domain.com http://google.com
@plugin=url_sig.so @pparam=sign_test.config @pparam=pristineurl
Restart traffic server or traffic_ctl config reload - verify there are
no errors in diags.log
diff --git a/plugins/experimental/url_sig/url_sig.c
b/plugins/experimental/url_sig/url_sig.c
index 6aa1fe0..f65ca89 100644
--- a/plugins/experimental/url_sig/url_sig.c
+++ b/plugins/experimental/url_sig/url_sig.c
@@ -23,9 +23,9 @@
_a < _b ? _a : _b; \
})
-#include "tscore/ink_defs.h"
#include "url_sig.h"
+#include
#include
#include
#include
@@ -538,7 +538,7 @@ TSRemapDoRemap(void *ih, TSHttpTxn txnp, TSRemapRequestInfo
*rri)
char sig_string[2 * MAX_SIG_SIZE + 1];
if (current_url_len >= MAX_REQ_LEN - 1) {
-err_log(url, "URL string too long");
+err_log(current_url, "Request Url string too long");
goto deny;
}
@@ -555,15 +555,12 @@ TSRemapDoRemap(void *ih, TSHttpTxn txnp,
TSRemapRequestInfo *rri)
err_log(url, "Pristine URL string too long.");
goto deny;
}
-
} else {
url_len = current_url_len;
}
TSDebug(PLUGIN_NAME, "%s", url);
- const char *query = strchr(url, '?');
-
if (cfg->regex) {
const int offset = 0, options = 0;
int ovector[30];
@@ -580,12 +577,17 @@ TSRemapDoRemap(void *ih, TSHttpTxn txnp,
TSRemapRequestInfo *rri)
}
}
+ const char *query = strchr(url, '?');
+
// check for path params.
if (query == NULL || strstr(query, "E=") == NULL) {
-if ((url = urlParse(url, cfg->sig_anchor, new_path, 8192, path_params,
8192)) == NULL) {
+char *const parsed = urlParse(url, cfg->sig_anchor, new_path, 8192,
path_params, 8192);
+if (NULL == parsed) {
err_log(url, "Has no signing query string or signing path parameters.");
goto deny;
}
+
+url = parsed;
has_path_params = true;
query = strstr(url, ";");
[trafficserver] branch 8.1.x updated (8316204 -> 3398a16)
This is an automated email from the ASF dual-hosted git repository. zwoop pushed a change to branch 8.1.x in repository https://gitbox.apache.org/repos/asf/trafficserver.git. from 8316204 Disables "virtual host not used with AWS auth v4" error in s3_auth new 0a2a286 Add autest for cache_range_request, add xdebug x-parentselection-key new 3398a16 cache_range_requests plugin: detect and handle TSCacheUrlSet failures which poison the cache The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../plugins/cache_range_requests.en.rst| 191 .../cache_range_requests/cache_range_requests.cc | 105 --- .../cache_range_requests.test.py | 344 + .../cache_range_requests_cachekey.test.py | 198 .../cache_range_requests/gold/404.stdout.gold | 8 + .../cache_range_requests/gold/frange.stderr.gold | 1 + .../cache_range_requests/gold/frange.stdout.gold | 11 + .../cache_range_requests/gold/full.stderr.gold | 1 + .../cache_range_requests/gold/full.stdout.gold | 9 + .../cache_range_requests/gold/inner.stderr.gold| 1 + .../cache_range_requests/gold/inner.stdout.gold| 11 + .../cache_range_requests/gold/last.stderr.gold | 1 + .../cache_range_requests/gold/last.stdout.gold | 11 + .../cache_range_requests/gold/pselect.stdout.gold | 11 + .../gold/pselect_none.stdout.gold | 10 + 15 files changed, 876 insertions(+), 37 deletions(-) create mode 100644 doc/admin-guide/plugins/cache_range_requests.en.rst create mode 100644 tests/gold_tests/pluginTest/cache_range_requests/cache_range_requests.test.py create mode 100644 tests/gold_tests/pluginTest/cache_range_requests/cache_range_requests_cachekey.test.py create mode 100644 tests/gold_tests/pluginTest/cache_range_requests/gold/404.stdout.gold create mode 100644 tests/gold_tests/pluginTest/cache_range_requests/gold/frange.stderr.gold create mode 100644 tests/gold_tests/pluginTest/cache_range_requests/gold/frange.stdout.gold create mode 100644 tests/gold_tests/pluginTest/cache_range_requests/gold/full.stderr.gold create mode 100644 tests/gold_tests/pluginTest/cache_range_requests/gold/full.stdout.gold create mode 100644 tests/gold_tests/pluginTest/cache_range_requests/gold/inner.stderr.gold create mode 100644 tests/gold_tests/pluginTest/cache_range_requests/gold/inner.stdout.gold create mode 100644 tests/gold_tests/pluginTest/cache_range_requests/gold/last.stderr.gold create mode 100644 tests/gold_tests/pluginTest/cache_range_requests/gold/last.stdout.gold create mode 100644 tests/gold_tests/pluginTest/cache_range_requests/gold/pselect.stdout.gold create mode 100644 tests/gold_tests/pluginTest/cache_range_requests/gold/pselect_none.stdout.gold
[trafficserver] 01/02: Add autest for cache_range_request, add xdebug x-parentselection-key
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 8.1.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
commit 0a2a2862bd94ef5a53d0d82c57acc08f49080137
Author: Brian Olsen
AuthorDate: Wed Nov 13 22:07:38 2019 +
Add autest for cache_range_request, add xdebug x-parentselection-key
(cherry picked from commit e02c55fb44c497137e60ac5cd14ab30da683be92)
---
.../cache_range_requests.test.py | 344 +
.../cache_range_requests/gold/404.stdout.gold | 8 +
.../cache_range_requests/gold/frange.stderr.gold | 1 +
.../cache_range_requests/gold/frange.stdout.gold | 11 +
.../cache_range_requests/gold/full.stderr.gold | 1 +
.../cache_range_requests/gold/full.stdout.gold | 9 +
.../cache_range_requests/gold/inner.stderr.gold| 1 +
.../cache_range_requests/gold/inner.stdout.gold| 11 +
.../cache_range_requests/gold/last.stderr.gold | 1 +
.../cache_range_requests/gold/last.stdout.gold | 11 +
.../cache_range_requests/gold/pselect.stdout.gold | 11 +
.../gold/pselect_none.stdout.gold | 10 +
12 files changed, 419 insertions(+)
diff --git
a/tests/gold_tests/pluginTest/cache_range_requests/cache_range_requests.test.py
b/tests/gold_tests/pluginTest/cache_range_requests/cache_range_requests.test.py
new file mode 100644
index 000..77fd29e
--- /dev/null
+++
b/tests/gold_tests/pluginTest/cache_range_requests/cache_range_requests.test.py
@@ -0,0 +1,344 @@
+'''
+'''
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import os
+
+Test.Summary = '''
+Basic cache_range_requests plugin test
+'''
+
+## Test description:
+# Preload the cache with the entire asset to be range requested.
+# Reload remap rule with cache_range_requests plugin
+# Request content through the cache_range_requests plugin
+
+Test.SkipUnless(
+Condition.PluginExists('cache_range_requests.so'),
+Condition.PluginExists('xdebug.so'),
+)
+Test.ContinueOnFail = False
+Test.testName = "cache_range_requests"
+
+# Define and configure ATS
+ts = Test.MakeATSProcess("ts", command="traffic_server")
+
+# Define and configure origin server
+server = Test.MakeOriginServer("server", lookup_key="{%uuid}")
+
+# default root
+req_chk = {"headers":
+ "GET / HTTP/1.1\r\n" +
+ "Host: www.example.com\r\n" +
+ "uuid: none\r\n" +
+ "\r\n",
+ "timestamp": "1469733493.993",
+ "body": ""
+}
+
+res_chk = {"headers":
+ "HTTP/1.1 200 OK\r\n" +
+ "Connection: close\r\n" +
+ "\r\n",
+ "timestamp": "1469733493.993",
+ "body": ""
+}
+
+server.addResponse("sessionlog.json", req_chk, res_chk)
+
+body = "lets go surfin now"
+
+req_full = {"headers":
+ "GET /path HTTP/1.1\r\n" +
+ "Host: www.example.com\r\n" +
+ "Accept: */*\r\n" +
+ "uuid: full\r\n" +
+ "\r\n",
+ "timestamp": "1469733493.993",
+ "body": ""
+}
+
+res_full = {"headers":
+ "HTTP/1.1 200 OK\r\n" +
+ "Cache-Control: max-age=500\r\n" +
+ "Connection: close\r\n" +
+ 'Etag: "772102f4-56f4bc1e6d417"\r\n' +
+ "Last-Modified: Sat, 23 Jun 2018 09:27:29 GMT\r\n" +
+ "\r\n",
+ "timestamp": "1469733493.993",
+ "body": body
+}
+
+server.addResponse("sessionlog.json", req_full, res_full)
+
+block_bytes = 7
+bodylen = len(body)
+
+inner_str = "7-15"
+
+req_inner = {"headers":
+ "GET /path HTTP/1.1\r\n" +
+ "Host: www.example.com\r\n" +
+ "Accept: */*\r\n" +
+ "Range: bytes={}\r\n".format(inner_str) +
+ "uuid: inner\r\n" +
+ "\r\n",
+ "timestamp": "1469733493.993",
+ "body": ""
+}
+
+res_inner = {"headers":
+ "HTTP/1.1 206 Partial Content\r\n" +
+ "Accept-Ranges: bytes\r\n" +
+ "Cache-Control: max-age=500\r\n" +
+ "Content-Range: bytes {0}/{1}\r\n".format(inner_str, bodylen) +
+ "Connection: close\r\n" +
+ 'Etag: "772102f4-56f4bc1e6d417"\r\n' +
+ "Last-Modified: Sat, 23 Jun 2018 09:27:29 GMT\r\n" +
+ "\r\n",
+ "timestamp": "1469733493.993",
+ "body": body[7:15]
+}
+
+server.addResponse("sessionlog.json", req_inner, res_inner)
+
+frange_str = "0-"
+
+req_frange = {"headers":
+ "GET /path HTTP/1.1\r\n" +
+ "Host: www.example.com\r\n" +
+ "Accept: */*\r\n" +
+ "Range: bytes={}\r\n".format(frange_str) +
+ "uuid: frange\r\n" +
+ "\r\n",
+
[trafficserver] 02/02: cache_range_requests plugin: detect and handle TSCacheUrlSet failures which poison the cache
This is an automated email from the ASF dual-hosted git repository. zwoop pushed a commit to branch 8.1.x in repository https://gitbox.apache.org/repos/asf/trafficserver.git commit 3398a1681f5e2eb5f5e8c469736e1697d9f02c2d Author: Brian Olsen AuthorDate: Fri Feb 28 16:49:06 2020 + cache_range_requests plugin: detect and handle TSCacheUrlSet failures which poison the cache (cherry picked from commit 8021a8e77723784b576592f5d93c703733706078) --- .../plugins/cache_range_requests.en.rst| 191 .../cache_range_requests/cache_range_requests.cc | 105 +++ .../cache_range_requests_cachekey.test.py | 198 + 3 files changed, 457 insertions(+), 37 deletions(-) diff --git a/doc/admin-guide/plugins/cache_range_requests.en.rst b/doc/admin-guide/plugins/cache_range_requests.en.rst new file mode 100644 index 000..6d11d21 --- /dev/null +++ b/doc/admin-guide/plugins/cache_range_requests.en.rst @@ -0,0 +1,191 @@ +.. Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + + +.. include:: ../../common.defs + +.. _admin-plugins-cache-range-requests: + + +Cache Range Requests Plugin +*** + +Description +=== + +Most origin servers support HTTP/1.1 range requests (rfc 7233). +ATS internally handles range request caching in one of 2 ways: + +* Don't cache range requests. +* Only server range requests from a wholly cached object. + +This plugin allows you to remap individual range requests so that they +are stored as individual objects in the ATS cache when subsequent range +requests are likely to use the same range. This spreads range requests +over multiple stripes thereby reducing I/O wait and system load averages. + +:program:`cache_range_requests` reads the range request header byte range +value and then creates a new ``cache key URL`` using the original request +url with the range value appended to it. The range header is removed +where appropriate from the requests and the origin server response code +is changed from a 206 to a 200 to insure that the object is written to +cache using the new cache key url. The response code sent to the client +will be changed back to a 206 and all requests to the origin server will +contain the range header so that the correct response is received. + +The :program:`cache_range_requests` plugin by itself has no logic to +efficiently manage overlapping ranges. It is best to use this plugin +in conjunction with a smart client that only requests predetermined +non overlapping cache ranges (request blocking) or as a helper for the +:program:`slice` plugin. + +Only requests which contain the ``Range: =`` GET header +will be served by the :program:`cache_range_requests` plugin. + +If/when ATS implements partial object caching this plugin will +become deprecated. + +*NOTE* Given a multi range request the :program:`cache_range_requests` +only processes the first range and ignores the rest. + +How to run the plugin += + +The plugin can run as a global plugin (a single global instance configured +using :file:`plugin.config`) or as per-remap plugin (a separate instance +configured per remap rule in :file:`remap.config`). + +Global instance +--- + +.. code:: + + $ cat plugin.config + cache_range_request.so + + +Per-remap instance +-- + +.. code:: + + $cat remap.config + map http://www.example.com http://www.origin.com \ + @plugin=cache_range_requests.so + + +If both global and per-remap instance are used the per-remap configuration +would take precedence (per-remap configuration would be applied and the +global configuration ignored). + +Plugin options +== + + +Parent Selection as Cache Key +- + +.. option:: --ps-cachekey +.. option:: -p + +Without this option parent selection is based solely on the hash of a +URL Path a URL is requested from the same upstream parent cache listed +in parent.config + + +With this option parent selection is based on the full ``cache key URL`` +which includes information about the partial content range. In this mode, +all requests (include partial content) will use consistent hashing method +for parent
[trafficserver] branch 8.1.x updated: Disables "virtual host not used with AWS auth v4" error in s3_auth
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 8.1.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
The following commit(s) were added to refs/heads/8.1.x by this push:
new 8316204 Disables "virtual host not used with AWS auth v4" error in
s3_auth
8316204 is described below
commit 8316204f0ab7dfd0fd6c27500908275f42d24dc0
Author: Randall Meyer
AuthorDate: Thu Mar 19 14:52:26 2020 -0700
Disables "virtual host not used with AWS auth v4" error in s3_auth
This would error on every request
(cherry picked from commit 05556de2a3b06400e7f587ab4accf1cbf8b2a88e)
---
plugins/s3_auth/s3_auth.cc | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
diff --git a/plugins/s3_auth/s3_auth.cc b/plugins/s3_auth/s3_auth.cc
index 6814859..242d5fe 100644
--- a/plugins/s3_auth/s3_auth.cc
+++ b/plugins/s3_auth/s3_auth.cc
@@ -202,9 +202,7 @@ public:
}
} else {
/* 4 == _version */
- if (_virt_host_modified) {
-TSError("[%s] virtual host not used with AWS auth v4, parameter
ignored", PLUGIN_NAME);
- }
+ // NOTE: virtual host not used with AWS auth v4, parameter ignored
}
return true;
}
[trafficserver] 01/02: A recipient MUST ignore If-Modified-Since if the request contains an If-None-Match header field, about rfc https://tools.ietf.org/html/rfc7232#section-3.3
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 8.1.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
commit 1c7f41135639ee65fb7304749ae3bae98ff4f668
Author: chenggang7
AuthorDate: Sun Apr 14 21:27:12 2019 +0800
A recipient MUST ignore If-Modified-Since if the request contains an
If-None-Match header field, about rfc
https://tools.ietf.org/html/rfc7232#section-3.3
(cherry picked from commit 91b05598bc07d4e8a97281f4d5a556b7956d803e)
---
proxy/http/HttpTransactCache.cc | 59 +++--
1 file changed, 27 insertions(+), 32 deletions(-)
diff --git a/proxy/http/HttpTransactCache.cc b/proxy/http/HttpTransactCache.cc
index 0b5ecdf..56adb7b 100644
--- a/proxy/http/HttpTransactCache.cc
+++ b/proxy/http/HttpTransactCache.cc
@@ -1288,6 +1288,33 @@
HttpTransactCache::match_response_to_request_conditionals(HTTPHdr *request, HTTP
return response->status_get();
}
+ // If-None-Match: may match weakly //
+ if (request->presence(MIME_PRESENCE_IF_NONE_MATCH)) {
+int raw_etags_len, comma_sep_tag_list_len;
+const char *raw_etags = response->value_get(MIME_FIELD_ETAG,
MIME_LEN_ETAG, _etags_len);
+const char *comma_sep_tag_list = nullptr;
+
+if (raw_etags) {
+ comma_sep_tag_list = request->value_get(MIME_FIELD_IF_NONE_MATCH,
MIME_LEN_IF_NONE_MATCH, _sep_tag_list_len);
+ if (!comma_sep_tag_list) {
+comma_sep_tag_list = "";
+comma_sep_tag_list_len = 0;
+ }
+
+
+ // If we have an etag and a if-none-match, we are talking to someone //
+ // who is doing a 1.1 revalidate. Since this is a GET request with no //
+ // sub-ranges, we can do a weak validation. //
+
+ if (do_strings_match_weakly(raw_etags, raw_etags_len,
comma_sep_tag_list, comma_sep_tag_list_len)) {
+// the response already failed If-modified-since (if one exists)
+return HTTP_STATUS_NOT_MODIFIED;
+ } else {
+return response->status_get();
+ }
+}
+ }
+
// If-Modified-Since //
if (request->presence(MIME_PRESENCE_IF_MODIFIED_SINCE)) {
if (response->presence(MIME_PRESENCE_LAST_MODIFIED)) {
@@ -1316,38 +1343,6 @@
HttpTransactCache::match_response_to_request_conditionals(HTTPHdr *request, HTTP
response_code = HTTP_STATUS_NOT_MODIFIED;
}
-
-// we cannot return NOT_MODIFIED yet, need to check If-none-match
-if (!request->presence(MIME_PRESENCE_IF_NONE_MATCH)) {
- return response_code;
-}
- }
-
- // If-None-Match: may match weakly //
- if (request->presence(MIME_PRESENCE_IF_NONE_MATCH)) {
-int raw_etags_len, comma_sep_tag_list_len;
-const char *raw_etags = response->value_get(MIME_FIELD_ETAG,
MIME_LEN_ETAG, _etags_len);
-const char *comma_sep_tag_list = nullptr;
-
-if (raw_etags) {
- comma_sep_tag_list = request->value_get(MIME_FIELD_IF_NONE_MATCH,
MIME_LEN_IF_NONE_MATCH, _sep_tag_list_len);
- if (!comma_sep_tag_list) {
-comma_sep_tag_list = "";
-comma_sep_tag_list_len = 0;
- }
-
-
- // If we have an etag and a if-none-match, we are talking to someone //
- // who is doing a 1.1 revalidate. Since this is a GET request with no //
- // sub-ranges, we can do a weak validation. //
-
- if (do_strings_match_weakly(raw_etags, raw_etags_len,
comma_sep_tag_list, comma_sep_tag_list_len)) {
-// the response already failed If-modified-since (if one exists)
-return HTTP_STATUS_NOT_MODIFIED;
- } else {
-return response->status_get();
- }
-}
}
// There is no If-none-match, and If-modified-since failed,
[trafficserver] branch 8.1.x updated (63b8092 -> 3877af7)
This is an automated email from the ASF dual-hosted git repository. zwoop pushed a change to branch 8.1.x in repository https://gitbox.apache.org/repos/asf/trafficserver.git. from 63b8092 Updated ChangeLOG new 1c7f411 A recipient MUST ignore If-Modified-Since if the request contains anIf-None-Match header field, about rfc https://tools.ietf.org/html/rfc7232#section-3.3 new 3877af7 fix If-Match and If-Unmodified-Since priority problem,about rfc https://tools.ietf.org/html/rfc7232#section-3.3 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: proxy/http/HttpTransactCache.cc | 92 ++--- 1 file changed, 40 insertions(+), 52 deletions(-)
[trafficserver] 02/02: fix If-Match and If-Unmodified-Since priority problem, about rfc https://tools.ietf.org/html/rfc7232#section-3.3
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 8.1.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
commit 3877af7a0439462cf53c4fe17c28e3a94b5d014c
Author: chenggang7
AuthorDate: Tue Jul 9 11:07:52 2019 +0800
fix If-Match and If-Unmodified-Since priority problem,about rfc
https://tools.ietf.org/html/rfc7232#section-3.3
(cherry picked from commit edd6ae44ab791fe88947995edba280db75a2d41e)
---
proxy/http/HttpTransactCache.cc | 35 ++-
1 file changed, 14 insertions(+), 21 deletions(-)
diff --git a/proxy/http/HttpTransactCache.cc b/proxy/http/HttpTransactCache.cc
index 56adb7b..05a8c3d 100644
--- a/proxy/http/HttpTransactCache.cc
+++ b/proxy/http/HttpTransactCache.cc
@@ -1307,7 +1307,6 @@
HttpTransactCache::match_response_to_request_conditionals(HTTPHdr *request, HTTP
// sub-ranges, we can do a weak validation. //
if (do_strings_match_weakly(raw_etags, raw_etags_len,
comma_sep_tag_list, comma_sep_tag_list_len)) {
-// the response already failed If-modified-since (if one exists)
return HTTP_STATUS_NOT_MODIFIED;
} else {
return response->status_get();
@@ -1351,24 +1350,6 @@
HttpTransactCache::match_response_to_request_conditionals(HTTPHdr *request, HTTP
return response_code;
}
- // return PRECONDITIONAL_FAILED if either If-unmodified-since
- // or If-match fails
- // BUT, return the original response code only if both pass
-
- // If-Unmodified-Since //
- if (request->presence(MIME_PRESENCE_IF_UNMODIFIED_SINCE)) {
-// lm_value is zero if Last-modified not exists
-ink_time_t lm_value = response->get_last_modified();
-
-// Condition fails if Last-modified not exists
-if ((request->get_if_unmodified_since() < lm_value) || (lm_value == 0)) {
- return HTTP_STATUS_PRECONDITION_FAILED;
-} else {
- // we cannot return yet, need to check If-match
- response_code = response->status_get();
-}
- }
-
// If-Match: must match strongly //
if (request->presence(MIME_PRESENCE_IF_MATCH)) {
int raw_etags_len, comma_sep_tag_list_len;
@@ -1390,13 +1371,25 @@
HttpTransactCache::match_response_to_request_conditionals(HTTPHdr *request, HTTP
}
if (do_strings_match_strongly(raw_etags, raw_etags_len,
comma_sep_tag_list, comma_sep_tag_list_len)) {
- // at the point, the response passed both If-unmodified-since
- // and If-match, so we can return the original response code
return response->status_get();
} else {
return HTTP_STATUS_PRECONDITION_FAILED;
}
}
+
+ // If-Unmodified-Since //
+ if (request->presence(MIME_PRESENCE_IF_UNMODIFIED_SINCE)) {
+// lm_value is zero if Last-modified not exists
+ink_time_t lm_value = response->get_last_modified();
+
+// Condition fails if Last-modified not exists
+if ((request->get_if_unmodified_since() < lm_value) || (lm_value == 0)) {
+ return HTTP_STATUS_PRECONDITION_FAILED;
+} else {
+ response_code = response->status_get();
+}
+ }
+
// There is no If-match, and If-unmodified-since passed,
// so return the original response code
if (response_code != HTTP_STATUS_NONE) {
[trafficserver] branch 8.1.x updated: Updated ChangeLOG
This is an automated email from the ASF dual-hosted git repository. zwoop pushed a commit to branch 8.1.x in repository https://gitbox.apache.org/repos/asf/trafficserver.git The following commit(s) were added to refs/heads/8.1.x by this push: new 63b8092 Updated ChangeLOG 63b8092 is described below commit 63b8092aadc894c04c6018063335330c5e64d45d Author: Leif Hedstrom AuthorDate: Thu Mar 26 15:46:04 2020 -0600 Updated ChangeLOG --- CHANGELOG-8.1.0 | 4 1 file changed, 4 insertions(+) diff --git a/CHANGELOG-8.1.0 b/CHANGELOG-8.1.0 index 22350e3..d2b1bcf 100644 --- a/CHANGELOG-8.1.0 +++ b/CHANGELOG-8.1.0 @@ -3,11 +3,14 @@ Changes with Apache Traffic Server 8.1.0 #4028 - proxy.config.http.connect_attempts_timeout tracks TTBF instead of connect #4203 - Allow enabling MPTCP on a listening socket through the records.config #4217 - Fix a regression in the traffic_ctl host status subcommand + #4242 - header_rewrite: Adds basic string concatenation #4253 - Add the capability to mark origins down with traffic_ctl. #4300 - Expand the post chunked tests. #4422 - TCL: Rmove TCL dependency from HostStatus #4437 - Adds the log tag cqint for internal requests #4441 - Corrects the wording of the usage messages for traffic_ctl host subcommand + #4461 - Fixes memory leaks in header_rewrite + #4568 - Adjusts the format for header_rewrite string concatenation #4664 - Use hash string when it is provided for hashing #4762 - Fix OCSP stapling with dual cert #4869 - Cleanup: remove lib/cppapi from clang-format, it's been moved. @@ -38,6 +41,7 @@ Changes with Apache Traffic Server 8.1.0 #6154 - tslua: Exposes set/get method for server request objects #6182 - tslua: Exposes set/get version for server request objects #6191 - Limit this check to Linux, where ldd always works + #6234 - Don't sleep if ProtectedQueue.localQueue is not empty. (#6234) #6264 - Change HTTP/2 error rate log to warning #6294 - Enhance Connection Collapse in ATS core - back port to 8.x #6328 - Document the order of the X-Cache header
[trafficserver] branch master updated: Moved printing the incoming headers for debug before remapping
This is an automated email from the ASF dual-hosted git repository.
bcall pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
The following commit(s) were added to refs/heads/master by this push:
new adc0248 Moved printing the incoming headers for debug before remapping
adc0248 is described below
commit adc02486fef9a720c7dc1c0b217a67848fce9745
Author: Bryan Call
AuthorDate: Wed Mar 25 16:35:29 2020 -0700
Moved printing the incoming headers for debug before remapping
---
proxy/http/HttpTransact.cc | 6 +-
1 file changed, 1 insertion(+), 5 deletions(-)
diff --git a/proxy/http/HttpTransact.cc b/proxy/http/HttpTransact.cc
index 18bf817..e6b11ea 100644
--- a/proxy/http/HttpTransact.cc
+++ b/proxy/http/HttpTransact.cc
@@ -851,6 +851,7 @@ HttpTransact::StartRemapRequest(State *s)
TxnDebug("http_trans", "Before Remapping:");
obj_describe(s->hdr_info.client_request.m_http, true);
}
+ DUMP_HEADER("http_hdrs", >hdr_info.client_request, s->state_machine_id,
"Incoming Request");
if (s->http_config_param->referer_filter_enabled) {
s->filter_mask = URL_REMAP_FILTER_REFERER;
@@ -1426,11 +1427,6 @@ HttpTransact::HandleRequest(State *s)
TRANSACT_RETURN(SM_ACTION_INTERNAL_REQUEST, nullptr);
}
- // this needs to be called after initializing state variables from request
- // it adds the client-ip to the incoming client request.
-
- DUMP_HEADER("http_hdrs", >hdr_info.client_request, s->state_machine_id,
"Incoming Request");
-
if (s->state_machine->plugin_tunnel_type == HTTP_PLUGIN_AS_INTERCEPT) {
setup_plugin_request_intercept(s);
return;
[trafficserver] branch 8.1.x updated: Don't sleep if ProtectedQueue.localQueue is not empty. (#6234)
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 8.1.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
The following commit(s) were added to refs/heads/8.1.x by this push:
new 46b6310 Don't sleep if ProtectedQueue.localQueue is not empty. (#6234)
46b6310 is described below
commit 46b63106c79daf420472685213088a9e3260388f
Author: Oknet Xu
AuthorDate: Wed Nov 27 22:28:33 2019 +0800
Don't sleep if ProtectedQueue.localQueue is not empty. (#6234)
(cherry picked from commit aa5b2eafda2667621f443b455829ce3b5c3be0b0)
---
iocore/eventsystem/P_UnixEThread.h | 2 +-
iocore/eventsystem/UnixEThread.cc | 8 +++-
2 files changed, 8 insertions(+), 2 deletions(-)
diff --git a/iocore/eventsystem/P_UnixEThread.h
b/iocore/eventsystem/P_UnixEThread.h
index 9328c3b..45931e6 100644
--- a/iocore/eventsystem/P_UnixEThread.h
+++ b/iocore/eventsystem/P_UnixEThread.h
@@ -33,7 +33,7 @@
#include "I_EThread.h"
#include "I_EventProcessor.h"
-const int DELAY_FOR_RETRY = HRTIME_MSECONDS(10);
+const ink_hrtime DELAY_FOR_RETRY = HRTIME_MSECONDS(10);
TS_INLINE Event *
EThread::schedule_imm(Continuation *cont, int callback_event, void *cookie)
diff --git a/iocore/eventsystem/UnixEThread.cc
b/iocore/eventsystem/UnixEThread.cc
index 225d81f..137a063 100644
--- a/iocore/eventsystem/UnixEThread.cc
+++ b/iocore/eventsystem/UnixEThread.cc
@@ -259,7 +259,13 @@ EThread::execute_regular()
next_time = EventQueue.earliest_timeout();
ink_hrtime sleep_time = next_time - Thread::get_hrtime_updated();
if (sleep_time > 0) {
- sleep_time = std::min(sleep_time,
HRTIME_MSECONDS(thread_max_heartbeat_mseconds));
+ if (EventQueueExternal.localQueue.empty()) {
+sleep_time = std::min(sleep_time,
HRTIME_MSECONDS(thread_max_heartbeat_mseconds));
+ } else {
+// Because of a missed lock, Timed-Event and Negative-Event have been
pushed into localQueue for retry in awhile.
+// Therefore, we have to set the limitation of sleep time in order to
handle the next retry in time.
+sleep_time = std::min(sleep_time, DELAY_FOR_RETRY);
+ }
++(current_metric->_wait);
} else {
sleep_time = 0;
[trafficserver] branch 8.1.x updated: Updated ChangeLog
This is an automated email from the ASF dual-hosted git repository. zwoop pushed a commit to branch 8.1.x in repository https://gitbox.apache.org/repos/asf/trafficserver.git The following commit(s) were added to refs/heads/8.1.x by this push: new 8dfe11c Updated ChangeLog 8dfe11c is described below commit 8dfe11cc697c438d7d74c0a8e0c81fb49823eace Author: Leif Hedstrom AuthorDate: Thu Mar 26 14:55:31 2020 -0600 Updated ChangeLog --- CHANGELOG-8.1.0 | 21 - 1 file changed, 20 insertions(+), 1 deletion(-) diff --git a/CHANGELOG-8.1.0 b/CHANGELOG-8.1.0 index 056af85..22350e3 100644 --- a/CHANGELOG-8.1.0 +++ b/CHANGELOG-8.1.0 @@ -1,7 +1,14 @@ Changes with Apache Traffic Server 8.1.0 + #3899 - Add a --reason tag option to the host subcommand #4028 - proxy.config.http.connect_attempts_timeout tracks TTBF instead of connect #4203 - Allow enabling MPTCP on a listening socket through the records.config + #4217 - Fix a regression in the traffic_ctl host status subcommand + #4253 - Add the capability to mark origins down with traffic_ctl. + #4300 - Expand the post chunked tests. + #4422 - TCL: Rmove TCL dependency from HostStatus #4437 - Adds the log tag cqint for internal requests + #4441 - Corrects the wording of the usage messages for traffic_ctl host subcommand + #4664 - Use hash string when it is provided for hashing #4762 - Fix OCSP stapling with dual cert #4869 - Cleanup: remove lib/cppapi from clang-format, it's been moved. #4874 - Avoid ats_malloc in unmarshal @@ -9,18 +16,25 @@ Changes with Apache Traffic Server 8.1.0 #5058 - Fixed use after free when running regression under ASAN #5122 - fix crash in CacheVC::openReadFromWriter #5131 - Do not run clang-format for @default_stack_size@ + #5143 - Fix a self detection issue where parents are not marked down during ATS startup. #5149 - Add nullptr check on ConfigProcessor::release() #5163 - Fix memleak in ProcessManager::stop() #5209 - Allows logging of negative values #5227 - Normalize on negative value for milestone metrics #5234 - Correct interpretation of proxy.config.ssl.client.verify.server - #5235 - Fix lost pending_actions causing actions on stale objects. #5236 - Update documentation for SSL VERIFY hooks. #5242 - Adds a missing header field for the tcpinfo log + #5252 - Make host status persistent across restarts #5329 - Fixes clang-analyzer error dereferencing nullptr in parent #5348 - Don't assign if this and other are the same object #5354 - Password can not be nullptr here, so don't check + #5417 - Change HostStatus to use only one stat per host. + #5585 - Do not change stream state from HALF_CLOSED_REMOTE with CONTINUATION frame #5811 - Fix no_activity timeout for server session reuse. + #5865 - cachekey: added --canonical-prefix parameter + #5872 - cachekey: added --key-type to allow parent selection URL to be modified + #5921 - Cleanup AuTest for HTTP/2 + #5945 - cachekey: allow multiple values for `--key-type` #6154 - tslua: Exposes set/get method for server request objects #6182 - tslua: Exposes set/get version for server request objects #6191 - Limit this check to Linux, where ldd always works @@ -29,5 +43,10 @@ Changes with Apache Traffic Server 8.1.0 #6328 - Document the order of the X-Cache header #6343 - Fixes an assertion failure when proxy.config.http.no_dns_just_forward_to_parent is enabled #6392 - Free TSMgmtString after using it. + #6442 - Disable the chunked_encoding test + #6518 - Fixes a bug where HostStatus::getHostStatus() creates too many stats. #6534 - Ignore MgmtMessageHdr if ProcessManager is not running #6535 - Add MIMEHdr Garbage Collection to HPACK Dynamic Table + #6541 - Add cachekey latest features to 8.1.x + #6553 - Options to roll empty logs and log trimming (8.1.x) + #6557 - free(map); -> delete map;
[trafficserver] branch 8.1.x updated (37c56b0 -> 556d173)
This is an automated email from the ASF dual-hosted git repository. zwoop pushed a change to branch 8.1.x in repository https://gitbox.apache.org/repos/asf/trafficserver.git. from 37c56b0 Adjusts the format for header_rewrite string concatenation add 2eb38aa back port of PR 3899: add a reason tag to traffic_control host subcommand. add 7d69676 backport of PR 4217: Fix a regression in the traffic_ctl host status subcommand. Host statuses were not being found as the reason tag was left off. add 6139bf4 backport of PR 4441: Corrects the wording of the usage messages for traffic_ctl host subcommand. add 6d9e3fa backport of PR 4422: add cc57201 Back port of PR 4664: Use optionally provided hash string for hashing add 434fd42 Back port of PR 5143: add ac2c105 This PR fixes #5248 - parent host status is not persistent accross restarts. add a788867 Adds a HostStatus::loadStats() function used to load persisted HostStatus stats after a restart of trafficserver. add becd388 For Nexthop, add the capability to mark down origin servers utilizing traffic_ctl. add 64ba5ba Change HostStatus to use only one stat per host. The host stat is now a string value containing all time and reason data so that it may be restored from persistent store when ATS is restarted. add 556d173 Fixes a bug where getHostStatus() will create a host status record when none is found. A Host status record should only be created when a host is marked up or down when traffic_ctl is used to mark a host up or down. No new revisions were added by this update. Summary of changes: doc/admin-guide/files/parent.config.en.rst | 8 + doc/appendices/command-line/traffic_ctl.en.rst | 41 +- iocore/cache/test/stub.cc | 264 + mgmt/LocalManager.cc | 4 +- mgmt/LocalManager.h| 2 +- mgmt/api/CoreAPI.cc| 11 +- mgmt/api/CoreAPI.h | 4 +- mgmt/api/CoreAPIRemote.cc | 12 +- mgmt/api/INKMgmtAPI.cc | 8 +- mgmt/api/NetworkMessage.cc | 4 +- mgmt/api/TSControlMain.cc | 13 +- mgmt/api/include/mgmtapi.h | 4 +- proxy/HostStatus.h | 161 +++- proxy/ParentConsistentHash.cc | 12 +- proxy/ParentRoundRobin.cc | 3 +- proxy/ParentSelection.cc | 144 --- proxy/ParentSelection.h| 1 + proxy/http/HttpTransact.cc | 40 +- src/traffic_ctl/Makefile.inc | 1 + src/traffic_ctl/host.cc| 68 +++- src/traffic_server/HostStatus.cc | 413 + src/traffic_server/traffic_server.cc | 2 +- .../header_rewrite/gold/header_rewrite-303.gold| 2 +- 23 files changed, 1014 insertions(+), 208 deletions(-) create mode 100644 iocore/cache/test/stub.cc
[trafficserver] branch 8.1.x updated: Do not change stream state from HALF_CLOSED_REMOTE with CONTINUATION frame
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 8.1.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
The following commit(s) were added to refs/heads/8.1.x by this push:
new b8a82a4 Do not change stream state from HALF_CLOSED_REMOTE with
CONTINUATION frame
b8a82a4 is described below
commit b8a82a4250e274ae6ed1603920a9bd54a9ef2c1c
Author: Susan Hinrichs
AuthorDate: Mon Sep 24 13:19:47 2018 -0500
Do not change stream state from HALF_CLOSED_REMOTE with CONTINUATION frame
This is a combination of 3 commits.
1. Expand the post chunked tests.
(cherry picked from commit a523a62e2f309694bd6811f8b35eecbc678c24fb)
2. Do not change stream state from HALF_CLOSED_REMOTE with CONTINUATION
frame
(cherry picked from commit d765d1a33328b43ca83ecc9d0ae39ac01a307a88)
3. Cleanup AuTest for HTTP/2
(cherry picked from commit 76cadf55ece2253c0ddf01a172c57c346f8cd89a)
Conflicts:
tests/gold_tests/h2/http2.test.py
---
proxy/http2/Http2Stream.cc| 3 +
tests/gold_tests/h2/gold/http2_8_stderr.gold | 18 +
tests/gold_tests/h2/gold/http2_8_stdout.gold | 1 +
tests/gold_tests/h2/http2.test.py | 96 ---
tests/gold_tests/h2/rules/huge_resp_hdrs.conf | 24 +++
5 files changed, 117 insertions(+), 25 deletions(-)
diff --git a/proxy/http2/Http2Stream.cc b/proxy/http2/Http2Stream.cc
index d1c37f6..5af0ef1 100644
--- a/proxy/http2/Http2Stream.cc
+++ b/proxy/http2/Http2Stream.cc
@@ -250,6 +250,9 @@ Http2Stream::change_state(uint8_t type, uint8_t flags)
} else if (type == HTTP2_FRAME_TYPE_HEADERS) { // w/o END_STREAM flag
// No state change here. Expect a following DATA frame with END_STREAM
flag.
return true;
+} else if (type == HTTP2_FRAME_TYPE_CONTINUATION) { // w/o END_STREAM flag
+ // No state change here. Expect a following DATA frame with END_STREAM
flag.
+ return true;
} else {
// Error, set state closed
_state = Http2StreamState::HTTP2_STREAM_STATE_CLOSED;
diff --git a/tests/gold_tests/h2/gold/http2_8_stderr.gold
b/tests/gold_tests/h2/gold/http2_8_stderr.gold
new file mode 100644
index 000..f5d57eb
--- /dev/null
+++ b/tests/gold_tests/h2/gold/http2_8_stderr.gold
@@ -0,0 +1,18 @@
+``
+> GET /huge_resp_hdrs HTTP/2
+> Host: ``
+> User-Agent: curl/``
+> Accept: */*
+``
+< HTTP/2 200 ``
+< server: ATS/``
+< content-length: 6
+< date: ``
+< age: ``
+< x-huge-0:
0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789ab
[...]
+< x-huge-1:
0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789ab
[...]
+< x-huge-2:
0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789ab
[...]
+< x-huge-3:
0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789ab
[...]
+< x-huge-4:
[trafficserver] 02/03: Fixes memory leaks in header_rewrite
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 8.1.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
commit a267fca18500a035235d1b505074a5a2adc06605
Author: Randall Meyer
AuthorDate: Mon Oct 22 16:27:52 2018 -0700
Fixes memory leaks in header_rewrite
(cherry picked from commit 8f820c067f92590a8fece61cce98b92f24adb99b)
Conflicts:
plugins/header_rewrite/value.h
---
plugins/header_rewrite/condition.h | 2 +-
plugins/header_rewrite/value.cc| 8
plugins/header_rewrite/value.h | 5 +
3 files changed, 10 insertions(+), 5 deletions(-)
diff --git a/plugins/header_rewrite/condition.h
b/plugins/header_rewrite/condition.h
index 1164271..73e3297 100644
--- a/plugins/header_rewrite/condition.h
+++ b/plugins/header_rewrite/condition.h
@@ -118,7 +118,7 @@ public:
return _qualifier;
}
- // Virtual methods, has to be implemented by each conditional
+ // Virtual methods, has to be implemented by each conditional;
void initialize(Parser ) override;
virtual void append_value(std::string , const Resources ) = 0;
diff --git a/plugins/header_rewrite/value.cc b/plugins/header_rewrite/value.cc
index 6dadde0..ef15e43 100644
--- a/plugins/header_rewrite/value.cc
+++ b/plugins/header_rewrite/value.cc
@@ -30,6 +30,14 @@
#include "parser.h"
#include "conditions.h"
+Value::~Value()
+{
+ TSDebug(PLUGIN_NAME_DBG, "Calling DTOR for Value");
+ for (auto it = _cond_vals.begin(); it != _cond_vals.end(); it++) {
+delete *it;
+ }
+}
+
void
Value::set_value(const std::string )
{
diff --git a/plugins/header_rewrite/value.h b/plugins/header_rewrite/value.h
index 76bee5f..23ec557 100644
--- a/plugins/header_rewrite/value.h
+++ b/plugins/header_rewrite/value.h
@@ -44,10 +44,7 @@ public:
TSDebug(PLUGIN_NAME_DBG, "Calling CTOR for Value");
}
- virtual ~Value()
- {
-TSDebug(PLUGIN_NAME_DBG, "Calling DTOR for Value");
- }
+ virtual ~Value();
void set_value(const std::string );
[trafficserver] branch 8.1.x updated (21381d7 -> 37c56b0)
This is an automated email from the ASF dual-hosted git repository. zwoop pushed a change to branch 8.1.x in repository https://gitbox.apache.org/repos/asf/trafficserver.git. from 21381d7 Revert "Fix lost pending_actions causing actions on stale objects." new 3d78a00 header_rewrite: Adds basic string concatenation new a267fca Fixes memory leaks in header_rewrite new 37c56b0 Adjusts the format for header_rewrite string concatenation The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: doc/admin-guide/plugins/header_rewrite.en.rst | 23 --- plugins/header_rewrite/Examples/Geo | 2 +- plugins/header_rewrite/Makefile.inc | 1 + plugins/header_rewrite/conditions.cc | 46 ++ plugins/header_rewrite/conditions.h | 34 ++ plugins/header_rewrite/factory.cc | 2 +- plugins/header_rewrite/header_rewrite.cc | 2 +- plugins/header_rewrite/header_rewrite_test.cc | 92 +-- plugins/header_rewrite/parser.cc | 78 +-- plugins/header_rewrite/parser.h | 15 + plugins/header_rewrite/regex_helper.cc| 12 plugins/header_rewrite/regex_helper.h | 2 - plugins/header_rewrite/value.cc | 73 + plugins/header_rewrite/value.h| 40 +++- 14 files changed, 354 insertions(+), 68 deletions(-) create mode 100644 plugins/header_rewrite/value.cc
[trafficserver] 03/03: Adjusts the format for header_rewrite string concatenation
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 8.1.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
commit 37c56b0b2fcd8b89b066baafdb8e498ea72bcc9e
Author: Randall Meyer
AuthorDate: Mon Nov 5 16:52:28 2018 -0800
Adjusts the format for header_rewrite string concatenation
Drop the "+" from the syntax. This syntax was never released and
therefore doesn't have backwards compatibilty requirements.
eg
add-header X-Party "let's party like it's %{NOW:YEAR}!"
(quotes optional)
(cherry picked from commit 3a5763b1c0f649ec9fb2fe90c8c740b28ce0874e)
Conflicts:
doc/admin-guide/plugins/header_rewrite.en.rst
---
doc/admin-guide/plugins/header_rewrite.en.rst | 9 ++-
plugins/header_rewrite/header_rewrite.cc | 4 +-
plugins/header_rewrite/header_rewrite_test.cc | 91 +--
plugins/header_rewrite/parser.cc | 71 +
plugins/header_rewrite/parser.h | 16 -
plugins/header_rewrite/regex_helper.cc| 12
plugins/header_rewrite/regex_helper.h | 2 -
plugins/header_rewrite/value.cc | 30 -
8 files changed, 186 insertions(+), 49 deletions(-)
diff --git a/doc/admin-guide/plugins/header_rewrite.en.rst
b/doc/admin-guide/plugins/header_rewrite.en.rst
index 128f457..12f9b4a 100644
--- a/doc/admin-guide/plugins/header_rewrite.en.rst
+++ b/doc/admin-guide/plugins/header_rewrite.en.rst
@@ -806,9 +806,14 @@ QSAAppend the results of the rule to the query string.
Values and Variable Expansion
-
-You can concatenate values using strings, condition values and variable
expansions via the ``+`` operator. Variables (eg %) in the concatenation
must be enclosed in double quotes ``"``::
-add-header CustomHeader "Hello from " + %{IP:SERVER} + ":" +
"%"
+You can concatenate values using strings, condition values and variable
expansions on the same line.
+
+add-header CustomHeader "Hello from %{IP:SERVER}:%"
+
+However, the above example is somewhat contrived to show the old tags, it
should instead be written as
+
+add-header CustomHeader "Hello from %{IP:SERVER}:%{INBOUND:LOCAL-PORT}"
Concatenation is not supported in condition testing.
diff --git a/plugins/header_rewrite/header_rewrite.cc
b/plugins/header_rewrite/header_rewrite.cc
index af19305..60caecd 100644
--- a/plugins/header_rewrite/header_rewrite.cc
+++ b/plugins/header_rewrite/header_rewrite.cc
@@ -184,7 +184,7 @@ RulesConfig::parse_config(const std::string ,
TSHttpHookID default_hook)
continue;
}
-Parser p(line, true); // Tokenize and parse this line preserving quotes
from input
+Parser p(line); // Tokenize and parse this line
if (p.empty()) {
continue;
}
@@ -470,6 +470,6 @@ TSRemapDoRemap(void *ih, TSHttpTxn rh, TSRemapRequestInfo
*rri)
rule = rule->next;
}
- TSDebug(PLUGIN_NAME_DBG, "Returing from TSRemapDoRemap with status: %d",
rval);
+ TSDebug(PLUGIN_NAME_DBG, "Returning from TSRemapDoRemap with status: %d",
rval);
return rval;
}
diff --git a/plugins/header_rewrite/header_rewrite_test.cc
b/plugins/header_rewrite/header_rewrite_test.cc
index c81113a..6a4e181 100644
--- a/plugins/header_rewrite/header_rewrite_test.cc
+++ b/plugins/header_rewrite/header_rewrite_test.cc
@@ -33,6 +33,12 @@ const char PLUGIN_NAME_DBG[] = "TEST_dbg_header_rewrite";
extern "C" void
TSError(const char *fmt, ...)
{
+ va_list args;
+
+ va_start(args, fmt);
+ vfprintf(stderr, fmt, args);
+ va_end(args);
+ fprintf(stderr, "\n");
}
class ParserTest : public Parser
@@ -40,7 +46,7 @@ class ParserTest : public Parser
public:
ParserTest(const std::string ) : Parser(line), res(true) { std::cout <<
"Finished parser test: " << line << std::endl; }
std::vector
- getTokens()
+ getTokens() const
{
return _tokens;
}
@@ -58,17 +64,38 @@ public:
bool res;
};
+class SimpleTokenizerTest : public SimpleTokenizer
+{
+public:
+ SimpleTokenizerTest(const std::string ) : SimpleTokenizer(line),
res(true)
+ {
+std::cout << "Finished tokenizer test: " << line << std::endl;
+ }
+
+ template
+ void
+ do_parser_check(T x, U y, int line = 0)
+ {
+if (x != y) {
+ std::cerr << "CHECK FAILED on line " << line << ": |" << x << "| != |"
<< y << "|" << std::endl;
+ res = false;
+}
+ }
+
+ bool res;
+};
+
#define CHECK_EQ(x, y) \
do { \
p.do_parser_check((x), (y), __LINE__); \
- } while (false);
+ } while (false)
#define END_TEST(s) \
do { \
if (!p.res) { \
++errors; \
} \
- } while (false);
+ } while (false)
int
test_parsing()
@@ -336,6 +363,17 @@ test_parsing()
END_TEST();
}
+ {
+ParserTest p(R"(set-header Alt-Svc "quic=\":443\"; v=\"35\"")");
+
+
[trafficserver] 01/03: header_rewrite: Adds basic string concatenation
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 8.1.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
commit 3d78a006d523e3ec7818bb457e71449d4d2af7ba
Author: Randall Meyer
AuthorDate: Tue Sep 11 10:56:56 2018 -0700
header_rewrite: Adds basic string concatenation
eg:
add-header X-Party "let's party like it's " + %{NOW:YEAR} + "!"
(cherry picked from commit ed1ded2623674bf6957a0faeb68a8ca10cb0725f)
Conflicts:
plugins/header_rewrite/value.h
---
doc/admin-guide/plugins/header_rewrite.en.rst | 18
plugins/header_rewrite/Examples/Geo | 2 +-
plugins/header_rewrite/Makefile.inc | 1 +
plugins/header_rewrite/condition.h| 2 +-
plugins/header_rewrite/conditions.cc | 46 +++
plugins/header_rewrite/conditions.h | 34 ++
plugins/header_rewrite/factory.cc | 2 +-
plugins/header_rewrite/header_rewrite.cc | 2 +-
plugins/header_rewrite/header_rewrite_test.cc | 1 +
plugins/header_rewrite/parser.cc | 29 +---
plugins/header_rewrite/parser.h | 3 +-
plugins/header_rewrite/value.cc | 65 +++
plugins/header_rewrite/value.h| 35 ---
13 files changed, 192 insertions(+), 48 deletions(-)
diff --git a/doc/admin-guide/plugins/header_rewrite.en.rst
b/doc/admin-guide/plugins/header_rewrite.en.rst
index 25f6c52..128f457 100644
--- a/doc/admin-guide/plugins/header_rewrite.en.rst
+++ b/doc/admin-guide/plugins/header_rewrite.en.rst
@@ -804,11 +804,15 @@ L Last rule, do not continue.
QSAAppend the results of the rule to the query string.
==
-Variable Expansion
---
+Values and Variable Expansion
+-
+You can concatenate values using strings, condition values and variable
expansions via the ``+`` operator. Variables (eg %) in the concatenation
must be enclosed in double quotes ``"``::
+
+add-header CustomHeader "Hello from " + %{IP:SERVER} + ":" +
"%"
+
+Concatenation is not supported in condition testing.
-Only limited variable expansion is supported in `add-header`_ and
`set-header`_ . Supported
-substitutions are currently:
+Supported substitutions are currently:
===
==
VariableDescription
@@ -833,12 +837,6 @@ VariableDescription
%The full protocol stack of the inbound connection
separated by ','.
===
==
-Variables to be expanded must be enclosed in double quotes ``"``, as in this
example using the arbitrary header
-``Custom-Client-IP``::
-
-set-header Custom-Client-IP "%"
-
-
Header Values
-
diff --git a/plugins/header_rewrite/Examples/Geo
b/plugins/header_rewrite/Examples/Geo
index 3ea3a60..d4261e1 100644
--- a/plugins/header_rewrite/Examples/Geo
+++ b/plugins/header_rewrite/Examples/Geo
@@ -16,7 +16,7 @@
# limitations under the License.
cond %{SEND_RESPONSE_HDR_HOOK} [AND]
-cond %${GEO:COUNTRY} =US
+cond %{GEO:COUNTRY} =US
set-header ATS-Geo-Country %{GEO:COUNTRY}
set-header ATS-Geo-Country-ISO %{GEO:COUNTRY-ISO}
set-header ATS-Geo-ASN %{GEO:ASN}
diff --git a/plugins/header_rewrite/Makefile.inc
b/plugins/header_rewrite/Makefile.inc
index 436e3da..c4b81d6 100644
--- a/plugins/header_rewrite/Makefile.inc
+++ b/plugins/header_rewrite/Makefile.inc
@@ -43,6 +43,7 @@ header_rewrite_header_rewrite_la_SOURCES = \
header_rewrite/ruleset.h \
header_rewrite/statement.cc \
header_rewrite/statement.h \
+ header_rewrite/value.cc \
header_rewrite/value.h
header_rewrite_parser_la_SOURCES = \
diff --git a/plugins/header_rewrite/condition.h
b/plugins/header_rewrite/condition.h
index 73e3297..1164271 100644
--- a/plugins/header_rewrite/condition.h
+++ b/plugins/header_rewrite/condition.h
@@ -118,7 +118,7 @@ public:
return _qualifier;
}
- // Virtual methods, has to be implemented by each conditional;
+ // Virtual methods, has to be implemented by each conditional
void initialize(Parser ) override;
virtual void append_value(std::string , const Resources ) = 0;
diff --git a/plugins/header_rewrite/conditions.cc
b/plugins/header_rewrite/conditions.cc
index 4b6cda6..c80d985 100644
--- a/plugins/header_rewrite/conditions.cc
+++ b/plugins/header_rewrite/conditions.cc
@@ -30,6 +30,7 @@
#include "ts/ts.h"
#include "conditions.h"
+#include "expander.h"
#include "lulu.h"
// ConditionStatus
@@ -1383,3 +1384,48 @@ ConditionInbound::append_value(std::string , const
Resources , NetworkSess
s += zret;
}
}
+
[trafficserver] branch 8.1.x updated (1aadff3 -> 21381d7)
This is an automated email from the ASF dual-hosted git repository. zwoop pushed a change to branch 8.1.x in repository https://gitbox.apache.org/repos/asf/trafficserver.git. from 1aadff3 Options to roll empty logs and log trimming (8.1.x) (#6553) add 21381d7 Revert "Fix lost pending_actions causing actions on stale objects." No new revisions were added by this update. Summary of changes: iocore/net/P_UnixNetVConnection.h | 7 --- proxy/http/HttpSM.cc | 29 +++-- 2 files changed, 7 insertions(+), 29 deletions(-)
[trafficserver] branch 8.1.x updated (2a70ee8 -> 1aadff3)
This is an automated email from the ASF dual-hosted git repository. zwoop pushed a change to branch 8.1.x in repository https://gitbox.apache.org/repos/asf/trafficserver.git. from 2a70ee8 cachekey plugin docs and clang-format/tidy changes add 1aadff3 Options to roll empty logs and log trimming (8.1.x) (#6553) No new revisions were added by this update. Summary of changes: doc/admin-guide/files/records.config.en.rst | 25 doc/admin-guide/logging/rotation.en.rst | 24 mgmt/RecordsConfig.cc | 4 ++ proxy/logging/LogConfig.cc | 11 proxy/logging/LogConfig.h | 2 + proxy/logging/LogFile.cc| 89 - proxy/logging/LogFile.h | 3 +- proxy/logging/LogObject.cc | 25 ++-- proxy/logging/LogObject.h | 16 +++--- proxy/logging/YamlLogConfig.cc | 32 --- src/traffic_server/InkAPI.cc| 8 +-- 11 files changed, 214 insertions(+), 25 deletions(-)
[trafficserver] branch master updated: Convert proxy/logging unit tests to use Catch.
This is an automated email from the ASF dual-hosted git repository.
rrm pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
The following commit(s) were added to refs/heads/master by this push:
new 55abbd7 Convert proxy/logging unit tests to use Catch.
55abbd7 is described below
commit 55abbd77910842979dec2f1e487f6aa2bba59103
Author: Walter Karas
AuthorDate: Fri Mar 20 20:16:15 2020 -0500
Convert proxy/logging unit tests to use Catch.
---
proxy/logging/Makefile.am | 24 +
proxy/logging/test_LogUtils.cc | 104 -
.../{test_LogUtils2.cc => test_LogUtils.cc}| 62 ++--
proxy/logging/unit-tests/test_LogUtils.h | 2 +-
4 files changed, 61 insertions(+), 131 deletions(-)
diff --git a/proxy/logging/Makefile.am b/proxy/logging/Makefile.am
index 58b6b55..b825094 100644
--- a/proxy/logging/Makefile.am
+++ b/proxy/logging/Makefile.am
@@ -69,37 +69,21 @@ liblogging_a_SOURCES = \
check_PROGRAMS = \
test_LogUtils \
- test_LogUtils2 \
test_RolledLogDeleter
-TESTS = \
- test_LogUtils \
- test_LogUtils2 \
- test_RolledLogDeleter
+TESTS = $(check_PROGRAMS)
test_LogUtils_CPPFLAGS = \
$(AM_CPPFLAGS) \
- -DTEST_LOG_UTILS
-
-test_LogUtils_SOURCES = \
- test_LogUtils.cc
-
-test_LogUtils_LDADD = \
- $(top_builddir)/src/tscore/libtscore.la \
- $(top_builddir)/src/tscpp/util/libtscpputil.la \
- $(top_builddir)/iocore/eventsystem/libinkevent.a
-
-test_LogUtils2_CPPFLAGS = \
- $(AM_CPPFLAGS) \
-DTEST_LOG_UTILS \
-I$(abs_top_srcdir)/tests/include
-test_LogUtils2_SOURCES = \
+test_LogUtils_SOURCES = \
LogUtils.cc \
unit-tests/BufferWriterFormat.cc \
- unit-tests/test_LogUtils2.cc
+ unit-tests/test_LogUtils.cc
-test_LogUtils2_LDADD = \
+test_LogUtils_LDADD = \
$(top_builddir)/src/tscore/libtscore.la \
$(top_builddir)/src/tscpp/util/libtscpputil.la \
$(top_builddir)/iocore/eventsystem/libinkevent.a
diff --git a/proxy/logging/test_LogUtils.cc b/proxy/logging/test_LogUtils.cc
deleted file mode 100644
index 02b8cfe..000
--- a/proxy/logging/test_LogUtils.cc
+++ /dev/null
@@ -1,104 +0,0 @@
-/** @file
-
- A brief file description
-
- @section license License
-
- Licensed to the Apache Software Foundation (ASF) under one
- or more contributor license agreements. See the NOTICE file
- distributed with this work for additional information
- regarding copyright ownership. The ASF licenses this file
- to you under the Apache License, Version 2.0 (the
- "License"); you may not use this file except in compliance
- with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
- */
-
-#include "tscore/TestBox.h"
-#include "LogUtils.cc"
-#include
-
-#if 0
-// Stub
-EThread *
-this_ethread()
-{
- return nullptr;
-}
-#endif
-
-REGRESSION_TEST(LogUtils_pure_escapify_url)(RegressionTest *t, int /* atype
ATS_UNUSED */, int *pstatus)
-{
- TestBox box(t, pstatus);
- box = REGRESSION_TEST_PASSED;
-
- char input[][32] = {
-" ",
-"%",
-"% ",
-"%20",
- };
- const char *expected[] = {
-"%20",
-"%25",
-"%25%20",
-"%2520",
- };
- char output[128];
- int output_len;
-
- int n = sizeof(input) / sizeof(input[0]);
- for (int i = 0; i < n; ++i) {
-LogUtils::pure_escapify_url(NULL, input[i], strlen(input[i]), _len,
output, 128);
-box.check(strcmp(output, expected[i]) == 0, "expected \"%s\", was \"%s\"",
expected[i], output);
- }
-}
-
-REGRESSION_TEST(LogUtils_escapify_url)(RegressionTest *t, int /* atype
ATS_UNUSED */, int *pstatus)
-{
- TestBox box(t, pstatus);
- box = REGRESSION_TEST_PASSED;
-
- char input[][32] = {
-" ",
-"%",
-"% ",
-"%20",
- };
- const char *expected[] = {
-"%20",
-"%25",
-"%25%20",
-"%20",
- };
- char output[128];
- int output_len;
-
- int n = sizeof(input) / sizeof(input[0]);
- for (int i = 0; i < n; ++i) {
-LogUtils::escapify_url(NULL, input[i], strlen(input[i]), _len,
output, 128);
-box.check(strcmp(output, expected[i]) == 0, "expected \"%s\", was \"%s\"",
expected[i], output);
- }
-}
-
-int
-main(int argc, const char **argv)
-{
- return RegressionTest::main(argc, argv, REGRESSION_TEST_QUICK);
-}
-
-//
-// Stub
-//
-void
-RecSignalManager(int, const char *, unsigned long)
-{
- ink_release_assert(false);
-}
diff --git a/proxy/logging/unit-tests/test_LogUtils2.cc
b/proxy/logging/unit-tests/test_LogUtils.cc
similarity index 80%
rename
[trafficserver] branch master updated: Convert single regression test in iocore/eventsystem/SocketManager.cc to Catch.
This is an automated email from the ASF dual-hosted git repository.
rrm pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
The following commit(s) were added to refs/heads/master by this push:
new 26daeab Convert single regression test in
iocore/eventsystem/SocketManager.cc to Catch.
26daeab is described below
commit 26daeab82f350404b0d097c332f2b66bc8f7bfba
Author: Walter Karas
AuthorDate: Wed Mar 25 13:09:31 2020 -0500
Convert single regression test in iocore/eventsystem/SocketManager.cc to
Catch.
---
iocore/eventsystem/SocketManager.cc | 16
iocore/eventsystem/unit_tests/test_EventSystem.cc | 21 +++--
2 files changed, 19 insertions(+), 18 deletions(-)
diff --git a/iocore/eventsystem/SocketManager.cc
b/iocore/eventsystem/SocketManager.cc
index 6dffbe1..9835519 100644
--- a/iocore/eventsystem/SocketManager.cc
+++ b/iocore/eventsystem/SocketManager.cc
@@ -29,7 +29,6 @@
#include "P_EventSystem.h"
#include "tscore/TextBuffer.h"
-#include "tscore/TestBox.h"
SocketManager socketManager;
@@ -127,18 +126,3 @@ SocketManager::fastopen_supported()
return value & TFO_CLIENT_ENABLE;
}
-
-REGRESSION_TEST(socket_fastopen)(RegressionTest *test, int level, int *pstatus)
-{
- TestBox box(test, pstatus);
-
- box = REGRESSION_TEST_PASSED;
-
- if (SocketManager::fastopen_supported()) {
-box.check(MSG_FASTOPEN != 0, "TCP Fast Open is supported, MSG_FASTOPEN
must not be 0");
- }
-
- if (::access("/proc/sys/net/ipv4/tcp_fastopen", F_OK) == 0) {
-box.check(MSG_FASTOPEN != 0, "TCP Fast Open is available, MSG_FASTOPEN
must not be 0");
- }
-}
diff --git a/iocore/eventsystem/unit_tests/test_EventSystem.cc
b/iocore/eventsystem/unit_tests/test_EventSystem.cc
index 75faad2..5de42b0 100644
--- a/iocore/eventsystem/unit_tests/test_EventSystem.cc
+++ b/iocore/eventsystem/unit_tests/test_EventSystem.cc
@@ -46,7 +46,7 @@ TEST_CASE("EventSystem", "[iocore]")
ink_atomic_increment((int *), 1);
EThread *e = this_ethread();
- printf("thread=%d (%p) count = %d\n", e->id, e, count);
+ std::printf("thread=%d (%p) count = %d\n", e->id, e, count);
return 0;
}
@@ -59,7 +59,7 @@ TEST_CASE("EventSystem", "[iocore]")
kill_function(int /* event ATS_UNUSED */, Event * /* e ATS_UNUSED */)
{
EThread *e = this_ethread();
- printf("thread=%d (%p) count is %d\n", e->id, e, count);
+ std::printf("thread=%d (%p) count is %d\n", e->id, e, count);
REQUIRE(count > 0);
REQUIRE(count <= TEST_TIME_SECOND * TEST_THREADS);
@@ -99,3 +99,20 @@ struct EventProcessorListener : Catch::TestEventListenerBase
{
};
CATCH_REGISTER_LISTENER(EventProcessorListener);
+
+TEST_CASE("EventSystemSocketManager", "[iocore][sock_mgr]")
+{
+ if (SocketManager::fastopen_supported()) {
+if (MSG_FASTOPEN == 0) {
+ std::printf("TCP Fast Open is supported, MSG_FASTOPEN must not be 0\n");
+ CHECK(false);
+}
+ }
+
+ if (::access("/proc/sys/net/ipv4/tcp_fastopen", F_OK) == 0) {
+if (MSG_FASTOPEN == 0) {
+ std::printf("TCP Fast Open is supported, MSG_FASTOPEN must not be 0\n");
+ CHECK(false);
+}
+ }
+}
