[trafficserver] branch 9.0.x updated: Fixed const issue with magick plugin

2019-08-30 Thread zwoop
This is an automated email from the ASF dual-hosted git repository.

zwoop pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git


The following commit(s) were added to refs/heads/9.0.x by this push:
 new 8afbe11  Fixed const issue with magick plugin
8afbe11 is described below

commit 8afbe11ff316e97c34eb1dd1bf69a4c945c5cf70
Author: Bryan Call 
AuthorDate: Tue Aug 27 15:58:51 2019 -0700

Fixed const issue with magick plugin

(cherry picked from commit 58a1451ba0ab6a06308dc6ac405da74395f9dff8)
---
 plugins/experimental/magick/magick.cc | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/plugins/experimental/magick/magick.cc 
b/plugins/experimental/magick/magick.cc
index 00f4755..2e6a18b 100644
--- a/plugins/experimental/magick/magick.cc
+++ b/plugins/experimental/magick/magick.cc
@@ -184,7 +184,7 @@ struct EVPKey {
   EVPKey() : key(EVP_PKEY_new()) { assert(nullptr != key); }
 
   bool
-  assign(const char *const k) const
+  assign(char *k) const
   {
 assert(nullptr != k);
 const int rc = EVP_PKEY_assign_RSA(key, k);
@@ -196,7 +196,7 @@ struct EVPKey {
   bool
   assign(T )
   {
-return assign(reinterpret_cast(t));
+return assign(reinterpret_cast(t));
   }
 };
 



[trafficserver] branch 9.0.x updated: cachekey: added --key-type (for parent selection)

2019-08-30 Thread zwoop
This is an automated email from the ASF dual-hosted git repository.

zwoop pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git


The following commit(s) were added to refs/heads/9.0.x by this push:
 new 57e5a4b  cachekey: added --key-type (for parent selection)
57e5a4b is described below

commit 57e5a4bd2619fd1a2a4e8568d4e84037e4e5df8a
Author: Gancho Tenev 
AuthorDate: Fri Aug 23 14:26:26 2019 -0700

cachekey: added --key-type (for parent selection)

Added ability to apply all transformations, available for modifying
the cache key, to parent selection URL:
* --key-type=cache_key - apply transformations to  cache key
* --key-type=parent_selection_url - apply transformations to
  parent selection URL

TODO/TBD: After this change all transformations can be applied not
only to the cache key but to parent selection URL as well. It would
make sense to give the cachekey plugin a new, more suitable name.

(cherry picked from commit af7299dd06a2a7b5e862edf5aea0693dd1bde2c1)
---
 doc/admin-guide/plugins/cachekey.en.rst | 167 +++-
 doc/admin-guide/plugins/index.en.rst|   4 +-
 plugins/cachekey/cachekey.cc|  69 ++---
 plugins/cachekey/cachekey.h |  10 +-
 plugins/cachekey/configs.cc |  62 +++-
 plugins/cachekey/configs.h  |  29 +-
 plugins/cachekey/plugin.cc  |   2 +-
 7 files changed, 248 insertions(+), 95 deletions(-)

diff --git a/doc/admin-guide/plugins/cachekey.en.rst 
b/doc/admin-guide/plugins/cachekey.en.rst
index 51f4ef1..3ee0948 100644
--- a/doc/admin-guide/plugins/cachekey.en.rst
+++ b/doc/admin-guide/plugins/cachekey.en.rst
@@ -21,22 +21,24 @@
 .. _admin-plugins-cachekey:
 
 
-Cache Key Manipulation Plugin
-*
+Cache Key and Parent Selection URL Manipulation Plugin
+**
 
 Description
 ===
 
-This plugin allows some common cache key manipulations based on various HTTP 
request components.  It can
+This plugin allows some common `cache key` or `parent selection URL` 
manipulations based on various HTTP request components.
+Although `cache key` is used everywhere in this document, the same 
manipulations can be applied to `parent selection URL`
+by switching `key type`_. The plugin can
 
 * sort query parameters to prevent query parameter reordering being a cache 
miss
-* ignore specific query parameters from the cache key by name or regular 
expression
-* ignore all query parameters from the cache key
-* only use specific query parameters in the cache key by name or regular 
expression
+* ignore specific query parameters from the `cache key` by name or regular 
expression
+* ignore all query parameters from the `cache key`
+* only use specific query parameters in the `cache key` by name or regular 
expression
 * include headers or cookies by name
 * capture values from the ``User-Agent`` header.
 * classify request using ``User-Agent`` and a list of regular expressions
-* capture and replace strings from the URI and include them in the cache key
+* capture and replace strings from the URI and include them in the `cache key`
 * do more - please find more examples below.
 
 URI type
@@ -46,6 +48,17 @@ The plugin manipulates the ``remap`` URI (value set during 
URI remap) by default
 
 * ``--uri-type=[remap|pristine]`` (default: ``remap``)
 
+Key type
+
+
+The plugin manipulates the `cache key` by default. If `parent selection URL` 
manipulation is needed the following option can be used:
+
+* ``--key-type=[cache_key|parent_selection_url]`` (default: ``cache_key``)
+
+One instance of this plugin can used either for `cache key` or `parent 
selection URL` manupulation but never both.
+If `simultaneous cache key and parent selection URL manipulation`_ is needed 
two separate instances of the plugin
+have to be loaded for each key type.
+
 Cache key structure and related plugin parameters
 =
 
@@ -59,11 +72,11 @@ Cache key structure and related plugin parameters
   │  (default)  |  (optional)  │  (optional)  │  (optional)  │  (default)  │  
(default)  │
   
└─┴──┴──┴──┴─┴─┘
 
-* The cache key set by the cachekey plugin can be considered as divided into 
several sections.
+* The `cache key` set by the cachekey plugin can be considered as divided into 
several sections.
 * Every section is manipulated separately by the related plugin parameters 
(more info in each section description below).
-* "User-Agent", "Headers" and "Cookies" sections are optional and will be 
missing from the cache key if no related plugin parameters are used.
+* "User-Agent", "Headers" and "Cookies" sections are optional and will be 
missing from the `cache key` if no related plugin parameters are used.
 * "Prefix", 

[trafficserver] branch 9.0.x updated: static linking asan, tsan, lsan

2019-08-30 Thread zwoop
This is an automated email from the ASF dual-hosted git repository.

zwoop pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git


The following commit(s) were added to refs/heads/9.0.x by this push:
 new 8d9d055  static linking asan, tsan, lsan
8d9d055 is described below

commit 8d9d0558b897e92f5f66d9583afa5d5b7fe80da0
Author: Fei Deng 
AuthorDate: Tue Aug 27 11:20:25 2019 -0500

static linking asan, tsan, lsan

(cherry picked from commit 211d6c9aaa5c9704bf8afed5b8b543f70002b2c8)
---
 configure.ac | 75 ++--
 1 file changed, 73 insertions(+), 2 deletions(-)

diff --git a/configure.ac b/configure.ac
index c77b284..112afa4 100644
--- a/configure.ac
+++ b/configure.ac
@@ -974,26 +974,97 @@ fi
 
 # Flags for ASAN
 if test "x${enable_asan}" = "xyes"; then
-  if test "x${enable_tsan}" = "xyes"; then
+  if test "x${enable_tsan}" = "xyes" -o "x${enable_tsan}" = "xstatic"; then
 AC_ERROR([Cannot have ASAN and TSAN options at the same time, pick one])
   fi
   TS_ADDTO(AM_CFLAGS, [-fno-omit-frame-pointer -fsanitize=address])
   TS_ADDTO(AM_CXXFLAGS, [-fno-omit-frame-pointer -fsanitize=address])
+elif test "x${enable_asan}" = "xstatic"; then
+  if test "x${enable_tsan}" = "xyes" -o "x${enable_tsan}" = "xstatic"; then
+AC_ERROR([Cannot have ASAN and TSAN options at the same time, pick one])
+  fi
+  asan_CXXFLAGS="$CXXFLAGS"
+  CXXFLAGS="$CXXFLAGS -fno-omit-frame-pointer -fsanitize=address 
-static-libasan"
+  AC_LANG_PUSH(C++)
+  AC_MSG_CHECKING([static ASAN library is available])
+  AC_LINK_IFELSE(
+[AC_LANG_PROGRAM([#include ], [])],
+[AC_MSG_RESULT([yes])],
+[
+  AC_MSG_RESULT([no])
+  AC_ERROR([Cannot find static ASAN library.])
+]
+  )
+  AC_LANG_POP
+  CXXFLAGS="$asan_CXXFLAGS"
+  TS_ADDTO(AM_CFLAGS, [-fno-omit-frame-pointer -fsanitize=address 
-static-libasan])
+  TS_ADDTO(AM_CXXFLAGS, [-fno-omit-frame-pointer -fsanitize=address 
-static-libasan])
 fi
 
 # Flags for LSAN stand-alone mode
 if test "x${enable_lsan}" = "xyes"; then
-  if test "x${enable_asan}" = "xyes"; then
+  if test "x${enable_asan}" = "xyes" -o "x${enable_asan}" = "xstatic"; then
 AC_ERROR([ASAN already specified, --enable-lsan is meant only for lsan 
stand-alone mode])
   fi
+  if test "x${enable_tsan}" = "xyes" -o "x${enable_tsan}" = "xstatic"; then
+AC_ERROR([Cannot have LSAN and TSAN options at the same time, pick one])
+  fi
   TS_ADDTO(AM_CFLAGS, [-fno-omit-frame-pointer -fsanitize=leak])
   TS_ADDTO(AM_CXXFLAGS, [-fno-omit-frame-pointer -fsanitize=leak])
+elif test "x${enable_lsan}" = "xstatic"; then
+  if test "x${enable_asan}" = "xyes" -o "x${enable_asan}" = "xstatic"; then
+AC_ERROR([ASAN already specified, --enable-lsan is meant only for lsan 
stand-alone mode])
+  fi
+  if test "x${enable_tsan}" = "xyes" -o "x${enable_tsan}" = "xstatic"; then
+AC_ERROR([Cannot have LSAN and TSAN options at the same time, pick one])
+  fi
+  AC_CHECK_LIB(lsan, _init, [lsan_have_libs=yes], [lsan_have_libs=no])
+  if test "x${lsan_have_libs}" == "xno"; then
+AC_ERROR([Cannot find LSAN static library])
+  fi
+  lsan_CXXFLAGS="$CXXFLAGS"
+  CXXFLAGS="$CXXFLAGS -fno-omit-frame-pointer -fsanitize=leak -static-liblsan"
+  AC_LANG_PUSH(C++)
+  AC_MSG_CHECKING([static LSAN library is available])
+  AC_LINK_IFELSE(
+[AC_LANG_PROGRAM([#include ], [])],
+[AC_MSG_RESULT([yes])],
+[
+  AC_MSG_RESULT([no])
+  AC_ERROR([Cannot find static LSAN library.])
+]
+  )
+  AC_LANG_POP
+  CXXFLAGS="$lsan_CXXFLAGS"
+  TS_ADDTO(AM_CFLAGS, [-fno-omit-frame-pointer -fsanitize=leak 
-static-liblsan])
+  TS_ADDTO(AM_CXXFLAGS, [-fno-omit-frame-pointer -fsanitize=leak 
-static-liblsan])
 fi
 
 # Flags for TSAN
 if test "x${enable_tsan}" = "xyes"; then
   TS_ADDTO(AM_CFLAGS, [-fsanitize=thread])
   TS_ADDTO(AM_CXXFLAGS, [-fsanitize=thread])
+elif test "x${enable_tsan}" = "xstatic"; then
+  AC_CHECK_LIB(tsan, _init, [tsan_have_libs=yes], [tsan_have_libs=no])
+  if test "x${tsan_have_libs}" == "xno"; then
+AC_ERROR([Cannot find TSAN static library])
+  fi
+  tsan_CXXFLAGS="$CXXFLAGS"
+  CXXFLAGS="$CXXFLAGS -fsanitize=thread -static-libtsan"
+  AC_LANG_PUSH(C++)
+  AC_MSG_CHECKING([static TSAN library is available])
+  AC_LINK_IFELSE(
+[AC_LANG_PROGRAM([#include ], [])],
+[AC_MSG_RESULT([yes])],
+[
+  AC_MSG_RESULT([no])
+  AC_ERROR([Cannot find static TSAN library.])
+]
+  )
+  AC_LANG_POP
+  CXXFLAGS="$tsan_CXXFLAGS"
+  TS_ADDTO(AM_CFLAGS, [-fsanitize=thread -static-libtsan])
+  TS_ADDTO(AM_CXXFLAGS, [-fsanitize=thread -static-libtsan])
 fi
 
 # Checks for pointer size.



[trafficserver] branch 9.0.x updated: Fix bad limit in poll loop. jtest -c1 now works again. Note: for poll() we must scan the entire array passed in because, unlink epoll() poll() does not populate t

2019-08-30 Thread zwoop
This is an automated email from the ASF dual-hosted git repository.

zwoop pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git


The following commit(s) were added to refs/heads/9.0.x by this push:
 new da2e0dc  Fix bad limit in poll loop.  jtest -c1 now works again. Note: 
for poll() we must scan the entire array passed in because, unlink epoll() 
poll() does not populate the output with only the ready fds, instead it sets 
event bits over the entire array which was passed in.
da2e0dc is described below

commit da2e0dc7a76855fadba7caeef76c5645219a4614
Author: John Plevyak 
AuthorDate: Fri Aug 23 11:44:42 2019 -0700

Fix bad limit in poll loop.  jtest -c1 now works again.
Note: for poll() we must scan the entire array passed in
because, unlink epoll() poll() does not populate the output
with only the ready fds, instead it sets event bits over the
entire array which was passed in.

Also fix port endian-ness in verbose message.

Signed-off-by: John Plevyak 
(cherry picked from commit 209a0b243303a60487ede65ae66b0c0c1cc94c02)
---
 tools/jtest/jtest.cc | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/tools/jtest/jtest.cc b/tools/jtest/jtest.cc
index 4253d04..b856469 100644
--- a/tools/jtest/jtest.cc
+++ b/tools/jtest/jtest.cc
@@ -606,7 +606,7 @@ static void
 poll_set(int sock, poll_cb read_cb, poll_cb write_cb = nullptr)
 {
   if (verbose) {
-printf("adding poll %d\n", sock);
+printf("adding poll %d %s %s\n", sock, read_cb ? "READ" : "-", write_cb ? 
"WRITE" : "-");
   }
   fd[sock].fd   = sock;
   fd[sock].read_cb  = read_cb;
@@ -1770,7 +1770,7 @@ open_server(unsigned short int port, accept_fn_t 
accept_fn)
   }
 
   if (verbose) {
-printf("opening server on %d port %d\n", sock, name.sin_port);
+printf("opening server on %d port %d\n", sock, port);
   }
 
   poll_init_set(sock, accept_fn);
@@ -1806,7 +1806,7 @@ poll_loop()
 if (ip >= POLL_GROUP_SIZE || i == last_fd) {
   int n = poll(pfd, ip, POLL_TIMEOUT);
   if (n > 0) {
-for (int j = 0; j < n; j++) {
+for (int j = 0; j < ip; j++) {
   if (pfd[j].revents & (POLLIN | POLLERR | POLLHUP | POLLNVAL)) {
 if (verbose) {
   printf("poll read %d %X\n", pfd[j].fd, pfd[j].revents);



[trafficserver] branch 9.0.x updated: Update HttpTransact.cc

2019-08-30 Thread zwoop
This is an automated email from the ASF dual-hosted git repository.

zwoop pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git


The following commit(s) were added to refs/heads/9.0.x by this push:
 new dbe00ba  Update HttpTransact.cc
dbe00ba is described below

commit dbe00ba98224c3acd5dd5064b9506f255cbff9ad
Author: YuanYingdong <1975643...@qq.com>
AuthorDate: Fri Aug 23 20:17:33 2019 +0800

Update HttpTransact.cc

Once we get hit stale and cannot connec t to source site but we decice to 
serve from cache ,we need to set s->source = SOURCE_CACHE or we will get 
coredump in HttpTransact::handle_content_length_header

(cherry picked from commit 5b700d270b021c9facf664914fdefb3887800975)
---
 proxy/http/HttpTransact.cc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/proxy/http/HttpTransact.cc b/proxy/http/HttpTransact.cc
index 3dbc176..2023688 100644
--- a/proxy/http/HttpTransact.cc
+++ b/proxy/http/HttpTransact.cc
@@ -3704,7 +3704,7 @@ HttpTransact::handle_server_connection_not_open(State *s)
 ink_assert(s->cache_info.object_read != nullptr);
 ink_assert(s->cache_info.action == CACHE_DO_UPDATE || s->cache_info.action 
== CACHE_DO_SERVE);
 ink_assert(s->internal_msg_buffer == nullptr);
-
+s->source = SOURCE_CACHE;
 TxnDebug("http_trans", "[hscno] serving stale doc to client");
 build_response_from_cache(s, HTTP_WARNING_CODE_REVALIDATION_FAILED);
   } else {



[trafficserver] branch 9.0.x updated: PR#5867: Explain how to use open_con().

2019-08-30 Thread zwoop
This is an automated email from the ASF dual-hosted git repository.

zwoop pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git


The following commit(s) were added to refs/heads/9.0.x by this push:
 new 93ee51a  PR#5867: Explain how to use open_con().
93ee51a is described below

commit 93ee51a65f0cef90be03d953b59066270619274f
Author: Oknet Xu 
AuthorDate: Fri Aug 23 17:06:30 2019 +0800

PR#5867: Explain how to use open_con().

The DNS sub-system has 2 work modes:

- Primary-Secondary mode
- Round-Robin mode

The `open_con()` and `open_cons()` are shared with these 2 modes.

Within the Primary-Secondary mode:

- The `DNSHandler::ip` is used to store the Primary DNS server address.
- The `validate_ip()` is the API to load the first DNS server address
into The `DNSHandler::ip`.
- The `open_con()` is the API to update `DNSHandler::ip` with `target`
and connect to the address, but you cannot update `DNSHandler::ip` with
it self.

Within the Round-Robin mode:

- The `DNSHandler::ip` is useless.
- The `open_con()` is the API to connect to `target`.

The SplitDNS is only available in the Primary-Secondary mode:

- I suspect that the Round-Robin mode is a feature that was added later.
- The author only made this functionality for the DNS sub-system, and
forgot to do it for SplitDNS.

(cherry picked from commit 678ab7a778d66f83475b478868f254aa62ff8344)
---
 iocore/dns/DNS.cc | 35 ---
 1 file changed, 32 insertions(+), 3 deletions(-)

diff --git a/iocore/dns/DNS.cc b/iocore/dns/DNS.cc
index 8d91744..6713003 100644
--- a/iocore/dns/DNS.cc
+++ b/iocore/dns/DNS.cc
@@ -468,6 +468,15 @@ DNSHandler::open_cons(sockaddr const *target, bool failed, 
int icon)
   Open (and close) connections as necessary and also assures that the
   epoll fd struct is properly updated.
 
+  target == nullptr :
+  open connection to DNSHandler::ip.
+  generally, the icon should be 0 if target == nullptr.
+
+  target != nullptr and icon == 0 :
+  open connection to target, and the target is assigned to DNSHandler::ip.
+
+  target != nullptr and icon > 0 :
+  open connection to target.
 */
 void
 DNSHandler::open_con(sockaddr const *target, bool failed, int icon, bool 
over_tcp)
@@ -475,6 +484,8 @@ DNSHandler::open_con(sockaddr const *target, bool failed, 
int icon, bool over_tc
   ip_port_text_buffer ip_text;
   PollDescriptor *pd = get_PollDescriptor(dnsProcessor.thread);
 
+  ink_assert(target != );
+
   if (!icon && target) {
 ats_ip_copy(, target);
   } else if (!target) {
@@ -549,6 +560,12 @@ DNSHandler::startEvent(int /* event ATS_UNUSED */, Event 
*e)
 dns_handler_initialized = 1;
 SET_HANDLER(::mainEvent);
 if (dns_ns_rr) {
+  /* Round Robin mode:
+   *   Establish a connection to each DNS server to make it a connection 
pool.
+   *   For each DNS Request, a connection is picked up from the pool by 
round robin method.
+   *
+   *   The first DNS server is assigned to DNSHandler::ip within 
open_con() function.
+   */
   int max_nscount = m_res->nscount;
   if (max_nscount > MAX_NAMED) {
 max_nscount = MAX_NAMED;
@@ -565,6 +582,18 @@ DNSHandler::startEvent(int /* event ATS_UNUSED */, Event 
*e)
   }
   dns_ns_rr_init_down = 0;
 } else {
+  /* Primary - Secondary mode:
+   *   Establish a connection to the Primary DNS server.
+   *   It always send DNS requests to the Primary DNS server.
+   *   If the Primary DNS server dies,
+   * - it will attempt to send DNS requests to the secondary DNS 
server until the Primary DNS server is back.
+   * - and keep to detect the health of the Primary DNS server.
+   *   If DNSHandler::recv_dns() got a valid DNS response from the Primary 
DNS server,
+   * - it means that the Primary DNS server returns.
+   * - it send all DNS requests to the Primary DNS server.
+   *
+   *   The first DNS server is the Primary DNS server, and it is assigned 
to DNSHandler::ip within validate_ip() function.
+   */
   open_cons(nullptr); // use current target address.
   n_con = 1;
 }
@@ -587,8 +616,8 @@ DNSHandler::startEvent_sdns(int /* event ATS_UNUSED */, 
Event *e)
   this->validate_ip();
 
   SET_HANDLER(::mainEvent);
-  open_cons(, false, n_con);
-  ++n_con; // TODO should n_con be zeroed?
+  open_cons(nullptr, false, 0);
+  n_con = 1;
 
   return EVENT_CONT;
 }
@@ -647,7 +676,7 @@ DNSHandler::try_primary_named(bool reopen)
   if (reopen && ((t - last_primary_reopen) > DNS_PRIMARY_REOPEN_PERIOD)) {
 Debug("dns", "try_primary_named: reopening primary DNS connection");
 last_primary_reopen = t;
-open_cons(, true, 0);
+open_cons(nullptr, true, 0);
   }
   if ((t - last_primary_retry) > DNS_PRIMARY_RETRY_PERIOD) {
 unsigned char 

[trafficserver] branch 9.0.x updated: cachekey: added --canonical-prefix parameter

2019-08-30 Thread zwoop
This is an automated email from the ASF dual-hosted git repository.

zwoop pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git


The following commit(s) were added to refs/heads/9.0.x by this push:
 new 5d5d4cc  cachekey: added --canonical-prefix parameter
5d5d4cc is described below

commit 5d5d4cc288e2101ae3987b4070f93bd41713c4b0
Author: Gancho Tenev 
AuthorDate: Wed Aug 21 16:11:48 2019 -0700

cachekey: added --canonical-prefix parameter

In certain use-cases when calculating the prefix (the initial value
of the new cache key) we need to have the scheme, host and port in
their original form from the request URI, i.e. when hosting.config
is used the cache key is expected to contain a valid URI authority
element used for volume selection.

More details about the new parameter and its functionality can be
found in doc/admin-guide/plugins/cachekey.en.rst

(cherry picked from commit 0f86efc7678248c09f87ec8b99bc026456edc3f7)
---
 doc/admin-guide/plugins/cachekey.en.rst | 21 +---
 plugins/cachekey/cachekey.cc| 90 -
 plugins/cachekey/cachekey.h |  3 +-
 plugins/cachekey/configs.cc | 11 
 plugins/cachekey/configs.h  |  6 +++
 plugins/cachekey/plugin.cc  |  2 +-
 6 files changed, 102 insertions(+), 31 deletions(-)

diff --git a/doc/admin-guide/plugins/cachekey.en.rst 
b/doc/admin-guide/plugins/cachekey.en.rst
index 5342d8b..51f4ef1 100644
--- a/doc/admin-guide/plugins/cachekey.en.rst
+++ b/doc/admin-guide/plugins/cachekey.en.rst
@@ -70,20 +70,29 @@ Cache key structure and related plugin parameters
 
 ::
 
-  Optional components  | 
┌─┬──┬──┐
+  Optional components  | ┌─┬── 
───┬──┐
   (included in this order) | │ --static-prefix | --capture-prefix │ 
--capture-prefix-uri │
| 
├─┴──┴──┤
-  Default values if no | │ /host/port  
  |
+  Default values if no | │ /host/port or scheme://host:port (see the table 
below)|
   optional components  | 
└───┘
   configured   |
 
+  ┌┬─┬──┐
+  │ --canonical-prefix |  default value if no│ input used for   │
+  │|  prefix parameters used │ --capture-prefix │
+  ├┴─┴──┤
+  │ fasle  | /host/port  | host:port|
+  ├┴─┴──┤
+  │ true   | scheme://host:port  | scheme://host:port   |
+  └──┴──┘
+
+
 * ``--static-prefix=`` (default: empty string) - if specified and not 
an empty string the  will be added to the cache key.
-* ``--capture-prefix=`` (default: empty string) - if 
specified and not empty then strings are captured from ``host:port`` based on 
the  and are added to the cache key.
+* ``--capture-prefix=`` (default: empty string) - if 
specified and not empty then strings are captured based on the value of 
``--canonical-prefix`` parameter (see the table above) and 
 and are added to the cache key.
 * ``--capture-prefix-uri=`` (default: empty string) - if 
specified and not empty then strings are captured from the entire URI based on 
the  and are added to the cache key.
 * If any of the "Prefix" related plugin parameters are used together in the 
plugin configuration they are added to the cache key in the order shown in the 
diagram.
-* ``--remove-prefix= default prefix */
+append(getCanonicalUrl(_buf, _url, canonicalPrefix, /* provideDefaultKey 
*/ true), /* useSeparator */ false);
 CacheKeyDebug("added default prefix, key: '%s'", _key.c_str());
   }
 }
diff --git a/plugins/cachekey/cachekey.h b/plugins/cachekey/cachekey.h
index 7ea058c..0922ed1 100644
--- a/plugins/cachekey/cachekey.h
+++ b/plugins/cachekey/cachekey.h
@@ -55,9 +55,10 @@ public:
 
   void append(unsigned number);
   void append(const String &);
+  void append(const String , bool useSeparator);
   void append(const char *s);
   void append(const char *n, unsigned s);
-  void appendPrefix(const String , Pattern , Pattern 
);
+  void appendPrefix(const String , Pattern , Pattern 
, bool canonicalPrefix);
   void appendPath(Pattern , Pattern );
   void appendHeaders(const ConfigHeaders );
   void appendQuery(const ConfigQuery );
diff --git a/plugins/cachekey/configs.cc b/plugins/cachekey/configs.cc
index 2320db1..a56565c 100644
--- a/plugins/cachekey/configs.cc
+++ b/plugins/cachekey/configs.cc
@@ -397,6 +397,8 @@ Configs::init(int argc, const char 

[trafficserver] branch 9.0.x updated: correct the size of DNS buffers

2019-08-30 Thread zwoop
This is an automated email from the ASF dual-hosted git repository.

zwoop pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git


The following commit(s) were added to refs/heads/9.0.x by this push:
 new 3c93d86  correct the size of DNS buffers
3c93d86 is described below

commit 3c93d86f48182b4defbe2d1b5dd0d42fa577e244
Author: Zizhong Zhang 
AuthorDate: Tue Aug 20 18:18:56 2019 -0700

correct the size of DNS buffers

(cherry picked from commit 1b98e7f4c786a0a1ff675e95d23a59ecd6a852c2)
---
 iocore/dns/DNS.cc   | 12 ++--
 iocore/dns/I_DNSProcessor.h | 19 ++-
 2 files changed, 16 insertions(+), 15 deletions(-)

diff --git a/iocore/dns/DNS.cc b/iocore/dns/DNS.cc
index 7a7df3b..8d91744 100644
--- a/iocore/dns/DNS.cc
+++ b/iocore/dns/DNS.cc
@@ -597,7 +597,7 @@ static inline int
 _ink_res_mkquery(ink_res_state res, char *qname, int qtype, unsigned char 
*buffer, bool over_tcp = false)
 {
   int offset = over_tcp ? tcp_data_length_offset : 0;
-  int r  = ink_res_mkquery(res, QUERY, qname, C_IN, qtype, nullptr, 0, 
nullptr, buffer + offset, MAX_DNS_PACKET_LEN);
+  int r  = ink_res_mkquery(res, QUERY, qname, C_IN, qtype, nullptr, 0, 
nullptr, buffer + offset, MAX_DNS_REQUEST_LEN - offset);
   if (over_tcp) {
 NS_PUT16(r, buffer);
   }
@@ -629,7 +629,7 @@ DNSHandler::retry_named(int ndx, ink_hrtime t, bool reopen)
   }
   bool over_tcp = dns_conn_mode == DNS_CONN_MODE::TCP_ONLY;
   int con_fd= over_tcp ? tcpcon[ndx].fd : udpcon[ndx].fd;
-  unsigned char buffer[MAX_DNS_PACKET_LEN];
+  unsigned char buffer[MAX_DNS_REQUEST_LEN];
   Debug("dns", "trying to resolve '%s' from DNS connection, ndx %d", 
try_server_names[try_servers], ndx);
   int r   = _ink_res_mkquery(m_res, try_server_names[try_servers], T_A, 
buffer, over_tcp);
   try_servers = (try_servers + 1) % countof(try_server_names);
@@ -650,7 +650,7 @@ DNSHandler::try_primary_named(bool reopen)
 open_cons(, true, 0);
   }
   if ((t - last_primary_retry) > DNS_PRIMARY_RETRY_PERIOD) {
-unsigned char buffer[MAX_DNS_PACKET_LEN];
+unsigned char buffer[MAX_DNS_REQUEST_LEN];
 bool over_tcp  = dns_conn_mode == DNS_CONN_MODE::TCP_ONLY;
 int con_fd = over_tcp ? tcpcon[0].fd : udpcon[0].fd;
 last_primary_retry = t;
@@ -824,7 +824,7 @@ DNSHandler::recv_dns(int /* event ATS_UNUSED */, Event * /* 
e ATS_UNUSED */)
 goto Lerror;
   }
   dnsc->tcp_data.total_length = ntohs(dnsc->tcp_data.total_length);
-  if (res != sizeof(dnsc->tcp_data.total_length) || 
dnsc->tcp_data.total_length > MAX_DNS_PACKET_LEN) {
+  if (res != sizeof(dnsc->tcp_data.total_length)) {
 goto Lerror;
   }
 }
@@ -852,7 +852,7 @@ DNSHandler::recv_dns(int /* event ATS_UNUSED */, Event * /* 
e ATS_UNUSED */)
 hostent_cache = dnsBufAllocator.alloc();
   }
 
-  res = socketManager.recvfrom(dnsc->fd, hostent_cache->buf, 
MAX_DNS_PACKET_LEN, 0, _ip.sa, _length);
+  res = socketManager.recvfrom(dnsc->fd, hostent_cache->buf, 
MAX_DNS_RESPONSE_LEN, 0, _ip.sa, _length);
   Debug("dns", "DNSHandler::recv_dns res = [%d]", res);
   if (res == -EAGAIN) {
 break;
@@ -1092,7 +1092,7 @@ static bool
 write_dns_event(DNSHandler *h, DNSEntry *e, bool over_tcp)
 {
   ProxyMutex *mutex = h->mutex.get();
-  unsigned char buffer[MAX_DNS_PACKET_LEN];
+  unsigned char buffer[MAX_DNS_REQUEST_LEN];
   int offset = over_tcp ? tcp_data_length_offset : 0;
   HEADER *header = reinterpret_cast(buffer + offset);
   int r  = 0;
diff --git a/iocore/dns/I_DNSProcessor.h b/iocore/dns/I_DNSProcessor.h
index a0ec651..83f4bd2 100644
--- a/iocore/dns/I_DNSProcessor.h
+++ b/iocore/dns/I_DNSProcessor.h
@@ -28,11 +28,12 @@
 const int DOMAIN_SERVICE_PORT= NAMESERVER_PORT;
 const int DEFAULT_DOMAIN_NAME_SERVER = 0;
 
-const int MAX_DNS_PACKET_LEN = 8192;
-const int DNS_RR_MAX_COUNT   = (MAX_DNS_PACKET_LEN - HFIXEDSZ + RRFIXEDSZ - 1) 
/ RRFIXEDSZ;
-const int DNS_MAX_ALIASES= DNS_RR_MAX_COUNT;
-const int DNS_MAX_ADDRS  = DNS_RR_MAX_COUNT;
-const int DNS_HOSTBUF_SIZE   = MAX_DNS_PACKET_LEN;
+const int MAX_DNS_REQUEST_LEN  = NS_PACKETSZ;
+const int MAX_DNS_RESPONSE_LEN = 65536;
+const int DNS_RR_MAX_COUNT = (MAX_DNS_RESPONSE_LEN - HFIXEDSZ + RRFIXEDSZ 
- 1) / RRFIXEDSZ;
+const int DNS_MAX_ALIASES  = DNS_RR_MAX_COUNT;
+const int DNS_MAX_ADDRS= DNS_RR_MAX_COUNT;
+const int DNS_HOSTBUF_SIZE = MAX_DNS_RESPONSE_LEN;
 
 /**
   All buffering required to handle a DNS receipt. For asynchronous DNS,
@@ -41,10 +42,10 @@ const int DNS_HOSTBUF_SIZE   = MAX_DNS_PACKET_LEN;
 
 */
 struct HostEnt : RefCountObj {
-  struct hostent ent   = {.h_name = nullptr, .h_aliases = nullptr, 
.h_addrtype = 0, .h_length = 0, .h_addr_list = nullptr};
-  uint32_t ttl = 0;
-  int packet_size  = 0;
-  char buf[MAX_DNS_PACKET_LEN] = {0};
+  struct hostent 

[trafficserver] branch 9.0.x updated: Fix no_activity timeout for server session reuse.

2019-08-30 Thread zwoop
This is an automated email from the ASF dual-hosted git repository.

zwoop pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git


The following commit(s) were added to refs/heads/9.0.x by this push:
 new 24daee0  Fix no_activity timeout for server session reuse.
24daee0 is described below

commit 24daee0b63fab4ca43501bf3c304de06bc5da6b3
Author: Susan Hinrichs 
AuthorDate: Fri Aug 9 20:27:49 2019 +

Fix no_activity timeout for server session reuse.

(cherry picked from commit 924c8f9f70a9d30d70f4b130a43af82b79c90eda)
---
 proxy/http/HttpSM.cc | 7 +++
 1 file changed, 7 insertions(+)

diff --git a/proxy/http/HttpSM.cc b/proxy/http/HttpSM.cc
index 523feec..346eab5 100644
--- a/proxy/http/HttpSM.cc
+++ b/proxy/http/HttpSM.cc
@@ -5975,6 +5975,13 @@ HttpSM::setup_server_send_request()
 
   milestones[TS_MILESTONE_SERVER_BEGIN_WRITE] = Thread::get_hrtime();
   server_entry->write_vio = 
server_entry->vc->do_io_write(this, hdr_length, buf_start);
+
+  // Make sure the VC is using correct timeouts.  We may be reusing a 
previously used server session
+  if (t_state.api_txn_no_activity_timeout_value != -1) {
+
server_session->get_netvc()->set_inactivity_timeout(HRTIME_MSECONDS(t_state.api_txn_no_activity_timeout_value));
+  } else {
+
server_session->get_netvc()->set_inactivity_timeout(HRTIME_SECONDS(t_state.txn_conf->transaction_no_activity_timeout_out));
+  }
 }
 
 void



[trafficserver] branch master updated (af7299d -> 7831fb0)

2019-08-30 Thread amc
This is an automated email from the ASF dual-hosted git repository.

amc pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/trafficserver.git.


from af7299d  cachekey: added --key-type (for parent selection)
 add 7831fb0  ProxySsn renaming member vars

No new revisions were added by this update.

Summary of changes:
 .../client-session-architecture.en.rst |  2 +-
 proxy/ProxyTransaction.cc  | 82 +--
 proxy/ProxyTransaction.h   |  6 +-
 proxy/http/Http1ClientSession.cc   | 10 +--
 proxy/http/Http1ClientSession.h|  4 +-
 proxy/http/Http1Transaction.cc | 46 +--
 proxy/http/HttpUpdateTester.cc |  6 +-
 proxy/http2/Http2ClientSession.cc  | 32 
 proxy/http2/Http2ClientSession.h   |  2 +-
 proxy/http2/Http2ConnectionState.cc| 10 +--
 proxy/http2/Http2Stream.cc | 92 +++---
 proxy/http3/Http3Transaction.cc| 38 -
 12 files changed, 165 insertions(+), 165 deletions(-)



[trafficserver] branch master updated (a53821e -> af7299d)

2019-08-30 Thread gancho
This is an automated email from the ASF dual-hosted git repository.

gancho pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/trafficserver.git.


from a53821e  Bumped the version of master to 10.0.0
 add af7299d  cachekey: added --key-type (for parent selection)

No new revisions were added by this update.

Summary of changes:
 doc/admin-guide/plugins/cachekey.en.rst | 167 +++-
 doc/admin-guide/plugins/index.en.rst|   4 +-
 plugins/cachekey/cachekey.cc|  69 ++---
 plugins/cachekey/cachekey.h |  10 +-
 plugins/cachekey/configs.cc |  62 +++-
 plugins/cachekey/configs.h  |  29 +-
 plugins/cachekey/plugin.cc  |   2 +-
 7 files changed, 248 insertions(+), 95 deletions(-)