Re: [trafficserver] branch master updated: remove bad check

[James Peach]

> On Jun 7, 2017, at 12:49 PM, dra...@apache.org wrote:
> 
> This is an automated email from the ASF dual-hosted git repository.
> 
> dragon pushed a commit to branch master
> in repository https://gitbox.apache.org/repos/asf/trafficserver.git
> 
> 
> The following commit(s) were added to refs/heads/master by this push:
> new 7ea644a  remove bad check
> 7ea644a is described below
> 
> commit 7ea644a9d4fdb91b6283a38e0d2fdb3b169f2f00
> Author: unknown 

Please fix your git user.email configuration.

Re: [trafficserver] branch master updated: Update testing extensions

[James Peach]

On 3 Jun 2017, at 7:42, dra...@apache.org wrote:

> This is an automated email from the ASF dual-hosted git repository.
>
> dragon pushed a commit to branch master
> in repository https://gitbox.apache.org/repos/asf/trafficserver.git
>
> The following commit(s) were added to refs/heads/master by this push:
>new  53d5da0   Update testing extensions
> 53d5da0 is described below
>
> commit 53d5da0b0a78c3911543bdd88b6e8e610e407316
> Author: Unknown 
> AuthorDate: Fri Jun 2 17:33:19 2017 -0500

Please fix your git config :)

Re: [trafficserver] branch master updated: Removes the tsqa tests, to avoid confusion. Use autest.

[James Peach]

On 24 May 2017, at 18:12, zw...@apache.org wrote:


This is an automated email from the ASF dual-hosted git repository.

zwoop pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/trafficserver.git

The following commit(s) were added to refs/heads/master by this push:
   new  1e41700   Removes the tsqa tests, to avoid confusion. Use 
autest.


Does autest cover all these tests now?

J

Re: [trafficserver] branch master updated: TS-5063: Fixes coverity warnings and cleans up the temp string

[James Peach]

> On Nov 27, 2016, at 2:19 PM, zw...@apache.org wrote:
> 
> This is an automated email from the ASF dual-hosted git repository.
> 
> zwoop pushed a commit to branch master
> in repository https://git-dual.apache.org/repos/asf/trafficserver.git
> 
> The following commit(s) were added to refs/heads/master by this push:
>   new  2e79b30   TS-5063: Fixes coverity warnings and cleans up the temp 
> string
> 2e79b30 is described below
> 
> commit 2e79b300df22985cf21ef070295d0f1ea0a32b33
> Author: Leif Hedstrom 
> AuthorDate: Wed Nov 23 14:23:00 2016 -0700
> 
>TS-5063: Fixes coverity warnings and cleans up the temp string
> 
>The Vec seems overkill here, and makes things less nice when
>managing the intermediary strings. The new code has issues with being
>potentially used without initialization.
> 
>CID 1365975, 1365974
> ---
> mgmt/LocalManager.cc | 31 +++
> 1 file changed, 15 insertions(+), 16 deletions(-)
> 
> diff --git a/mgmt/LocalManager.cc b/mgmt/LocalManager.cc
> index 75652a1..e10250d 100644
> --- a/mgmt/LocalManager.cc
> +++ b/mgmt/LocalManager.cc
> @@ -858,6 +858,8 @@ LocalManager::processEventQueue()
>  *   onetime_options: one time options that traffic_server should be started 
> with (ie
>  *these options do not persist across reboots)
>  */
> +static const size_t OPTIONS_SIZE = 16384; // Arbitrary max size for command 
> line option string
> +
> bool
> LocalManager::startProxy(const char *onetime_options)
> {
> @@ -924,19 +926,18 @@ LocalManager::startProxy(const char *onetime_options)
> int i = 0;
> char *options[32], *last, *tok;
> bool open_ports_p = false;
> +char real_proxy_options[OPTIONS_SIZE];
> 
> -Vec real_proxy_options;
> -
> -real_proxy_options.append(proxy_options, strlen(proxy_options));
> +ink_strlcpy(real_proxy_options, proxy_options, strlen(proxy_options));


ink_strlcpy(real_proxy_options, proxy, options, OPTIONS_SIZE);

Re: [trafficserver] branch master updated: TS-4858: fix memory leak problem of global_default_keyblock

[James Peach]

> On Oct 17, 2016, at 3:53 PM, shinr...@apache.org wrote:
> 
> This is an automated email from the ASF dual-hosted git repository.
> 
> shinrich pushed a commit to branch master
> in repository https://git-dual.apache.org/repos/asf/trafficserver.git
> 
> The following commit(s) were added to refs/heads/master by this push:
>   new  34b5535   TS-4858: fix memory leak problem of 
> global_default_keyblock
> 34b5535 is described below
> 
> commit 34b55356d1cfb38744bb4b4c080fc758e96c6381
> Author: Persia Aziz 
> AuthorDate: Tue Sep 20 14:40:28 2016 -0500
> 
>TS-4858: fix memory leak problem of global_default_keyblock
> ---
> iocore/net/P_SSLCertLookup.h |  4 +--
> iocore/net/P_SSLConfig.h |  6 +++-
> iocore/net/SSLCertLookup.cc  | 70 
> iocore/net/SSLConfig.cc  | 23 --
> iocore/net/SSLUtils.cc   | 76 +++-
> proxy/InkAPI.cc  |  1 -
> 6 files changed, 103 insertions(+), 77 deletions(-)
> 
> diff --git a/iocore/net/P_SSLCertLookup.h b/iocore/net/P_SSLCertLookup.h
> index b408626..97c11ff 100644
> --- a/iocore/net/P_SSLCertLookup.h
> +++ b/iocore/net/P_SSLCertLookup.h
> @@ -40,7 +40,6 @@ struct ssl_ticket_key_block {
>   unsigned num_keys;
>   ssl_ticket_key_t keys[];
> };
> -
> /** A certificate context.
> 
> This holds data about a certificate and how it is used by the SSL logic. 
> Current this is mainly
> @@ -110,5 +109,6 @@ struct SSLCertLookup : public ConfigInfo {
> 
> void ticket_block_free(void *ptr);
> ssl_ticket_key_block *ticket_block_alloc(unsigned count);
> -
> +ssl_ticket_key_block *ticket_block_create(char *ticket_key_data, int 
> ticket_key_len);
> +ssl_ticket_key_block *ssl_create_ticket_keyblock(const char 
> *ticket_key_path);

As noted in the original review, this should be called ticket_block_read() to 
be consistent with the other APIs above.

> #endif /* __P_SSLCERTLOOKUP_H__ */
> diff --git a/iocore/net/P_SSLConfig.h b/iocore/net/P_SSLConfig.h
> index 35fdff1..6adc2b5 100644
> --- a/iocore/net/P_SSLConfig.h
> +++ b/iocore/net/P_SSLConfig.h
> @@ -34,9 +34,11 @@
> #include "ProxyConfig.h"
> #include "SSLSessionCache.h"
> #include "ts/ink_inet.h"
> +#include 
> +#include "P_SSLCertLookup.h"
> 
> struct SSLCertLookup;
> -
> +struct ssl_ticket_key_block;
> /
> //
> // struct SSLConfigParams
> @@ -67,6 +69,8 @@ struct SSLConfigParams : public ConfigInfo {
>   char *dhparamsFile;
>   char *cipherSuite;
>   char *client_cipherSuite;
> +  char *ticket_key_filename;
> +  ssl_ticket_key_block *default_global_keyblock;
>   int configExitOnLoadError;
>   int clientCertLevel;
>   int verify_depth;
> diff --git a/iocore/net/SSLCertLookup.cc b/iocore/net/SSLCertLookup.cc
> index b9d7381..972071d 100644
> --- a/iocore/net/SSLCertLookup.cc
> +++ b/iocore/net/SSLCertLookup.cc
> @@ -28,10 +28,18 @@
> #include "P_SSLConfig.h"
> #include "I_EventSystem.h"
> #include "ts/I_Layout.h"
> +#include "ts/MatcherUtils.h"
> #include "ts/Regex.h"
> #include "ts/Trie.h"
> #include "ts/TestBox.h"
> 
> +// Check if the ticket_key callback #define is available, and if so, enable 
> session tickets.
> +#ifdef SSL_CTX_set_tlsext_ticket_key_cb
> +
> +#define HAVE_OPENSSL_SESSION_TICKETS 1
> +
> +#endif /* SSL_CTX_set_tlsext_ticket_key_cb */
> +
> struct SSLAddressLookupKey {
>   explicit SSLAddressLookupKey(const IpEndpoint &ip) : sep(0)
>   {
> @@ -160,7 +168,69 @@ ticket_block_alloc(unsigned count)
> 
>   return ptr;
> }
> +ssl_ticket_key_block *
> +ticket_block_create(char *ticket_key_data, int ticket_key_len)
> +{
> +  ssl_ticket_key_block *keyblock = NULL;
> +  unsigned num_ticket_keys   = ticket_key_len / sizeof(ssl_ticket_key_t);
> +  if (num_ticket_keys == 0) {
> +Error("SSL session ticket key is too short (>= 48 bytes are required)");
> +goto fail;
> +  }
> +
> +  keyblock = ticket_block_alloc(num_ticket_keys);
> 
> +  // Slurp all the keys in the ticket key file. We will encrypt with the 
> first key, and decrypt
> +  // with any key (for rotation purposes).
> +  for (unsigned i = 0; i < num_ticket_keys; ++i) {
> +const char *data = (const char *)ticket_key_data + (i * 
> sizeof(ssl_ticket_key_t));
> +
> +memcpy(keyblock->keys[i].key_name, data, 
> sizeof(keyblock->keys[i].key_name));
> +memcpy(keyblock->keys[i].hmac_secret, data + 
> sizeof(keyblock->keys[i].key_name), sizeof(keyblock->keys[i].hmac_secret));
> +memcpy(keyblock->keys[i].aes_key, data + 
> sizeof(keyblock->keys[i].key_name) + sizeof(keyblock->keys[i].hmac_secret),
> +   sizeof(keyblock->keys[i].aes_key));
> +  }
> +
> +  return keyblock;
> +
> +fail:
> +  ticket_block_free(keyblock);
> +  return NULL;
> +}
> +
> +ssl_ticket_key_block *
> +ssl_create_ticket_keyblock(const char *ticket_key_path)
> +{
> +#if HAVE_OPENSSL_SESSION_TICKETS
> +  ats_scoped_str ticket_key_data;
> +  int ticket_key_len;
> +  ssl_ticket_key_block *keyblock =

Re: [trafficserver] branch master updated: Fix possibility of NULL assignment to std::string

[James Peach]

> On Oct 2, 2016, at 5:36 PM, sor...@apache.org wrote:
> 
> This is an automated email from the ASF dual-hosted git repository.
> 
> sorber pushed a commit to branch master
> in repository https://git-dual.apache.org/repos/asf/trafficserver.git
> 
> The following commit(s) were added to refs/heads/master by this push:
>   new  91bfe21   Fix possibility of NULL assignment to std::string
> 91bfe21 is described below
> 
> commit 91bfe214cc2356ae19b78776815d0344d1a0906d
> Author: Phil Sorber 
> AuthorDate: Sun Oct 2 18:18:35 2016 -0600
> 
>Fix possibility of NULL assignment to std::string
> ---
> plugins/header_rewrite/expander.cc | 7 ++-
> 1 file changed, 6 insertions(+), 1 deletion(-)
> 
> diff --git a/plugins/header_rewrite/expander.cc 
> b/plugins/header_rewrite/expander.cc
> index 22dca9c..c62a7d0 100644
> --- a/plugins/header_rewrite/expander.cc
> +++ b/plugins/header_rewrite/expander.cc
> @@ -67,7 +67,12 @@ VariableExpander::expand(const Resources &res)
>   // Protocol of the incoming request
>   if (TSHttpTxnPristineUrlGet(res.txnp, &bufp, &url_loc) == TS_SUCCESS) {
> int len;
> -resolved_variable = TSUrlSchemeGet(bufp, url_loc, &len);
> +const char *tmp = TSUrlSchemeGet(bufp, url_loc, &len);
> +if ((tmp != NULL) && (len > 0)) {
> +  resolved_variable.assign(tmp, len);
> +} else {
> +  resolved_variable.assign("");
> +}

FWIW every other place in this file does

if (tmp && len) {
resolved_variable.assign(tmp, len);
}

Re: [trafficserver] 07/08: TS-4738: docs: update logs_xml.config doc to note its replacement by logging.config (to preserve search engine hits)

[James Peach]

> On Aug 23, 2016, at 12:52 AM, js...@apache.org wrote:
> 
> This is an automated email from the ASF dual-hosted git repository.
> 
> jsime pushed a commit to branch master
> in repository https://git-dual.apache.org/repos/asf/trafficserver.git
> 
> commit ba71f829df79fc1f9d71699865b479896f55712b
> Author: Jon Sime 
> AuthorDate: Tue Aug 23 07:42:17 2016 +
> 
>TS-4738: docs: update logs_xml.config doc to note its replacement by 
> logging.config (to preserve search engine hits)

Good point. Do you want to link this into the index as well?

J

Re: [trafficserver] branch master updated: TS-3922: Websocket Timeouts, transaction overrideable & clang format

[James Peach]

This commit broke the Lua build. Did you run ./ci/regression before committing?

https://ci.trafficserver.apache.org/job/out_of_tree-master/1631/changes


> On Apr 21, 2016, at 7:45 PM, bri...@apache.org wrote:
> 
> This is an automated email from the ASF dual-hosted git repository.
> 
> briang pushed a commit to branch master
> in repository https://git-dual.apache.org/repos/asf/trafficserver.git
> 
> The following commit(s) were added to refs/heads/master by this push:
>   new  4fdf8ac   TS-3922: Websocket Timeouts, transaction overrideable & 
> clang format
> 4fdf8ac is described below
> 
> commit 4fdf8ac21e4dcac97297f6afb03c50a5c997f996
> Author: Brian Geffon 
> AuthorDate: Thu Apr 21 04:35:36 2016 -0700
> 
>TS-3922: Websocket Timeouts, transaction overrideable & clang format
> ---
> plugins/experimental/ts_lua/ts_lua_http_config.c | 198 ---
> proxy/InkAPITest.cc  |   4 +-
> 2 files changed, 103 insertions(+), 99 deletions(-)
> 
> diff --git a/plugins/experimental/ts_lua/ts_lua_http_config.c 
> b/plugins/experimental/ts_lua/ts_lua_http_config.c
> index 8b0ddf4..31abb1e 100644
> --- a/plugins/experimental/ts_lua/ts_lua_http_config.c
> +++ b/plugins/experimental/ts_lua/ts_lua_http_config.c
> @@ -114,6 +114,8 @@ typedef enum {
>   TS_LUA_CONFIG_HTTP_REDIRECT_USE_ORIG_CACHE_KEY = 
> TS_CONFIG_HTTP_REDIRECT_USE_ORIG_CACHE_KEY,
>   TS_LUA_CONFIG_HTTP_ATTACH_SERVER_SESSION_TO_CLIENT = 
> TS_CONFIG_HTTP_ATTACH_SERVER_SESSION_TO_CLIENT,
>   TS_LUA_CONFIG_HTTP_ORIGIN_MAX_CONNECTIONS_QUEUE = 
> TS_CONFIG_HTTP_ORIGIN_MAX_CONNECTIONS_QUEUE,
> +  TS_LUA_CONFIG_WEBSOCKET_NO_ACTIVITY_TIMEOUT = 
> TS_CONFIG_WEBSOCKET_NO_ACTIVITY_TIMEOUT,
> +  TS_LUA_CONFIG_WEBSOCKET_ACTIVE_TIMEOUT = 
> TS_CONFIG_WEBSOCKET_ACTIVE_TIMEOUT,
>   TS_LUA_CONFIG_LAST_ENTRY = TS_CONFIG_LAST_ENTRY,
> } TSLuaOverridableConfigKey;
> 
> @@ -124,104 +126,104 @@ typedef enum {
>   TS_LUA_TIMEOUT_NO_ACTIVITY = 3
> } TSLuaTimeoutKey;
> 
> -ts_lua_var_item ts_lua_http_config_vars[] = {
> -  TS_LUA_MAKE_VAR_ITEM(TS_LUA_CONFIG_URL_REMAP_PRISTINE_HOST_HDR),
> -  TS_LUA_MAKE_VAR_ITEM(TS_LUA_CONFIG_HTTP_CHUNKING_ENABLED),
> -  TS_LUA_MAKE_VAR_ITEM(TS_LUA_CONFIG_HTTP_NEGATIVE_CACHING_ENABLED),
> -  TS_LUA_MAKE_VAR_ITEM(TS_LUA_CONFIG_HTTP_NEGATIVE_CACHING_LIFETIME),
> -  TS_LUA_MAKE_VAR_ITEM(TS_LUA_CONFIG_HTTP_CACHE_WHEN_TO_REVALIDATE),
> -  TS_LUA_MAKE_VAR_ITEM(TS_LUA_CONFIG_HTTP_KEEP_ALIVE_ENABLED_IN),
> -  TS_LUA_MAKE_VAR_ITEM(TS_LUA_CONFIG_HTTP_KEEP_ALIVE_ENABLED_OUT),
> -  TS_LUA_MAKE_VAR_ITEM(TS_LUA_CONFIG_HTTP_KEEP_ALIVE_POST_OUT),
> -  TS_LUA_MAKE_VAR_ITEM(TS_LUA_CONFIG_HTTP_SERVER_SESSION_SHARING_MATCH),
> -  TS_LUA_MAKE_VAR_ITEM(TS_LUA_CONFIG_NET_SOCK_RECV_BUFFER_SIZE_OUT),
> -  TS_LUA_MAKE_VAR_ITEM(TS_LUA_CONFIG_NET_SOCK_SEND_BUFFER_SIZE_OUT),
> -  TS_LUA_MAKE_VAR_ITEM(TS_LUA_CONFIG_NET_SOCK_OPTION_FLAG_OUT),
> -  TS_LUA_MAKE_VAR_ITEM(TS_LUA_CONFIG_HTTP_FORWARD_PROXY_AUTH_TO_PARENT),
> -  TS_LUA_MAKE_VAR_ITEM(TS_LUA_CONFIG_HTTP_ANONYMIZE_REMOVE_FROM),
> -  TS_LUA_MAKE_VAR_ITEM(TS_LUA_CONFIG_HTTP_ANONYMIZE_REMOVE_REFERER),
> -  TS_LUA_MAKE_VAR_ITEM(TS_LUA_CONFIG_HTTP_ANONYMIZE_REMOVE_USER_AGENT),
> -  TS_LUA_MAKE_VAR_ITEM(TS_LUA_CONFIG_HTTP_ANONYMIZE_REMOVE_COOKIE),
> -  TS_LUA_MAKE_VAR_ITEM(TS_LUA_CONFIG_HTTP_ANONYMIZE_REMOVE_CLIENT_IP),
> -  TS_LUA_MAKE_VAR_ITEM(TS_LUA_CONFIG_HTTP_ANONYMIZE_INSERT_CLIENT_IP),
> -  TS_LUA_MAKE_VAR_ITEM(TS_LUA_CONFIG_HTTP_RESPONSE_SERVER_ENABLED),
> -  TS_LUA_MAKE_VAR_ITEM(TS_LUA_CONFIG_HTTP_INSERT_SQUID_X_FORWARDED_FOR),
> -  TS_LUA_MAKE_VAR_ITEM(TS_LUA_CONFIG_HTTP_SERVER_TCP_INIT_CWND),
> -  TS_LUA_MAKE_VAR_ITEM(TS_LUA_CONFIG_HTTP_SEND_HTTP11_REQUESTS),
> -  TS_LUA_MAKE_VAR_ITEM(TS_LUA_CONFIG_HTTP_CACHE_HTTP),
> -  TS_LUA_MAKE_VAR_ITEM(TS_LUA_CONFIG_HTTP_CACHE_CLUSTER_CACHE_LOCAL),
> -  TS_LUA_MAKE_VAR_ITEM(TS_LUA_CONFIG_HTTP_CACHE_IGNORE_CLIENT_NO_CACHE),
> -  TS_LUA_MAKE_VAR_ITEM(TS_LUA_CONFIG_HTTP_CACHE_IGNORE_CLIENT_CC_MAX_AGE),
> -  TS_LUA_MAKE_VAR_ITEM(TS_LUA_CONFIG_HTTP_CACHE_IMS_ON_CLIENT_NO_CACHE),
> -  TS_LUA_MAKE_VAR_ITEM(TS_LUA_CONFIG_HTTP_CACHE_IGNORE_SERVER_NO_CACHE),
> -  TS_LUA_MAKE_VAR_ITEM(TS_LUA_CONFIG_HTTP_CACHE_CACHE_RESPONSES_TO_COOKIES),
> -  TS_LUA_MAKE_VAR_ITEM(TS_LUA_CONFIG_HTTP_CACHE_IGNORE_AUTHENTICATION),
> -  
> TS_LUA_MAKE_VAR_ITEM(TS_LUA_CONFIG_HTTP_CACHE_CACHE_URLS_THAT_LOOK_DYNAMIC),
> -  TS_LUA_MAKE_VAR_ITEM(TS_LUA_CONFIG_HTTP_CACHE_REQUIRED_HEADERS),
> -  TS_LUA_MAKE_VAR_ITEM(TS_LUA_CONFIG_HTTP_INSERT_REQUEST_VIA_STR),
> -  TS_LUA_MAKE_VAR_ITEM(TS_LUA_CONFIG_HTTP_INSERT_RESPONSE_VIA_STR),
> -  TS_LUA_MAKE_VAR_ITEM(TS_LUA_CONFIG_HTTP_CACHE_HEURISTIC_MIN_LIFETIME),
> -  TS_LUA_MAKE_VAR_ITEM(TS_LUA_CONFIG_HTTP_CACHE_HEURISTIC_MAX_LIFETIME),
> -  TS_LUA_MAKE_VAR_ITEM(TS_LUA_CONFIG_HTTP_CACHE_GUARANTEED_MIN_LIFETIME),
> -  TS_LUA_MAKE_VAR_ITEM(TS_LUA_CONFIG_HTTP_CACHE_GUARANTEED_MAX_LIFETIME),
> -  TS_LUA_MAKE_VAR_ITEM(TS_LUA_CONFIG_HTTP_CACHE_MAX_STALE_AGE),
> -  TS_LUA_MAKE_VAR_ITEM(TS_LUA_CONFIG_HTTP_KEEP_ALIVE_NO_ACTIVITY_TI

Re: [trafficserver] branch master updated: TS-4312 Add config to strictly parse URL according to RFC 3986. This closes #574

[James Peach]

> On Apr 18, 2016, at 1:05 AM, bri...@apache.org wrote:
> 
> This is an automated email from the ASF dual-hosted git repository.
> 
> briang pushed a commit to branch master
> in repository https://git-dual.apache.org/repos/asf/trafficserver.git
> 
> The following commit(s) were added to refs/heads/master by this push:
>   new  9f9dc38   TS-4312 Add config to strictly parse URL according to 
> RFC 3986. This closes #574
> 9f9dc38 is described below
> 
> 
[snip]
> +
> +REGRESSION_TEST(ParseRules_strict_URI)(RegressionTest *t, int /* level 
> ATS_UNUSED */, int *pstatus)
> +{
> +  const struct {
> +const char *const uri;
> +bool valid;
> +  } http_strict_uri_parsing_test_case[] = {{"/home", true},
> +   {"/path/data?key=value#id", true},
> +   {"/ABCDEFGHIJKLMNOPQRSTUVWXYZ", 
> true},
> +   {"/abcdefghijklmnopqrstuvwxyz", 
> true},
> +   {"/0123456789", true},
> +   {":/?#[]@", true},
> +   {"!$&'()*+,;=", true},
> +   {"-._~", true},
> +   {"%", true},
> +   {"\n", false},
> +   {"\"", false},
> +   {"<", false},
> +   {">", false},
> +   {"\\", false},
> +   {"^", false},
> +   {"`", false},
> +   {"{", false},
> +   {"|", false},
> +   {"}", false},
> +   {"é", false}};
> +
> +  TestBox box(t, pstatus);
> +  box = REGRESSION_TEST_PASSED;
> +
> +  for (unsigned int i = 0; i < sizeof(http_strict_uri_parsing_test_case) / 
> sizeof(http_strict_uri_parsing_test_case[0]); ++i) {

countof(http_strict_uri_parsing_test_case)

Re: [trafficserver] 01/01: Merge pull request #567 from sudheerv/ts-3857

[James Peach]

I can't find an email with the actual diff? Is there one, or does that only 
happen on squashed merges?

> On Apr 14, 2016, at 3:06 PM, sudhe...@apache.org wrote:
> 
> This is an automated email from the ASF dual-hosted git repository.
> 
> sudheerv pushed a commit to branch master
> in repository https://git-dual.apache.org/repos/asf/trafficserver.git
> 
> commit acb3343a568981cb4fa9c9052d01bfbb648a9980
> Merge: 8d4c256 26b4b43
> Author: sudheerv 
> AuthorDate: Thu Apr 14 15:06:39 2016 -0700
> 
>Merge pull request #567 from sudheerv/ts-3857
> 
>[TS-3857] Change ERROR default to not log into syslogs.
> 
> proxy/logging/LogFile.cc | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
> 
> -- 
> To stop receiving notification emails like this one, please contact
> "commits@trafficserver.apache.org" .

Re: [trafficserver] branch master updated: TS-4340: fix small issue related to match typ e NONE

[James Peach]

> On Apr 14, 2016, at 1:44 PM, bri...@apache.org wrote:
> 
> This is an automated email from the ASF dual-hosted git repository.
> 
> briang pushed a commit to branch master
> in repository https://git-dual.apache.org/repos/asf/trafficserver.git
> 
> The following commit(s) were added to refs/heads/master by this push:
>   new  8d4c256   TS-4340: fix small issue related to match typ e NONE
> 8d4c256 is described below
> 
> commit 8d4c25653bb038f70651b8675a5969e522668d43
> Author: Brian Geffon 
> AuthorDate: Thu Mar 3 15:45:41 2016 -0800
> 
>TS-4340: fix small issue related to match typ e NONE
> ---
> proxy/http/HttpConnectionCount.h | 8 
> proxy/http/HttpServerSession.cc  | 4 ++--
> 2 files changed, 10 insertions(+), 2 deletions(-)
> 
> diff --git a/proxy/http/HttpConnectionCount.h 
> b/proxy/http/HttpConnectionCount.h
> index b025e8f..eea0d1a 100644
> --- a/proxy/http/HttpConnectionCount.h
> +++ b/proxy/http/HttpConnectionCount.h
> @@ -58,6 +58,10 @@ public:
>   int
>   getCount(const IpEndpoint &addr, const INK_MD5 &hostname_hash, 
> TSServerSessionSharingMatchType match_type)
>   {
> +if (TS_SERVER_SESSION_SHARING_MATCH_NONE == match_type) {
> +  return 0; // We can never match a node if match type is NONE
> +}
> +
> ink_mutex_acquire(&_mutex);
> int count = _hostCount.get(ConnAddr(addr, hostname_hash, match_type));
> ink_mutex_release(&_mutex);
> @@ -73,6 +77,10 @@ public:
>   incrementCount(const IpEndpoint &addr, const INK_MD5 &hostname_hash, 
> TSServerSessionSharingMatchType match_type,
>  const int delta = 1)
>   {
> +if (TS_SERVER_SESSION_SHARING_MATCH_NONE == match_type) {
> +  return; // We can never match a node if match type is NONE.
> +}
> +
> ConnAddr caddr(addr, hostname_hash, match_type);
> ink_mutex_acquire(&_mutex);
> int count = _hostCount.get(caddr);
> diff --git a/proxy/http/HttpServerSession.cc b/proxy/http/HttpServerSession.cc
> index 763c6b8..d2f2386 100644
> --- a/proxy/http/HttpServerSession.cc
> +++ b/proxy/http/HttpServerSession.cc
> @@ -140,14 +140,14 @@ HttpServerSession::do_io_close(int alerrno)
>   // Check to see if we are limiting the number of connections
>   // per host
>   if (enable_origin_connection_limiting == true) {
> -if (connection_count->getCount(server_ip, hostname_hash, sharing_match) 
> > 0) {
> +if (connection_count->getCount(server_ip, hostname_hash, sharing_match) 
> >= 0) {

Surely a count of -1 is a bug? It should be a release_assert?

>   connection_count->incrementCount(server_ip, hostname_hash, 
> sharing_match, -1);
>   ip_port_text_buffer addrbuf;
>   Debug("http_ss", "[%" PRId64 "] connection closed, ip: %s, count: %u", 
> con_id,
> ats_ip_nptop(&server_ip.sa, addrbuf, sizeof(addrbuf)),
> connection_count->getCount(server_ip, hostname_hash, 
> sharing_match));
> } else {
> -  Error("[%" PRId64 "] number of connections should be greater than 
> zero: %u", con_id,
> +  Error("[%" PRId64 "] number of connections should be greater than or 
> equal to zero: %u", con_id,
> connection_count->getCount(server_ip, hostname_hash, 
> sharing_match));
> }
>   }
> 
> -- 
> To stop receiving notification emails like this one, please contact
> ['"commits@trafficserver.apache.org" '].

Re: trafficserver git commit: TS-4087: Reduce SETTINGS_MAX_CONCURRENT_STREAMS when too many streams

[James Peach]

> On Mar 24, 2016, at 2:57 AM, Masaori Koshiba  wrote:
> 
> 2016年3月23日(水) 14:23 James Peach :
> 
>> 

[snip]

>>>  }
>>> +
>>> +return Http2::min_concurrent_streams_in;
>>> +  } else {
>>> +if (Http2::throttling) {
>>> +  Note("revert SETTINGS_MAX_CONCURRENT_STREAMS to %d",
>> Http2::max_concurrent_streams_in);
>> 
>> I’m not sure that we want to log these. Consider a throttling metric
>> instead?
>> 
> throttling metric? Would you give me more details?
> 
> ATS logs "too many connections, throttling" when connections are over
> proxy.config.net.connections_throttle. I followed this.

I don't think that people read logs much :) I am wondering whether we should 
introduce some metrics around connection and stream throttling so that it is 
easier to monitor?

J

Re: trafficserver git commit: TS-4087: Reduce SETTINGS_MAX_CONCURRENT_STREAMS when too many streams

[James Peach]

> On Mar 21, 2016, at 6:57 PM, masa...@apache.org wrote:
> 
> Repository: trafficserver
> Updated Branches:
>  refs/heads/master 1e9c9484c -> 0e6e5c151
> 
> 
> TS-4087: Reduce SETTINGS_MAX_CONCURRENT_STREAMS when too many streams
> 
> Add below variables in records.config
> - proxy.config.http2.min_concurrent_streams_in
> - proxy.config.http2.max_active_streams_in
> 
> When connection wide active stream are larger than 
> proxy.config.http2.max_active_streams_in,
> SETTINGS_MAX_CONCURRENT_STREAMS is reduced to 
> proxy.config.http2.min_concurrent_streams_in
> in new connections.
> 
> If the value of proxy.config.http2.max_active_streams_in is 0, there is no 
> limit.
> 
> This closes #485
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/0e6e5c15
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/0e6e5c15
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/0e6e5c15
> 
> Branch: refs/heads/master
> Commit: 0e6e5c151cde5f06c15e295f663a98b2b7d37a6d
> Parents: 1e9c948
> Author: Masaori Koshiba 
> Authored: Mon Feb 15 20:57:25 2016 +0900
> Committer: Masaori Koshiba 
> Committed: Tue Mar 22 10:52:37 2016 +0900
> 
> --
> mgmt/RecordsConfig.cc   |  4 
> proxy/http2/HTTP2.cc| 12 ---
> proxy/http2/HTTP2.h |  5 -
> proxy/http2/Http2ConnectionState.cc | 35 
> proxy/http2/Http2ConnectionState.h  |  4 +++-
> 5 files changed, 55 insertions(+), 5 deletions(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/0e6e5c15/mgmt/RecordsConfig.cc
> --
> diff --git a/mgmt/RecordsConfig.cc b/mgmt/RecordsConfig.cc
> index 0e963c9..b0e0e6a 100644
> --- a/mgmt/RecordsConfig.cc
> +++ b/mgmt/RecordsConfig.cc
> @@ -1977,6 +1977,10 @@ static const RecordElement RecordsConfig[] =
>   ,
>   {RECT_CONFIG, "proxy.config.http2.max_concurrent_streams_in", RECD_INT, 
> "100", RECU_DYNAMIC, RR_NULL, RECC_STR, "^[0-9]+$", RECA_NULL}
>   ,
> +  {RECT_CONFIG, "proxy.config.http2.min_concurrent_streams_in", RECD_INT, 
> "10", RECU_DYNAMIC, RR_NULL, RECC_STR, "^[0-9]+$", RECA_NULL}
> +  ,
> +  {RECT_CONFIG, "proxy.config.http2.max_active_streams_in", RECD_INT, "0", 
> RECU_DYNAMIC, RR_NULL, RECC_STR, "^[0-9]+$", RECA_NULL}
> +  ,

Please document these new settings.

>   {RECT_CONFIG, "proxy.config.http2.initial_window_size_in", RECD_INT, 
> "1048576", RECU_DYNAMIC, RR_NULL, RECC_STR, "^[0-9]+$", RECA_NULL}
>   ,
>   {RECT_CONFIG, "proxy.config.http2.max_frame_size", RECD_INT, "16384", 
> RECU_DYNAMIC, RR_NULL, RECC_STR, "^[0-9]+$", RECA_NULL}
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/0e6e5c15/proxy/http2/HTTP2.cc
> --
> diff --git a/proxy/http2/HTTP2.cc b/proxy/http2/HTTP2.cc
> index 5cb9d79..c9ea2bc 100644
> --- a/proxy/http2/HTTP2.cc
> +++ b/proxy/http2/HTTP2.cc
> @@ -727,7 +727,10 @@ http2_decode_header_blocks(HTTPHdr *hdr, const uint8_t 
> *buf_start, const uint8_t
> }
> 
> // Initialize this subsystem with librecords configs (for now)
> -uint32_t Http2::max_concurrent_streams = 100;
> +uint32_t Http2::max_concurrent_streams_in = 100;
> +uint32_t Http2::min_concurrent_streams_in = 10;
> +uint32_t Http2::max_active_streams_in = 0;
> +bool Http2::throttling = false;
> uint32_t Http2::initial_window_size = 1048576;
> uint32_t Http2::max_frame_size = 16384;
> uint32_t Http2::header_table_size = 4096;
> @@ -740,7 +743,9 @@ uint32_t Http2::active_timeout_in = 0;
> void
> Http2::init()
> {
> -  REC_EstablishStaticConfigInt32U(max_concurrent_streams, 
> "proxy.config.http2.max_concurrent_streams_in");
> +  REC_EstablishStaticConfigInt32U(max_concurrent_streams_in, 
> "proxy.config.http2.max_concurrent_streams_in");
> +  REC_EstablishStaticConfigInt32U(min_concurrent_streams_in, 
> "proxy.config.http2.min_concurrent_streams_in");
> +  REC_EstablishStaticConfigInt32U(max_active_streams_in, 
> "proxy.config.http2.max_active_streams_in");
>   REC_EstablishStaticConfigInt32U(initial_window_size, 
> "proxy.config.http2.initial_window_size_in");
>   REC_EstablishStaticConfigInt32U(max_frame_size, 
> "proxy.config.http2.max_frame_size");
>   REC_EstablishStaticConfigInt32U(header_table_size, 
> "proxy.config.http2.header_table_size");
> @@ -751,7 +756,8 @@ Http2::init()
>   REC_EstablishStaticConfigInt32U(active_timeout_in, 
> "proxy.config.http2.active_timeout_in");
> 
>   // If any settings is broken, ATS should not start
> -  
> ink_release_assert(http2_settings_parameter_is_valid({HTTP2_SETTINGS_MAX_CONCURRENT_STREAMS,
>  max_concurrent_streams}) &&
> +  
> ink_release_assert(http2_settings_parameter_is_valid({HTTP2_SETTINGS_MAX_CONCURR

Re: trafficserver git commit: TS-3977: Add cache-key-genid to experimental plugins. This closes #309.

[James Peach]

> On Mar 3, 2016, at 12:16 PM, a...@apache.org wrote:
> 
> Repository: trafficserver
> Updated Branches:
>  refs/heads/master 1d1cb8fae -> fbdf0221d
> 
> 
> TS-3977: Add cache-key-genid to experimental plugins.
> This closes #309.
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/fbdf0221
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/fbdf0221
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/fbdf0221
> 
> Branch: refs/heads/master
> Commit: fbdf0221df113029bbfb4b35b650c768c900e515
> Parents: 1d1cb8f
> Author: Steven Feltner 
> Authored: Thu Oct 22 10:55:05 2015 -0700
> Committer: Alan M. Carroll 
> Committed: Thu Mar 3 14:12:56 2016 -0600
> 
> --
> NOTICE  |   7 +-
> configure.ac|  14 ++
> plugins/experimental/Makefile.am|   4 +
> .../experimental/cache_key_genid/Makefile.am|  22 +++
> .../cache_key_genid/cache_key_genid.c   | 174 +++
> 5 files changed, 218 insertions(+), 3 deletions(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/fbdf0221/NOTICE
> --
> diff --git a/NOTICE b/NOTICE
> index 15d94b6..7ee263f 100644
> --- a/NOTICE
> +++ b/NOTICE
> @@ -9,6 +9,7 @@ This product includes software developed at
>   - Comcast
>   - LinkedIn
>   - Mike Pall
> +  - GoDaddy
> 
> ~~~
> 
> @@ -44,8 +45,8 @@ Copyright (C) 2012 Oregon Health & Science University
> 
> ~~~
> 
> -healthcheck Plugin developed by GoDaddy.
> -Copyright (C) 2012 GoDaddy.
> +cache-key-genid Plugin developed by GoDaddy
> +Copyright (C) 2013 GoDaddy Operating Company, LLC
> 
> ~~~
> 
> @@ -84,7 +85,7 @@ Copyright (C) 2016 Yahoo! Inc.  All rights reserved.
> ~~~
> 
> healthchecks: Plugin for ATS healthchecks.
> -Copyright (C) 2012 Go Daddy Operating Company, LLC
> +Copyright (C) 2012 GoDaddy Operating Company, LLC
> 
> ~~~
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/fbdf0221/configure.ac
> --
> diff --git a/configure.ac b/configure.ac
> index 530d8c7..26e446e 100644
> --- a/configure.ac
> +++ b/configure.ac
> @@ -1454,6 +1454,19 @@ 
> AC_CHECK_LIB([mysqlclient],[mysql_info],[AC_SUBST([LIB_MYSQLCLIENT],["-lmysqlcli
> AC_SUBST(has_mysql)
> AM_CONDITIONAL([HAS_MYSQL], [ test "x${has_mysql}" = "x1" ])
> 
> +AC_CHECK_HEADERS([kclangc.h], [
> +  AC_CHECK_LIB([kyotocabinet], [kcdbopen], [
> +AC_SUBST([LIB_KYOTOCABINET], ["-lkyotocabinet"])
> +has_kyotocabinet=1
> +  ], [
> +has_kyotocabinet=0

The convention is to use "yes" and "no". Can we please be consistent here?

J

Re: trafficserver git commit: [TS-4024] wire tracing enhancements. This closes #337.

[James Peach]

This doesn't look like it addresses Alan's last comment about "unifying the SSL 
error reporting logic"

> On Jan 12, 2016, at 7:48 AM, shinr...@apache.org wrote:
> 
> Repository: trafficserver
> Updated Branches:
>  refs/heads/master 9399a7641 -> bb40d788b
> 
> 
> [TS-4024] wire tracing enhancements.  This closes #337.
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/bb40d788
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/bb40d788
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/bb40d788
> 
> Branch: refs/heads/master
> Commit: bb40d788bb092670b05e3d084bc14e330e3afa96
> Parents: 9399a76
> Author: ericcarlschwartz 
> Authored: Sat Nov 14 13:26:58 2015 -0800
> Committer: shinrich 
> Committed: Tue Jan 12 09:44:00 2016 -0600
> 
> --
> iocore/net/SSLConfig.cc |  8 
> iocore/net/SSLNetVConnection.cc | 39 ++--
> mgmt/RecordsConfig.cc   |  8 
> proxy/http/HttpSM.cc| 31 ++--
> 4 files changed, 52 insertions(+), 34 deletions(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/bb40d788/iocore/net/SSLConfig.cc
> --
> diff --git a/iocore/net/SSLConfig.cc b/iocore/net/SSLConfig.cc
> index 7bb60fe..63540ce 100644
> --- a/iocore/net/SSLConfig.cc
> +++ b/iocore/net/SSLConfig.cc
> @@ -307,10 +307,10 @@ SSLConfigParams::initialize()
>   REC_ReadConfigInt32(ssl_allow_client_renegotiation, 
> "proxy.config.ssl.allow_client_renegotiation");
> 
>   // SSL Wire Trace configurations
> -  REC_ReadConfigInteger(ssl_wire_trace_enabled, 
> "proxy.config.ssl.wire_trace_enabled");
> +  REC_EstablishStaticConfigInt32(ssl_wire_trace_enabled, 
> "proxy.config.ssl.wire_trace_enabled");
>   if (ssl_wire_trace_enabled) {
> // wire trace specific source ip
> -REC_ReadConfigStringAlloc(ssl_wire_trace_addr, 
> "proxy.config.ssl.wire_trace_addr");
> +REC_EstablishStaticConfigStringAlloc(ssl_wire_trace_addr, 
> "proxy.config.ssl.wire_trace_addr");
> if (ssl_wire_trace_addr) {
>   ssl_wire_trace_ip = new IpAddr();
>   ssl_wire_trace_ip->load(ssl_wire_trace_addr);
> @@ -318,8 +318,8 @@ SSLConfigParams::initialize()
>   ssl_wire_trace_ip = NULL;
> }
> // wire trace percentage of requests
> -REC_ReadConfigInteger(ssl_wire_trace_percentage, 
> "proxy.config.ssl.wire_trace_percentage");
> -REC_ReadConfigStringAlloc(ssl_wire_trace_server_name, 
> "proxy.config.ssl.wire_trace_server_name");
> +REC_EstablishStaticConfigInt32(ssl_wire_trace_percentage, 
> "proxy.config.ssl.wire_trace_percentage");
> +REC_EstablishStaticConfigStringAlloc(ssl_wire_trace_server_name, 
> "proxy.config.ssl.wire_trace_server_name");
>   } else {
> ssl_wire_trace_addr = NULL;
> ssl_wire_trace_ip = NULL;
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/bb40d788/iocore/net/SSLNetVConnection.cc
> --
> diff --git a/iocore/net/SSLNetVConnection.cc b/iocore/net/SSLNetVConnection.cc
> index 7199efa..dc48c63 100644
> --- a/iocore/net/SSLNetVConnection.cc
> +++ b/iocore/net/SSLNetVConnection.cc
> @@ -286,12 +286,16 @@ ssl_read_from_net(SSLNetVConnection *sslvc, EThread 
> *lthread, int64_t &ret)
> Debug("ssl.error", "[SSL_NetVConnection::ssl_read_from_net] 
> SSL_ERROR_ZERO_RETURN");
> break;
>   case SSL_ERROR_SSL:
> -  default:
> -TraceIn(trace, sslvc->get_remote_addr(), sslvc->get_remote_port(), 
> "SSL Error: sslErr=%d, errno=%d", sslErr, errno);
> +  default: {
> +char buf[512];
> +unsigned long e = ERR_peek_last_error();
> +ERR_error_string_n(e, buf, sizeof(buf));
> +TraceIn(trace, sslvc->get_remote_addr(), sslvc->get_remote_port(), 
> "SSL Error: sslErr=%d, ERR_get_error=%ld (%s) errno=%d",
> +sslErr, e, buf, errno);
> event = SSL_READ_ERROR;
> ret = errno;
> SSL_CLR_ERR_INCR_DYN_STAT(sslvc, ssl_error_ssl, 
> "[SSL_NetVConnection::ssl_read_from_net]: errno=%d", errno);
> -break;
> +  } break;
>   } // switch
>   break;
> } // while( block_write_avail > 0 )
> @@ -833,11 +837,15 @@ SSLNetVConnection::load_buffer_and_write(int64_t 
> towrite, int64_t &wattempted, i
>   Debug("ssl.error", "SSL_write-SSL_ERROR_ZERO_RETURN");
>   break;
> case SSL_ERROR_SSL:
> -default:
> -  TraceOut(trace, get_remote_addr(), get_remote_port(), "SSL Error: 
> sslErr=%d, errno=%d", err, errno);
> +default: {
> +  char buf[512];
> +  unsigned long e = ERR_peek_last_error();
> +  ERR_error_string_n(e, buf, sizeof(buf));
> +  TraceIn(trace, get_remot

Re: trafficserver git commit: TS-4007: ClassAllocator: don't attempt to destruct the prototype object when ClassAllocator goes out of scope

[James Peach]

> On Nov 9, 2015, at 10:38 PM, bri...@apache.org wrote:
> 
> Repository: trafficserver
> Updated Branches:
>  refs/heads/master c7610467e -> e4cb30531
> 
> 
> TS-4007: ClassAllocator: don't attempt to destruct the prototype object when 
> ClassAllocator goes out of scope
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/e4cb3053
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/e4cb3053
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/e4cb3053
> 
> Branch: refs/heads/master
> Commit: e4cb305315e4969748c9dc8414fd96c8d36b4091
> Parents: c761046
> Author: Can Selcik 
> Authored: Mon Nov 9 22:29:37 2015 -0800
> Committer: Brian Geffon 
> Committed: Mon Nov 9 22:38:24 2015 -0800
> 
> --
> lib/ts/Allocator.h | 4 +++-
> 1 file changed, 3 insertions(+), 1 deletion(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/e4cb3053/lib/ts/Allocator.h
> --
> diff --git a/lib/ts/Allocator.h b/lib/ts/Allocator.h
> index 3b489a7..a21ff90 100644
> --- a/lib/ts/Allocator.h
> +++ b/lib/ts/Allocator.h
> @@ -40,6 +40,7 @@
> #ifndef _Allocator_h_
> #define _Allocator_h_
> 
> +#include 
> #include 
> #include "ts/ink_queue.h"
> #include "ts/ink_defs.h"
> @@ -193,11 +194,12 @@ public:
>   */
>   ClassAllocator(const char *name, unsigned int chunk_size = 128, unsigned 
> int alignment = 16)
>   {
> +::new ((void*)&proto.typeObject) C();
> ink_freelist_init(&this->fl, name, RND16(sizeof(C)), chunk_size, 
> RND16(alignment));
>   }
> 
>   struct {
> -C typeObject;
> +uint8_t typeObject[sizeof(C)];

I think this should be union'ed to guarantee correct alignment?

> int64_t space_holder;
>   } proto;
> };
> 

Re: trafficserver git commit: TS-315: Add switch to disable config file generation/runtime behavior changing

[James Peach]

Hey Bryan,

I think that proxy.config.disable_configuration_modification should prevent 
rewriting the config, but you should still be able to update 
proxy.config.disable_configuration_modification out of band and have it update 
without a restart. Apart from that, disabling the update threads prevents 
proxy.node.config.reconfigure_required working correctly.


> On Oct 22, 2015, at 4:35 PM, bc...@apache.org wrote:
> 
> Repository: trafficserver
> Updated Branches:
>  refs/heads/master 3ce7dfa06 -> 3e7504eec
> 
> 
> TS-315: Add switch to disable config file generation/runtime behavior changing
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/3e7504ee
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/3e7504ee
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/3e7504ee
> 
> Branch: refs/heads/master
> Commit: 3e7504eec85bd69c91d694591968369d16ffe1c3
> Parents: 3ce7dfa
> Author: Bryan Call 
> Authored: Thu Oct 22 16:33:35 2015 -0700
> Committer: Bryan Call 
> Committed: Thu Oct 22 16:33:56 2015 -0700
> 
> --
> lib/records/RecLocal.cc   | 11 ---
> lib/records/RecProcess.cc | 17 -
> mgmt/RecordsConfig.cc |  3 ++-
> mgmt/api/TSControlMain.cc |  9 +
> 4 files changed, 31 insertions(+), 9 deletions(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/3e7504ee/lib/records/RecLocal.cc
> --
> diff --git a/lib/records/RecLocal.cc b/lib/records/RecLocal.cc
> index c97f93e..8846691 100644
> --- a/lib/records/RecLocal.cc
> +++ b/lib/records/RecLocal.cc
> @@ -209,9 +209,14 @@ RecLocalInitMessage()
> int
> RecLocalStart(FileManager *configFiles)
> {
> -  ink_thread_create(sync_thr, configFiles);
> -  ink_thread_create(config_update_thr, NULL);
> -
> +  RecInt disable_modification = 0;
> +  RecGetRecordInt("proxy.config.disable_configuration_modification", 
> &disable_modification);
> +  if (disable_modification == 1) {
> +RecDebug(DL_Debug, "Disable configuration modification");
> +  } else {
> +ink_thread_create(sync_thr, configFiles);
> +ink_thread_create(config_update_thr, NULL);
> +  }

This should start the threads but sync_thr() should check the configuration.

>   return REC_ERR_OKAY;
> }
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/3e7504ee/lib/records/RecProcess.cc
> --
> diff --git a/lib/records/RecProcess.cc b/lib/records/RecProcess.cc
> index 5cefe0f..ed95d6d 100644
> --- a/lib/records/RecProcess.cc
> +++ b/lib/records/RecProcess.cc
> @@ -491,15 +491,22 @@ RecProcessStart(void)
> 
>   Debug("statsproc", "Starting sync continuations:");
>   raw_stat_sync_cont *rssc = new raw_stat_sync_cont(new_ProxyMutex());
> -  Debug("statsproc", "\traw-stat syncer");
> +  Debug("statsproc", "raw-stat syncer");
>   raw_stat_sync_cont_event = eventProcessor.schedule_every(rssc, 
> HRTIME_MSECONDS(g_rec_raw_stat_sync_interval_ms), ET_TASK);
> 
> -  config_update_cont *cuc = new config_update_cont(new_ProxyMutex());
> -  Debug("statsproc", "\tconfig syncer");
> -  config_update_cont_event = eventProcessor.schedule_every(cuc, 
> HRTIME_MSECONDS(g_rec_config_update_interval_ms), ET_TASK);
> +  RecInt disable_modification = 0;
> +  RecGetRecordInt("proxy.config.disable_configuration_modification", 
> &disable_modification);
> +  // Schedule continuation to call the configuration callbacks if we are 
> allowed to modify configuration in RAM
> +  if (disable_modification == 1) {
> +Debug("statsproc", "Disabled configuration modification");
> +  } else {
> +config_update_cont *cuc = new config_update_cont(new_ProxyMutex());
> +Debug("statsproc", "config syncer");
> +config_update_cont_event = eventProcessor.schedule_every(cuc, 
> HRTIME_MSECONDS(g_rec_config_update_interval_ms), ET_TASK);
> +  }

This should always start the syncer, but the syncer should check the config and 
decide to do nothing.

>   sync_cont *sc = new sync_cont(new_ProxyMutex());
> -  Debug("statsproc", "\tremote syncer");
> +  Debug("statsproc", "remote syncer");
>   sync_cont_event = eventProcessor.schedule_every(sc, 
> HRTIME_MSECONDS(g_rec_remote_sync_interval_ms), ET_TASK);
> 
>   g_started = true;
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/3e7504ee/mgmt/RecordsConfig.cc
> --
> diff --git a/mgmt/RecordsConfig.cc b/mgmt/RecordsConfig.cc
> index 66c5ee6..b8bced0 100644
> --- a/mgmt/RecordsConfig.cc
> +++ b/mgmt/RecordsConfig.cc
> @@ -1069,7 +1069,8 @@ static const RecordElement RecordsConfig[] =
>   ,
>   {RECT_CONFIG, "proxy.config.hostdb.host_file.interval", RECD_INT, "86400",

Re: trafficserver git commit: [TS-3949] Prevent active timeout event from firing after PluginVC is closed.

[James Peach]

> On Sep 26, 2015, at 10:41 PM, sudhe...@apache.org wrote:
> 
> Repository: trafficserver
> Updated Branches:
>  refs/heads/master f9a6930fc -> ae9a343a9
> 
> 
> [TS-3949] Prevent active timeout event from firing
> after PluginVC is closed.
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/ae9a343a
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/ae9a343a
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/ae9a343a
> 
> Branch: refs/heads/master
> Commit: ae9a343a9ba861240d938c3c47391e3de5011a25
> Parents: f9a6930
> Author: Sudheer Vinukonda 
> Authored: Sun Sep 27 05:41:10 2015 +
> Committer: Sudheer Vinukonda 
> Committed: Sun Sep 27 05:41:10 2015 +
> 
> --
> proxy/PluginVC.cc | 6 ++
> 1 file changed, 6 insertions(+)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ae9a343a/proxy/PluginVC.cc
> --
> diff --git a/proxy/PluginVC.cc b/proxy/PluginVC.cc
> index d1072ad..fb29696 100644
> --- a/proxy/PluginVC.cc
> +++ b/proxy/PluginVC.cc
> @@ -743,6 +743,12 @@ PluginVC::process_timeout(Event **e, int event_to_send)
> {
>   ink_assert(*e == inactive_event || *e == active_event);
> 
> +  if (closed) {
> +// already closed, ignore the timeout event
> +// to avoid handle_event asserting use-after-free
> +return;
> +  }

Then isn't reading PluginVC::closed also a use-after-free?

J

Re: trafficserver git commit: TS-3928: Make TSRemapNewInstance not required in remap plugins

[James Peach]

Phil, can you please update the man page for TSRemapNewInstance?

https://docs.trafficserver.apache.org/en/latest/reference/api/TSRemap.en.html

thanks!

> On Sep 21, 2015, at 1:14 PM, sor...@apache.org wrote:
> 
> Repository: trafficserver
> Updated Branches:
>  refs/heads/master 0a58767f7 -> 90432575b
> 
> 
> TS-3928: Make TSRemapNewInstance not required in remap plugins
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/90432575
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/90432575
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/90432575
> 
> Branch: refs/heads/master
> Commit: 90432575baed23aa9e01f87cb8d8c5cedad4eb40
> Parents: 0a58767
> Author: Phil Sorber 
> Authored: Mon Sep 21 14:07:58 2015 -0600
> Committer: Phil Sorber 
> Committed: Mon Sep 21 14:07:58 2015 -0600
> 
> --
> proxy/http/remap/RemapConfig.cc | 9 -
> 1 file changed, 4 insertions(+), 5 deletions(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/90432575/proxy/http/remap/RemapConfig.cc
> --
> diff --git a/proxy/http/remap/RemapConfig.cc b/proxy/http/remap/RemapConfig.cc
> index 2013009..7715282 100644
> --- a/proxy/http/remap/RemapConfig.cc
> +++ b/proxy/http/remap/RemapConfig.cc
> @@ -786,9 +786,6 @@ remap_load_plugin(const char **argv, int argc, 
> url_mapping *mp, char *errbuf, in
>   if (!pi->fp_tsremap_init) {
> snprintf(errbuf, errbufsize, "Can't find \"%s\" function in remap 
> plugin \"%s\"", TSREMAP_FUNCNAME_INIT, c);
> retcode = -10;
> -  } else if (!pi->fp_tsremap_new_instance) {
> -snprintf(errbuf, errbufsize, "Can't find \"%s\" function in remap 
> plugin \"%s\"", TSREMAP_FUNCNAME_NEW_INSTANCE, c);
> -retcode = -11;
>   } else if (!pi->fp_tsremap_do_remap) {
> snprintf(errbuf, errbufsize, "Can't find \"%s\" function in remap 
> plugin \"%s\"", TSREMAP_FUNCNAME_DO_REMAP, c);
> retcode = -12;
> @@ -862,8 +859,10 @@ remap_load_plugin(const char **argv, int argc, 
> url_mapping *mp, char *errbuf, in
> 
>   Debug("remap_plugin", "creating new plugin instance");
> 
> -  TSReturnCode res = TS_ERROR;
> -  res = pi->fp_tsremap_new_instance(parc, parv, &ih, tmpbuf, sizeof(tmpbuf) 
> - 1);
> +  TSReturnCode res = TS_SUCCESS;
> +  if (pi->fp_tsremap_new_instance) {
> +res = pi->fp_tsremap_new_instance(parc, parv, &ih, tmpbuf, 
> sizeof(tmpbuf) - 1);
> +  }
> 
>   Debug("remap_plugin", "done creating new plugin instance");
> 
> 

Re: trafficserver git commit: Coverity issues: making the sanity check calls useful!

[James Peach]

This has multiple, independent changes, so it should have been a series of 
separate commits. I think it is also worth referencing the Coverity CID (I 
assume that this fixes a number of them?)


> On Aug 11, 2015, at 2:18 PM, a...@apache.org wrote:
> 
> Repository: trafficserver
> Updated Branches:
>  refs/heads/master f65b867bd -> 3d0d2564c
> 
> 
> Coverity issues: making the sanity check calls useful!
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/3d0d2564
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/3d0d2564
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/3d0d2564
> 
> Branch: refs/heads/master
> Commit: 3d0d2564cc91bcb32174f1b8a61d903a30eda8b5
> Parents: f65b867
> Author: Syeda Persia Aziz 
> Authored: Mon Jul 20 18:16:17 2015 -0500
> Committer: Alan M. Carroll 
> Committed: Tue Aug 11 16:00:59 2015 -0500
> 
> --
> iocore/cache/RamCacheCLFUS.cc   |  2 +
> lib/atscppapi/src/Plugin.cc |  3 +-
> lib/ts/ParseRules.cc| 83 
> lib/ts/ParseRules.h |  2 -
> mgmt/Alarms.cc  |  1 +
> mgmt/LocalManager.cc|  2 +-
> mgmt/MultiFile.cc   |  1 +
> mgmt/api/EventCallback.cc   | 82 ++-
> .../stale_while_revalidate.c|  1 +
> proxy/InkAPI.cc | 20 ++---
> proxy/SocksProxy.cc |  6 +-
> proxy/congest/CongestionTest.cc |  5 ++
> proxy/http/HttpClientSession.cc |  2 +-
> proxy/http2/Http2ClientSession.cc   |  1 +
> proxy/logging/Log.cc|  1 +
> 15 files changed, 72 insertions(+), 140 deletions(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/3d0d2564/iocore/cache/RamCacheCLFUS.cc
> --
> diff --git a/iocore/cache/RamCacheCLFUS.cc b/iocore/cache/RamCacheCLFUS.cc
> index 0e051b2..8eb72ce 100644
> --- a/iocore/cache/RamCacheCLFUS.cc
> +++ b/iocore/cache/RamCacheCLFUS.cc
> @@ -310,6 +310,8 @@ RamCacheCLFUS::tick()
> return;
>   e = lru[1].dequeue();
> Lfree:
> +  if (!e) // e may be NULL after e= lru[1].dequeue()
> +return;
>   e->flag_bits.lru = 0;
>   history--;
>   uint32_t b = e->key.slice32(3) % nbuckets;
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/3d0d2564/lib/atscppapi/src/Plugin.cc
> --
> diff --git a/lib/atscppapi/src/Plugin.cc b/lib/atscppapi/src/Plugin.cc
> index 6de4391..176a030 100644
> --- a/lib/atscppapi/src/Plugin.cc
> +++ b/lib/atscppapi/src/Plugin.cc
> @@ -34,5 +34,6 @@ atscppapi::RegisterGlobalPlugin(std::string name, 
> std::string vendor, std::strin
>   info.plugin_name = const_cast(name.c_str());
>   info.vendor_name = const_cast(vendor.c_str());
>   info.support_email = const_cast(email.c_str());
> -  TSPluginRegister(&info);
> +  if (TSPluginRegister(&info) != TS_SUCCESS)
> +TSError("[Plugin.cc] Plugin registration failed.");
> }
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/3d0d2564/lib/ts/ParseRules.cc
> --
> diff --git a/lib/ts/ParseRules.cc b/lib/ts/ParseRules.cc
> index ffc4a4e..6bd61b7 100644
> --- a/lib/ts/ParseRules.cc
> +++ b/lib/ts/ParseRules.cc
> @@ -41,89 +41,6 @@ const char parseRulesCTypeToLower[256] = {
> #include "ParseRulesCTypeToLower"
> };
> 
> -unsigned char *
> -ParseRules::scan_while(unsigned char *ptr, unsigned int n, uint32_t bitmask)
> -{
> -  unsigned int i;
> -  uint32_t *wptr;
> -  unsigned char *align_ptr;
> -  uintptr_t f_bytes, b_bytes, words, align_off;
> -
> -  align_off = ((uintptr_t)ptr & 3);
> -  align_ptr = (unsigned char *)(((uintptr_t)ptr) & ~3);
> -
> -  f_bytes = (align_off ? 4 - align_off : 0);
> -
> -  words = (n - f_bytes) >> 2;
> -
> -  if (words == 0) {
> -for (i = 0; i < n; i++)
> -  if (!is_type(ptr[i], bitmask))
> -return (&ptr[i]);
> -  } else {
> -wptr = ((uint32_t *)align_ptr) + (align_off ? 1 : 0);
> -switch (align_off) {
> -case 1:
> -  if (!is_type(align_ptr[1], bitmask))
> -return (&ptr[1]);
> -case 2:
> -  if (!is_type(align_ptr[2], bitmask))
> -return (&ptr[2]);
> -case 3:
> -  if (!is_type(align_ptr[3], bitmask))
> -return (&ptr[3]);
> -  break;
> -default:
> -  break;
> -}
> -
> -b_bytes = n - ((words << 2) + f_bytes);
> -
> -for (i = 0; i < words; i++) {
> -  uint32_t word = wptr[i];
> -  uint32_t result = (is_type(((word >> 0)

Re: trafficserver git commit: TS-3497: Define Http2Error to classify errors

[James Peach]

> On Aug 5, 2015, at 9:05 AM, zw...@apache.org wrote:
> 
> Repository: trafficserver
> Updated Branches:
>  refs/heads/master 7d63eae28 -> 0fc4dcdea
> 
> 
> TS-3497: Define Http2Error to classify errors
> 
> This closes #267
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/0fc4dcde
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/0fc4dcde
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/0fc4dcde
> 
> Branch: refs/heads/master
> Commit: 0fc4dcdea9449929c440e54d301144edefbb721f
> Parents: 7d63eae
> Author: Masakazu Kitajo 
> Authored: Wed Aug 5 01:08:19 2015 +0900
> Committer: Leif Hedstrom 
> Committed: Wed Aug 5 10:03:49 2015 -0600
> 
> --
> proxy/http2/HTTP2.h |  20 +++
> proxy/http2/Http2ConnectionState.cc | 211 ++-
> 2 files changed, 145 insertions(+), 86 deletions(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/0fc4dcde/proxy/http2/HTTP2.h
> --
> diff --git a/proxy/http2/HTTP2.h b/proxy/http2/HTTP2.h
> index a398e0e..3183ec0 100644
> --- a/proxy/http2/HTTP2.h
> +++ b/proxy/http2/HTTP2.h
> @@ -78,6 +78,14 @@ extern RecRawStatBlock *http2_rsb; // Container for 
> statistics.
> // 6.9.1 The Flow Control Window
> static const Http2WindowSize HTTP2_MAX_WINDOW_SIZE = 0x7FFF;
> 
> +// 5.4.  Error Handling
> +enum Http2ErrorClass {
> +  HTTP2_ERROR_CLASS_NONE,
> +  HTTP2_ERROR_CLASS_CONNECTION,
> +  HTTP2_ERROR_CLASS_STREAM,
> +};
> +
> +// 7.  Error Codes
> enum Http2ErrorCode {
>   HTTP2_ERROR_NO_ERROR = 0,
>   HTTP2_ERROR_PROTOCOL_ERROR = 1,
> @@ -213,6 +221,18 @@ struct Http2FrameHeader {
>   Http2StreamId streamid;
> };
> 
> +// 5.4.  Error Handling
> +struct Http2Error {
> +  Http2Error(const Http2ErrorClass error_class = HTTP2_ERROR_CLASS_NONE, 
> const Http2ErrorCode error_code = HTTP2_ERROR_NO_ERROR)
> +  {
> +cls = error_class;
> +code = error_code;
> +  };

Add a 
static Http2Error success() {
return Http2Error(HTTP2_ERROR_CLASS_NONE, HTTP2_ERROR_NO_ERROR);
}

then all the times you return Http2Error(HTTP2_ERROR_CLASS_NONE) will read 
Http2Error::success(), which is easier to read. I'd also add

bool is_success() const { ... }


> +
> +  Http2ErrorClass cls;
> +  Http2ErrorCode code;
> +};
> +
> // 6.5.1. SETTINGS Format
> struct Http2SettingsParameter {
>   uint16_t id;
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/0fc4dcde/proxy/http2/Http2ConnectionState.cc
> --
> diff --git a/proxy/http2/Http2ConnectionState.cc 
> b/proxy/http2/Http2ConnectionState.cc
> index 3f61b4b..c30dd7a 100644
> --- a/proxy/http2/Http2ConnectionState.cc
> +++ b/proxy/http2/Http2ConnectionState.cc
> @@ -30,7 +30,7 @@
> // Currently use only HTTP/1.1 for requesting to origin server
> const static char *HTTP2_FETCHING_HTTP_VERSION = "HTTP/1.1";
> 
> -typedef Http2ErrorCode (*http2_frame_dispatch)(Http2ClientSession &, 
> Http2ConnectionState &, const Http2Frame &);
> +typedef Http2Error (*http2_frame_dispatch)(Http2ClientSession &, 
> Http2ConnectionState &, const Http2Frame &);
> 
> static const int buffer_size_index[HTTP2_FRAME_TYPE_MAX] = {
>   BUFFER_SIZE_INDEX_8K,  // HTTP2_FRAME_TYPE_DATA
> @@ -60,7 +60,7 @@ read_rcv_buffer(char *buf, size_t bufsize, unsigned 
> &nbytes, const Http2Frame &f
>   return end - buf;
> }
> 
> -static Http2ErrorCode
> +static Http2Error
> rcv_data_frame(Http2ClientSession &cs, Http2ConnectionState &cstate, const 
> Http2Frame &frame)
> {
>   char buf[BUFFER_SIZE_FOR_INDEX(buffer_size_index[HTTP2_FRAME_TYPE_DATA])];
> @@ -71,24 +71,25 @@ rcv_data_frame(Http2ClientSession &cs, 
> Http2ConnectionState &cstate, const Http2
> 
>   DebugSsn(&cs, "http2_cs", "[%" PRId64 "] Received DATA frame.", 
> cs.connection_id());
> 
> +  // If a DATA frame is received whose stream identifier field is 0x0, the 
> recipient MUST
> +  // respond with a connection error of type PROTOCOL_ERROR.
>   if (!http2_is_client_streamid(id)) {
> -return HTTP2_ERROR_PROTOCOL_ERROR;
> +return Http2Error(HTTP2_ERROR_CLASS_CONNECTION, 
> HTTP2_ERROR_PROTOCOL_ERROR);
>   }
> 
>   Http2Stream *stream = cstate.find_stream(id);
>   if (stream == NULL) {
> if (id <= cstate.get_latest_stream_id()) {
> -  return HTTP2_ERROR_STREAM_CLOSED;
> +  return Http2Error(HTTP2_ERROR_CLASS_CONNECTION, 
> HTTP2_ERROR_STREAM_CLOSED);
> } else {
> -  return HTTP2_ERROR_PROTOCOL_ERROR;
> +  return Http2Error(HTTP2_ERROR_CLASS_CONNECTION, 
> HTTP2_ERROR_PROTOCOL_ERROR);
> }
>   }
> 
>   // If a DATA frame is received whose stream is not in "open" or "half 
> closed (local)" state,
>   // the recipient M

Re: [1/6] trafficserver git commit: Document which configurations are overwritable

[James Peach]

Do you mean "overridable"? TSHttpOverridableConfig().


> On Jun 8, 2015, at 3:46 PM, mlib...@apache.org wrote:
> 
> Repository: trafficserver
> Updated Branches:
>  refs/heads/master 9bb97769e -> dfb14525a
> 
> 
> Document which configurations are overwritable
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/2198745f
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/2198745f
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/2198745f
> 
> Branch: refs/heads/master
> Commit: 2198745f15902832b8d110a2a4eefcd31901b3c3
> Parents: f0a6575
> Author: Miles Libbey 
> Authored: Tue Apr 7 10:03:49 2015 -0700
> Committer: Miles Libbey 
> Committed: Mon Jun 8 15:44:46 2015 -0700
> 
> --
> doc/ext/traffic-server.py   |  3 +
> .../configuration/records.config.en.rst | 77 ++--
> 2 files changed, 75 insertions(+), 5 deletions(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/2198745f/doc/ext/traffic-server.py
> --
> diff --git a/doc/ext/traffic-server.py b/doc/ext/traffic-server.py
> index 4b2d9ec..30cb26c 100644
> --- a/doc/ext/traffic-server.py
> +++ b/doc/ext/traffic-server.py
> @@ -49,6 +49,7 @@ class TSConfVar(std.Target):
> 'class' : rst.directives.class_option,
> 'reloadable' : rst.directives.flag,
> 'deprecated' : rst.directives.flag,
> +'overwritable' : rst.directives.flag,
> 'metric' : rst.directives.unchanged,
> }
> required_arguments = 3
> @@ -117,6 +118,8 @@ class TSConfVar(std.Target):
> fl.append(self.make_field('Metric', self.options['metric']))
> if ('reloadable' in self.options):
> fl.append(self.make_field('Reloadable', 'Yes'))
> +if ('overwritable' in self.options):
> +fl.append(self.make_field('Overwritable', 'Yes'))
> if ('deprecated' in self.options):
> fl.append(self.make_field('Deprecated', 'Yes'))
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/2198745f/doc/reference/configuration/records.config.en.rst
> --
> diff --git a/doc/reference/configuration/records.config.en.rst 
> b/doc/reference/configuration/records.config.en.rst
> index 93b8f7b..e3e7579 100644
> --- a/doc/reference/configuration/records.config.en.rst
> +++ b/doc/reference/configuration/records.config.en.rst
> @@ -53,6 +53,9 @@ A variable marked as ``Reloadable`` can be updated via the 
> command::
> 
>traffic_line -x
> 
> +A variable marked as ``Overwritable`` can be changed on a per-remap basis 
> using plugins
> +(like the :ref:`conf-remap-plugin`).
> +
> ``INT`` type configurations are expressed as any normal integer,
> e.g. *32768*. They can also be expressed using more human readable values
> using standard prefixes, e.g. *32K*. The following prefixes are supported
> @@ -140,7 +143,7 @@ System Variables
> 
>The name of the product.
> 
> -.. ts:cv:: CONFIG proxy.config.proxy_name STRING ``build_machine``
> +.. ts:cv:: CONFIG proxy.config.proxy_name STRING build_machine
>:reloadable:
> 
>The name of the Traffic Server node.
> @@ -389,6 +392,7 @@ this be local"
>Set the interface to use for cluster communications.
> 
> .. ts:cv:: CONFIG proxy.config.http.cache.cluster_cache_local INT 0
> +   :overwritable:
> 
>This turns on the local caching of objects in cluster mode. The point of
>this is to allow for popular or **hot** content to be cached on all nodes
> @@ -627,6 +631,7 @@ Supports both wildcards ('\*') and ranges ("0-1023").
> 
> .. ts:cv:: CONFIG proxy.config.http.insert_request_via_str INT 1
>:reloadable:
> +   :overwritable:
> 
>Set how the ``Via`` field is handled on a request to the origin server.
> 
> @@ -645,6 +650,7 @@ Value Effect
> 
> .. ts:cv:: CONFIG proxy.config.http.insert_response_via_str INT 0
>:reloadable:
> +   :overwritable:
> 
>Set how the ``Via`` field is handled on the response to the client.
> 
> @@ -671,6 +677,7 @@ Value Effect
> 
> .. ts:cv:: CONFIG proxy.config.http.response_server_enabled INT 1
>:reloadable:
> +   :overwritable:
> 
>You can specify one of the following:
> 
> @@ -680,6 +687,7 @@ Value Effect
> 
> .. ts:cv:: CONFIG proxy.config.http.insert_age_in_response INT 1
>:reloadable:
> +   :overwritable:
> 
>This option specifies whether Traffic Server should insert an ``Age`` 
> header in the response. The Age field value is the cache's
>estimate of the amount of time since the response was generated or 
> revalidated by the origin server.
> @@ -689,6 +697,7 @@ Value Effect
> 
> .. ts:cv:: CONFIG proxy.config.http.response_server_str STR

Re: trafficserver git commit: TS-3122: Add support for hugepages on Linux

[James Peach]

> On Jun 8, 2015, at 8:40 AM, sor...@apache.org wrote:
> 
> Repository: trafficserver
> Updated Branches:
>  refs/heads/master 1a0832b3b -> bba557870
> 
> 
> TS-3122: Add support for hugepages on Linux
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/bba55787
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/bba55787
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/bba55787
> 
> Branch: refs/heads/master
> Commit: bba557870c05222d302a05ec948871cdde8bf63b
> Parents: 1a0832b
> Author: Phil Sorber 
> Authored: Thu Oct 16 19:58:08 2014 -0600
> Committer: Phil Sorber 
> Committed: Mon Jun 8 09:28:56 2015 -0600
> 
> --
> .../configuration/records.config.en.rst |  12 ++
> iocore/cache/Cache.cc   |  10 +-
> iocore/cache/CacheDir.cc|  53 +--
> iocore/cache/P_CacheDir.h   |   4 +-
> lib/ts/Makefile.am  |   2 +
> lib/ts/hugepages.cc | 144 +++
> lib/ts/hugepages.h  |  32 +
> lib/ts/ink_queue.cc |  21 ++-
> lib/ts/libts.h  |   1 +
> mgmt/RecordsConfig.cc   |   2 +
> proxy/Main.cc   |   7 +
> 11 files changed, 267 insertions(+), 21 deletions(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/bba55787/doc/reference/configuration/records.config.en.rst
> --
> diff --git a/doc/reference/configuration/records.config.en.rst 
> b/doc/reference/configuration/records.config.en.rst
> index 694d338..ccced34 100644
> --- a/doc/reference/configuration/records.config.en.rst
> +++ b/doc/reference/configuration/records.config.en.rst
> @@ -2849,6 +2849,18 @@ Sockets
>Sets the minimum number of items a ProxyAllocator (per-thread) will 
> guarantee to be
>holding at any one time.
> 
> +.. ts:cv:: CONFIG proxy.config.allocator.hugepages INT 0
> +
> +   Enable (1) the use of huge pages on supported platforms. (Currently only 
> Linux)
> +
> +   You must also enable hugepages at the OS level. In a modern linux Kernel
> +   this can be done by setting ``/proc/sys/vm/nr_overcommit_hugepages`` to a
> +   sufficiently large value. It is reasonable to use (system
> +   memory/hugepage size) because these pages are only created on demand.
> +
> +   For more information on the implications of enabling huge pages, see
> +   `Wikipedia 
> _`.
> +
> .. ts:cv:: CONFIG proxy.config.http.enabled INT 1

I think it would be very helpful to hear about your experiences with this 
feature ... how about starting a thread on dev@?

> 
>Turn on or off support for HTTP proxying. This is rarely used, the one
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/bba55787/iocore/cache/Cache.cc
> --
> diff --git a/iocore/cache/Cache.cc b/iocore/cache/Cache.cc
> index becf713..370c516 100644
> --- a/iocore/cache/Cache.cc
> +++ b/iocore/cache/Cache.cc
> @@ -38,6 +38,8 @@
> #include "P_CacheBC.h"
> #endif
> 
> +#include "hugepages.h"
> +
> // Compilation Options
> #define USELESS_REENABLES // allow them for now
> // #define VERIFY_JTEST_DATA
> @@ -1495,7 +1497,13 @@ Vol::init(char *s, off_t blocks, off_t dir_skip, bool 
> clear)
> 
>   Debug("cache_init", "allocating %zu directory bytes for a %lld byte volume 
> (%lf%%)", vol_dirlen(this), (long long)this->len,
> (double)vol_dirlen(this) / (double)this->len * 100.0);
> -  raw_dir = (char *)ats_memalign(ats_pagesize(), vol_dirlen(this));
> +
> +  raw_dir = NULL;
> +  if (ats_hugepage_enabled())
> +raw_dir = (char *)ats_alloc_hugepage(vol_dirlen(this));
> +  if (raw_dir == NULL)
> +raw_dir = (char *)ats_memalign(ats_pagesize(), vol_dirlen(this));
> +
>   dir = (Dir *)(raw_dir + vol_headerlen(this));
>   header = (VolHeaderFooter *)raw_dir;
>   footer = (VolHeaderFooter *)(raw_dir + vol_dirlen(this) - 
> ROUND_TO_STORE_BLOCK(sizeof(VolHeaderFooter)));
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/bba55787/iocore/cache/CacheDir.cc
> --
> diff --git a/iocore/cache/CacheDir.cc b/iocore/cache/CacheDir.cc
> index 3a7b9c4..e0f20d3 100644
> --- a/iocore/cache/CacheDir.cc
> +++ b/iocore/cache/CacheDir.cc
> @@ -24,6 +24,8 @@
> 
> #include "P_Cache.h"
> 
> +#include "hugepages.h"
> +
> // #define LOOP_CHECK_MODE 1
> #ifdef LOOP_CHECK_MODE
> #define DIR_LOOP_THRESHOLD 1000
> @@ -1011,6 +1013,7 @@ sync_cache_dir_on_shutdown(void

Re: [1/2] trafficserver git commit: TS-3545: Make traffic_line and traffic_ctl more verbose Add message when setting config options that don't require a restart

[James Peach]

> On Apr 24, 2015, at 1:45 PM, bc...@apache.org wrote:
> 
> 
[snip]
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/27cd3df7/cmd/traffic_ctl/config.cc
> --
> diff --git a/cmd/traffic_ctl/config.cc b/cmd/traffic_ctl/config.cc
> index ce19f3e..ec9019a 100644
> --- a/cmd/traffic_ctl/config.cc
> +++ b/cmd/traffic_ctl/config.cc
> @@ -232,7 +232,7 @@ config_set(unsigned argc, const char **argv)
> printf("set %s, restart required\n", file_arguments[0]);
> break;
>   case TS_ACTION_RECONFIGURE:
> -// printf("Set %s, reconfiguration required\n", file_arguments[0]);
> +printf("set %s, please wait 10 seconds for traffic server to sync 
> configuration, restart is not required\n", file_arguments[0]);

10s is not true is it? AFAIK this depends on 
proxy.config.config_update_interval_ms (and possibly other stuff).

J

Re: trafficserver git commit: TS-3479 Remove verbose Warning() from HTTP/2 connections

[James Peach]

> On Apr 2, 2015, at 8:55 PM, Leif Hedstrom  wrote:
> 
>> 
>> On Apr 2, 2015, at 9:51 PM, Leif Hedstrom  wrote:
>> 
>>> 
>>> On Apr 1, 2015, at 6:08 PM, James Peach  wrote:
>>> 
>>> 
>>>> On Apr 1, 2015, at 3:50 PM, zw...@apache.org wrote:
>>>> 
>>> [snip]
>>>> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/26f438d4/proxy/http2/Http2SessionAccept.cc
>>>> --
>>>> diff --git a/proxy/http2/Http2SessionAccept.cc 
>>>> b/proxy/http2/Http2SessionAccept.cc
>>>> index 5fe2ce7..fbb25db 100644
>>>> --- a/proxy/http2/Http2SessionAccept.cc
>>>> +++ b/proxy/http2/Http2SessionAccept.cc
>>>> @@ -40,7 +40,7 @@ Http2SessionAccept::accept(NetVConnection *netvc, 
>>>> MIOBuffer *iobuf, IOBufferRead
>>>> {
>>>> // XXX we need to refactor the ACL checks from HttpSessionAccept so that 
>>>> we can invoke them here, and also in
>>>> // the SPDY protocol layer ...
>>>> -  Warning("skipping access control checks for HTTP/2 connection");
>>>> +  // Warning("skipping access control checks for HTTP/2 connection");
>>> 
>>> I added this warning so that someone would wire up the access control 
>>> checks for HTTP/2. I don't think we should remove the warning without doing 
>>> that :)
>> 
>> 
>> I disagree, the Warning() makes the entire HTTP2 feature useless since it’ll 
>> fill up the disk :/.

yeh filling up the disk is bad

>> Feel free to make it a single warning on startup maybe ?
> 
> 
> I filed a Jira though, since there was none.

thanks .. and thanks for the other patch ;)

J

Re: trafficserver git commit: TS-3470 traffic_server --help segfaults when SPDY is enabled

[James Peach]

> On Mar 31, 2015, at 6:51 PM, Leif Hedstrom  wrote:
> 
>> 
>> On Mar 31, 2015, at 1:39 PM, James Peach  wrote:
>> 
>> 
>>> On Mar 31, 2015, at 10:17 AM, zw...@apache.org wrote:
>>> 
>>> Repository: trafficserver
>>> Updated Branches:
>>> refs/heads/master e13d8a6e8 -> e0de1d6ed
>>> 
>>> 
>>> TS-3470 traffic_server --help segfaults when SPDY is enabled
>>> 
>>> 
>> [snip]
>>> diff --git a/lib/ts/ink_args.cc b/lib/ts/ink_args.cc
>>> index 12ab5c0..91cebd8 100644
>>> --- a/lib/ts/ink_args.cc
>>> +++ b/lib/ts/ink_args.cc
>>> @@ -323,5 +323,5 @@ usage(const ArgumentDescription *argument_descriptions, 
>>> unsigned n_argument_desc
>>>   }
>>>   fprintf(stderr, " %s\n", argument_descriptions[i].description);
>>> }
>>> -  exit(EX_USAGE);
>>> +  _exit(EX_USAGE);
>>> }
>> 
>> What's the underlying bug here? Note that using _exit() everywhere defeats 
>> tcmalloc leak detection and seems to be a bit of a cargo cult in our code ...
> 
> 
> 
> It’d segfault in SPDY as part of the destructor for some global object (I 
> think). I didn’t feel it worthwhile to try to fix SPDY, since we’re killing 
> it, and exiting out of the “usage()” like this hardly can have an impact on 
> e.g. tcmalloc (would we care to test for memory leaks in “traffic_server 
> —help”?).

FWIW, we have ~70 instances of _exit(). It's currently not possible to leak 
check with tcmalloc without hacking the build.

> 
> We can certainly clean up the use of _exit(), I’m all for that.

yes that would be good imho

J

Re: trafficserver git commit: TS-3479 Remove verbose Warning() from HTTP/2 connections

[James Peach]

> On Apr 1, 2015, at 3:50 PM, zw...@apache.org wrote:
> 
[snip]
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/26f438d4/proxy/http2/Http2SessionAccept.cc
> --
> diff --git a/proxy/http2/Http2SessionAccept.cc 
> b/proxy/http2/Http2SessionAccept.cc
> index 5fe2ce7..fbb25db 100644
> --- a/proxy/http2/Http2SessionAccept.cc
> +++ b/proxy/http2/Http2SessionAccept.cc
> @@ -40,7 +40,7 @@ Http2SessionAccept::accept(NetVConnection *netvc, MIOBuffer 
> *iobuf, IOBufferRead
> {
>   // XXX we need to refactor the ACL checks from HttpSessionAccept so that we 
> can invoke them here, and also in
>   // the SPDY protocol layer ...
> -  Warning("skipping access control checks for HTTP/2 connection");
> +  // Warning("skipping access control checks for HTTP/2 connection");

I added this warning so that someone would wire up the access control checks 
for HTTP/2. I don't think we should remove the warning without doing that :)

J

Re: trafficserver git commit: TS-3470 traffic_server --help segfaults when SPDY is enabled

[James Peach]

> On Mar 31, 2015, at 10:17 AM, zw...@apache.org wrote:
> 
> Repository: trafficserver
> Updated Branches:
>  refs/heads/master e13d8a6e8 -> e0de1d6ed
> 
> 
> TS-3470 traffic_server --help segfaults when SPDY is enabled
> 
> 
[snip]
> diff --git a/lib/ts/ink_args.cc b/lib/ts/ink_args.cc
> index 12ab5c0..91cebd8 100644
> --- a/lib/ts/ink_args.cc
> +++ b/lib/ts/ink_args.cc
> @@ -323,5 +323,5 @@ usage(const ArgumentDescription *argument_descriptions, 
> unsigned n_argument_desc
> }
> fprintf(stderr, " %s\n", argument_descriptions[i].description);
>   }
> -  exit(EX_USAGE);
> +  _exit(EX_USAGE);
> }

What's the underlying bug here? Note that using _exit() everywhere defeats 
tcmalloc leak detection and seems to be a bit of a cargo cult in our code ...

J

Re: trafficserver git commit: TS-2709: ATS does not send close-notify on shutdown.

[James Peach]

> On Mar 26, 2015, at 1:18 PM, shinr...@apache.org wrote:
> 
[snip]
> --
> diff --git a/iocore/net/SSLNetVConnection.cc b/iocore/net/SSLNetVConnection.cc
> index 3d58072..a1988f2 100644
> --- a/iocore/net/SSLNetVConnection.cc
> +++ b/iocore/net/SSLNetVConnection.cc
> @@ -762,6 +762,43 @@ SSLNetVConnection::SSLNetVConnection()
> {
> }
> 
> +void 
> +SSLNetVConnection::do_io_close(int lerrno)
> +{
> +  if (this->ssl != NULL && sslHandShakeComplete) {
> +int new_shutdown_mode = 0, shutdown_mode = 0;
> +if (this->lerrno < 0) {

Why do you look at VConnection::lerrno instead of the lerrno argument?

> +  new_shutdown_mode = SSL_SENT_SHUTDOWN|SSL_RECEIVED_SHUTDOWN;
> +} else {
> +  shutdown_mode = SSL_get_shutdown(ssl);
> +  Debug("ssl-shutdown", "previous shutdown state 0x%x", shutdown_mode);
> +  new_shutdown_mode = shutdown_mode | SSL_RECEIVED_SHUTDOWN;
> +}
> +if (new_shutdown_mode != shutdown_mode) {
> +  // We do not need to sit around and wait for the client's close-notify 
> if
> +  // they have not already sent it.  We will still be standards compliant
> +  Debug("ssl-shutdown", "new SSL_set_shutdown 0x%x", new_shutdown_mode);
> +  SSL_set_shutdown(ssl, new_shutdown_mode);
> +}
> +
> +// If the peer has already sent a FIN, don't bother with the shutdown
> +// They will just send us a RST for our troubles
> +// This test is not foolproof.  The client's fin could be on the wire 
> +// at the same time we send the close-notify.  If so, the client will 
> likely
> +// send RST anyway
> +char c;
> +ssize_t x = recv(this->con.fd, &c, 1, MSG_PEEK);
> +// x < 0 means error.  x == 0 means fin sent
> +if (x != 0) {
> +  // Send the close-notify
> +  int ret = SSL_shutdown(ssl);
> +  Debug("ssl-shutdown", "SSL_shutdown %s", (ret)?"success":"failed");
> +}
> +  }
> +  // Go on and do the unix socket cleanups
> +  super::do_io_close(lerrno);
> +}
> +
> void
> SSLNetVConnection::free(EThread *t)
> {
> @@ -780,8 +817,6 @@ SSLNetVConnection::free(EThread *t)
>   closed = 0;
>   ink_assert(con.fd == NO_FD);
>   if (ssl != NULL) {
> -/*if (sslHandShakeComplete)
> -   SSL_set_shutdown(ssl, SSL_SENT_SHUTDOWN|SSL_RECEIVED_SHUTDOWN); */
> SSL_free(ssl);
> ssl = NULL;
>   }
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/03734d05/iocore/net/SSLUtils.cc
> --
> diff --git a/iocore/net/SSLUtils.cc b/iocore/net/SSLUtils.cc
> index b813aee..75a44a7 100644
> --- a/iocore/net/SSLUtils.cc
> +++ b/iocore/net/SSLUtils.cc
> @@ -1238,8 +1238,6 @@ SSLInitServerContext(const SSLConfigParams *params, 
> const ssl_user_config &sslMu
>   SSL_CTX_set_options(ctx, SSL_OP_SAFARI_ECDHE_ECDSA_BUG);
> #endif
> 
> -  SSL_CTX_set_quiet_shutdown(ctx, 1);
> -
>   // pass phrase dialog configuration
>   passphrase_cb_userdata ud(params, sslMultCertSettings.dialog, 
> sslMultCertSettings.first_cert, sslMultCertSettings.key);
> 
> 

Re: [1/3] trafficserver git commit: TS-3437: Make DH params configurable

[James Peach]

The added configuration is "proxy.config.ssl.server.enable_dhparams", but the 
code checks "proxy.config.ssl.client.enable_dhparams".

The added configuration is marked RECU_RESTART_TS, but will actually take 
effect when ssl_multicert.config is reloaded. So I think this should be 
RECU_DYNAMIC (though this is almost an intermediate state).

> On Mar 11, 2015, at 3:17 PM, bri...@apache.org wrote:
> 
> Repository: trafficserver
> Updated Branches:
>  refs/heads/master 66bdd406f -> 4361f4d0d
> 
> 
> TS-3437: Make DH params configurable
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/091b59ca
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/091b59ca
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/091b59ca
> 
> Branch: refs/heads/master
> Commit: 091b59ca3f772ebc4a6cbc832b57fb0794c6b82e
> Parents: 66bdd40
> Author: Brian Geffon 
> Authored: Wed Mar 11 15:16:07 2015 -0700
> Committer: Brian Geffon 
> Committed: Wed Mar 11 15:16:07 2015 -0700
> 
> --
> iocore/net/P_SSLConfig.h | 1 +
> iocore/net/SSLConfig.cc  | 2 ++
> iocore/net/SSLUtils.cc   | 4 +++-
> mgmt/RecordsConfig.cc| 2 ++
> 4 files changed, 8 insertions(+), 1 deletion(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/091b59ca/iocore/net/P_SSLConfig.h
> --
> diff --git a/iocore/net/P_SSLConfig.h b/iocore/net/P_SSLConfig.h
> index cda2dcb..376036d 100644
> --- a/iocore/net/P_SSLConfig.h
> +++ b/iocore/net/P_SSLConfig.h
> @@ -81,6 +81,7 @@ struct SSLConfigParams : public ConfigInfo
>   char *  clientKeyPath;
>   char *  clientCACertFilename;
>   char *  clientCACertPath;
> +  int enable_dhparams;
>   int clientVerify;
>   int client_verify_depth;
>   longssl_ctx_options;
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/091b59ca/iocore/net/SSLConfig.cc
> --
> diff --git a/iocore/net/SSLConfig.cc b/iocore/net/SSLConfig.cc
> index 627ccd2..1e1c0df 100644
> --- a/iocore/net/SSLConfig.cc
> +++ b/iocore/net/SSLConfig.cc
> @@ -74,6 +74,7 @@ SSLConfigParams::SSLConfigParams()
> 
>   clientCertLevel = client_verify_depth = verify_depth = clientVerify = 0;
> 
> +  enable_dhparams = 0;
>   ssl_ctx_options = 0;
>   ssl_client_ctx_protocols = 0;
>   ssl_session_cache = SSL_SESSION_CACHE_MODE_SERVER_ATS_IMPL;
> @@ -159,6 +160,7 @@ SSLConfigParams::initialize()
>   REC_ReadConfigStringAlloc(cipherSuite, 
> "proxy.config.ssl.server.cipher_suite");
>   REC_ReadConfigStringAlloc(client_cipherSuite, 
> "proxy.config.ssl.client.cipher_suite");
>   dhparamsFile = 
> RecConfigReadConfigPath("proxy.config.ssl.server.dhparams_file");
> +  REC_ReadConfigInt32(enable_dhparams, 
> "proxy.config.ssl.client.enable_dhparams");
> 
>   int options;
>   int client_ssl_options;
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/091b59ca/iocore/net/SSLUtils.cc
> --
> diff --git a/iocore/net/SSLUtils.cc b/iocore/net/SSLUtils.cc
> index 361d344..83ce5e5 100644
> --- a/iocore/net/SSLUtils.cc
> +++ b/iocore/net/SSLUtils.cc
> @@ -1473,8 +1473,10 @@ SSLInitServerContext(const SSLConfigParams * params, 
> const ssl_user_config & ssl
>   SSL_CTX_set_default_passwd_cb_userdata(CTX, NULL);\
>   }
>   SSL_CLEAR_PW_REFERENCES(ud,ctx)
> -  if (!ssl_context_enable_dhe(params->dhparamsFile, ctx)) {
> +  if (params->enable_dhparams && 
> !ssl_context_enable_dhe(params->dhparamsFile, ctx)) {
> goto fail;
> +  } else if (!params->enable_dhparams) {
> +Debug("ssl", "Not using dhparams");
>   }
>   return ssl_context_enable_ecdh(ctx);
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/091b59ca/mgmt/RecordsConfig.cc
> --
> diff --git a/mgmt/RecordsConfig.cc b/mgmt/RecordsConfig.cc
> index b360aa0..76d6cb2 100644
> --- a/mgmt/RecordsConfig.cc
> +++ b/mgmt/RecordsConfig.cc
> @@ -1322,6 +1322,8 @@ static const RecordElement RecordsConfig[] =
>   ,
>   {RECT_CONFIG, "proxy.config.ssl.allow_client_renegotiation", RECD_INT, "0", 
> RECU_DYNAMIC, RR_NULL, RECC_INT, "[0-1]", RECA_NULL}
>   ,
> +  {RECT_CONFIG, "proxy.config.ssl.server.enable_dhparams", RECD_INT, "0", 
> RECU_RESTART_TS, RR_NULL, RECC_INT, "[0-1]", RECA_NULL}
> +  ,
>   {RECT_CONFIG, "proxy.config.ssl.server.dhparams_file", RECD_STRING, NULL, 
> RECU_RESTART_TS, RR_NULL, RECC_NULL, NULL, RECA_NULL}
>   ,
>   
> //##
> 

Re: trafficserver git commit: TS-3417: Add MADV_DONTDUMP capability

[James Peach]

> On Mar 9, 2015, at 6:54 PM, sor...@apache.org wrote:
> 
> Repository: trafficserver
> Updated Branches:
>  refs/heads/master 3b24ea14e -> fe219a131
> 
> 
> TS-3417: Add MADV_DONTDUMP capability
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/fe219a13
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/fe219a13
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/fe219a13
> 
> Branch: refs/heads/master
> Commit: fe219a131af7d9af3c96a9079830141ba03e0378
> Parents: 3b24ea1
> Author: Phil Sorber 
> Authored: Sat Feb 28 12:35:40 2015 -0700
> Committer: Phil Sorber 
> Committed: Mon Mar 9 19:45:06 2015 -0600
> 
> --
> CHANGES|  2 ++
> iocore/eventsystem/IOBuffer.cc | 14 +-
> lib/ts/Allocator.h |  5 ++---
> lib/ts/ink_queue.cc| 18 ++
> lib/ts/ink_queue.h |  4 ++--
> mgmt/RecordsConfig.cc  |  2 ++
> 6 files changed, 35 insertions(+), 10 deletions(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/fe219a13/CHANGES
> --
> diff --git a/CHANGES b/CHANGES
> index 2ea6b7f..79b0fa5 100644
> --- a/CHANGES
> +++ b/CHANGES
> @@ -1,6 +1,8 @@
>  -*- coding: utf-8 -*-
> Changes with Apache Traffic Server 5.3.0
> 
> +  *) [TS-3417] Add MADV_DONTDUMP capability
> +
>   *) [TS-3036] Add a new log tag, 'chm', which shows cache-hit-miss specific
>information. For example, was this a RAM cache hit.
>Co-Author: Ryan Franz.
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/fe219a13/iocore/eventsystem/IOBuffer.cc
> --
> diff --git a/iocore/eventsystem/IOBuffer.cc b/iocore/eventsystem/IOBuffer.cc
> index 879e8ba..9a86895 100644
> --- a/iocore/eventsystem/IOBuffer.cc
> +++ b/iocore/eventsystem/IOBuffer.cc
> @@ -25,7 +25,9 @@
>   UIOBuffer.cc
> 
> **/
> +#include "ink_defs.h"
> #include "P_EventSystem.h"
> +#include "I_RecCore.h"
> 
> //
> // General Buffer Allocator
> @@ -45,6 +47,16 @@ void
> init_buffer_allocators()
> {
>   char *name;
> +  int advice = 0;
> +
> +#ifdef MADV_DONTDUMP // This should only exist on Linux 3.4 and higher.
> +bool dont_dump_enabled;
> +RecGetRecordBool("proxy.config.allocator.dontdump_iobuffers", 
> &dont_dump_enabled, false);
> +
> +if (dont_dump_enabled) {
> +  advice = MADV_DONTDUMP;
> +}
> +#endif
> 
>   for (int i = 0; i < DEFAULT_BUFFER_SIZES; i++) {
> int64_t s = DEFAULT_BUFFER_BASE_SIZE * (((int64_t)1) << i);
> @@ -55,7 +67,7 @@ init_buffer_allocators()
> 
> name = new char[64];
> snprintf(name, 64, "ioBufAllocator[%d]", i);
> -ioBufAllocator[i].re_init(name, s, n, a);
> +ioBufAllocator[i].re_init(name, s, n, a, advice);
>   }
> }
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/fe219a13/lib/ts/Allocator.h
> --
> diff --git a/lib/ts/Allocator.h b/lib/ts/Allocator.h
> index 1e8bba6..9f4edde 100644
> --- a/lib/ts/Allocator.h
> +++ b/lib/ts/Allocator.h
> @@ -96,10 +96,9 @@ public:
> 
>   /** Re-initialize the parameters of the allocator. */
>   void
> -  re_init(const char *name, unsigned int element_size,
> -  unsigned int chunk_size, unsigned int alignment)
> +  re_init(const char *name, unsigned int element_size, unsigned int 
> chunk_size, unsigned int alignment, int advice)
>   {
> -ink_freelist_init(&this->fl, name, element_size, chunk_size, alignment);
> +ink_freelist_madvise_init(&this->fl, name, element_size, chunk_size, 
> alignment, advice);
>   }
> 
> protected:
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/fe219a13/lib/ts/ink_queue.cc
> --
> diff --git a/lib/ts/ink_queue.cc b/lib/ts/ink_queue.cc
> index 49207a9..9e0b7dc 100644
> --- a/lib/ts/ink_queue.cc
> +++ b/lib/ts/ink_queue.cc
> @@ -111,10 +111,18 @@ ink_freelist_init(InkFreeList **fl, const char *name, 
> uint32_t type_size,
>   f->allocated = 0;
>   f->allocated_base = 0;
>   f->used_base = 0;
> +  f->advice = 0;
>   *fl = f;
> #endif
> }
> 
> +void
> +ink_freelist_madvise_init(InkFreeList **fl, const char *name, uint32_t 
> type_size, uint32_t chunk_size, uint32_t alignment, int advice)
> +{
> +  ink_freelist_init(fl, name, type_size, chunk_size, alignment);
> +  (*fl)->advice = advice;
> +}
> +
> InkFreeList *
> ink_freelist_create(const char *name, uint32_t type_size, uint32_t chunk_size,
> uint32_t alignment)
> @@ -165,6 +173,8 @@ ink

Re: [1/3] trafficserver git commit: TS-3421 Add a -t range option for proxyauth, which asks for a bytes=0-0 Range

[James Peach]

Please update the documentation

> On Mar 2, 2015, at 5:10 PM, zw...@apache.org wrote:
> 
> Repository: trafficserver
> Updated Branches:
>  refs/heads/master 0c697b238 -> ab4da857d
> 
> 
> TS-3421 Add a -t range option for proxyauth, which asks for a bytes=0-0 Range
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/68c6dd8c
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/68c6dd8c
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/68c6dd8c
> 
> Branch: refs/heads/master
> Commit: 68c6dd8cda5ec53aa78905b93e76e4e4a53e5734
> Parents: 0c697b2
> Author: Leif Hedstrom 
> Authored: Mon Mar 2 18:04:00 2015 -0700
> Committer: Leif Hedstrom 
> Committed: Mon Mar 2 18:05:01 2015 -0700
> 
> --
> plugins/experimental/authproxy/authproxy.cc | 59 
> 1 file changed, 49 insertions(+), 10 deletions(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/68c6dd8c/plugins/experimental/authproxy/authproxy.cc
> --
> diff --git a/plugins/experimental/authproxy/authproxy.cc 
> b/plugins/experimental/authproxy/authproxy.cc
> index 4e22d14..697b5c8 100644
> --- a/plugins/experimental/authproxy/authproxy.cc
> +++ b/plugins/experimental/authproxy/authproxy.cc
> @@ -159,14 +159,14 @@ struct AuthRequestContext
>   TSHttpParser hparser; // HTTP response header parser.
>   HttpHeader rheader;   // HTTP response header.
>   HttpIoBuffer iobuf;
> -  bool is_head; // This is a HEAD request
> +  const char* method;   // Client request method (e.g. GET)
>   bool read_body;
> 
>   const StateTransition* state;
> 
>   AuthRequestContext()
> : txn(NULL), cont(NULL), vconn(NULL), hparser(TSHttpParserCreate()), 
> rheader(),
> -  iobuf(TS_IOBUFFER_SIZE_INDEX_4K), is_head(false), read_body(true), 
> state(NULL)
> +  iobuf(TS_IOBUFFER_SIZE_INDEX_4K), method(NULL), read_body(true), 
> state(NULL)
>   {
> this->cont = TSContCreate(dispatch, TSMutexCreate());
> TSContDataSet(this->cont, this);
> @@ -255,20 +255,20 @@ pump:
> }
> 
> // Return whether the client request was a HEAD request.
> -static bool
> -AuthRequestIsHead(TSHttpTxn txn)
> +const char*
> +AuthRequestGetMethod(TSHttpTxn txn)
> {
>   TSMBuffer mbuf;
>   TSMLoc mhdr;
>   int len;
> -  bool is_head;
> +  const char* method;
> 
>   TSReleaseAssert(TSHttpTxnClientReqGet(txn, &mbuf, &mhdr) == TS_SUCCESS);
> 
> -  is_head = (TSHttpHdrMethodGet(mbuf, mhdr, &len) == TS_HTTP_METHOD_HEAD);
> +  method = TSHttpHdrMethodGet(mbuf, mhdr, &len);
>   TSHandleMLocRelease(mbuf, TS_NULL_MLOC, mhdr);
> 
> -  return is_head;
> +  return method;
> }
> 
> // Chain the response header hook to send the proxy's authorization response.
> @@ -317,6 +317,43 @@ AuthWriteHeadRequest(AuthRequestContext* auth)
>   return true;
> }
> 
> +// Transform the client request into a GET Range: bytes=0-0 request. This is 
> useful
> +// for example of the authentication service is a caching proxy which might 
> not
> +// cache HEAD requests.
> +static bool
> +AuthWriteRangeRequest(AuthRequestContext* auth)
> +{
> +  HttpHeader rq;
> +  TSMBuffer mbuf;
> +  TSMLoc mhdr;
> +
> +  TSReleaseAssert(TSHttpTxnClientReqGet(auth->txn, &mbuf, &mhdr) == 
> TS_SUCCESS);
> +
> +  // First, copy the whole client request to our new auth proxy request.
> +  TSReleaseAssert(TSHttpHdrCopy(rq.buffer, rq.header, mbuf, mhdr) == 
> TS_SUCCESS);
> +
> +  // Next, assure that the request to the auth server is a GET request, 
> since we'll send a Range:
> +  if (TS_HTTP_METHOD_GET != auth->method) {
> +TSReleaseAssert(TSHttpHdrMethodSet(rq.buffer, rq.header, 
> TS_HTTP_METHOD_GET, -1) == TS_SUCCESS);
> +  }
> +
> +  HttpSetMimeHeader(rq.buffer, rq.header, TS_MIME_FIELD_CONTENT_LENGTH, 0u);
> +  HttpSetMimeHeader(rq.buffer, rq.header, TS_MIME_FIELD_RANGE, "bytes=0-0");
> +  HttpSetMimeHeader(rq.buffer, rq.header, TS_MIME_FIELD_CACHE_CONTROL, 
> "no-cache");
> +
> +  HttpDebugHeader(rq.buffer, rq.header);
> +
> +  // Serialize the HTTP request to the write IO buffer.
> +  TSHttpHdrPrint(rq.buffer, rq.header, auth->iobuf.buffer);
> +
> +  // We have to tell the auth context not to try to ready the response
> +  // body, since we'are asking for a zero length Range.
> +  auth->read_body = false;
> +
> +  TSHandleMLocRelease(mbuf, TS_NULL_MLOC, mhdr);
> +  return true;
> +}
> +
> // Transform the client request into a form that the auth proxy can consume 
> and
> // write it out.
> static bool
> @@ -374,8 +411,8 @@ StateAuthProxyConnect(AuthRequestContext* auth, void* /* 
> edata ATS_UNUSED */)
> 
>   TSReleaseAssert(ip); // We must have a client IP.
> 
> -  auth->is_head = AuthRequestIsHead(auth->

Re: trafficserver git commit: TS-3409: Adjust new and old metric names to avoid corrupting existing *.snap files.

[James Peach]

> On Feb 26, 2015, at 1:15 PM, shinr...@apache.org wrote:
> 
> Repository: trafficserver
> Updated Branches:
>  refs/heads/master 4ab0ea32b -> 115eabd1e
> 
> 
> TS-3409: Adjust new and old metric names to avoid corrupting existing *.snap 
> files.

How does changing metric names corrupt the snap files? Is this fixable in 
librecords without these renaming shenanigans?

> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/115eabd1
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/115eabd1
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/115eabd1
> 
> Branch: refs/heads/master
> Commit: 115eabd1e8b728e1e2a3bd14ec04a3fdef0741e2
> Parents: 4ab0ea3
> Author: shinrich 
> Authored: Thu Feb 26 15:14:22 2015 -0600
> Committer: shinrich 
> Committed: Thu Feb 26 15:14:22 2015 -0600
> 
> --
> iocore/net/SSLUtils.cc| 2 +-
> mgmt/RecordsConfig.cc | 2 +-
> proxy/config/stats.config.xml.default | 4 ++--
> 3 files changed, 4 insertions(+), 4 deletions(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/115eabd1/iocore/net/SSLUtils.cc
> --
> diff --git a/iocore/net/SSLUtils.cc b/iocore/net/SSLUtils.cc
> index 410915e..361d344 100644
> --- a/iocore/net/SSLUtils.cc
> +++ b/iocore/net/SSLUtils.cc
> @@ -908,7 +908,7 @@ SSLInitializeStatistics()
>   RecRegisterRawStat(ssl_rsb, RECT_PROCESS, 
> "proxy.process.ssl.total_handshake_time",
>  RECD_INT, RECP_PERSISTENT, (int) 
> ssl_total_handshake_time_stat,
>  RecRawStatSyncSum);
> -  RecRegisterRawStat(ssl_rsb, RECT_PROCESS, 
> "proxy.process.ssl.total_success_handshake_count_in",
> +  RecRegisterRawStat(ssl_rsb, RECT_PROCESS, 
> "proxy.process.ssl.total_success_handshake_count",
>  RECD_INT, RECP_PERSISTENT, (int) 
> ssl_total_success_handshake_count_in_stat,
>  RecRawStatSyncCount);
>   RecRegisterRawStat(ssl_rsb, RECT_PROCESS, 
> "proxy.process.ssl.total_success_handshake_count_out",
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/115eabd1/mgmt/RecordsConfig.cc
> --
> diff --git a/mgmt/RecordsConfig.cc b/mgmt/RecordsConfig.cc
> index 75cc5a8..7a3ba34 100644
> --- a/mgmt/RecordsConfig.cc
> +++ b/mgmt/RecordsConfig.cc
> @@ -1800,7 +1800,7 @@ static const RecordElement RecordsConfig[] =
>   ,
> 
> 
> -  {RECT_NODE, "proxy.process.ssl.total_success_handshake_count", RECD_INT, 
> "0", RECU_NULL, RR_NULL, RECC_NULL, NULL, RECA_NULL}
> +  {RECT_NODE, "proxy.process.ssl.total_success_handshake_count_in", 
> RECD_INT, "0", RECU_NULL, RR_NULL, RECC_NULL, NULL, RECA_NULL}
>   ,
> 
>   //#
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/115eabd1/proxy/config/stats.config.xml.default
> --
> diff --git a/proxy/config/stats.config.xml.default 
> b/proxy/config/stats.config.xml.default
> index 507ba9f..430e2be 100644
> --- a/proxy/config/stats.config.xml.default
> +++ b/proxy/config/stats.config.xml.default
> @@ -1948,9 +1948,9 @@
> 
>minimum="0">
> -
> proxy.process.ssl.total_success_handshake_count
> +
> proxy.process.ssl.total_success_handshake_count_in
> 
> -proxy.process.ssl.total_success_handshake_count_in
> +proxy.process.ssl.total_success_handshake_count
> 
> 
> 
> 

Re: [1/2] trafficserver git commit: TS-3375 - Further refining error actions on bad SSL configuration.

[James Peach]

> On Feb 16, 2015, at 9:53 AM, shinr...@apache.org wrote:
> 
> Repository: trafficserver
> Updated Branches:
>  refs/heads/master 53d5c6dfb -> a8e0c5e17
> 
> 
> TS-3375 - Further refining error actions on bad SSL configuration.
> 
[snip]
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/8152dbfc/iocore/net/SSLConfig.cc
> --
> diff --git a/iocore/net/SSLConfig.cc b/iocore/net/SSLConfig.cc
> index 98214d2..627ccd2 100644
> --- a/iocore/net/SSLConfig.cc
> +++ b/iocore/net/SSLConfig.cc
> @@ -342,7 +342,10 @@ SSLCertificateConfig::startup()
>   sslCertUpdate->attach("proxy.config.ssl.server.private_key.path");
>   sslCertUpdate->attach("proxy.config.ssl.server.cert_chain.filename");
> 
> -  return reconfigure();
> +  if (!reconfigure()) {
> +_exit(1);
> +  }
> +  return true;
> }

This should just return the status. It's up to the caller to decide whether to 
exit or not. It looks like we don't typically plumb these failures all the way 
to main(), but I think it would we reasonable for the SSLNetProcessor to exit. 
It should exit by using Fatal().

> bool
> @@ -360,7 +363,8 @@ SSLCertificateConfig::reconfigure()
> ink_hrtime_sleep(HRTIME_SECONDS(secs));
>   }
> 
> -  if (SSLParseCertificateConfiguration(params, lookup)) {
> +  SSLParseCertificateConfiguration(params, lookup);

You still need to check the return value of SSLParseCertificateConfiguration() 
here.

> +  if (lookup->is_valid) {
> configid = configProcessor.set(configid, lookup);
>   } else {
> retStatus = false;

Re: trafficserver git commit: TS-3380: Add configure time constant to enable openssl 1.0.2 changes to compile with libressl

[James Peach]

> On Feb 9, 2015, at 4:12 PM, shinr...@apache.org wrote:
> 
> 
[snip]
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/2dbdd9c2/build/crypto.m4
> --
> diff --git a/build/crypto.m4 b/build/crypto.m4
> index 32276a2..2c83793 100644
> --- a/build/crypto.m4
> +++ b/build/crypto.m4
> @@ -89,7 +89,12 @@ AC_DEFUN([TS_CHECK_CRYPTO_SNI], [
>   enable_tls_sni=yes
> 
>   TS_ADDTO(LIBS, [$OPENSSL_LIBS])
> -  AC_CHECK_HEADERS(openssl/tls1.h openssl/ssl.h openssl/ts.h)
> +  AC_CHECK_HEADERS(openssl/ssl.h openssl/ts.h)
> +  AC_CHECK_HEADERS(openssl/tls1.h, [], [], 
> +[ #ifdef HEADER_SSL_H
> +#include 
> +#endif ])

Where does HEADER_SSL_H come from? The previous check for  
works on my systems ... what configuration was this change for?

> +
>   # We are looking for SSL_CTX_set_tlsext_servername_callback, but it's a
>   # macro, so AC_CHECK_FUNCS is not going to do the business.
>   AC_MSG_CHECKING([for SSL_CTX_set_tlsext_servername_callback])
> @@ -122,3 +127,79 @@ AC_DEFUN([TS_CHECK_CRYPTO_SNI], [
>   TS_ARG_ENABLE_VAR([use], [tls-sni])
>   AC_SUBST(use_tls_sni)
> ])
> +
> +AC_DEFUN([TS_CHECK_CRYPTO_CERT_CB], [

SSL_CTX_set_cert_cb is a function, so you should just be able to use 
AC_SEARCH_LIBS here ...

[snip]
> +
> +AC_DEFUN([TS_CHECK_CRYPTO_SET_RBIO], [
> +  _rbio_saved_LIBS=$LIBS
> +  enable_set_rbio=yes

Can you explain this one to me? SSL_set_rbio is not an OpenSSL API ... is it 
planned for a future version?

thanks,
James

Re: trafficserver git commit: TS-2480: Fix to work in the case where there are no ticket key files but tickets have not been disabled. Also fix RHEL 5 compile error.

[James Peach]

> On Feb 5, 2015, at 7:32 PM, shinr...@apache.org wrote:
> 
> Repository: trafficserver
> Updated Branches:
>  refs/heads/master 1d617582b -> 5fe69772a
> 
> 
> TS-2480: Fix to work in the case where there are no ticket key files but 
> tickets have not been disabled.

Doesn't OpenSSL do this implicitly? Or is the problem that we end up setting 
the callback without having a keyblock?

>  Also fix RHEL 5 compile error.
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/5fe69772
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/5fe69772
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/5fe69772
> 
> Branch: refs/heads/master
> Commit: 5fe69772aa7e5e841349f3426a997930b44c0ff5
> Parents: 1d61758
> Author: shinrich 
> Authored: Thu Feb 5 19:24:08 2015 -0600
> Committer: shinrich 
> Committed: Thu Feb 5 21:32:26 2015 -0600
> 
> --
> iocore/net/SSLUtils.cc | 45 ++---
> 1 file changed, 26 insertions(+), 19 deletions(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/5fe69772/iocore/net/SSLUtils.cc
> --
> diff --git a/iocore/net/SSLUtils.cc b/iocore/net/SSLUtils.cc
> index 055d396..f0265c6 100644
> --- a/iocore/net/SSLUtils.cc
> +++ b/iocore/net/SSLUtils.cc
> @@ -543,28 +543,34 @@ ssl_context_enable_tickets(SSL_CTX * ctx, const char * 
> ticket_key_path)
>   Error("failed to read SSL session ticket key from %s", (const char 
> *)ticket_key_path);
>   goto fail;
> }
> +  } else {
> + // Generate a random ticket key
> + ticket_key_len = 48;
> + ticket_key_data = (char *)ats_malloc(ticket_key_len);
> + char *tmp_ptr = ticket_key_data;
> + RAND_bytes(reinterpret_cast(tmp_ptr), ticket_key_len);
> +  }
> 
> -num_ticket_keys = ticket_key_len / sizeof(ssl_ticket_key_t);
> -if (num_ticket_keys == 0) {
> -  Error("SSL session ticket key from %s is too short (>= 48 bytes are 
> required)", (const char *)ticket_key_path);
> -  goto fail;
> -}
> +  num_ticket_keys = ticket_key_len / sizeof(ssl_ticket_key_t);
> +  if (num_ticket_keys == 0) {
> +Error("SSL session ticket key from %s is too short (>= 48 bytes are 
> required)", (const char *)ticket_key_path);
> +goto fail;
> +  }
> 
> -// Increase the stats.
> -if (ssl_rsb != NULL) { // ssl_rsb is not initialized during the first 
> run.
> -  SSL_INCREMENT_DYN_STAT(ssl_total_ticket_keys_renewed_stat);
> -}
> +  // Increase the stats.
> +  if (ssl_rsb != NULL) { // ssl_rsb is not initialized during the first run.
> +SSL_INCREMENT_DYN_STAT(ssl_total_ticket_keys_renewed_stat);
> +  }
> 
> -keyblock = ticket_block_alloc(num_ticket_keys);
> +  keyblock = ticket_block_alloc(num_ticket_keys);
> 
> -// Slurp all the keys in the ticket key file. We will encrypt with the 
> first key, and decrypt
> -// with any key (for rotation purposes).
> -for (unsigned i = 0; i < num_ticket_keys; ++i) {
> -  const char * data = (const char *)ticket_key_data + (i * 
> sizeof(ssl_ticket_key_t));
> -  memcpy(keyblock->keys[i].key_name, data, 
> sizeof(ssl_ticket_key_t::key_name));
> -  memcpy(keyblock->keys[i].hmac_secret, data + 
> sizeof(ssl_ticket_key_t::key_name), sizeof(ssl_ticket_key_t::hmac_secret));
> -  memcpy(keyblock->keys[i].aes_key, data + 
> sizeof(ssl_ticket_key_t::key_name) + sizeof(ssl_ticket_key_t::hmac_secret), 
> sizeof(ssl_ticket_key_t::aes_key));
> -}
> +  // Slurp all the keys in the ticket key file. We will encrypt with the 
> first key, and decrypt
> +  // with any key (for rotation purposes).
> +  for (unsigned i = 0; i < num_ticket_keys; ++i) {
> +const char * data = (const char *)ticket_key_data + (i * 
> sizeof(ssl_ticket_key_t));
> +memcpy(keyblock->keys[i].key_name, data, 
> sizeof(ssl_ticket_key_t::key_name));
> +memcpy(keyblock->keys[i].hmac_secret, data + 
> sizeof(ssl_ticket_key_t::key_name), sizeof(ssl_ticket_key_t::hmac_secret));
> +memcpy(keyblock->keys[i].aes_key, data + 
> sizeof(ssl_ticket_key_t::key_name) + sizeof(ssl_ticket_key_t::hmac_secret), 
> sizeof(ssl_ticket_key_t::aes_key));
>   }
> 
>   // Setting the callback can only fail if OpenSSL does not recognize the
> @@ -1771,10 +1777,11 @@ ssl_store_ssl_context(
>   if (SSLConfigParams::init_ssl_ctx_cb) {
> SSLConfigParams::init_ssl_ctx_cb(ctx, true);
>   }
> +#if HAVE_OPENSSL_SESSION_TICKETS
>   if (!inserted && keyblock != NULL) {
> ticket_block_free(keyblock);
>   }
> -
> +#endif
>   return ctx;
> }
> 
> 

Re: trafficserver git commit: Fix linking to external ssl libs for traffic_crashlog

[James Peach]

> On Feb 5, 2015, at 9:41 AM, James Peach  wrote:
> 
> 
>> On Feb 4, 2015, at 1:54 PM, James Peach  wrote:
>> 
>> Wait, how does traffic_crashlog drag in OpenSSL?
> 
> I believe that this happens because libtsutil claims to depend on OpenSSL 
> libraries. Automake sees that and dutifully drags them in whenever anything 
> links libtsutil. Thomas, can you take a crack at removing the OpenSSL linkage 
> from libtsutil?

Ok, libtsutil drags in libcrypto to get MD5. It seems unfortunate to have to 
link everything against OpenSSL, but necessary :(

J

Re: trafficserver git commit: Fix linking to external ssl libs for traffic_crashlog

[James Peach]

> On Feb 4, 2015, at 1:54 PM, James Peach  wrote:
> 
> Wait, how does traffic_crashlog drag in OpenSSL?

I believe that this happens because libtsutil claims to depend on OpenSSL 
libraries. Automake sees that and dutifully drags them in whenever anything 
links libtsutil. Thomas, can you take a crack at removing the OpenSSL linkage 
from libtsutil?

> 
>> On Feb 4, 2015, at 12:01 PM, jackso...@apache.org wrote:
>> 
>> Repository: trafficserver
>> Updated Branches:
>> refs/heads/master b220c23d6 -> d95ab615a
>> 
>> 
>> Fix linking to external ssl libs for traffic_crashlog
>> 
>> 
>> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
>> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/d95ab615
>> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/d95ab615
>> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/d95ab615
>> 
>> Branch: refs/heads/master
>> Commit: d95ab615a2d74a4f7250ad568477896441479810
>> Parents: b220c23
>> Author: Thomas Jackson 
>> Authored: Wed Feb 4 11:31:49 2015 -0800
>> Committer: Thomas Jackson 
>> Committed: Wed Feb 4 12:01:15 2015 -0800
>> 
>> --
>> cmd/traffic_crashlog/Makefile.am | 4 
>> 1 file changed, 4 insertions(+)
>> --
>> 
>> 
>> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/d95ab615/cmd/traffic_crashlog/Makefile.am
>> --
>> diff --git a/cmd/traffic_crashlog/Makefile.am 
>> b/cmd/traffic_crashlog/Makefile.am
>> index fdb79f0..9a2ff96 100644
>> --- a/cmd/traffic_crashlog/Makefile.am
>> +++ b/cmd/traffic_crashlog/Makefile.am
>> @@ -18,6 +18,7 @@
>> bin_PROGRAMS = traffic_crashlog
>> 
>> AM_CPPFLAGS = \
>> +  @OPENSSL_INCLUDES@ \
>>  $(iocore_include_dirs) \
>>  -I$(top_srcdir)/lib \
>>  -I$(top_builddir)/lib \
>> @@ -33,6 +34,9 @@ traffic_crashlog_SOURCES = \
>>  traffic_crashlog.h
>> 
>> traffic_crashlog_LDADD = \
>> +  @LIBTOOL_LINK_FLAGS@ \
>> +  @OPENSSL_LIBS@ \
>> +  @OPENSSL_LDFLAGS@ \
>>  $(top_builddir)/lib/records/librecords_p.a \
>>  $(top_builddir)/mgmt/libmgmt_p.la \
>>  $(top_builddir)/iocore/eventsystem/libinkevent.a \
>> 
> 

Re: trafficserver git commit: Fix linking to external ssl libs for traffic_crashlog

[James Peach]

Wait, how does traffic_crashlog drag in OpenSSL?

> On Feb 4, 2015, at 12:01 PM, jackso...@apache.org wrote:
> 
> Repository: trafficserver
> Updated Branches:
>  refs/heads/master b220c23d6 -> d95ab615a
> 
> 
> Fix linking to external ssl libs for traffic_crashlog
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/d95ab615
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/d95ab615
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/d95ab615
> 
> Branch: refs/heads/master
> Commit: d95ab615a2d74a4f7250ad568477896441479810
> Parents: b220c23
> Author: Thomas Jackson 
> Authored: Wed Feb 4 11:31:49 2015 -0800
> Committer: Thomas Jackson 
> Committed: Wed Feb 4 12:01:15 2015 -0800
> 
> --
> cmd/traffic_crashlog/Makefile.am | 4 
> 1 file changed, 4 insertions(+)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/d95ab615/cmd/traffic_crashlog/Makefile.am
> --
> diff --git a/cmd/traffic_crashlog/Makefile.am 
> b/cmd/traffic_crashlog/Makefile.am
> index fdb79f0..9a2ff96 100644
> --- a/cmd/traffic_crashlog/Makefile.am
> +++ b/cmd/traffic_crashlog/Makefile.am
> @@ -18,6 +18,7 @@
> bin_PROGRAMS = traffic_crashlog
> 
> AM_CPPFLAGS = \
> +  @OPENSSL_INCLUDES@ \
>   $(iocore_include_dirs) \
>   -I$(top_srcdir)/lib \
>   -I$(top_builddir)/lib \
> @@ -33,6 +34,9 @@ traffic_crashlog_SOURCES = \
>   traffic_crashlog.h
> 
> traffic_crashlog_LDADD = \
> +  @LIBTOOL_LINK_FLAGS@ \
> +  @OPENSSL_LIBS@ \
> +  @OPENSSL_LDFLAGS@ \
>   $(top_builddir)/lib/records/librecords_p.a \
>   $(top_builddir)/mgmt/libmgmt_p.la \
>   $(top_builddir)/iocore/eventsystem/libinkevent.a \
> 

Re: [1/2] trafficserver git commit: TS-3303 tcpinfo: we can close the log file object multiple times

[James Peach]

> On Jan 15, 2015, at 5:29 PM, zw...@apache.org wrote:
> 
> 
[snip]
> diff --git a/plugins/tcpinfo/tcpinfo.cc b/plugins/tcpinfo/tcpinfo.cc
> index 481f553..1937eed 100644
> --- a/plugins/tcpinfo/tcpinfo.cc
> +++ b/plugins/tcpinfo/tcpinfo.cc
> @@ -171,12 +171,9 @@ log_tcp_info(Config * config, const char * event_name, 
> TSHttpSsn ssnp)
>  info.tcpi_rtt);
>   }
> 
> -  // It's really not clear how we should handle logging failures. It a 
> failure transient
> -  // or persistent? Should we try to re-open the logs? How frequently should 
> we do that?
>   if (ret != TS_SUCCESS) {
> -TSError("[tcpinfo] log write failed, disabling logging");
> -TSTextLogObjectDestroy(config->log);
> -config->log = NULL;
> +  // ToDo: This could be due to a failure, or logs full. Should we 
> consider
> +  // closing / reopening the log? If so, how often do we do that ?
>   }
> }

Yeh I think the original code was a bit aggressive here. A failing log write 
seems like a poor excuse to give up altogether. Do you know whether the log 
writing will recover from this failure?

J

Re: trafficserver git commit: TS-3219: Separate WCCP client process

[James Peach]

Should this be cmd/traffic_wccp?

> On Jan 8, 2015, at 12:00 PM, shinr...@apache.org wrote:
> 
> Repository: trafficserver
> Updated Branches:
>  refs/heads/master aaf5d6bfa -> 5f2fc30d7
> 
> 
> TS-3219: Separate WCCP client process
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/5f2fc30d
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/5f2fc30d
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/5f2fc30d
> 
> Branch: refs/heads/master
> Commit: 5f2fc30d7d1b4f2107026ede93c05df8360e4d75
> Parents: aaf5d6b
> Author: shinrich 
> Authored: Thu Nov 20 15:14:55 2014 -0600
> Committer: shinrich 
> Committed: Thu Jan 8 13:54:10 2015 -0600
> 
> --
> CHANGES|   2 +
> lib/wccp/Makefile.am   |   8 +-
> lib/wccp/Wccp.h|   2 +-
> lib/wccp/WccpConfig.cc |  11 +
> lib/wccp/WccpEndPoint.cc   | 139 +++
> lib/wccp/WccpLocal.h   |  17 ++
> lib/wccp/wccp-test-cache.cc| 196 ---
> lib/wccp/wccp-test-router.cc   |  14 +-
> tools/Makefile.am  |  17 +-
> tools/wccp_client/Makefile.am  |  33 +++
> tools/wccp_client/readme.txt   |  42 
> tools/wccp_client/service-nogre-example.config |  66 +
> tools/wccp_client/wccp_client.cc   | 256 
> 13 files changed, 544 insertions(+), 259 deletions(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/5f2fc30d/CHANGES
> --
> diff --git a/CHANGES b/CHANGES
> index eac1de9..3b06244 100644
> --- a/CHANGES
> +++ b/CHANGES
> @@ -1,6 +1,8 @@
>  -*- coding: utf-8 -*-
> Changes with Apache Traffic Server 5.3.0
> 
> +  *) [TS-3219] Create WCCP client process.
> +
>   *) [TS-3272] Fix to ensure that SSL_SNI callback only called when state
>changes.
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/5f2fc30d/lib/wccp/Makefile.am
> --
> diff --git a/lib/wccp/Makefile.am b/lib/wccp/Makefile.am
> index f782c8b..abbf632 100644
> --- a/lib/wccp/Makefile.am
> +++ b/lib/wccp/Makefile.am
> @@ -26,7 +26,7 @@ AM_CPPFLAGS = \
> #DEFS += $(WCCP_DEFS)
> 
> noinst_LIBRARIES = libwccp.a
> -#noinst_PROGRAMS = test-cache
> +# noinst_PROGRAMS = test-cache 
> 
> libwccp_a_SOURCES = \
>   Wccp.h \
> @@ -38,5 +38,7 @@ libwccp_a_SOURCES = \
>   WccpStatic.cc \
>   WccpUtil.h
> 
> -#test_cache_SOURCES = \
> -#  wccp-test-cache.cc wccp-test-router.cc
> +# test_cache_SOURCES = \
> +#   wccp-test-cache.cc
> +
> +# test_cache_LDADD = $(LDADD) -L$(top_builddir)/lib/tsconfig -ltsconfig 
> -L$(top_builddir)/lib/wccp -lwccp -L$(top_builddir)/lib/ts -ltsutil
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/5f2fc30d/lib/wccp/Wccp.h
> --
> diff --git a/lib/wccp/Wccp.h b/lib/wccp/Wccp.h
> index 4e885e5..cfe99c7 100644
> --- a/lib/wccp/Wccp.h
> +++ b/lib/wccp/Wccp.h
> @@ -27,6 +27,7 @@
> # include 
> # include 
> # include 
> +# include 
> // Nasty, defining this with no prefix. The value is still available
> // in TS_VERSION_STRING.
> # undef VERSION
> @@ -421,7 +422,6 @@ inline ServiceGroup::Type
> ServiceGroup::getSvcType() const {
>   return static_cast(m_svc_type);
> }
> -
> inline uint8_t
> ServiceGroup::getSvcId() const {
>   return m_svc_id;
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/5f2fc30d/lib/wccp/WccpConfig.cc
> --
> diff --git a/lib/wccp/WccpConfig.cc b/lib/wccp/WccpConfig.cc
> index 443fa8f..fca9966 100644
> --- a/lib/wccp/WccpConfig.cc
> +++ b/lib/wccp/WccpConfig.cc
> @@ -67,6 +67,7 @@ static char const * const SVC_PROP_ROUTERS = "routers";
> static char const * const SVC_PROP_FORWARD = "forward";
> static char const * const SVC_PROP_RETURN = "return";
> static char const * const SVC_PROP_ASSIGN = "assignment";
> +static char const * const SVC_PROP_PROC = "proc-name";
> 
> static char const * const SECURITY_PROP_OPTION = "option";
> static char const * const SECURITY_PROP_KEY = "key";
> @@ -725,6 +726,16 @@ CacheImpl::loadServicesFromFile(char const* path) {
> 
> // Properties after this are optional so we can proceed if they fail.
> GroupData& svc = this->defineServiceGroup(svc_info);
> +
> +// Is there a process we should track?
> +if ((prop = svc_cfg[SVC_PROP_PROC]).hasValue()) {
> +  if (ts::config::StringValue == prop.getType()) {
> + 

Re: trafficserver git commit: Enable a few more options for the base build

[James Peach]

> On Dec 17, 2014, at 8:24 AM, Leif Hedstrom  wrote:
> 
> 
>> On Dec 17, 2014, at 9:18 AM, James Peach  wrote:
>> 
>> Does this mean that all our CI builds now use --enable-reclaimable-freelist?
> 
> 
> No, only 2 of them. I didn’t want to do it for everything, because this is 
> the “uncommon” path.

Great, I agree with that :)

J

Re: trafficserver git commit: Enable a few more options for the base build

[James Peach]

Does this mean that all our CI builds now use --enable-reclaimable-freelist?


> On Dec 17, 2014, at 7:07 AM, zw...@apache.org wrote:
> 
> Repository: trafficserver
> Updated Branches:
>  refs/heads/master 8993142d4 -> 76db63d70
> 
> 
> Enable a few more options for the base build
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/76db63d7
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/76db63d7
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/76db63d7
> 
> Branch: refs/heads/master
> Commit: 76db63d70cde6dd08d495372202df0b07e560b1b
> Parents: 8993142
> Author: Leif Hedstrom 
> Authored: Wed Dec 17 08:07:13 2014 -0700
> Committer: Leif Hedstrom 
> Committed: Wed Dec 17 08:07:13 2014 -0700
> 
> --
> ci/jenkins/bin/in_tree.sh | 4 +++-
> ci/jenkins/bin/out_of_tree.sh | 2 ++
> 2 files changed, 5 insertions(+), 1 deletion(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/76db63d7/ci/jenkins/bin/in_tree.sh
> --
> diff --git a/ci/jenkins/bin/in_tree.sh b/ci/jenkins/bin/in_tree.sh
> index 1aff446..5859774 100755
> --- a/ci/jenkins/bin/in_tree.sh
> +++ b/ci/jenkins/bin/in_tree.sh
> @@ -29,9 +29,11 @@ autoreconf -fi
> --enable-ccache \
> --enable-werror \
> --enable-experimental-plugins \
> -${enable_cppapi} \
> --enable-example-plugins \
> --enable-test-tools \
> +--enable-reclaimable-freelist \
> +--enable-wccp \
> +${enable_cppapi} \
> CORES=2
> 
> ${ATS_MAKE} -j5 V=1
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/76db63d7/ci/jenkins/bin/out_of_tree.sh
> --
> diff --git a/ci/jenkins/bin/out_of_tree.sh b/ci/jenkins/bin/out_of_tree.sh
> index f04b276..09fdec5 100755
> --- a/ci/jenkins/bin/out_of_tree.sh
> +++ b/ci/jenkins/bin/out_of_tree.sh
> @@ -28,6 +28,8 @@ mkdir -p BUILDS && cd BUILDS
> --enable-experimental-plugins \
> --enable-example-plugins \
> --enable-test-tools \
> +--enable-reclaimable-freelist \
> +--enable-wccp \
> CORES=2
> 
> ${ATS_MAKE} -j5 V=1
> 

Re: trafficserver git commit: TS-3224: fix problem of ts_lua coredump

[James Peach]

> On Dec 10, 2014, at 6:18 AM, kic...@apache.org wrote:
> 
> Repository: trafficserver
> Updated Branches:
>  refs/heads/master 156b7be9c -> 60c97c6f0
> 
> 
> TS-3224: fix problem of ts_lua coredump
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/60c97c6f
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/60c97c6f
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/60c97c6f
> 
> Branch: refs/heads/master
> Commit: 60c97c6f04538a1cd176f412dfc2622b9008936e
> Parents: 156b7be
> Author: Kit Chan 
> Authored: Wed Dec 10 06:17:50 2014 -0800
> Committer: Kit Chan 
> Committed: Wed Dec 10 06:17:50 2014 -0800
> 
> --
> CHANGES  |  2 ++
> plugins/experimental/ts_lua/ts_lua_cached_response.c |  8 ++--
> plugins/experimental/ts_lua/ts_lua_client_request.c  | 14 +++---
> plugins/experimental/ts_lua/ts_lua_client_response.c |  8 ++--
> plugins/experimental/ts_lua/ts_lua_server_request.c  |  6 +-
> plugins/experimental/ts_lua/ts_lua_server_response.c |  9 +++--
> 6 files changed, 37 insertions(+), 10 deletions(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/60c97c6f/CHANGES
> --
> diff --git a/CHANGES b/CHANGES
> index c091270..51f119e 100644
> --- a/CHANGES
> +++ b/CHANGES
> @@ -1,6 +1,8 @@
>  -*- coding: utf-8 -*-
> Changes with Apache Traffic Server 5.3.0
> 
> +  *) [TS-3224] fix ts_lua core dump issue.
> +
>   *) [TS-3229] Filter unsupported epic metric names.
> 
>   *) [TS-3230] Remove unused ink_error APIs.
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/60c97c6f/plugins/experimental/ts_lua/ts_lua_cached_response.c
> --
> diff --git a/plugins/experimental/ts_lua/ts_lua_cached_response.c 
> b/plugins/experimental/ts_lua/ts_lua_cached_response.c
> index abc30b7..74935fb 100644
> --- a/plugins/experimental/ts_lua/ts_lua_cached_response.c
> +++ b/plugins/experimental/ts_lua/ts_lua_cached_response.c
> @@ -131,8 +131,12 @@ ts_lua_cached_response_get_version(lua_State * L)
> 
>   version = TSHttpHdrVersionGet(http_ctx->cached_response_bufp, 
> http_ctx->cached_response_hdrp);
> 
> -  n = snprintf(buf, sizeof(buf) - 1, "%d.%d", TS_HTTP_MAJOR(version), 
> TS_HTTP_MINOR(version));
> -  lua_pushlstring(L, buf, n);
> +  n = snprintf(buf, sizeof(buf), "%d.%d", TS_HTTP_MAJOR(version), 
> TS_HTTP_MINOR(version));
> +  if(n >= sizeof(buf)) {
> +lua_pushlstring(L, buf, sizeof(buf) - 1);
> +  } else {
> +lua_pushlstring(L, buf, n);
> +  }

Maybe a common function, instead of 4 copies of this code?

> 
>   return 1;
> }
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/60c97c6f/plugins/experimental/ts_lua/ts_lua_client_request.c
> --
> diff --git a/plugins/experimental/ts_lua/ts_lua_client_request.c 
> b/plugins/experimental/ts_lua/ts_lua_client_request.c
> index 9d6cd26..14a5353 100644
> --- a/plugins/experimental/ts_lua/ts_lua_client_request.c
> +++ b/plugins/experimental/ts_lua/ts_lua_client_request.c
> @@ -499,7 +499,11 @@ ts_lua_client_request_get_uri(lua_State * L)
> 
>   uri_len = snprintf(uri, TS_LUA_MAX_URL_LENGTH, "/%.*s", path_len, path);
> 
> -  lua_pushlstring(L, uri, uri_len);
> +  if(uri_len >= TS_LUA_MAX_URL_LENGTH) {
> +lua_pushlstring(L, uri, TS_LUA_MAX_URL_LENGTH - 1);
> +  } else {
> +lua_pushlstring(L, uri, uri_len);
> +  }
> 
>   return 1;
> }
> @@ -762,8 +766,12 @@ ts_lua_client_request_get_version(lua_State * L)
> 
>   version = TSHttpHdrVersionGet(http_ctx->client_request_bufp, 
> http_ctx->client_request_hdrp);
> 
> -  n = snprintf(buf, sizeof(buf) - 1, "%d.%d", TS_HTTP_MAJOR(version), 
> TS_HTTP_MINOR(version));
> -  lua_pushlstring(L, buf, n);
> +  n = snprintf(buf, sizeof(buf), "%d.%d", TS_HTTP_MAJOR(version), 
> TS_HTTP_MINOR(version));
> +  if (n >= sizeof(buf)) {
> +lua_pushlstring(L, buf, sizeof(buf) - 1);
> +  } else {
> +lua_pushlstring(L, buf, n);
> +  }
> 
>   return 1;
> }
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/60c97c6f/plugins/experimental/ts_lua/ts_lua_client_response.c
> --
> diff --git a/plugins/experimental/ts_lua/ts_lua_client_response.c 
> b/plugins/experimental/ts_lua/ts_lua_client_response.c
> index ac98869..9c8030d 100644
> --- a/plugins/experimental/ts_lua/ts_lua_client_response.c
> +++ b/plugins/experimental/ts_lua/ts_lua_client_response.c
> @@ -309,8 +309,12 @@ ts_lua_client_response_get_version(lua_State * L)
> 
>   version = TSHttpHdrV

Re: trafficserver git commit: Fixing AddressSanitizer issue where sizeof(size_t) > sizeof(int).

[James Peach]

> On Nov 20, 2014, at 11:53 AM, bri...@apache.org wrote:
> 
> Repository: trafficserver
> Updated Branches:
>  refs/heads/master 74d29be0e -> 90b635481
> 
> 
> Fixing AddressSanitizer issue where sizeof(size_t) > sizeof(int).
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/90b63548
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/90b63548
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/90b63548
> 
> Branch: refs/heads/master
> Commit: 90b635481aa9a819852fbfeafdf334bf06513181
> Parents: 74d29be
> Author: Brian Geffon 
> Authored: Thu Nov 20 11:53:24 2014 -0800
> Committer: Brian Geffon 
> Committed: Thu Nov 20 11:53:24 2014 -0800
> 
> --
> proxy/logging/LogAccess.cc | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/90b63548/proxy/logging/LogAccess.cc
> --
> diff --git a/proxy/logging/LogAccess.cc b/proxy/logging/LogAccess.cc
> index 17cfed8..ce5f04c 100644
> --- a/proxy/logging/LogAccess.cc
> +++ b/proxy/logging/LogAccess.cc
> @@ -861,7 +861,7 @@ LogAccess::marshal_ip(char* dest, sockaddr const* ip) {
> inline int
> LogAccess::unmarshal_with_map(int64_t code, char *dest, int len, 
> Ptr map, const char *msg)
> {
> -  int codeStrLen;
> +  long int codeStrLen = 0;

Huh? Why not size_t?

> 
>   switch (map->asString(code, dest, len, (size_t *) & codeStrLen)) {
>   case LogFieldAliasMap::INVALID_INT:
> 

Re: [1/2] trafficserver git commit: [TS-3153]: Ability to disable or modify npn advertisement based on SNI

[James Peach]

> On Nov 17, 2014, at 1:12 PM, sudhe...@apache.org wrote:
> 
> Repository: trafficserver
> Updated Branches:
>  refs/heads/master 08f19da9b -> 34ca6f2dc
> 
> 
> [TS-3153]: Ability to disable or modify npn advertisement based on SNI
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/24262d8f
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/24262d8f
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/24262d8f
> 
> Branch: refs/heads/master
> Commit: 24262d8f6a14b6bb7bf7288f6309a68f6dc8589b
> Parents: 08f19da
> Author: Sudheer Vinukonda 
> Authored: Mon Nov 17 21:10:59 2014 +
> Committer: Sudheer Vinukonda 
> Committed: Mon Nov 17 21:10:59 2014 +
> 
> --
> configure.ac|   1 +
> iocore/net/P_SSLNetVConnection.h|   6 +
> iocore/net/SSLNetVConnection.cc |  82 +++-
> iocore/net/SSLUtils.cc  |   5 +
> plugins/experimental/Makefile.am|   1 +
> plugins/experimental/sni_proto_nego/Makefile.am |  21 ++
> .../sni_proto_nego/sni_proto_nego.cc| 194 +++
> proxy/InkAPI.cc |  10 +
> proxy/api/ts/ts.h   |   1 +
> 9 files changed, 320 insertions(+), 1 deletion(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/24262d8f/configure.ac
> --
> diff --git a/configure.ac b/configure.ac
> index 3e4465b..91e9874 100644
> --- a/configure.ac
> +++ b/configure.ac
> @@ -1945,6 +1945,7 @@ AS_IF([test "x$enable_experimental_plugins" = xyes], [
> plugins/experimental/regex_revalidate/Makefile
> plugins/experimental/remap_stats/Makefile
> plugins/experimental/s3_auth/Makefile
> +plugins/experimental/sni_proto_nego/Makefile
> plugins/experimental/sslheaders/Makefile
> plugins/experimental/ssl_cert_loader/Makefile
> plugins/experimental/stale_while_revalidate/Makefile
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/24262d8f/iocore/net/P_SSLNetVConnection.h
> --
> diff --git a/iocore/net/P_SSLNetVConnection.h 
> b/iocore/net/P_SSLNetVConnection.h
> index c481c8b..1dc7071 100644
> --- a/iocore/net/P_SSLNetVConnection.h
> +++ b/iocore/net/P_SSLNetVConnection.h
> @@ -122,6 +122,9 @@ public:
>   static int advertise_next_protocol(SSL * ssl, const unsigned char ** out, 
> unsigned * outlen, void *);
>   static int select_next_protocol(SSL * ssl, const unsigned char ** out, 
> unsigned char * outlen, const unsigned char * in, unsigned inlen, void *);
> 
> +  bool modify_npn_advertisement(const unsigned char ** list, unsigned cnt);
> +  bool setAdvertiseProtocols(const unsigned char ** list, unsigned cnt);
> +
>   Continuation * endpoint() const {
> return npnEndpoint;
>   }
> @@ -198,6 +201,9 @@ private:
> 
>   const SSLNextProtocolSet * npnSet;
>   Continuation * npnEndpoint;
> +  unsigned char * npnAdvertised;
> +  size_t npnszAdvertised;
> +  int npnAdvertisedBufIndex;
> };
> 
> typedef int (SSLNetVConnection::*SSLNetVConnHandler) (int, void *);
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/24262d8f/iocore/net/SSLNetVConnection.cc
> --
> diff --git a/iocore/net/SSLNetVConnection.cc b/iocore/net/SSLNetVConnection.cc
> index 4a9ec29..60fcbf9 100644
> --- a/iocore/net/SSLNetVConnection.cc
> +++ b/iocore/net/SSLNetVConnection.cc
> @@ -27,6 +27,8 @@
> #include "P_SSLUtils.h"
> #include "InkAPIInternal.h"   // Added to include the ssl_hook definitions
> 
> +extern unsigned char * append_protocol(const char * proto, unsigned char * 
> buf);

This is non-static by accident. Please make a static helper API in 
SSLNextProtocolSet to construct the advertisement. Maybe something like this:

size_t SSLNextProtocolSet::create_advertisement(const char **, unsigned, 
unsigned char * buf, size_t bufsz) 

> +
> // Defined in SSLInternal.c, should probably make a separate include
> // file for this at some point
> void SSL_set_rbio(SSLNetVConnection *sslvc, BIO *rbio);
> @@ -776,7 +778,10 @@ SSLNetVConnection::SSLNetVConnection():
>   sslPreAcceptHookState(SSL_HOOKS_INIT),
>   sslSNIHookState(SNI_HOOKS_INIT),
>   npnSet(NULL),
> -  npnEndpoint(NULL)
> +  npnEndpoint(NULL),
> +  npnAdvertised(NULL),
> +  npnszAdvertised(0),
> +  npnAdvertisedBufIndex(-1)

You don't need to store npnAdvertisedBufIndex, since the size tells you which 
iobuf allocator to use.

> {
> }
> 
> @@ -815,6 +820,9 @@ SSLNetVConnection::free(EThread * t) {
>   hookOpRequested = TS_SSL_HOOK_OP_DEFAULT;
>   npnSet = NULL;
>   npnEndpoint= NULL;
> +  np

Re: [1/2] trafficserver git commit: [TS-3153]: Add documentation for new API TSSslAdvertiseProtocolSet

[James Peach]

Why didn't this go through API review?

https://cwiki.apache.org/confluence/display/TS/API+Review+Process

> On Nov 17, 2014, at 1:27 PM, sudhe...@apache.org wrote:
> 
> Repository: trafficserver
> Updated Branches:
>  refs/heads/master be70f1cab -> f7d8f1b54
> 
> 
> [TS-3153]: Add documentation for new API TSSslAdvertiseProtocolSet
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/f19eba3c
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/f19eba3c
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/f19eba3c
> 
> Branch: refs/heads/master
> Commit: f19eba3ca70159af1ced63cc987b4a2508c570ba
> Parents: 1b35859
> Author: Sudheer Vinukonda 
> Authored: Mon Nov 17 21:26:24 2014 +
> Committer: Sudheer Vinukonda 
> Committed: Mon Nov 17 21:26:24 2014 +
> 
> --
> .../api/TSSslAdvertiseProtocolSet.en.rst| 32 
> 1 file changed, 32 insertions(+)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/f19eba3c/doc/reference/api/TSSslAdvertiseProtocolSet.en.rst
> --
> diff --git a/doc/reference/api/TSSslAdvertiseProtocolSet.en.rst 
> b/doc/reference/api/TSSslAdvertiseProtocolSet.en.rst
> new file mode 100644
> index 000..7684242
> --- /dev/null
> +++ b/doc/reference/api/TSSslAdvertiseProtocolSet.en.rst
> @@ -0,0 +1,32 @@
> +.. Licensed to the Apache Software Foundation (ASF) under one or more
> +   contributor license agreements.  See the NOTICE file distributed
> +   with this work for additional information regarding copyright
> +   ownership.  The ASF licenses this file to you under the Apache
> +   License, Version 2.0 (the "License"); you may not use this file
> +   except in compliance with the License.  You may obtain a copy of
> +   the License at
> +
> +  http://www.apache.org/licenses/LICENSE-2.0
> +
> +   Unless required by applicable law or agreed to in writing, software
> +   distributed under the License is distributed on an "AS IS" BASIS,
> +   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> +   implied.  See the License for the specific language governing
> +   permissions and limitations under the License.
> +
> +
> +TSSslAdvertiseProtocolSet
> +
> +
> +Synopsis
> +
> +
> +`#include `
> +
> +.. c:function:: TSReturnCode TSSslAdvertiseProtocolSet(TSVConn sslp, const 
> unsigned char ** list, unsigned int count);
> +
> +Description
> +---
> +
> +   Modifies the NPN advertisement list for a given SSL connection with 
> :arg:`list`. If :arg:`count` is 0, sets the NPN advertisement list to the 
> default registered protocol list for the end point. Note that, the plugin 
> that uses this API owns the :arg:`list` and is responsible for making sure it 
> points to a valid memory.
> +
> 

Re: trafficserver git commit: [TS-3199]: Do not wait for body for HEAD method

[James Peach]

> On Nov 14, 2014, at 4:27 PM, sudhe...@apache.org wrote:
> 
> Repository: trafficserver
> Updated Branches:
>  refs/heads/master 5c37860c7 -> d26674750
> 
> 
> [TS-3199]: Do not wait for body for HEAD method
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/d2667475
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/d2667475
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/d2667475
> 
> Branch: refs/heads/master
> Commit: d26674750e10c89029d752e86de4dc6ef5bf3588
> Parents: 5c37860
> Author: Sudheer Vinukonda 
> Authored: Sat Nov 15 00:26:16 2014 +
> Committer: Sudheer Vinukonda 
> Committed: Sat Nov 15 00:26:16 2014 +
> 
> --
> proxy/FetchSM.cc| 9 -
> proxy/FetchSM.h | 4 ++--
> proxy/spdy/SpdyClientSession.cc | 7 ++-
> 3 files changed, 12 insertions(+), 8 deletions(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/d2667475/proxy/FetchSM.cc
> --
> diff --git a/proxy/FetchSM.cc b/proxy/FetchSM.cc
> index 91d..ad7e655 100644
> --- a/proxy/FetchSM.cc
> +++ b/proxy/FetchSM.cc
> @@ -105,6 +105,8 @@ FetchSM::has_body()
>   if (!header_done)
> return false;
> 
> +  if (is_method_head)
> +return false;
>   //
>   // The following code comply with HTTP/1.1:
>   // http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.4
> @@ -550,12 +552,17 @@ FetchSM::ext_init(Continuation *cont, const char 
> *method,
>   memset(&callback_options, 0, sizeof(callback_options));
>   memset(&callback_events, 0, sizeof(callback_events));
> 
> -  req_buffer->write(method, strlen(method));
> +  int method_len = strlen(method);
> +  req_buffer->write(method, method_len);
>   req_buffer->write(" ", 1);
>   req_buffer->write(url, strlen(url));
>   req_buffer->write(" ", 1);
>   req_buffer->write(version, strlen(version));
>   req_buffer->write("\r\n", 2);
> +
> +  if ((method_len == strlen("HEAD")) && !memcmp(method, "HEAD", method_len)) 
> {

Shouldn't this be:

if ((method_len == HTTP_LEN_HEAD) && (memcmp(method, HTTP_METHOD_HEAD, 
method_len) == 0))

J

Re: [1/2] git commit: TS-2986: Adding stats to TLS errors

[James Peach]

> On Aug 6, 2014, at 10:20 AM, James Peach  wrote:
> 
> On Aug 5, 2014, at 12:16 PM, Brian Geffon  wrote:
> 
>> Technically you'd probably aggregate all of these into a single variable
>> for production monitoring, but having insight into the individual failure
>> scenarios could prove to be very valuable.
> 
> I'm still very dubious about these stats.
> 
> SSL_ERROR_WANT_WRITE, SSL_ERROR_WANT_READ, and SSL_ERROR_WANT_X509_LOOKUP 
> boil down to rescheduling the I/O. Hard to see how a metric for that is 
> useful. The other things here end up as an error or EOF. Again, hard to see 
> how this is helpful as a metric.
> 
> Handshake errors (client and server) seems like they could be interesting to 
> monitor, but these are bundled in "ssl_error_ssl", so you'll really have no 
> idea what is happening.

Well, I now have a handshake error in production and no logging :-/ Does anyone 
know how I can debug this?

The only thing I can get out of logs now is this:

[Nov 14 23:58:13.204] Server {0x2aaaba64a700} DEBUG:  (ssl.error) 
SSLNetVConnection::sslClientHandShakeEvent, SSL_ERROR_SSL

J

Re: git commit: [TS-2682] Add per remap support for background fetch plugin

[James Peach]

> On Oct 30, 2014, at 4:09 PM, sudhe...@apache.org wrote:
> 
> Repository: trafficserver
> Updated Branches:
>  refs/heads/master 589d8013e -> 42c89c713
> 
> 
> [TS-2682] Add per remap support for background fetch plugin
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/42c89c71
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/42c89c71
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/42c89c71
> 
> Branch: refs/heads/master
> Commit: 42c89c7133ee8dc8d0b3f71154f44b034c2de611
> Parents: 589d801
> Author: Sudheer Vinukonda 
> Authored: Thu Oct 30 23:09:04 2014 +
> Committer: Sudheer Vinukonda 
> Committed: Thu Oct 30 23:09:04 2014 +
> 
> --
> .../background_fetch/background_fetch.cc| 124 +--
> 1 file changed, 114 insertions(+), 10 deletions(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/42c89c71/plugins/experimental/background_fetch/background_fetch.cc
> --
> diff --git a/plugins/experimental/background_fetch/background_fetch.cc 
> b/plugins/experimental/background_fetch/background_fetch.cc
> index f6a3e0e..440e295 100644
> --- a/plugins/experimental/background_fetch/background_fetch.cc
> +++ b/plugins/experimental/background_fetch/background_fetch.cc
> @@ -51,12 +51,19 @@ const char PLUGIN_NAME[] = "background_fetch";
> 
> typedef std::set stringSet;
> 
> +static int g_background_fetch_ArgIndex = 0;
> static  stringSet contentTypeSet;
> static  stringSet userAgentSet;
> static  stringSet clientIpSet;
> 
> +typedef struct {
> +  stringSet contentTypeSet;
> +  stringSet userAgentSet;
> +  stringSet clientIpSet;
> +} RemapInstance;
> +
> static
> -bool read_config(char* config_file) {
> +bool read_config(char* config_file, RemapInstance* ri=NULL) {
>   char file_path[1024];
>   TSFile file;
>   if (config_file == NULL) {
> @@ -73,6 +80,17 @@ bool read_config(char* config_file) {
> return 0;
>   }
> 
> +  stringSet* contentTypeSetP = &contentTypeSet;
> +  stringSet* userAgentSetP = &userAgentSet;
> +  stringSet* clientIpSetP = &clientIpSet;
> +
> +  if (ri) {
> +TSDebug(PLUGIN_NAME, "setting per-remap filters");
> +contentTypeSetP = &(ri->contentTypeSet);
> +userAgentSetP = &(ri->userAgentSet);
> +clientIpSetP = &(ri->clientIpSet);
> +  }

This is really ugly. You should just create another RemapInstance when the 
global plugin loads. Then you have the same code path everywhere.

> +
>   char buffer[1024];
>   memset(buffer, 0, sizeof(buffer));
>   while (TSfgets(file, buffer, sizeof(buffer) - 1) != NULL) {
> @@ -103,13 +121,13 @@ bool read_config(char* config_file) {
> if (cfg_type && cfg_value) {
>   if (!strcmp(cfg_type, "Content-Type")) {
> TSDebug(PLUGIN_NAME, "adding content-type %s", cfg_value);
> -contentTypeSet.insert(cfg_value);
> +contentTypeSetP->insert(cfg_value);
>   } else if (!strcmp(cfg_type, "User-Agent")) {
> TSDebug(PLUGIN_NAME, "adding user-agent %s", cfg_value);
> -userAgentSet.insert(cfg_value);
> +userAgentSetP->insert(cfg_value);
>   } else if (!strcmp(cfg_type, "Client-Ip")) {
> TSDebug(PLUGIN_NAME, "adding client-ip %s", cfg_value);
> -clientIpSet.insert(cfg_value);
> +clientIpSetP->insert(cfg_value);
>   }
> }
> 
> @@ -649,7 +667,7 @@ check_hdr_configured(TSMBuffer hdr_bufp, TSMLoc req_hdrs, 
> const char* field_type
> }
> 
> static bool
> -is_background_fetch_allowed(TSHttpTxn txnp)
> +is_background_fetch_allowed(TSHttpTxn txnp, RemapInstance* ri=NULL)
> {
>   bool allow_bg_fetch = true;
>   TSDebug(PLUGIN_NAME, "Testing: request is internal?");
> @@ -657,6 +675,17 @@ is_background_fetch_allowed(TSHttpTxn txnp)
> return false;
>   }
> 
> +  stringSet* contentTypeSetP = &contentTypeSet;
> +  stringSet* userAgentSetP = &userAgentSet;
> +  stringSet* clientIpSetP = &clientIpSet;
> +
> +  if (ri) {
> +TSDebug(PLUGIN_NAME, "setting per-remap filters");
> +contentTypeSetP = &(ri->contentTypeSet);
> +userAgentSetP = &(ri->userAgentSet);
> +clientIpSetP = &(ri->clientIpSet);
> +  }
> +
>   const sockaddr* client_ip = TSHttpTxnClientAddrGet(txnp);
>   char* ip_buf = NULL;
> 
> @@ -670,8 +699,8 @@ is_background_fetch_allowed(TSHttpTxn txnp)
> 
>   if (ip_buf) {
> TSDebug(PLUGIN_NAME,"client_ip %s", ip_buf);
> -stringSet::iterator it = clientIpSet.begin();
> -while(it!=clientIpSet.end()) {
> +stringSet::iterator it = clientIpSetP->begin();
> +while(it!=clientIpSetP->end()) {
>   if (NULL != strstr(ip_buf, (*it).c_str())) {
> TSDebug(PLUGIN_NAME,"excluding bg fetch for ip %s, configured 

Re: git commit: [TS-2683]: Enhance the bg fetch plugin to support specifyng various exclusion criteria

[James Peach]

> On Oct 30, 2014, at 2:38 PM, sudhe...@apache.org wrote:
> 
> Repository: trafficserver
> Updated Branches:
>  refs/heads/master 35a71e919 -> e49344c15
> 
> 
> [TS-2683]: Enhance the bg fetch plugin to support specifyng various exclusion 
> criteria
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/e49344c1
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/e49344c1
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/e49344c1
> 
> Branch: refs/heads/master
> Commit: e49344c152cd7ff7eb92c38f00f76c6f7179a8d1
> Parents: 35a71e9
> Author: Sudheer Vinukonda 
> Authored: Thu Oct 30 21:36:12 2014 +
> Committer: Sudheer Vinukonda 
> Committed: Thu Oct 30 21:36:12 2014 +
> 
> --
> doc/reference/plugins/background_fetch.en.rst   |  16 +-
> .../background_fetch/background_fetch.cc| 183 ++-
> 2 files changed, 192 insertions(+), 7 deletions(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/e49344c1/doc/reference/plugins/background_fetch.en.rst
> --
> diff --git a/doc/reference/plugins/background_fetch.en.rst 
> b/doc/reference/plugins/background_fetch.en.rst
> index 3df7d3c..8465045 100644
> --- a/doc/reference/plugins/background_fetch.en.rst
> +++ b/doc/reference/plugins/background_fetch.en.rst
> @@ -58,6 +58,21 @@ original client request, which continues as normal.
> Only one background fetch per URL is ever performed, making sure we do not
> accidentally put pressure on the origin servers.
> 
> +The plugin now supports a config file that can specify exclusion of 
> background
> +fetch based on the below criteria:
> +1. Client-Ip
> +2. Content-Type
> +3. User-Agent
> +
> +To specify the exclusion criteria, the plugin needs to be activated as below:
> +
> +background_fetch.so --config 

A general comment ... "--config" implies a general plugin configuration, but 
this is actually a list of exclusions. Since that's the case, I suggest the 
option be called "--exclusions", or the config file format be modified to be 
able to express more than a single list.

> +
> +The contents of the config-file could be as below:
> +
> +Client-Ip 127.0.0.1

This should be Client-IP

> +User-Agent ABCDEF
> +Content-Type text
> 
> 
> Future additions
> @@ -66,7 +81,6 @@ Future additions
> The infrastructure is in place for providing global and per-remap
> configurations. This could include:
> 
> -- Limiting the background fetches to certain Content-Types
> - Limiting the background fetches to content of certain sizes
> 
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/e49344c1/plugins/experimental/background_fetch/background_fetch.cc
> --
> diff --git a/plugins/experimental/background_fetch/background_fetch.cc 
> b/plugins/experimental/background_fetch/background_fetch.cc
> index 4c18a4b..f6a3e0e 100644
> --- a/plugins/experimental/background_fetch/background_fetch.cc
> +++ b/plugins/experimental/background_fetch/background_fetch.cc
> @@ -34,6 +34,7 @@
> #include "ts/ts.h"
> #include "ts/remap.h"
> #include "ink_defs.h"
> +#include 
> 
> 
> // Some wonkiness around compiler version and the unordered map (hash)
> @@ -48,6 +49,80 @@
> // Constants
> const char PLUGIN_NAME[] = "background_fetch";
> 
> +typedef std::set stringSet;
> +
> +static  stringSet contentTypeSet;
> +static  stringSet userAgentSet;
> +static  stringSet clientIpSet;
> +
> +static
> +bool read_config(char* config_file) {
> +  char file_path[1024];
> +  TSFile file;
> +  if (config_file == NULL) {
> +TSError("invalid config file");
> +return false;
> +  }
> +  snprintf(file_path, sizeof(file_path), "%s/%s", TSInstallDirGet(), 
> config_file);

This needs to test for an absolute path. An absolute path should be used 
directly; a relative path must be relative to TSConfigDirGet().

> +
> +  TSDebug(PLUGIN_NAME, "trying to open config file in this path: %s", 
> file_path);
> +
> +  file = TSfopen(file_path, "r");
> +  if (file == NULL) {
> +TSError("Failed to open config file %s", config_file);
> +return 0;

s/0/false/

> +  }
> +
> +  char buffer[1024];
> +  memset(buffer, 0, sizeof(buffer));
> +  while (TSfgets(file, buffer, sizeof(buffer) - 1) != NULL) {
> +char *eol = 0;
> +// make sure line was not bigger than buffer
> +if ((eol = strchr(buffer, '\n')) == NULL && (eol = strstr(buffer, 
> "\r\n")) == NULL) {

Huh? DOS line endings?

> +  TSError("sni_proto_nego line too long, did not get a good line in cfg, 
> skipping, line: %s", buffer);
> +  memset(buffer, 0, sizeof(buffer));
> +  continue;
> +}
> +// make sure line has something useful on it
> +if (eol

Re: git commit: [TS-2503]: Dynamic TLS Record Sizing for better page load latencies. The strategy used is essentially to use small TLS records that fit into a single TCP segment for the first ~1 MB of

[James Peach]

> On Oct 17, 2014, at 7:59 AM, sudhe...@apache.org wrote:
> 
> Repository: trafficserver
> Updated Branches:
>  refs/heads/master 609443dbe -> dee36e716
> 
> 
> [TS-2503]: Dynamic TLS Record Sizing for better page load latencies.
> The strategy used is essentially to use small TLS records that fit into a
> single TCP segment for the first ~1 MB of data, increase record size to 16 KB
> after that to optimize throughput, and then reset record size back to a single
> segment after ~1 second of inactivity—lather, rinse, repeat.
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/dee36e71
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/dee36e71
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/dee36e71
> 
> Branch: refs/heads/master
> Commit: dee36e7163373c1d61bc6b97e529269dcde134c5
> Parents: 609443d
> Author: Sudheer Vinukonda 
> Authored: Fri Oct 17 14:57:15 2014 +
> Committer: Sudheer Vinukonda 
> Committed: Fri Oct 17 14:57:15 2014 +
> 
> --
> .../configuration/records.config.en.rst |  7 
> iocore/net/P_SSLNetVConnection.h| 13 ++
> iocore/net/P_SSLUtils.h |  2 +
> iocore/net/SSLNetVConnection.cc | 43 +++-
> iocore/net/SSLUtils.cc  |  4 ++
> 5 files changed, 68 insertions(+), 1 deletion(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/dee36e71/doc/reference/configuration/records.config.en.rst
> --
> diff --git a/doc/reference/configuration/records.config.en.rst 
> b/doc/reference/configuration/records.config.en.rst
> index 07244b7..ca30055 100644
> --- a/doc/reference/configuration/records.config.en.rst
> +++ b/doc/reference/configuration/records.config.en.rst
> @@ -2168,6 +2168,13 @@ SSL Termination
>   buffering at the SSL layer. The default of ``0`` means to always
>   write all available data into a single SSL record.
> 
> +  A value of ``-1`` means TLS record size is dynamically determined. The
> +  strategy employed is to use small TLS records that fit into a single
> +  TCP segment for the first ~1 MB of data, but, increase the record size to
> +  16 KB after that to optimize throughput. The record size is reset back to
> +  a single segment after ~1 second of inactivity and the record size ramping
> +  mechanism is repeated again.
> +
> .. ts:cv:: CONFIG proxy.config.ssl.session_cache INT 2
> 
>   Enables the SSL Session Cache:
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/dee36e71/iocore/net/P_SSLNetVConnection.h
> --
> diff --git a/iocore/net/P_SSLNetVConnection.h 
> b/iocore/net/P_SSLNetVConnection.h
> index 51b7391..4a6d7c8 100644
> --- a/iocore/net/P_SSLNetVConnection.h
> +++ b/iocore/net/P_SSLNetVConnection.h
> @@ -52,6 +52,17 @@
> #define SSL_TLSEXT_ERR_NOACK 3
> #endif
> 
> +// TS-2503: dynamic TLS record sizing
> +// For smaller records, we should also reserve space for various TCP options
> +// (timestamps, SACKs.. up to 40 bytes [1]), and account for TLS record 
> overhead
> +// (another 20-60 bytes on average, depending on the negotiated ciphersuite 
> [2]).
> +// All in all: 1500 - 40 (IP) - 20 (TCP) - 40 (TCP options) - TLS overhead 
> (60-100)
> +// For larger records, the size is determined by TLS protocol record size
> +#define SSL_DEF_TLS_RECORD_SIZE   1300 // 1500 - 40 (IP) - 20 
> (TCP) - 40 (TCP options) - TLS overhead (60-100)
> +#define SSL_MAX_TLS_RECORD_SIZE  16383 // 2^14 - 1
> +#define SSL_DEF_TLS_RECORD_BYTE_THRESHOLD  100
> +#define SSL_DEF_TLS_RECORD_MSEC_THRESHOLD 1000
> +
> class SSLNextProtocolSet;
> struct SSLCertLookup;
> 
> @@ -105,6 +116,8 @@ public:
> 
>   SSL *ssl;
>   ink_hrtime sslHandshakeBeginTime;
> +  ink_hrtime sslLastWriteTime;
> +  int64_tsslTotalBytesSent;
> 
>   static int advertise_next_protocol(SSL * ssl, const unsigned char ** out, 
> unsigned * outlen, void *);
>   static int select_next_protocol(SSL * ssl, const unsigned char ** out, 
> unsigned char * outlen, const unsigned char * in, unsigned inlen, void *);
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/dee36e71/iocore/net/P_SSLUtils.h
> --
> diff --git a/iocore/net/P_SSLUtils.h b/iocore/net/P_SSLUtils.h
> index ec7b0b9..3b77e7d 100644
> --- a/iocore/net/P_SSLUtils.h
> +++ b/iocore/net/P_SSLUtils.h
> @@ -70,6 +70,8 @@ enum SSL_Stats
>   ssl_total_tickets_verified_stat,
>   ssl_total_tickets_not_found_stat,
>   ssl_total_tickets_renewed_stat,
> +  ssl_total_dyn_def_tls_record_count,
> +  ssl_total_dyn_max_tls_record_coun

Re: git commit: TS-3139] New script, traffic_primer, to prime a set of boxes with content

[James Peach]

On Oct 15, 2014, at 1:47 PM, zw...@apache.org wrote:

> Repository: trafficserver
> Updated Branches:
>  refs/heads/master 3e64d405c -> cdd1d15b4
> 
> 
> TS-3139] New script, traffic_primer, to prime a set of boxes with content

I think it wold be better to have 1 script that works in both modes. 2 separate 
scripts seems like overkill

> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/cdd1d15b
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/cdd1d15b
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/cdd1d15b
> 
> Branch: refs/heads/master
> Commit: cdd1d15b48fe94158a59454ef75233f1ee827172
> Parents: 3e64d40
> Author: Leif Hedstrom 
> Authored: Wed Oct 15 14:47:01 2014 -0600
> Committer: Leif Hedstrom 
> Committed: Wed Oct 15 14:47:01 2014 -0600
> 
> --
> CHANGES  |  4 +++
> tools/traffic_primer | 79 +++
> 2 files changed, 83 insertions(+)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/cdd1d15b/CHANGES
> --
> diff --git a/CHANGES b/CHANGES
> index 100261c..a7b3d0c 100644
> --- a/CHANGES
> +++ b/CHANGES
> @@ -1,6 +1,10 @@
>  -*- coding: utf-8 -*-
> Changes with Apache Traffic Server 5.2.0
> 
> +  *) [TS-3139] New script, traffic_primer, which will fetch a URL from origin
> +   (or another proxy) and PUSH the same object to a given set of
> +   caches. Useful for priming a pool of servers with the same object.
> +
>   *) [TS-3135] Disable SSLv3 by default. This can be enabled again by adding a
>line to records.config for proxy.config.ssl.SSLv3.
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/cdd1d15b/tools/traffic_primer
> --
> diff --git a/tools/traffic_primer b/tools/traffic_primer
> new file mode 100755
> index 000..1a39de7
> --- /dev/null
> +++ b/tools/traffic_primer
> @@ -0,0 +1,79 @@
> +#!/bin/sh
> +#
> +# Licensed to the Apache Software Foundation (ASF) under one
> +# or more contributor license agreements.  See the NOTICE file
> +# distributed with this work for additional information
> +# regarding copyright ownership.  The ASF licenses this file
> +# to you under the Apache License, Version 2.0 (the
> +# "License"); you may not use this file except in compliance
> +# with the License.  You may obtain a copy of the License at
> +#
> +#  http://www.apache.org/licenses/LICENSE-2.0
> +#
> +# Unless required by applicable law or agreed to in writing, software
> +# distributed under the License is distributed on an "AS IS" BASIS,
> +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> +# See the License for the specific language governing permissions and
> +# limitations under the License.
> +
> +# Simple script to fetch a URL through one proxy, and then PUSH that 
> response (headers
> +# and body) to a set of hosts. The host:port defaults to localhost:80 for 
> fetching
> +# the URL, but can be overriden with -h/-p.
> +
> +
> +# Print some help text
> +usage() {
> +echo 'Usage: traffic_primer -u  [-h Host] [-p port] host1 host2 ...'
> +exit 2
> +}
> +
> +# Default values for command line options
> +url=""
> +host="localhost"
> +port="80"
> +
> +# Parse command line arguments
> +PARGS=$(getopt u:h:p: $@)
> +[ $? != 0 ] && usage
> +
> +set -- $PARGS
> +while true; do
> +case "$1" in
> +-u)
> +url="$2"
> +shift 2 ;;
> +-h)
> +host="$2"
> +shift 2 ;;
> +-p)
> +port="$2"
> +shift 2 ;;
> +--)
> +shift
> +break ;;
> +*)
> +usage
> +break
> +;;
> +esac
> +done
> +
> +[ "" == "$url" ] && usage
> +
> +tmpfile=$(mktemp /tmp/pusher-XX)
> +
> +# CLeanup just in case
> +trap "rm -f $tmpfile; exit 0" 0 1 2 3 15
> +
> +# Fetch the URL through the proxy on localhost, and create the file for PUSH
> +curl -x ${host}:${port} -s -i -o ${tmpfile} $url
> +
> +for h in $@; do
> +curl -f -I -x ${h}:${port} -s -o /dev/null -H "Cache-Control: 
> only-if-cached" $url > /dev/null
> +if [ $? -gt 0 ]; then
> +echo "PUSHing to $h..."
> +curl -x ${h}:80 -s -o /dev/null -X PUSH --data-binary @${tmpfile} 
> $url
> +fi
> +done
> +
> +rm -f ${tmpfile}
> 

Re: [1/3] git commit: TS-3080: Optimized SSL Session Cache

[James Peach]

I thought you were going to format to our style guidelines before committing?

On Oct 8, 2014, at 11:34 AM, bri...@apache.org wrote:

> Repository: trafficserver
> Updated Branches:
>  refs/heads/master 195259b16 -> f1bedb41e
> 
> 
> TS-3080: Optimized SSL Session Cache
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/53bf5d1e
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/53bf5d1e
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/53bf5d1e
> 
> Branch: refs/heads/master
> Commit: 53bf5d1e7618ae38b0a8b49263a047282eec68d1
> Parents: 72b7c05
> Author: Brian Geffon 
> Authored: Tue Oct 7 18:51:34 2014 -0700
> Committer: Brian Geffon 
> Committed: Tue Oct 7 18:52:34 2014 -0700
> 
> --
> iocore/net/Makefile.am|   1 +
> iocore/net/P_SSLConfig.h  |  12 +-
> iocore/net/P_SSLUtils.h   |   4 +
> iocore/net/SSLConfig.cc   |  19 ++-
> iocore/net/SSLSessionCache.cc | 246 +
> iocore/net/SSLSessionCache.h  | 149 ++
> iocore/net/SSLUtils.cc|  98 ++-
> lib/ts/ink_mutex.h|  29 +
> mgmt/RecordsConfig.cc |   8 +-
> proxy/Makefile.am |   2 +-
> 10 files changed, 559 insertions(+), 9 deletions(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/53bf5d1e/iocore/net/Makefile.am
> --
> diff --git a/iocore/net/Makefile.am b/iocore/net/Makefile.am
> index 0120528..da7a476 100644
> --- a/iocore/net/Makefile.am
> +++ b/iocore/net/Makefile.am
> @@ -88,6 +88,7 @@ libinknet_a_SOURCES = \
>   P_UnixUDPConnection.h \
>   Socks.cc \
>   SSLCertLookup.cc \
> +  SSLSessionCache.cc \
>   SSLConfig.cc \
>   SSLNetAccept.cc \
>   SSLNetProcessor.cc \
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/53bf5d1e/iocore/net/P_SSLConfig.h
> --
> diff --git a/iocore/net/P_SSLConfig.h b/iocore/net/P_SSLConfig.h
> index aa4926f..0cad7d9 100644
> --- a/iocore/net/P_SSLConfig.h
> +++ b/iocore/net/P_SSLConfig.h
> @@ -32,6 +32,7 @@
> #define __P_SSLCONFIG_H__
> 
> #include "ProxyConfig.h"
> +#include "SSLSessionCache.h"
> 
> struct SSLCertLookup;
> 
> @@ -51,7 +52,8 @@ struct SSLConfigParams : public ConfigInfo
>   enum SSL_SESSION_CACHE_MODE
>   {
> SSL_SESSION_CACHE_MODE_OFF = 0,
> -SSL_SESSION_CACHE_MODE_SERVER = 1
> +SSL_SESSION_CACHE_MODE_SERVER_OPENSSL_IMPL = 1,
> +SSL_SESSION_CACHE_MODE_SERVER_ATS_IMPL = 2
>   };
> 
>   SSLConfigParams();
> @@ -69,6 +71,8 @@ struct SSLConfigParams : public ConfigInfo
>   int verify_depth;
>   int ssl_session_cache; // SSL_SESSION_CACHE_MODE
>   int ssl_session_cache_size;
> +  int ssl_session_cache_num_buckets;
> +  int ssl_session_cache_skip_on_contention;
>   int ssl_session_cache_timeout;
> 
>   char *  clientCertPath;
> @@ -88,6 +92,10 @@ struct SSLConfigParams : public ConfigInfo
>   static int  ssl_ocsp_request_timeout;
>   static int  ssl_ocsp_update_period;
> 
> +  static size_t session_cache_number_buckets;
> +  static size_t session_cache_max_bucket_size;
> +  static bool session_cache_skip_on_lock_contention;
> +
>   static init_ssl_ctx_func init_ssl_ctx_cb;
> 
>   void initialize();
> @@ -126,4 +134,6 @@ private:
>   static int configid;
> };
> 
> +extern SSLSessionCache *session_cache;
> +
> #endif
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/53bf5d1e/iocore/net/P_SSLUtils.h
> --
> diff --git a/iocore/net/P_SSLUtils.h b/iocore/net/P_SSLUtils.h
> index 3cf0c20..1c9f0b8 100644
> --- a/iocore/net/P_SSLUtils.h
> +++ b/iocore/net/P_SSLUtils.h
> @@ -70,6 +70,10 @@ enum SSL_Stats
>   ssl_total_tickets_verified_stat,
>   ssl_total_tickets_not_found_stat,
>   ssl_total_tickets_renewed_stat,
> +  ssl_session_cache_hit,
> +  ssl_session_cache_miss,
> +  ssl_session_cache_eviction,
> +  ssl_session_cache_lock_contention,
> 
>   /* error stats */
>   ssl_error_want_write,
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/53bf5d1e/iocore/net/SSLConfig.cc
> --
> diff --git a/iocore/net/SSLConfig.cc b/iocore/net/SSLConfig.cc
> index 402664a..3aaddc1 100644
> --- a/iocore/net/SSLConfig.cc
> +++ b/iocore/net/SSLConfig.cc
> @@ -37,6 +37,7 @@
> #include "P_SSLConfig.h"
> #include "P_SSLUtils.h"
> #include "P_SSLCertLookup.h"
> +#include "SSLSessionCache.h"
> #include 
> 
> int SSLConfig::configid = 0;
> @@ -47,6 +48,10 @@ bool SSLConfigParams::ssl_ocsp_enabled = false;
> int SSLConfigParams::ssl_ocsp_cache_timeout = 3600;
> int SSLConfigParams::ssl_ocsp_request_ti

Re: git commit: [TS-3121] Add method is_valid_response()

[James Peach]

On Oct 8, 2014, at 9:42 AM, sudhe...@apache.org wrote:

> Repository: trafficserver
> Updated Branches:
>  refs/heads/master a33073505 -> 6548088ec
> 
> 
> [TS-3121] Add method is_valid_response()
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/6548088e
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/6548088e
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/6548088e
> 
> Branch: refs/heads/master
> Commit: 6548088ec940c35115c2272c736bd2fde3731ba9
> Parents: a330735
> Author: Sudheer Vinukonda 
> Authored: Wed Oct 8 16:42:22 2014 +
> Committer: Sudheer Vinukonda 
> Committed: Wed Oct 8 16:42:22 2014 +
> 
> --
> proxy/spdy/SpdyCommon.h | 5 -
> 1 file changed, 4 insertions(+), 1 deletion(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/6548088e/proxy/spdy/SpdyCommon.h
> --
> diff --git a/proxy/spdy/SpdyCommon.h b/proxy/spdy/SpdyCommon.h
> index e14f232..5f43433 100644
> --- a/proxy/spdy/SpdyCommon.h
> +++ b/proxy/spdy/SpdyCommon.h
> @@ -83,7 +83,10 @@ public:
>   SpdyNV(TSFetchSM fetch_sm);
>   ~SpdyNV();
> 
> -  bool is_valid_response();
> +  bool 
> +  is_valid_response() {

bool is_valid_response() const {

> +return valid_response; 
> +  }
> 
> public:
>   const char **nv;
> 

Re: git commit: [TS-3112] - Add null pointer check for contp to prevent core dump after handleEvent(TS_FETCH_EVENT_EXT_HEAD_DONE)

[James Peach]

On Oct 3, 2014, at 9:01 AM, Sudheer Vinukonda  
wrote:

> Yeah, true - But, I scanned through the rest of the function and only
> found this place as missing the check. The rest of branches all use “goto
> out” conveniently which checks “contp” again.

Is this loop guaranteed to only execute once?

do {
  if (chunked_handler.state == ChunkedHandler::CHUNK_FLOW_CONTROL) {
chunked_handler.state = ChunkedHandler::CHUNK_READ_SIZE_START;
  }

  event = dechunk_body();
  if (!event) {
read_vio->reenable();
goto out;
  }

  contp->handleEvent(event, this);
} while (chunked_handler.state == ChunkedHandler::CHUNK_FLOW_CONTROL);


> 
> Thanks,
> 
> Sudheer
> 
> On 10/3/14, 8:57 AM, "James Peach"  wrote:
> 
>> On Oct 3, 2014, at 8:35 AM, Sudheer Vinukonda
>>  wrote:
>> 
>>> Hi James,
>>> 
>>> handleEvent() effectively calls the plugin (or in this case, SPDY layer)
>>> which may call TSFetchDestroy in error conditions. TSFetchDestroy sets
>>> contp to NULL, but, doesn¹t destroy FetchSM yet, since, it¹s in a tight
>>> loop protected by ³recursion² counter. When handleEvent returns,
>>> recursion
>>> is decremented and contp is already null, so, FetchSM gets destroyed.
>> 
>> Ah. In that case, there's a number of other places in InvokePluginExt
>> where that can happen ...
>> 
>>> 
>>> Thanks,
>>> 
>>> Sudheer
>>> 
>>> On 10/3/14, 8:28 AM, "James Peach"  wrote:
>>> 
>>>> On Oct 3, 2014, at 6:29 AM, sudhe...@apache.org wrote:
>>>> 
>>>>> Repository: trafficserver
>>>>> Updated Branches:
>>>>> refs/heads/master 33f651c90 -> d1b3dc66b
>>>>> 
>>>>> 
>>>>> [TS-3112] - Add null pointer check for contp to prevent core dump
>>>>> after
>>>>> handleEvent(TS_FETCH_EVENT_EXT_HEAD_DONE)
>>>>> 
>>>>> 
>>>>> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
>>>>> Commit: 
>>>>> http://git-wip-us.apache.org/repos/asf/trafficserver/commit/d1b3dc66
>>>>> Tree: 
>>>>> http://git-wip-us.apache.org/repos/asf/trafficserver/tree/d1b3dc66
>>>>> Diff: 
>>>>> http://git-wip-us.apache.org/repos/asf/trafficserver/diff/d1b3dc66
>>>>> 
>>>>> Branch: refs/heads/master
>>>>> Commit: d1b3dc66b5725879949350890ab014cf235cae64
>>>>> Parents: 33f651c
>>>>> Author: Sudheer Vinukonda 
>>>>> Authored: Fri Oct 3 13:29:03 2014 +
>>>>> Committer: Sudheer Vinukonda 
>>>>> Committed: Fri Oct 3 13:29:03 2014 +
>>>>> 
>>>>> --
>>>>> proxy/FetchSM.cc | 3 +++
>>>>> 1 file changed, 3 insertions(+)
>>>>> --
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/d1b3dc66/prox
>>>>> y/
>>>>> FetchSM.cc
>>>>> --
>>>>> diff --git a/proxy/FetchSM.cc b/proxy/FetchSM.cc
>>>>> index d7b187a..4a79db4 100644
>>>>> --- a/proxy/FetchSM.cc
>>>>> +++ b/proxy/FetchSM.cc
>>>>> @@ -249,6 +249,9 @@ FetchSM::InvokePluginExt(int fetch_event)
>>>>>   has_sent_header = true;
>>>>> }
>>>>> 
>>>>> +  if (!contp)
>>>>> +goto out;
>>>>> +
>>>> 
>>>> There's a check for contp being NULL just 10 lines above here ... how
>>>> can
>>>> it become NULL now?
>>>> 
>>>> 
>>>>> if (!has_body()) {
>>>>>   contp->handleEvent(TS_FETCH_EVENT_EXT_BODY_DONE, this);
>>>>>   goto out;
>>>>> 
>>>> 
>>> 
>> 
> 

Re: git commit: [TS-3112] - Add null pointer check for contp to prevent core dump after handleEvent(TS_FETCH_EVENT_EXT_HEAD_DONE)

[James Peach]

On Oct 3, 2014, at 8:28 AM, James Peach  wrote:

> On Oct 3, 2014, at 6:29 AM, sudhe...@apache.org wrote:
> 
>> Repository: trafficserver
>> Updated Branches:
>> refs/heads/master 33f651c90 -> d1b3dc66b
>> 
>> 
>> [TS-3112] - Add null pointer check for contp to prevent core dump after 
>> handleEvent(TS_FETCH_EVENT_EXT_HEAD_DONE)
>> 
>> 
>> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
>> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/d1b3dc66
>> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/d1b3dc66
>> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/d1b3dc66
>> 
>> Branch: refs/heads/master
>> Commit: d1b3dc66b5725879949350890ab014cf235cae64
>> Parents: 33f651c
>> Author: Sudheer Vinukonda 
>> Authored: Fri Oct 3 13:29:03 2014 +
>> Committer: Sudheer Vinukonda 
>> Committed: Fri Oct 3 13:29:03 2014 +
>> 
>> --
>> proxy/FetchSM.cc | 3 +++
>> 1 file changed, 3 insertions(+)
>> --
>> 
>> 
>> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/d1b3dc66/proxy/FetchSM.cc
>> --
>> diff --git a/proxy/FetchSM.cc b/proxy/FetchSM.cc
>> index d7b187a..4a79db4 100644
>> --- a/proxy/FetchSM.cc
>> +++ b/proxy/FetchSM.cc
>> @@ -249,6 +249,9 @@ FetchSM::InvokePluginExt(int fetch_event)
>>has_sent_header = true;
>>  }
>> 
>> +  if (!contp)
>> +goto out;
>> +
> 
> There's a check for contp being NULL just 10 lines above here ... how can it 
> become NULL now?

Also, InvokePluginExt weirdly mixes sending plugin events and de-chunking ... 
if there's no continuation, don't you still need to de-chunk? Any why is there 
even an InvokePlugin and InkvokePluginExt?

J

Re: git commit: [TS-3112] - Add null pointer check for contp to prevent core dump after handleEvent(TS_FETCH_EVENT_EXT_HEAD_DONE)

[James Peach]

On Oct 3, 2014, at 6:29 AM, sudhe...@apache.org wrote:

> Repository: trafficserver
> Updated Branches:
>  refs/heads/master 33f651c90 -> d1b3dc66b
> 
> 
> [TS-3112] - Add null pointer check for contp to prevent core dump after 
> handleEvent(TS_FETCH_EVENT_EXT_HEAD_DONE)
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/d1b3dc66
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/d1b3dc66
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/d1b3dc66
> 
> Branch: refs/heads/master
> Commit: d1b3dc66b5725879949350890ab014cf235cae64
> Parents: 33f651c
> Author: Sudheer Vinukonda 
> Authored: Fri Oct 3 13:29:03 2014 +
> Committer: Sudheer Vinukonda 
> Committed: Fri Oct 3 13:29:03 2014 +
> 
> --
> proxy/FetchSM.cc | 3 +++
> 1 file changed, 3 insertions(+)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/d1b3dc66/proxy/FetchSM.cc
> --
> diff --git a/proxy/FetchSM.cc b/proxy/FetchSM.cc
> index d7b187a..4a79db4 100644
> --- a/proxy/FetchSM.cc
> +++ b/proxy/FetchSM.cc
> @@ -249,6 +249,9 @@ FetchSM::InvokePluginExt(int fetch_event)
> has_sent_header = true;
>   }
> 
> +  if (!contp)
> +goto out;
> +

There's a check for contp being NULL just 10 lines above here ... how can it 
become NULL now?


>   if (!has_body()) {
> contp->handleEvent(TS_FETCH_EVENT_EXT_BODY_DONE, this);
> goto out;
> 

Re: git commit: TS-3108: Add port matching condition to header_rewrite

[James Peach]

On Oct 2, 2014, at 12:47 PM, sor...@apache.org wrote:

> Repository: trafficserver
> Updated Branches:
>  refs/heads/master adae7cd16 -> 5054186f9
> 
> 
> TS-3108: Add port matching condition to header_rewrite
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/5054186f
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/5054186f
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/5054186f
> 
> Branch: refs/heads/master
> Commit: 5054186f9083640583d366e732f18f846b65a6c2
> Parents: adae7cd
> Author: Phil Sorber 
> Authored: Thu Oct 2 13:47:12 2014 -0600
> Committer: Phil Sorber 
> Committed: Thu Oct 2 13:47:12 2014 -0600
> 
> --
> CHANGES  |  4 +++-
> plugins/header_rewrite/conditions.cc | 29 +
> plugins/header_rewrite/conditions.h  | 18 ++
> plugins/header_rewrite/factory.cc|  2 ++
> plugins/header_rewrite/lulu.cc   | 23 +++
> plugins/header_rewrite/lulu.h|  2 +-
> 6 files changed, 76 insertions(+), 2 deletions(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/5054186f/CHANGES
> --
> diff --git a/CHANGES b/CHANGES
> index c21cedc..867e8a3 100644
> --- a/CHANGES
> +++ b/CHANGES
> @@ -1,7 +1,9 @@
>  -*- coding: utf-8 -*-
> Changes with Apache Traffic Server 5.2.0
> 
> -  *) [TS=3068] Remove usage of Boost.
> +  *) [TS-3108] Add port matching condition to header_rewrite.
> +
> +  *) [TS-3068] Remove usage of Boost.
> 
>   *) [TS-2289] Removed old unused AIO modes.
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/5054186f/plugins/header_rewrite/conditions.cc
> --
> diff --git a/plugins/header_rewrite/conditions.cc 
> b/plugins/header_rewrite/conditions.cc
> index 0f78ace..be3d28b 100644
> --- a/plugins/header_rewrite/conditions.cc
> +++ b/plugins/header_rewrite/conditions.cc
> @@ -491,3 +491,32 @@ ConditionClientIp::append_value(std::string &s, const 
> Resources &res)
> s.append(ip);
>   }
> }
> +
> +void
> +ConditionIncomingPort::initialize(Parser &p)
> +{
> +  Condition::initialize(p);
> +
> +  Matchers* match = new Matchers(_cond_op);
> +  match->set(static_cast(strtoul(p.get_arg().c_str(), NULL, 10)));
> +  _matcher = match;
> +}
> +
> +bool
> +ConditionIncomingPort::eval(const Resources &res)
> +{
> +  uint16_t port = getPort(TSHttpTxnIncomingAddrGet(res.txnp));
> +  bool rval = static_cast*>(_matcher)->test(port);
> +  TSDebug(PLUGIN_NAME, "Evaluating INCOMING-PORT(): %d: rval: %d", port, 
> rval);
> +  return rval;
> +}
> +
> +void
> +ConditionIncomingPort::append_value(std::string &s, const Resources &res)
> +{
> +  std::ostringstream oss;
> +  uint16_t port = getPort(TSHttpTxnIncomingAddrGet(res.txnp));
> +  oss << port;
> +  s += oss.str();
> +  TSDebug(PLUGIN_NAME, "Appending %d to evaluation value -> %s", port, 
> s.c_str());
> +}
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/5054186f/plugins/header_rewrite/conditions.h
> --
> diff --git a/plugins/header_rewrite/conditions.h 
> b/plugins/header_rewrite/conditions.h
> index 0b76f42..fbb843d 100644
> --- a/plugins/header_rewrite/conditions.h
> +++ b/plugins/header_rewrite/conditions.h
> @@ -348,4 +348,22 @@ protected:
>   bool eval(const Resources &res);
> };
> 
> +class ConditionIncomingPort : public Condition
> +{
> +public:
> +  ConditionIncomingPort()
> +  {
> +TSDebug(PLUGIN_NAME_DBG, "Calling CTOR for ConditionIncomingPort");
> +  }
> +
> +  void initialize(Parser& p);
> +  void append_value(std::string &s, const Resources &res);
> +
> +protected:
> +  bool eval(const Resources &res);
> +
> +private:
> +  DISALLOW_COPY_AND_ASSIGN(ConditionIncomingPort);
> +};
> +
> #endif // __CONDITIONS_H
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/5054186f/plugins/header_rewrite/factory.cc
> --
> diff --git a/plugins/header_rewrite/factory.cc 
> b/plugins/header_rewrite/factory.cc
> index 1d99ea5..eb44369 100644
> --- a/plugins/header_rewrite/factory.cc
> +++ b/plugins/header_rewrite/factory.cc
> @@ -113,6 +113,8 @@ condition_factory(const std::string& cond)
> c = new ConditionInternalTransaction();
>   } else if (c_name == "CLIENT-IP") {
> c = new ConditionClientIp();
> +  } else if (c_name == "INCOMING-PORT") {

I think "SERVER-PORT" would be clearer ... also it would not take long to add 
CLIENT-PORT and SERVER-IP, so make it symmetric :)


J

Re: git commit: TS-1475: Coverity 1242015 Buffer not null terminated

[James Peach]

On Sep 30, 2014, at 7:28 AM, zw...@apache.org wrote:

> Repository: trafficserver
> Updated Branches:
>  refs/heads/master 98ea180cc -> f16c7615a
> 
> 
> TS-1475: Coverity 1242015 Buffer not null terminated
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/f16c7615
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/f16c7615
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/f16c7615
> 
> Branch: refs/heads/master
> Commit: f16c7615a4f9a824f011e759789331f04b945551
> Parents: 98ea180
> Author: Leif Hedstrom 
> Authored: Tue Sep 30 08:28:50 2014 -0600
> Committer: Leif Hedstrom 
> Committed: Tue Sep 30 08:28:50 2014 -0600
> 
> --
> iocore/net/SSLUtils.cc | 2 +-
> lib/ts/ink_string.cc   | 2 +-
> 2 files changed, 2 insertions(+), 2 deletions(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/f16c7615/iocore/net/SSLUtils.cc
> --
> diff --git a/iocore/net/SSLUtils.cc b/iocore/net/SSLUtils.cc
> index a807e2b..c399be7 100644
> --- a/iocore/net/SSLUtils.cc
> +++ b/iocore/net/SSLUtils.cc
> @@ -197,7 +197,7 @@ ssl_servername_callback(SSL * ssl, int * ad, void * 
> /*arg*/)
> netvc->getSSLHandShakeComplete());
> 
>   if (servername != NULL) {
> -strncpy(netvc->sniServername, servername, TS_MAX_HOST_NAME_LEN);
> +ink_strlcpy(netvc->sniServername, servername, TS_MAX_HOST_NAME_LEN);
>   }

Can you remove SSLNetVConnection::sniServername? It's not used anywhere, so it 
just makes SSLNetVConnections larger ...

Re: git commit: Build error fixes (Bison/Flex version check, ssl_cert_loader)

[James Peach]

On Sep 24, 2014, at 1:56 PM, a...@apache.org wrote:

> Repository: trafficserver
> Updated Branches:
>  refs/heads/master 60e8d9764 -> 676482c58
> 
> 
> Build error fixes (Bison/Flex version check, ssl_cert_loader)

oarfish:trafficserver.git jpeach$ ./configure
...
configure: error: Need bison version 2.4.1 or better to enable WCCP (found 
version 2.3.0)

OS X and RHEL 5 both have Bison 2.3 (also the error message is now misleading 
because I did not --enable-wccp)

> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/676482c5
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/676482c5
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/676482c5
> 
> Branch: refs/heads/master
> Commit: 676482c5851ebe51393ed017f1a9e0b590e98746
> Parents: 60e8d97
> Author: Alan M. Carroll 
> Authored: Wed Sep 24 15:56:07 2014 -0500
> Committer: Alan M. Carroll 
> Committed: Wed Sep 24 15:56:07 2014 -0500
> 
> --
> configure.ac | 15 +--
> .../experimental/ssl_cert_loader/ssl-cert-loader.cc  |  1 -
> 2 files changed, 5 insertions(+), 11 deletions(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/676482c5/configure.ac
> --
> diff --git a/configure.ac b/configure.ac
> index e3b18ae..e2f5276 100644
> --- a/configure.ac
> +++ b/configure.ac
> @@ -783,17 +783,12 @@ AS_IF([test -n "$LEX"],
>   ]
> )
> 
> -AS_IF([test "x$enable_wccp" = "xyes"],
> -  [
> -AS_IF([test -z "$LEX"  ],
> -   AC_MSG_ERROR([Need flex version $FLEX_MAJOR.$FLEX_MINOR.$FLEX_POINT 
> or better to enable WCCP (found $flex_version_check)])
> -)
> -
> -AS_IF([test -z "$YACC" ],
> -AC_MSG_ERROR([Need bison version 
> $BISON_MAJOR.$BISON_MINOR.$BISON_POINT or better to enable WCCP (found 
> $bison_version_check)])
> -)
> +AS_IF([test -z "$LEX"  ],
> +AC_MSG_ERROR([Need flex version $FLEX_MAJOR.$FLEX_MINOR.$FLEX_POINT or 
> better to enable WCCP (found $flex_version_check)])
> +)
> 
> -  ]
> +AS_IF([test -z "$YACC" ],
> +AC_MSG_ERROR([Need bison version $BISON_MAJOR.$BISON_MINOR.$BISON_POINT 
> or better to enable WCCP (found $bison_version_check)])
> )
> 
> # Check for Perl and Doxygen
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/676482c5/plugins/experimental/ssl_cert_loader/ssl-cert-loader.cc
> --
> diff --git a/plugins/experimental/ssl_cert_loader/ssl-cert-loader.cc 
> b/plugins/experimental/ssl_cert_loader/ssl-cert-loader.cc
> index 1385468..61a3236 100644
> --- a/plugins/experimental/ssl_cert_loader/ssl-cert-loader.cc
> +++ b/plugins/experimental/ssl_cert_loader/ssl-cert-loader.cc
> @@ -8,7 +8,6 @@
> # include 
> # include 
> # include 
> -# include 
> # include 
> # include 
> # include 
> 

Re: git commit: [TS-3085] Large POSTs over (relatively) slower connections failing in ats5 Call ERR_get_error_line_data() via SSL_CLR_ERR_INCR_DYN_STAT to clean up the error queue after processing an

[James Peach]

On Sep 24, 2014, at 6:40 AM, sudhe...@apache.org wrote:

> Repository: trafficserver
> Updated Branches:
>  refs/heads/master 17bef772a -> d12327d84
> 
> 
> [TS-3085] Large POSTs over (relatively) slower connections failing in ats5
> Call ERR_get_error_line_data() via SSL_CLR_ERR_INCR_DYN_STAT to clean up
> the error queue after processing an SSL_ERROR_SSL. Also, added consistent
> wrappers to SSL I/O functions and removed errno based looping on SSL_Write
> per James Peach's recommendations.  Note that, this change removes the 
> apparent
> logic of reattempting SSL_Write on transient error cases (such as ENOBUF), 
> since
> openSSL documentation doesn't indicate that errno is set during SSL_Write.
> https://www.openssl.org/docs/ssl/SSL_write.html
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/d12327d8
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/d12327d8
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/d12327d8
> 
> Branch: refs/heads/master
> Commit: d12327d841d07bfe1fa772c42272fbc908a39324
> Parents: 17bef77
> Author: Sudheer Vinukonda 
> Authored: Wed Sep 24 13:32:56 2014 +
> Committer: Sudheer Vinukonda 
> Committed: Wed Sep 24 13:32:56 2014 +
> 
> --
> iocore/net/P_SSLUtils.h | 12 +++
> iocore/net/SSLNetVConnection.cc | 69 
> iocore/net/SSLUtils.cc  | 59 ++
> 3 files changed, 93 insertions(+), 47 deletions(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/d12327d8/iocore/net/P_SSLUtils.h
> --
> diff --git a/iocore/net/P_SSLUtils.h b/iocore/net/P_SSLUtils.h
> index 82c41b4..269f009 100644
> --- a/iocore/net/P_SSLUtils.h
> +++ b/iocore/net/P_SSLUtils.h
> @@ -38,6 +38,8 @@ struct SSLCertLookup;
> class SSLNetVConnection;
> struct RecRawStatBlock;
> 
> +typedef int ssl_error_t;
> +
> enum SSL_Stats
> {
>   ssl_origin_server_expired_cert_stat,
> @@ -113,6 +115,12 @@ void SSLInitializeStatistics();
> // Release SSL_CTX and the associated data
> void SSLReleaseContext(SSL_CTX* ctx);
> 
> +// Wrapper functions to SSL I/O routines
> +ssl_error_t SSLWriteBuffer(SSL * ssl, const void * buf, size_t nbytes, 
> size_t& nwritten);
> +ssl_error_t SSLReadBuffer(SSL * ssl, void * buf, size_t nbytes, size_t& 
> nread);
> +ssl_error_t SSLAccept(SSL *ssl);
> +ssl_error_t SSLConnect(SSL * ssl);
> +
> // Log an SSL error.
> #define SSLError(fmt, ...) SSLDiagnostic(DiagsMakeLocation(), false, NULL, 
> fmt, ##__VA_ARGS__)
> #define SSLErrorVC(vc,fmt, ...) SSLDiagnostic(DiagsMakeLocation(), false, vc, 
> fmt, ##__VA_ARGS__)
> @@ -120,6 +128,10 @@ void SSLReleaseContext(SSL_CTX* ctx);
> #define SSLDebug(fmt, ...) SSLDiagnostic(DiagsMakeLocation(), true, NULL, 
> fmt, ##__VA_ARGS__)
> #define SSLDebugVC(vc,fmt, ...) SSLDiagnostic(DiagsMakeLocation(), true, vc, 
> fmt, ##__VA_ARGS__)
> 
> +#define SSL_CLR_ERR_INCR_DYN_STAT(x, fmt, ...) \
> +  SSLDiagnostic(DiagsMakeLocation(), true, NULL, fmt, ##__VA_ARGS__); \
> +  RecIncrRawStat(ssl_rsb, NULL, (int) x, 1);
> +

Pleas wrap this in a do { } while(0) ...

> void SSLDiagnostic(const SrcLoc& loc, bool debug, SSLNetVConnection * vc, 
> const char * fmt, ...) TS_PRINTFLIKE(4, 5);
> 
> // Return a static string name for a SSL_ERROR constant.
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/d12327d8/iocore/net/SSLNetVConnection.cc
> --
> diff --git a/iocore/net/SSLNetVConnection.cc b/iocore/net/SSLNetVConnection.cc
> index b4269e7..0f26679 100644
> --- a/iocore/net/SSLNetVConnection.cc
> +++ b/iocore/net/SSLNetVConnection.cc
> @@ -172,24 +172,6 @@ debug_certificate_name(const char * msg, X509_NAME * 
> name)
>   BIO_free(bio);
> }
> 
> -static inline int
> -do_SSL_write(SSL * ssl, void *buf, int size)
> -{
> -  int r = 0;
> -  do {
> -// need to check into SSL error handling
> -// to see if this is good enough.
> -r = SSL_write(ssl, (const char *) buf, size);
> -if (r >= 0)
> -  return r;
> -else
> -  r = -errno;
> -  } while (r == -EINTR || r == -ENOBUFS || r == -ENOMEM);
> -
> -  return r;
> -}
> -
> -
> static int
> ssl_read_from_net(SSLNetVConnection * sslvc, EThread * lthread, int64_t &ret)
> {
> @@ -199,7 +181,8 @@ ssl_read_from_net(SSLNetVConnection * sslvc, EThread * 
> lthread, int64_t &ret)
>   int event = SSL_READ_ERROR_NONE;
>   int64_t bytes_read;
>   int64_t block_write_avail;
> -  int sslErr = SSL_ERROR_NONE;
> +  ssl_error_t sslErr = SSL_ERROR_NONE;
> +  int nread = 0;
> 
>   for (bytes_read = 0; (b != 0) && (sslErr == SSL_ERROR_NONE); b = b->next) {
> block_write_avail = b->write_avail();
> @@ -209,23 +192,22 

Re: git commit: TS-3073: Forward FIN from client to server in tr-pass handling.

[James Peach]

On Sep 21, 2014, at 6:19 PM, a...@apache.org wrote:

> Repository: trafficserver
> Updated Branches:
>  refs/heads/master 9e46e4e19 -> 514c9e1a6
> 
> 
> TS-3073: Forward FIN from client to server in tr-pass handling.
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/514c9e1a
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/514c9e1a
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/514c9e1a
> 
> Branch: refs/heads/master
> Commit: 514c9e1a6a77bf0b7b956be9f9cc79306e98dedc
> Parents: 9e46e4e
> Author: Alan M. Carroll 
> Authored: Sun Sep 21 17:25:25 2014 -0500
> Committer: Alan M. Carroll 
> Committed: Sun Sep 21 20:18:49 2014 -0500
> 
> --
> CHANGES|  2 ++
> proxy/http/HttpClientSession.h |  1 +
> proxy/http/HttpSM.cc   | 22 --
> proxy/http/HttpTunnel.cc   | 11 +++
> 4 files changed, 30 insertions(+), 6 deletions(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/514c9e1a/CHANGES
> --
> diff --git a/CHANGES b/CHANGES
> index bccfff6..b401177 100644
> --- a/CHANGES
> +++ b/CHANGES
> @@ -1,6 +1,8 @@
>  -*- coding: utf-8 -*-
> Changes with Apache Traffic Server 5.2.0
> 
> +  *) [TS-3073] Fix FIN forwarding issue with transparent pass-through.
> +
>   *) [TS-3059] Add the TSTextLogObjectRollingSizeMbSet API function.
>Author: Brian Rectanus 
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/514c9e1a/proxy/http/HttpClientSession.h
> --
> diff --git a/proxy/http/HttpClientSession.h b/proxy/http/HttpClientSession.h
> index ef0768d..7da29d9 100644
> --- a/proxy/http/HttpClientSession.h
> +++ b/proxy/http/HttpClientSession.h
> @@ -72,6 +72,7 @@ public:
>   void new_transaction();
> 
>   void set_half_close_flag() { half_close = true; };
> +  bool get_half_close_flag() { return half_close; };


bool get_half_close_flag() const { ... };

>   virtual void release(IOBufferReader * r);
>   NetVConnection *get_netvc() const { return client_vc;  };
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/514c9e1a/proxy/http/HttpSM.cc
> --
> diff --git a/proxy/http/HttpSM.cc b/proxy/http/HttpSM.cc
> index eadfe1d..3dde3c3 100644
> --- a/proxy/http/HttpSM.cc
> +++ b/proxy/http/HttpSM.cc
> @@ -670,7 +670,10 @@ HttpSM::state_read_client_request_header(int event, void 
> *data)
> state = PARSE_ERROR;
>   }
> 
> -  if (event == VC_EVENT_READ_READY &&
> +  // We need to handle EOS as well as READ_READY because the client
> +  // may have sent all of the data already followed by a fIN and that
> +  // should be OK.
> +  if ((event == VC_EVENT_READ_READY || event == VC_EVENT_EOS) &&
>   state == PARSE_ERROR &&
>   is_transparent_passthrough_allowed() &&
>   ua_raw_buffer_reader != NULL) {
> @@ -686,6 +689,12 @@ HttpSM::state_read_client_request_header(int event, void 
> *data)
> 
>   /* establish blind tunnel */
>   setup_blind_tunnel_port();
> +
> +  // Setting half close means we will send the FIN when we've written 
> all of the data.
> +  if (event == VC_EVENT_EOS) {
> +this->set_ua_half_close_flag();  
> +t_state.client_info.keep_alive = HTTP_NO_KEEPALIVE;
> +  }
>   return 0;
>   }
> 
> @@ -3590,11 +3599,6 @@ HttpSM::tunnel_handler_ssl_consumer(int event, 
> HttpTunnelConsumer * c)
> c->write_success = true;
> if (c->self_producer->alive == true) {
>   c->vc->do_io_shutdown(IO_SHUTDOWN_WRITE);
> -  if (!c->producer->alive) {
> -tunnel.close_vc(c);
> -tunnel.local_finish_all(c->self_producer);
> -break;
> -  }
> } else {
>   c->vc->do_io_close();
> }
> @@ -6393,6 +6397,12 @@ HttpSM::setup_blind_tunnel(bool send_response_hdr)
>   server_entry->in_tunnel = true;
> 
>   tunnel.tunnel_run();
> +
> +  // If we're half closed, we got a FIN from the client. Forward it on to 
> the origin server
> +  // now that we have the tunnel operational.
> +  if (ua_session->get_half_close_flag()) {
> +p_ua->vc->do_io_shutdown(IO_SHUTDOWN_READ);
> +  }
> }
> 
> void
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/514c9e1a/proxy/http/HttpTunnel.cc
> --
> diff --git a/proxy/http/HttpTunnel.cc b/proxy/http/HttpTunnel.cc
> index d71f6c8..d7f362c 100644
> --- a/proxy/http/HttpTunnel.cc
> +++ b/proxy/http/HttpTunnel.cc
> @@ -891,6 +891,17 @@ HttpTunnel::producer_run(HttpTunnelProducer * p)
>   c->write_vio = NULL;
> 

Re: [1/2] git commit: TS-2802: Additional fixups. Changed NetVCOptions to use ats_scoped_str instead of a raw pointer. Added comments.

[James Peach]

On Sep 19, 2014, at 6:30 AM, a...@apache.org wrote:

> Repository: trafficserver
> Updated Branches:
>  refs/heads/master a9905459d -> 4a143e584
> 
> 
> TS-2802: Additional fixups.
> Changed NetVCOptions to use ats_scoped_str instead of a raw pointer.
> Added comments.
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/4b48b2c2
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/4b48b2c2
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/4b48b2c2
> 
> Branch: refs/heads/master
> Commit: 4b48b2c208f7e5607c27653bb407784e2cac1926
> Parents: a990545
> Author: Alan M. Carroll 
> Authored: Thu Sep 18 15:45:26 2014 -0500
> Committer: Alan M. Carroll 
> Committed: Thu Sep 18 15:45:26 2014 -0500
> 
> --
> iocore/net/I_NetVConnection.h | 30 ++
> iocore/net/P_UnixNetVConnection.h |  1 -
> iocore/net/SSLNetVConnection.cc   |  4 ++--
> 3 files changed, 20 insertions(+), 15 deletions(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/4b48b2c2/iocore/net/I_NetVConnection.h
> --
> diff --git a/iocore/net/I_NetVConnection.h b/iocore/net/I_NetVConnection.h
> index dcf86a7..cf97c64 100644
> --- a/iocore/net/I_NetVConnection.h
> +++ b/iocore/net/I_NetVConnection.h
> @@ -159,7 +159,9 @@ struct NetVCOptions {
> 
>   EventType etype;
> 
> -  char * sni_servername; // SSL SNI to origin
> +  /** Server name to use for SNI data on an outbound connection.
> +   */
> +  ats_scoped_str sni_servername;
> 
>   /// Reset all values to defaults.
>   void reset();
> @@ -167,31 +169,35 @@ struct NetVCOptions {
>   void set_sock_param(int _recv_bufsize, int _send_bufsize, unsigned long 
> _opt_flags,
>   unsigned long _packet_mark = 0, unsigned long 
> _packet_tos = 0);
> 
> -  NetVCOptions() : sni_servername(NULL) {
> +  NetVCOptions() {
> reset();
>   }
> 
>   ~NetVCOptions() {
> -ats_free(sni_servername);
>   }
> 
> -  void set_sni_servername(const char * name, size_t len) {
> +  /** Set the SNI server name.
> +  A local copy is made of @a name.
> +  */
> +  self& set_sni_servername(const char * name, size_t len) {
> IpEndpoint ip;
> 
> -ats_free(sni_servername);
> -sni_servername = NULL;
> // Literal IPv4 and IPv6 addresses are not permitted in 
> "HostName".(rfc6066#section-3)
> if (ats_ip_pton(ts::ConstBuffer(name, len), &ip) != 0) {
>   sni_servername = ats_strndup(name, len);
> +} else {
> +  sni_servername = NULL;
> }
> +return *this;
>   }
> 
> -  NetVCOptions & operator=(const NetVCOptions & opt) {
> -if (&opt != this) {
> -  ats_free(this->sni_servername);
> -  memcpy(this, &opt, sizeof(opt));
> -  if (opt.sni_servername) {
> -this->sni_servername = ats_strdup(opt.sni_servername);
> +  self& operator=(self const& that) {
> +if (&that != this) {
> +  sni_servername = NULL; // release any current name.
> +  memcpy(this, &that, sizeof(self));
> +  if (that.sni_servername) {
> + sni_servername.release(); // otherwise we'll free the source string.


Why don't you want to free the source string in this case?

Re: [1/2] git commit: [TS-3069] Add mysql_remap to autoconf

[James Peach]

This breaks the build for every system that doesn't have mysql development 
headers installed


On Sep 9, 2014, at 2:31 PM, bri...@apache.org wrote:

> Repository: trafficserver
> Updated Branches:
>  refs/heads/master 3b68602ff -> ef6eed1cb
> 
> 
> [TS-3069] Add mysql_remap to autoconf
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/ae60ad7a
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/ae60ad7a
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/ae60ad7a
> 
> Branch: refs/heads/master
> Commit: ae60ad7a7ecb487b9c5699a0446acd9c0bc19a44
> Parents: 3b68602
> Author: Brian Geffon 
> Authored: Tue Sep 9 14:30:35 2014 -0700
> Committer: Brian Geffon 
> Committed: Tue Sep 9 14:30:35 2014 -0700
> 
> --
> configure.ac |  1 +
> plugins/experimental/Makefile.am |  1 +
> plugins/experimental/mysql_remap/Makefile.am | 21 +
> 3 files changed, 23 insertions(+)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ae60ad7a/configure.ac
> --
> diff --git a/configure.ac b/configure.ac
> index 4f9ad08..9ce7718 100644
> --- a/configure.ac
> +++ b/configure.ac
> @@ -1943,6 +1943,7 @@ AS_IF([test "x$enable_experimental_plugins" = xyes], [
> plugins/experimental/healthchecks/Makefile
> plugins/experimental/hipes/Makefile
> plugins/experimental/metalink/Makefile
> +plugins/experimental/mysql_remap/Makefile
> plugins/experimental/regex_revalidate/Makefile
> plugins/experimental/remap_stats/Makefile
> plugins/experimental/s3_auth/Makefile
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ae60ad7a/plugins/experimental/Makefile.am
> --
> diff --git a/plugins/experimental/Makefile.am 
> b/plugins/experimental/Makefile.am
> index 57fec8c..661e80d 100644
> --- a/plugins/experimental/Makefile.am
> +++ b/plugins/experimental/Makefile.am
> @@ -22,6 +22,7 @@ SUBDIRS = \
>  channel_stats \
>  collapsed_connection \
>  custom_redirect \
> + mysql_remap \
>  epic \
>  escalate \
>  esi \
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ae60ad7a/plugins/experimental/mysql_remap/Makefile.am
> --
> diff --git a/plugins/experimental/mysql_remap/Makefile.am 
> b/plugins/experimental/mysql_remap/Makefile.am
> new file mode 100644
> index 000..d4cdfd5
> --- /dev/null
> +++ b/plugins/experimental/mysql_remap/Makefile.am
> @@ -0,0 +1,21 @@
> +#  Licensed to the Apache Software Foundation (ASF) under one
> +#  or more contributor license agreements.  See the NOTICE file
> +#  distributed with this work for additional information
> +#  regarding copyright ownership.  The ASF licenses this file
> +#  to you under the Apache License, Version 2.0 (the
> +#  "License"); you may not use this file except in compliance
> +#  with the License.  You may obtain a copy of the License at
> +#
> +#  http://www.apache.org/licenses/LICENSE-2.0
> +#
> +#  Unless required by applicable law or agreed to in writing, software
> +#  distributed under the License is distributed on an "AS IS" BASIS,
> +#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> +#  See the License for the specific language governing permissions and
> +#  limitations under the License.
> +
> +include $(top_srcdir)/build/plugins.mk
> +
> +pkglib_LTLIBRARIES = mysql_remap.la
> +mysql_remap_la_SOURCES = mysql_remap.cc
> +mysql_remap_la_LDFLAGS = $(TS_PLUGIN_LDFLAGS)
> 

Re: [1/6] git commit: TS-1800: Add new hash function base class

[James Peach]

On Aug 8, 2014, at 3:08 PM, sor...@apache.org wrote:

> Repository: trafficserver
> Updated Branches:
>  refs/heads/master 5c0bc7c94 -> 7fa5c5c7a
> 
> 
> TS-1800: Add new hash function base class

Any tests?

> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/9e7ceddb
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/9e7ceddb
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/9e7ceddb
> 
> Branch: refs/heads/master
> Commit: 9e7ceddbad56370643ab514d9ad918d7740f3db8
> Parents: 5c0bc7c
> Author: Phil Sorber 
> Authored: Fri Aug 8 12:30:54 2014 -0600
> Committer: Phil Sorber 
> Committed: Fri Aug 8 15:26:02 2014 -0600
> 
> --
> CHANGES|  2 ++
> lib/ts/Hash.cc | 48 ++
> lib/ts/Hash.h  | 51 +
> lib/ts/Makefile.am |  2 ++
> lib/ts/libts.h |  1 +
> 5 files changed, 104 insertions(+)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/9e7ceddb/CHANGES
> --
> diff --git a/CHANGES b/CHANGES
> index d8dd8bf..0050a2d 100644
> --- a/CHANGES
> +++ b/CHANGES
> @@ -1,6 +1,8 @@
>  -*- coding: utf-8 -*-
> Changes with Apache Traffic Server 5.1.0
> 
> +  *) [TS-1800] Create one hashing infrastructure.
> +
>   *) [TS-2860] Add AArch64 support.
> 
>   *) [TS-2916] set response header properly for combo_handler plugin.
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/9e7ceddb/lib/ts/Hash.cc
> --
> diff --git a/lib/ts/Hash.cc b/lib/ts/Hash.cc
> new file mode 100644
> index 000..a0931ef
> --- /dev/null
> +++ b/lib/ts/Hash.cc
> @@ -0,0 +1,48 @@
> +/** @file
> +
> +  @section license License
> +
> +  Licensed to the Apache Software Foundation (ASF) under one
> +  or more contributor license agreements.  See the NOTICE file
> +  distributed with this work for additional information
> +  regarding copyright ownership.  The ASF licenses this file
> +  to you under the Apache License, Version 2.0 (the
> +  "License"); you may not use this file except in compliance
> +  with the License.  You may obtain a copy of the License at
> +
> +  http://www.apache.org/licenses/LICENSE-2.0
> +
> +  Unless required by applicable law or agreed to in writing, software
> +  distributed under the License is distributed on an "AS IS" BASIS,
> +  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> +  See the License for the specific language governing permissions and
> +  limitations under the License.
> + */
> +
> +#include "Hash.h"
> +#include 
> +
> +ATSHashBase::~ATSHashBase() {
> +}
> +
> +bool
> +ATSHash::operator==(const ATSHash & other) const {
> +if (this->size() != other.size()) {
> +return false;
> +}
> +if (memcmp(this->get(), other.get(), this->size()) == 0) {
> +return true;
> +} else {
> +return false;
> +}
> +}
> +
> +bool
> +ATSHash32::operator==(const ATSHash32 & other) const {
> +return this->get() == other.get();
> +}
> +
> +bool
> +ATSHash64::operator==(const ATSHash64 & other) const {
> +return this->get() == other.get();
> +}
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/9e7ceddb/lib/ts/Hash.h
> --
> diff --git a/lib/ts/Hash.h b/lib/ts/Hash.h
> new file mode 100644
> index 000..c0e621c
> --- /dev/null
> +++ b/lib/ts/Hash.h
> @@ -0,0 +1,51 @@
> +/** @file
> +
> +  @section license License
> +
> +  Licensed to the Apache Software Foundation (ASF) under one
> +  or more contributor license agreements.  See the NOTICE file
> +  distributed with this work for additional information
> +  regarding copyright ownership.  The ASF licenses this file
> +  to you under the Apache License, Version 2.0 (the
> +  "License"); you may not use this file except in compliance
> +  with the License.  You may obtain a copy of the License at
> +
> +  http://www.apache.org/licenses/LICENSE-2.0
> +
> +  Unless required by applicable law or agreed to in writing, software
> +  distributed under the License is distributed on an "AS IS" BASIS,
> +  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> +  See the License for the specific language governing permissions and
> +  limitations under the License.
> + */
> +
> +#ifndef __HASH_H__
> +#define __HASH_H__
> +
> +#include 
> +#include 
> +
> +struct ATSHashBase {
> +virtual void update(const void *, size_t) = 0;
> +virtual void final(void) = 0;
> +virtual void clear(void) = 0;
> +virtual ~ATSHashBase();
> +

Re: [1/2] git commit: TS-2986: Adding stats to TLS errors

[James Peach]

On Aug 5, 2014, at 12:16 PM, Brian Geffon  wrote:

> Technically you'd probably aggregate all of these into a single variable
> for production monitoring, but having insight into the individual failure
> scenarios could prove to be very valuable.

I'm still very dubious about these stats.

SSL_ERROR_WANT_WRITE, SSL_ERROR_WANT_READ, and SSL_ERROR_WANT_X509_LOOKUP boil 
down to rescheduling the I/O. Hard to see how a metric for that is useful. The 
other things here end up as an error or EOF. Again, hard to see how this is 
helpful as a metric.

Handshake errors (client and server) seems like they could be interesting to 
monitor, but these are bundled in "ssl_error_ssl", so you'll really have no 
idea what is happening.

"ssl_sni_name_set_failure" would be an application error, not a metric. It 
would only happen when setting a hostname of > 255 bytes. If that happens, an 
error log is much more helpful.

I'm +1 on stats, but these could be much more useful.

> On Tue, Aug 5, 2014 at 12:11 PM, James Peach  wrote:
> 
>> What is the use case for these stats? I can see how the logs might be too
>> noisy, but they don't seem like good candidates for metrics. How would use
>> these for monitoring?
>> 
>>> On Aug 5, 2014, at 11:34 AM, bri...@apache.org wrote:
>>> 
>>> Repository: trafficserver
>>> Updated Branches:
>>> refs/heads/master b4343175e -> d9aba01de
>>> 
>>> 
>>> TS-2986: Adding stats to TLS errors
>>> 
>>> 
>>> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
>>> Commit:
>> http://git-wip-us.apache.org/repos/asf/trafficserver/commit/a8070bbb
>>> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/a8070bbb
>>> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/a8070bbb
>>> 
>>> Branch: refs/heads/master
>>> Commit: a8070bbb80f2e709f91d4c8b47d9ded4d55bdcdc
>>> Parents: b434317
>>> Author: Brian Geffon 
>>> Authored: Tue Aug 5 11:34:07 2014 -0700
>>> Committer: Brian Geffon 
>>> Committed: Tue Aug 5 11:34:07 2014 -0700
>>> 
>>> --
>>> iocore/net/P_SSLUtils.h | 10 ++
>>> iocore/net/SSLNetVConnection.cc | 60 +---
>>> iocore/net/SSLUtils.cc  | 28 +
>>> 3 files changed, 80 insertions(+), 18 deletions(-)
>>> --
>>> 
>>> 
>>> 
>> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/a8070bbb/iocore/net/P_SSLUtils.h
>>> --
>>> diff --git a/iocore/net/P_SSLUtils.h b/iocore/net/P_SSLUtils.h
>>> index b1bf01c..6e44be3 100644
>>> --- a/iocore/net/P_SSLUtils.h
>>> +++ b/iocore/net/P_SSLUtils.h
>>> @@ -69,6 +69,16 @@ enum SSL_Stats
>>>  ssl_total_tickets_not_found_stat,
>>>  ssl_total_tickets_renewed_stat,
>>> 
>>> +  /* error stats */
>>> +  ssl_error_want_write,
>>> +  ssl_error_want_read,
>>> +  ssl_error_want_x509_lookup,
>>> +  ssl_error_syscall,
>>> +  ssl_error_read_eos,
>>> +  ssl_error_zero_return,
>>> +  ssl_error_ssl,
>>> +  ssl_sni_name_set_failure,
>>> +
>>>  ssl_cipher_stats_start = 100,
>>>  ssl_cipher_stats_end = 300,
>>> 
>>> 
>>> 
>> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/a8070bbb/iocore/net/SSLNetVConnection.cc
>>> --
>>> diff --git a/iocore/net/SSLNetVConnection.cc
>> b/iocore/net/SSLNetVConnection.cc
>>> index 0f4a6b3..d3aa858 100644
>>> --- a/iocore/net/SSLNetVConnection.cc
>>> +++ b/iocore/net/SSLNetVConnection.cc
>>> @@ -140,22 +140,26 @@ ssl_read_from_net(SSLNetVConnection * sslvc,
>> EThread * lthread, int64_t &ret)
>>> 
>>>  case SSL_ERROR_WANT_WRITE:
>>>event = SSL_WRITE_WOULD_BLOCK;
>>> -Debug("ssl", "[SSL_NetVConnection::ssl_read_from_net]
>> SSL_ERROR_WOULD_BLOCK(write)");
>>> +SSL_INCREMENT_DYN_STAT(ssl_error_want_write);
>>> +Debug("ssl.error", "[SSL_NetVConnection::ssl_read_from_net]
>> SSL_ERROR_WOULD_BLOCK(write)");
>>>break;
>>>  case SSL_ERROR_WANT_READ:
>>>event = SSL_READ_WOULD_BLOCK;
>>> -Debu

Re: [1/3] ats_speed: PageSpeed optimization plugin

[James Peach]

Great! Thanks a lot Otto :)

On Aug 4, 2014, at 6:57 AM, osch...@apache.org wrote:

> Repository: trafficserver
> Updated Branches:
> refs/heads/master fab202517 -> 083abd4ff
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/083abd4f/plugins/experimental/ats_speed/gzip/README
> --
> diff --git a/plugins/experimental/ats_speed/gzip/README 
> b/plugins/experimental/ats_speed/gzip/README
> new file mode 100644
> index 000..2e74681
> --- /dev/null
> +++ b/plugins/experimental/ats_speed/gzip/README
> @@ -0,0 +1,4 @@
> +This gzip is not compiled, but only here to be able to diff later
> +with the official gzip plugin. It is very slightly modified no 
> +be able to disable caching of its compressed output through a 
> +response header
> \ No newline at end of file
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/083abd4f/plugins/experimental/ats_speed/gzip/configuration.cc
> --
> diff --git a/plugins/experimental/ats_speed/gzip/configuration.cc 
> b/plugins/experimental/ats_speed/gzip/configuration.cc
> new file mode 100644
> index 000..b1c499d
> --- /dev/null
> +++ b/plugins/experimental/ats_speed/gzip/configuration.cc
> @@ -0,0 +1,264 @@
> +/** @file
> +
> +  Transforms content using gzip or deflate
> +
> +  @section license License
> +
> +  Licensed to the Apache Software Foundation (ASF) under one
> +  or more contributor license agreements.  See the NOTICE file
> +  distributed with this work for additional information
> +  regarding copyright ownership.  The ASF licenses this file
> +  to you under the Apache License, Version 2.0 (the
> +  "License"); you may not use this file except in compliance
> +  with the License.  You may obtain a copy of the License at
> +
> +  http://www.apache.org/licenses/LICENSE-2.0
> +
> +  Unless required by applicable law or agreed to in writing, software
> +  distributed under the License is distributed on an "AS IS" BASIS,
> +  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> +  See the License for the specific language governing permissions and
> +  limitations under the License.
> + */
> +
> +#include "configuration.h"
> +#include 
> +#include 
> +#include 
> +#include 
> +
> +namespace Gzip {
> +  using namespace std;
> +
> +  void ltrim_if(string& s, int (* fp) (int)) {
> +for (size_t i = 0; i < s.size();) {
> +  if (fp(s[i])) {
> +s.erase(i,1);
> +  } else  {
> +break;
> +  }
> +}
> +  }
> +
> +  void rtrim_if(string& s, int (* fp) (int)) {
> +for (ssize_t i = (ssize_t)s.size() - 1; i >= 0; i--) {
> +  if (fp(s[i])) {
> +s.erase(i,1);
> +  } else  {
> +break;
> +  }
> +}
> +  }
> +
> +  void trim_if(string& s, int (* fp) (int)) {
> +ltrim_if(s, fp);
> +rtrim_if(s, fp);
> +  }
> +
> +  vector tokenize(const string &s, int (* fp) (int)) {
> +vector r;
> +string tmp;
> +
> +for (size_t i = 0; i < s.size(); i++) {
> +  if ( fp(s[i]) ) {
> +if ( tmp.size()  ) {
> +  r.push_back(tmp);
> +  tmp = "";
> +}
> +  } else {
> +  tmp += s[i];
> +  }
> +}
> +
> +if ( tmp.size()  ) {
> +  r.push_back(tmp);
> +}
> +
> +return r;
> +  }
> +
> +  enum ParserState {
> +kParseStart,
> +kParseCompressibleContentType,
> +kParseRemoveAcceptEncoding,
> +kParseEnable,
> +kParseCache,
> +kParseDisallow,
> +  };
> +
> +  void Configuration::AddHostConfiguration(HostConfiguration * hc){
> +host_configurations_.push_back(hc);
> +  }
> +
> +  void HostConfiguration::add_disallow(const std::string & disallow) {
> +disallows_.push_back(disallow);
> +  }
> +
> +  void HostConfiguration::add_compressible_content_type(const std::string & 
> content_type) {
> +compressible_content_types_.push_back(content_type);
> +  }
> +
> +  HostConfiguration * Configuration::Find(const char * host, int 
> host_length) {
> +HostConfiguration * host_configuration = host_configurations_[0];
> +
> +std::string shost(host, host_length);
> +
> +for (size_t i = 1; i < host_configurations_.size(); i++ ) {
> +  if (host_configurations_[i]->host() == shost){
> +host_configuration = host_configurations_[i];
> +break;
> +  }
> +}
> +
> +return host_configuration;
> +  }
> +
> +  bool HostConfiguration::IsUrlAllowed(const char * url, int url_len) {
> +string surl(url, url_len);
> +
> +for (size_t i = 0; i < disallows_.size(); i++) {
> +  if ( fnmatch (disallows_[i].c_str(), surl.c_str(), 0) == 0 ) {
> +info("url [%s] disabled for compression, matched on pattern [%s]",
> +surl.c_str(), disallows_[i].c_str());
> +return false;
> +  }
> +}
> +
> +return true;
> +  }
> +
> +  bool HostConfiguration::ContentTypeIsCompressible(const char * 
> co

Re: git commit: TS-2802: SNI support for origin servers - fix WCCP compile errors.

[James Peach]

Thanks Alan!

On Aug 1, 2014, at 7:20 AM, a...@apache.org wrote:

> Repository: trafficserver
> Updated Branches:
>  refs/heads/master 4b2429b60 -> 426879358
> 
> 
> TS-2802: SNI support for origin servers - fix WCCP compile errors.
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/42687935
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/42687935
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/42687935
> 
> Branch: refs/heads/master
> Commit: 426879358105f80785ceeb4ca375581b0c7cf79a
> Parents: 4b2429b
> Author: Alan M. Carroll 
> Authored: Fri Aug 1 09:17:08 2014 -0500
> Committer: Alan M. Carroll 
> Committed: Fri Aug 1 09:17:08 2014 -0500
> 
> --
> lib/ts/TsBuffer.h  | 26 +-
> lib/tsconfig/TsValue.cc|  6 +++---
> lib/tsconfig/TsValue.h |  6 +++---
> lib/wccp/WccpLocal.h   |  2 +-
> proxy/http/HttpTransact.cc |  1 -
> 5 files changed, 16 insertions(+), 25 deletions(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/42687935/lib/ts/TsBuffer.h
> --
> diff --git a/lib/ts/TsBuffer.h b/lib/ts/TsBuffer.h
> index 1abc6fe..4ed8ccc 100644
> --- a/lib/ts/TsBuffer.h
> +++ b/lib/ts/TsBuffer.h
> @@ -52,7 +52,7 @@ namespace ts {
> char * _ptr; ///< Pointer to base of memory chunk.
> size_t _size; ///< Size of memory chunk.
> 
> -/// Default constructor.
> +/// Default constructor (empty buffer).
> Buffer();
> 
> /** Construct from pointer and size.
> @@ -65,13 +65,11 @@ namespace ts {
> );
> /** Construct from two pointers.
>   @note This presumes a half open range, (start, end]
> - @note Due to ambiguity issues do not invoke this with
> - @a start == 0.
> */
> Buffer(
> -char* start, ///< First valid character.
> -char* end ///< First invalid character.
> -);
> + char* start, ///< First valid character.
> + char* end ///< First invalid character.
> +);
> 
> /** Equality.
> @return @c true if @a that refers to the same memory as @a this,
> @@ -129,10 +127,6 @@ namespace ts {
> 
>   /** A chunk of read only memory.
>   A convenience class because we pass this kind of pair frequently.
> -
> -  @note The default construct leaves the object
> -  uninitialized. This is for performance reasons. To construct an
> -  empty @c Buffer use @c Buffer(0).
>*/
>   struct ConstBuffer {
> typedef ConstBuffer self; ///< Self reference type.
> @@ -141,12 +135,10 @@ namespace ts {
> char const * _ptr; ///< Pointer to base of memory chunk.
> size_t _size; ///< Size of memory chunk.
> 
> -/// Default constructor.
> +/// Default constructor (empty buffer).
> ConstBuffer();
> 
> /** Construct from pointer and size.
> - @note Due to ambiguity issues do not call this with
> - two arguments if the first argument is 0.
>  */
> ConstBuffer(
>   char const * ptr, ///< Pointer to buffer.
> @@ -158,12 +150,12 @@ namespace ts {
>   @a start == 0.
> */
> ConstBuffer(
> -char const* start, ///< First valid character.
> -char const* end ///< First invalid character.
> -);
> +  char const* start, ///< First valid character.
> +  char const* end ///< First invalid character.
> +);
> /// Construct from writable buffer.
> ConstBuffer(
> -Buffer const& buffer ///< Buffer to copy.
> +  Buffer const& buffer ///< Buffer to copy.
> );
> 
> /** Equality.
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/42687935/lib/tsconfig/TsValue.cc
> --
> diff --git a/lib/tsconfig/TsValue.cc b/lib/tsconfig/TsValue.cc
> index d7c1db1..06beaec 100644
> --- a/lib/tsconfig/TsValue.cc
> +++ b/lib/tsconfig/TsValue.cc
> @@ -37,8 +37,8 @@
> // ---
> namespace ts { namespace config {
> // ---
> -Buffer const detail::NULL_BUFFER(0);
> -ConstBuffer const detail::NULL_CONST_BUFFER(0);
> +Buffer const detail::NULL_BUFFER;
> +ConstBuffer const detail::NULL_CONST_BUFFER;
> detail::ValueItem detail::ValueTableImpl::NULL_ITEM(VoidValue);
> detail::PseudoBool::Type const detail::PseudoBool::FALSE = 0;
> detail::PseudoBool::Type const detail::PseudoBool::TRUE = 
> &detail::PseudoBool::operator !;
> @@ -332,7 +332,7 @@ Configuration::getRoot() const {
> Rv
> Configuration::loadFromPath(char const* path) {
> Rv zret;
> -Buffer buffer(0);
> +Buffer buffer;
> FILE* in = fopen(path, "r");
> 
> if (in) {
> 
> http://git-wip-us

Re: git commit: TS-2863: Enable FQDN selection for shared sessions.

[James Peach]

Still reviewing, but I wish this had been a number of smaller patches. For 
example, const'ing the MD5 stuff and adding ~PluginIdentity() could have been 
independent patches ...

On Jul 24, 2014, at 7:20 PM, a...@apache.org wrote:

> Repository: trafficserver
> Updated Branches:
>  refs/heads/master be428d6f4 -> 0c358a4b1
> 
> 
> TS-2863: Enable FQDN selection for shared sessions.

[snip]

> +bool
> +ServerSessionPool::match(HttpServerSession* ss, sockaddr const* addr, 
> INK_MD5 const& hostname_hash, TSServerSessionSharingMatchType match_style)
> +{
> +  return TS_SERVER_SESSION_SHARING_MATCH_NONE != match_style && // if no 
> matching allowed, fail immediately.
> +// The hostname matches if we're not checking it or it (and the port!) 
> is a match.
> +(TS_SERVER_SESSION_SHARING_MATCH_IP == match_style ||  
> (ats_ip_port_cast(addr) == ats_ip_port_cast(ss->server_ip) && 
> ss->hostname_hash == hostname_hash)) &&
> +// The IP address matches if we're not checking it or it is a match.
> +(TS_SERVER_SESSION_SHARING_MATCH_HOST == match_style || 
> ats_ip_addr_port_eq(ss->server_ip, addr))
> +;
> +}

The comments help, but IMHO this would still be easier to read in a more 
declarative style:

switch (match_style) {
case TS_SERVER_SESSION_SHARING_MATCH_IP:
return ats_ip_port_cast(addr) == ats_ip_port_cast(ss->server_ip) && 
ss->hostname_hash == hostname_hash;
case TS_SERVER_SESSION_SHARING_MATCH_HOST:
return ats_ip_addr_port_eq(ss->server_ip, addr);
default:
ink_assert(match_style == TS_SERVER_SESSION_SHARING_MATCH_NONE);
return false;
}

Re: [2/3] git commit: TS-1146: added counters to TLS ticket callback

[James Peach]

On Jul 17, 2014, at 10:24 AM, bri...@apache.org wrote:

> TS-1146: added counters to TLS ticket callback

This is a new feature, so it should have had a new Jira ticket.

> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/a65742cd
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/a65742cd
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/a65742cd
> 
> Branch: refs/heads/master
> Commit: a65742cd81de5f21ed65d7bc8d7ece2046c5ff6d
> Parents: 5762069
> Author: Alexey Ivanov 
> Authored: Fri Jul 4 21:05:01 2014 -0700
> Committer: Brian Geffon 
> Committed: Thu Jul 17 10:23:51 2014 -0700
> 
> --
> iocore/net/P_SSLUtils.h |  4 
> iocore/net/SSLUtils.cc  | 19 ++-
> 2 files changed, 22 insertions(+), 1 deletion(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/a65742cd/iocore/net/P_SSLUtils.h
> --
> diff --git a/iocore/net/P_SSLUtils.h b/iocore/net/P_SSLUtils.h
> index 5145cb7..b1bf01c 100644
> --- a/iocore/net/P_SSLUtils.h
> +++ b/iocore/net/P_SSLUtils.h
> @@ -64,6 +64,10 @@ enum SSL_Stats
>   ssl_user_agent_session_timeout_stat,
>   ssl_total_handshake_time_stat,
>   ssl_total_success_handshake_count_stat,
> +  ssl_total_tickets_created_stat,
> +  ssl_total_tickets_verified_stat,
> +  ssl_total_tickets_not_found_stat,
> +  ssl_total_tickets_renewed_stat,
> 
>   ssl_cipher_stats_start = 100,
>   ssl_cipher_stats_end = 300,
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/a65742cd/iocore/net/SSLUtils.cc
> --
> diff --git a/iocore/net/SSLUtils.cc b/iocore/net/SSLUtils.cc
> index 2d26adc..5ee77e2 100644
> --- a/iocore/net/SSLUtils.cc
> +++ b/iocore/net/SSLUtils.cc
> @@ -644,6 +644,21 @@ SSLInitializeStatistics()
>  RECD_INT, RECP_PERSISTENT, (int) 
> ssl_total_success_handshake_count_stat,
>  RecRawStatSyncCount);
> 
> +  // TLS tickets
> +  RecRegisterRawStat(ssl_rsb, RECT_PROCESS, 
> "proxy.process.ssl.total_tickets_created",
> + RECD_INT, RECP_PERSISTENT, (int) 
> ssl_total_tickets_created_stat,
> + RecRawStatSyncCount);
> +  RecRegisterRawStat(ssl_rsb, RECT_PROCESS, 
> "proxy.process.ssl.total_tickets_verified",
> + RECD_INT, RECP_PERSISTENT, (int) 
> ssl_total_tickets_verified_stat,
> + RecRawStatSyncCount);
> +  RecRegisterRawStat(ssl_rsb, RECT_PROCESS, 
> "proxy.process.ssl.total_tickets_not_found",
> + RECD_INT, RECP_PERSISTENT, (int) 
> ssl_total_tickets_not_found_stat,
> + RecRawStatSyncCount);
> +  // TODO: ticket renewal is not used right now.
> +  RecRegisterRawStat(ssl_rsb, RECT_PROCESS, 
> "proxy.process.ssl.total_tickets_renewed",
> + RECD_INT, RECP_PERSISTENT, (int) 
> ssl_total_tickets_renewed_stat,
> + RecRawStatSyncCount);

I think that better stat names would be:

proxy.process.ssl.session_tickets.created
proxy.process.ssl.session_tickets.verified
proxy.process.ssl.session_tickets.not_found
proxy.process.ssl.session_tickets.renewed

This namespacing makes it easier to collect the group in metrics systems, ie. 
proxy.process.ssl.session_tickets.*

> +
>   // Get and register the SSL cipher stats. Note that we are using the 
> default SSL context to obtain
>   // the cipher list. This means that the set of ciphers is fixed by the 
> build configuration and not
>   // filtered by proxy.config.ssl.server.cipher_suite. This keeps the set of 
> cipher suites stable across
> @@ -1493,11 +1508,12 @@ ssl_callback_session_ticket(
> EVP_EncryptInit_ex(cipher_ctx, EVP_aes_128_cbc(), NULL, 
> ssl_ticket_key->aes_key, iv);
> HMAC_Init_ex(hctx, ssl_ticket_key->hmac_secret, 16, evp_md_func, NULL);
> Debug("ssl", "create ticket for a new session");
> -
> +SSL_INCREMENT_DYN_STAT(ssl_total_tickets_created_stat);
> return 0;
>   } else if (enc == 0) {
> if (memcmp(keyname, ssl_ticket_key->key_name, 16)) {
>   Error("keyname is not consistent.");
> +  SSL_INCREMENT_DYN_STAT(ssl_total_tickets_not_found_stat);
>   return 0;
> }
> 
> @@ -1505,6 +1521,7 @@ ssl_callback_session_ticket(
> HMAC_Init_ex(hctx, ssl_ticket_key->hmac_secret, 16, evp_md_func, NULL);
> 
> Debug("ssl", "verify the ticket for an existing session.");
> +SSL_INCREMENT_DYN_STAT(ssl_total_tickets_verified_stat);
> return 1;
>   }
> 
> 

Re: git commit: TS-2780: Core dump in SpdyRequest::clear() in production testing of SPDY

[James Peach]

A helper function would make this cleaner:

SpdyRequest *
SpdyClientSession::find_request(int streamid) {
  map::iterator iter = this->req_map.find(streamid);
  return iter == this->req_map.end() ? NULL : iter->second;
}

On Jul 8, 2014, at 12:29 PM, bc...@apache.org wrote:

> Repository: trafficserver
> Updated Branches:
>  refs/heads/master 0c18f7bcd -> c9d443353
> 
> 
> TS-2780: Core dump in SpdyRequest::clear() in production testing of SPDY
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/c9d44335
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/c9d44335
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/c9d44335
> 
> Branch: refs/heads/master
> Commit: c9d4433531767c9b5f6db42b488af4552bc5c4a9
> Parents: 0c18f7b
> Author: Sudheer Vinukonda 
> Authored: Tue Jul 8 12:28:57 2014 -0700
> Committer: Bryan Call 
> Committed: Tue Jul 8 12:28:57 2014 -0700
> 
> --
> proxy/spdy/SpdyCallbacks.cc | 10 ++
> proxy/spdy/SpdyClientSession.cc |  3 ++-
> 2 files changed, 8 insertions(+), 5 deletions(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/c9d44335/proxy/spdy/SpdyCallbacks.cc
> --
> diff --git a/proxy/spdy/SpdyCallbacks.cc b/proxy/spdy/SpdyCallbacks.cc
> index 5a0564f..830c336 100644
> --- a/proxy/spdy/SpdyCallbacks.cc
> +++ b/proxy/spdy/SpdyCallbacks.cc
> @@ -372,12 +372,13 @@ spdy_on_data_chunk_recv_callback(spdylay_session * 
> /*session*/, uint8_t /*flags*
>  size_t len, void *user_data)
> {
>   SpdyClientSession *sm = (SpdyClientSession *)user_data;
> -  SpdyRequest *req = sm->req_map[stream_id];
> +  SpdyRequest *req = NULL;
> +  map::iterator iter = sm->req_map.find(stream_id);
> 
>   //
>   // SpdyRequest has been deleted on error, drop this data;
>   //
> -  if (!req)
> +  if ((iter == sm->req_map.end()) || ((req=iter->second) == NULL))
> return;
> 
>   Debug("spdy", "Fetcher Append Data, len:%zu", len);
> @@ -391,7 +392,8 @@ spdy_on_data_recv_callback(spdylay_session *session, 
> uint8_t flags,
>int32_t stream_id, int32_t length, void *user_data)
> {
>   SpdyClientSession *sm = (SpdyClientSession *)user_data;
> -  SpdyRequest *req = sm->req_map[stream_id];
> +  SpdyRequest *req = NULL;
> +  map::iterator iter = sm->req_map.find(stream_id);
> 
>   spdy_show_data_frame("RECV", session, flags, stream_id, length, 
> user_data);
> 
> @@ -400,7 +402,7 @@ spdy_on_data_recv_callback(spdylay_session *session, 
> uint8_t flags,
>   // client might continue to send POST data, We should reenable
>   // sm->write_vio so that WINDOW_UPDATE has a chance to be sent.
>   //
> -  if (!req) {
> +  if ((iter == sm->req_map.end()) || ((req=iter->second) == NULL)) {
> TSVIOReenable(sm->write_vio);
> return;
>   }
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/c9d44335/proxy/spdy/SpdyClientSession.cc
> --
> diff --git a/proxy/spdy/SpdyClientSession.cc b/proxy/spdy/SpdyClientSession.cc
> index 6367e67..f816ba9 100644
> --- a/proxy/spdy/SpdyClientSession.cc
> +++ b/proxy/spdy/SpdyClientSession.cc
> @@ -378,11 +378,12 @@ spdy_read_fetch_body_callback(spdylay_session * 
> /*session*/, int32_t stream_id,
> 
>   SpdyClientSession *sm = (SpdyClientSession *)user_data;
>   SpdyRequest *req = (SpdyRequest *)source->ptr;
> +  map::iterator iter = sm->req_map.find(stream_id);
> 
>   //
>   // req has been deleted, ignore this data.
>   //
> -  if (req != sm->req_map[stream_id]) {
> +  if ((iter == sm->req_map.end()) || (req != iter->second)) {
> Debug("spdy", "stream_id:%d, call:%d, req has been deleted, return 0",
>   stream_id, g_call_cnt);
> *eof = 1;
> 

Re: git commit: Doc: Plugin identity and related logging tags.

[James Peach]

On Jun 9, 2014, at 5:35 AM, Alan M. Carroll  
wrote:

> James,
> 
> 
>> The correct syntax for referring to function arguments is :c:arg:`foo`
> 
> Except sockaddr in this case is not a functiona argument, it is a type. The 
> argument is "addr".

Ah, good point.

> 
>> :data:`NULL`
> 
> WARNING: py:data reference target not found: NULL

Weird, that's used in other places. Does :c:data:`NULL` generate an error too?

> 
> I would argue against using constructs that generate errors. We could use 
> :literal:`NULL` but that's synonymous with ``NULL``.
> 
> The real problem here is the lack of :c:const: and support for enums. We'll 
> have to fix that someday.
> 
> 
>> Well, there is not SPDY plugin, and there is no code that actually uses 
>> TSHttpConnectWithPluginId,
> 
> It is used in proxy/FetchSM.cc.

Well, that's not a plugin ;)

SPDY never makes the call directly; it happens implicitly since 
SpdyClientSession is both a continuation and a PluginIdentity. That trick is 
not available to plugins.

>> We invariably close with a See also section, which should at least refer to 
>> :manpage:`TSAPI(3ts)`
> 
> Not quite invariably - see TSHttpConnect. I'll fix it anyway.

thanks!

Re: git commit: Doc: Plugin identity and related logging tags.

[James Peach]

On Jun 5, 2014, at 6:52 PM, a...@apache.org wrote:

> Repository: trafficserver
> Updated Branches:
>  refs/heads/master caf514b25 -> 13843023a
> 
> 
> Doc: Plugin identity and related logging tags.
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/13843023
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/13843023
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/13843023
> 
> Branch: refs/heads/master
> Commit: 13843023a1eb45f9ee85b5df24a350eafc92db11
> Parents: caf514b
> Author: Alan M. Carroll 
> Authored: Thu Jun 5 16:30:10 2014 -0700
> Committer: Alan M. Carroll 
> Committed: Thu Jun 5 18:51:49 2014 -0700
> 
> --
> doc/admin/event-logging-formats.en.rst  | 10 
> .../api/TSHttpConnectWithPluginId.en.rst| 56 
> 2 files changed, 66 insertions(+)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/13843023/doc/admin/event-logging-formats.en.rst
> --
> diff --git a/doc/admin/event-logging-formats.en.rst 
> b/doc/admin/event-logging-formats.en.rst
> index 751ec35..81557f3 100644
> --- a/doc/admin/event-logging-formats.en.rst
> +++ b/doc/admin/event-logging-formats.en.rst
> @@ -220,6 +220,16 @@ The following list describes Traffic Server custom 
> logging fields.
> ``phr``
> The proxy hierarchy route; the route Traffic Server used to retrieve
> the object.
> +
> +.. _piid:
> +
> +``piid``
> +   The plugin ID for the transaction. This is set for plugin driven 
> transactions via :c:func:`TSHttpConnectWithPluginId`.
> +
> +.. _pitag:
> +
> +``pitag``
> +   The plugin tag for the transaction. This is set for plugin driven 
> transactions via :c:func:`TSHttpConnectWithPluginId`.

Nice!

> 
> ``pqbl``
> The proxy request transfer length; the body length in Traffic
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/13843023/doc/reference/api/TSHttpConnectWithPluginId.en.rst
> --
> diff --git a/doc/reference/api/TSHttpConnectWithPluginId.en.rst 
> b/doc/reference/api/TSHttpConnectWithPluginId.en.rst
> new file mode 100644
> index 000..2ddc7aa
> --- /dev/null
> +++ b/doc/reference/api/TSHttpConnectWithPluginId.en.rst
> @@ -0,0 +1,56 @@
> +.. Licensed to the Apache Software Foundation (ASF) under one or more
> +   contributor license agreements.  See the NOTICE file distributed
> +   with this work for additional information regarding copyright
> +   ownership.  The ASF licenses this file to you under the Apache
> +   License, Version 2.0 (the "License"); you may not use this file
> +   except in compliance with the License.  You may obtain a copy of
> +   the License at
> +
> +  http://www.apache.org/licenses/LICENSE-2.0
> +
> +   Unless required by applicable law or agreed to in writing, software
> +   distributed under the License is distributed on an "AS IS" BASIS,
> +   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> +   implied.  See the License for the specific language governing
> +   permissions and limitations under the License.
> +
> +
> +TSHttpConnectWithPluginID
> +=
> +
> +Allows the plugin to initiate an http connection. This will tag the HTTP 
> state machine with extra data that can be accessed by the logging interface. 
> The connection is treated as an HTTP transaction as if it came from a client.
> +
> +
> +Synopsis
> +
> +
> +`#include `
> +
> +.. c:function:: TSVConn TSHttpConnectWithPluginId(sockaddr const *addr, char 
> const* tag, int64_t id)
> +
> +
> +Description
> +---
> +
> +This call attempts to create an HTTP state machine and a virtual connection 
> to that state machine. This is more efficient than using 
> :c:func:`TSNetConnect` because it avoids using the operating system stack via 
> the loopback interface.
> +
> +*addr*
> +   This is the network address of the target of the connection. This 
> includes the port which should be stored in the :c:type:`sockaddr` structure.

The correct syntax for referring to function arguments is :c:arg:`foo`

> +
> +*tag*
> +   This is a tag that is passed through to the HTTP state machine. It must 
> be a persistent string that has a lifetime longer than the connection. It is 
> accessible via the log field :ref:`pitag `. This is intended as a 
> class or type identifier that is consistent across all connections for this 
> plugin. In effect, the name of the plugin. This can be ``NULL``.

:data:`NULL`

> +
> +*id*
> +   This is a numeric identifier that is passed through to the HTTP state 
> machine. It is accessible via the log field :ref:`piid `. This is 
> intended as a connection identifier and sh

Re: git commit: Update docs for ip_allow and remap.config to reflect changes to allow artibtrary methods

[James Peach]

On May 30, 2014, at 3:33 PM, bri...@apache.org wrote:

> Repository: trafficserver
> Updated Branches:
>  refs/heads/master 61b12a323 -> 82c14a99b
> 
> 
> Update docs for ip_allow and remap.config to reflect changes to allow 
> artibtrary methods
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/82c14a99
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/82c14a99
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/82c14a99
> 
> Branch: refs/heads/master
> Commit: 82c14a99bc243ea2cb70baf882a19206365fb744
> Parents: 61b12a3
> Author: Thomas Jackson 
> Authored: Fri May 30 15:25:43 2014 -0700
> Committer: Thomas Jackson 
> Committed: Fri May 30 15:25:43 2014 -0700
> 
> --
> .../configuration/ip_allow.config.en.rst| 13 +++-
> doc/reference/configuration/remap.config.en.rst | 21 
> 2 files changed, 29 insertions(+), 5 deletions(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/82c14a99/doc/reference/configuration/ip_allow.config.en.rst
> --
> diff --git a/doc/reference/configuration/ip_allow.config.en.rst 
> b/doc/reference/configuration/ip_allow.config.en.rst
> index 49812a1..857e3e6 100644
> --- a/doc/reference/configuration/ip_allow.config.en.rst
> +++ b/doc/reference/configuration/ip_allow.config.en.rst
> @@ -5,9 +5,9 @@
>   to you under the Apache License, Version 2.0 (the
>   "License"); you may not use this file except in compliance
>   with the License.  You may obtain a copy of the License at
> - 
> +
>http://www.apache.org/licenses/LICENSE-2.0
> - 
> +
>   Unless required by applicable law or agreed to in writing,
>   software distributed under the License is distributed on an
>   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
> @@ -43,9 +43,12 @@ access the Traffic Server proxy cache, and ``ip_deny`` 
> denies the
> specified client(s) to access the Traffic Server proxy cache. Multiple
> method keywords can be specified (method=GET method=HEAD), or multiple
> methods can be separated by an '\|' (method=GET\|HEAD). The method
> -keyword is optional and it is defaulted to ALL. Available methods: ALL,
> -GET, CONNECT, DELETE, HEAD, ICP_QUERY, OPTIONS, POST, PURGE, PUT,
> -TRACE, PUSH
> +keyword is optional and it is defaulted to ALL. This supports ANY string

... The ALL keyword allows any string ...

> +as the HTTP method, meaning no validation is done to check wether it

s/wether/whether/

> +is a valid HTTP method. This allows you to create filters for any method
> +that your origin may require, this is especially useful if you use newer

s/this/which/

> +methods that aren't know to trafficserver (such as PROPFIND) or if your
> +origin uses an http-ish protocol.

s/http/HTTP/

> 
> By default, the :file:`ip_allow.config` file contains the following lines,
> which allows all methods to localhost to access the Traffic Server proxy
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/82c14a99/doc/reference/configuration/remap.config.en.rst
> --
> diff --git a/doc/reference/configuration/remap.config.en.rst 
> b/doc/reference/configuration/remap.config.en.rst
> index fd430a8..e888fb4 100644
> --- a/doc/reference/configuration/remap.config.en.rst
> +++ b/doc/reference/configuration/remap.config.en.rst
> @@ -375,6 +375,27 @@ This will pass "1" and "2" to plugin1.so and "3" to 
> plugin2.so
> 
> .. _remap-config-named-filters:
> 
> +Acl Filters
> +===
> +
> +Acl filters can be created to control access of specific remap lines. The 
> markup
> +is very similar to that of :file:`ip_allow.config`, with slight changes to
> +accomodate remap markup
> +
> +Examples
> +
> +
> +::
> +map http://foo.example.com/neverpost  http://foo.example.com/neverpost 
> @action=deny @method=post
> +map http://foo.example.com/onlypost  http://foo.example.com/onlypost 
> @action=allow @method=post
> +
> +map http://foo.example.com/  http://foo.example.com/ @action=deny 
> @src_ip=1.2.3.4
> +map http://foo.example.com/  http://foo.example.com/ @action=allow 
> @src_ip=127.0.0.1
> +
> +map http://foo.example.com/  http://foo.example.com/ @action=allow 
> @src_ip=127.0.0.1 @method=post @method=get @method=head
> +
> +Note that these Acl filters will return a 403 response if the resource is 
> restricted.
> +
> Named Filters
> =
> 
> 

Re: [2/2] git commit: TS-2845: Commit e6b9cb533 causes problems with stats_over_http

[James Peach]

This seems like a fairly subtle bug. Is it possibly to construct a test to 
ensure we don't regress it? At the least, could you add a source code comment 
so that future changes are aware of how to manage squid error codes?

On May 30, 2014, at 7:13 PM, bc...@apache.org wrote:

> TS-2845: Commit e6b9cb533 causes problems with stats_over_http
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/2bf6ef38
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/2bf6ef38
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/2bf6ef38
> 
> Branch: refs/heads/5.0.x
> Commit: 2bf6ef38f9c5d0efc3b89ad2ebb3322f97c27782
> Parents: 82c14a9
> Author: Bryan Call 
> Authored: Fri May 30 17:43:03 2014 -0700
> Committer: Bryan Call 
> Committed: Fri May 30 17:43:03 2014 -0700
> 
> --
> CHANGES  | 4 +++-
> proxy/http/HttpSM.cc | 2 +-
> 2 files changed, 4 insertions(+), 2 deletions(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/2bf6ef38/CHANGES
> --
> diff --git a/CHANGES b/CHANGES
> index 6b46da2..c35a52b 100644
> --- a/CHANGES
> +++ b/CHANGES
> @@ -1,6 +1,8 @@
>  -*- coding: utf-8 -*-
> Changes with Apache Traffic Server 5.0.0
> -  
> +
> +  *) [TS-2845] Commit e6b9cb533 causes problems with stats_over_http
> +
>   *) [TS-1981] Url remap and IPAllow method filtering is broken with non-wks 
> method
>   and add support for arbitrary methods
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/2bf6ef38/proxy/http/HttpSM.cc
> --
> diff --git a/proxy/http/HttpSM.cc b/proxy/http/HttpSM.cc
> index 02afe4a..858499f 100644
> --- a/proxy/http/HttpSM.cc
> +++ b/proxy/http/HttpSM.cc
> @@ -2792,7 +2792,6 @@ HttpSM::tunnel_handler_server(int event, 
> HttpTunnelProducer * p)
>   break;
> case VC_EVENT_EOS:
>   t_state.current.server->state = HttpTransact::TRANSACTION_COMPLETE;
> -  t_state.squid_codes.log_code = SQUID_LOG_ERR_READ_ERROR;
>   break;
> }
> 
> @@ -2805,6 +2804,7 @@ HttpSM::tunnel_handler_server(int event, 
> HttpTunnelProducer * p)
>   t_state.current.server->abort = HttpTransact::ABORTED;
>   t_state.client_info.keep_alive = HTTP_NO_KEEPALIVE;
>   t_state.current.server->keep_alive = HTTP_NO_KEEPALIVE;
> +  t_state.squid_codes.log_code = SQUID_LOG_ERR_READ_ERROR;
> } else {
>   DebugSM("http", "[%" PRId64 "] [HttpSM::tunnel_handler_server] 
> finishing HTTP tunnel", sm_id);
>   p->read_success = true;
> 

Re: git commit: TS-2753: Add more SPDY and HTTPS statistics Reviewed: Bryan Call

[James Peach]

On May 29, 2014, at 12:19 AM, Wei Sun  wrote:

> 
> 
> On 5/29/14, 1:36 AM, "Bryan Call"  wrote:
> 
>> 
>> On May 26, 2014, at 6:00 PM, James Peach  wrote:
>> 
>>> Bryan,
>>> 
>>> How are you expecting proxy.process.ssl.total_handshake_time to be
>>> used? What units is it in? Unless I'm missing something, the aggregate
>>> handshake time does not seem very interesting.
>> 
>> We wanted to get a better picture of the SSL handshake time so we can
>> included it into the total request time and also look at it to see how
>> much time we are saving on average by using session tickets and session
>> caching.  Measuring SSL handshake time should ideally be measured by the
>> client, so we are trying to do the best we can with this approach.
>> 
>>> 
>>> What doesproxy.process.spdy.total_time mean, and how would I use it?
>> 
>> This is the total time spent on SPDY requests.  I think it is self
>> explanatory.
>> 
>>> 
>>> Why does proxy.process.ssl.total_success_handshake_count only count
>>> successes? What about failed handshakes?
>> 
>> When I was looking through the stats I saw that Ron was keeping track of
>> the error cases.  I would have to double check to make sure he is keeping
>> track of the failed handshakes.
>> 
>>> 
>>> Where is proxy.process.spdy.total_streams decremented?
>> 
>> Total stats are normally not decremented and are a total count since the
>> system has been running or since the last time the stats were reset.  We
>> don't want to decrement it and there is another stat to see the number of
>> active streams.
>> 
>>> 
>>> Where is proxy.process.https.connection_count decremented? Why is it
>>> persistent? If it's measuring the total number of https connections
>>> accepted, where is proxy.process.http.connection_count?
>> 
>> There is a proxy.process.http.total_client_connections already.  After
>> the commit I was thinking that proxy.process.https.connection_count name
>> should be changed to be more inline with the http stat.
>> 
>>> 
>>> SSL_INCREMENT_DYN_STAT_EX is not a great name, and it's only used once.
>>> Does it really need a global macro?
>>> 
>>> I'm not sure that sslHandshakeBeginTime is needed;
>>> UnixNetVConnection::submit_time seems to be there to mark the birth time
>>> of a netvc.
>> 
>> Yes, this can be cleaned up.
> 
> Looks like submit_time is created before ::accept() in
> do_blocking_accept(), might not be accurate to measure the starting time.

I wonder if that's a bug. NetAccept::acceptFastEvent() and net_accept() both 
set it after the accept.

> 
>> 
>> -Bryan
>> 
>>> 
>>> On May 23, 2014, at 1:14 PM, bc...@apache.org wrote:
>>> 
>>>> Repository: trafficserver
>>>> Updated Branches:
>>>> refs/heads/master 33a477145 -> 12bde5b9c
>>>> 
>>>> 
>>>> TS-2753: Add more SPDY and HTTPS statistics
>>>> Reviewed: Bryan Call
>>>> 
>>>> 
>>>> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
>>>> Commit: 
>>>> http://git-wip-us.apache.org/repos/asf/trafficserver/commit/12bde5b9
>>>> Tree: 
>>>> http://git-wip-us.apache.org/repos/asf/trafficserver/tree/12bde5b9
>>>> Diff: 
>>>> http://git-wip-us.apache.org/repos/asf/trafficserver/diff/12bde5b9
>>>> 
>>>> Branch: refs/heads/master
>>>> Commit: 12bde5b9cfbc99a6bc9227166b0107cee35d8f9f
>>>> Parents: 33a4771
>>>> Author: Wei Sun 
>>>> Authored: Fri May 23 13:14:31 2014 -0700
>>>> Committer: Bryan Call 
>>>> Committed: Fri May 23 13:14:31 2014 -0700
>>>> 
>>>> --
>>>> CHANGES | 2 ++
>>>> iocore/net/P_SSLNetVConnection.h| 1 +
>>>> iocore/net/P_SSLUtils.h | 3 +++
>>>> iocore/net/SSLNetVConnection.cc | 9 +
>>>> iocore/net/SSLNextProtocolAccept.cc | 1 +
>>>> iocore/net/SSLUtils.cc  | 8 
>>>> proxy/http/HttpClientSession.cc | 4 
>>>> proxy/http/HttpConfig.cc| 6 ++
>>>> proxy/http/HttpConfig.h | 3 +++
>>>> proxy/http/HttpTransact.cc  | 4 
>>>> proxy/spdy/SpdyClientSessi

Re: git commit: TS-2856: Remove proxy.config.spdy.verbose_in and use diags instead

[James Peach]

Nice!

On May 28, 2014, at 5:46 PM, bc...@apache.org wrote:

> Repository: trafficserver
> Updated Branches:
>  refs/heads/master d22f98910 -> d0f4d5c47
> 
> 
> TS-2856: Remove proxy.config.spdy.verbose_in and use diags instead
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/d0f4d5c4
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/d0f4d5c4
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/d0f4d5c4
> 
> Branch: refs/heads/master
> Commit: d0f4d5c47272dd2ab87fc3db6271553864822887
> Parents: d22f989
> Author: Bryan Call 
> Authored: Wed May 28 17:29:10 2014 -0700
> Committer: Bryan Call 
> Committed: Wed May 28 17:29:10 2014 -0700
> 
> --
> CHANGES   | 2 ++
> doc/reference/configuration/records.config.en.rst | 5 -
> mgmt/RecordsConfig.cc | 2 --
> proxy/spdy/SpdyClientSession.cc   | 4 ++--
> proxy/spdy/SpdyCommon.cc  | 1 -
> proxy/spdy/SpdyCommon.h   | 1 -
> 6 files changed, 4 insertions(+), 11 deletions(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/d0f4d5c4/CHANGES
> --
> diff --git a/CHANGES b/CHANGES
> index 50ce778..237a519 100644
> --- a/CHANGES
> +++ b/CHANGES
> @@ -1,6 +1,8 @@
>  -*- coding: utf-8 -*-
> Changes with Apache Traffic Server 5.0.0
> 
> +  *) [TS-2856] Remove proxy.config.spdy.verbose_in and use diags instead
> +
>   *) [TS-2391] Traffic Server tries to reverse resolve 127.0.0.1
> 
>   *) [TS-2837] Dangling pointer in URLImpl which may cause core dump
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/d0f4d5c4/doc/reference/configuration/records.config.en.rst
> --
> diff --git a/doc/reference/configuration/records.config.en.rst 
> b/doc/reference/configuration/records.config.en.rst
> index df8dcd4..21a1407 100644
> --- a/doc/reference/configuration/records.config.en.rst
> +++ b/doc/reference/configuration/records.config.en.rst
> @@ -2264,11 +2264,6 @@ SPDY Configuration
> 
>.. note:: Reloading this value affects only new SPDY connections, not 
> existing connects.
> 
> -.. ts:cv:: CONFIG proxy.config.spdy.verbose_in INT 0
> -   :reloadable:
> -
> -   Set the verbose flag for SPDY streams on inbound connections. ``0`` means 
> disable, any other value is enable.
> -
> Scheduled Update Configuration
> ==
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/d0f4d5c4/mgmt/RecordsConfig.cc
> --
> diff --git a/mgmt/RecordsConfig.cc b/mgmt/RecordsConfig.cc
> index b94e44c..8f80480 100644
> --- a/mgmt/RecordsConfig.cc
> +++ b/mgmt/RecordsConfig.cc
> @@ -1937,8 +1937,6 @@ RecordElement RecordsConfig[] = {
>   ,
>   {RECT_CONFIG, "proxy.config.spdy.initial_window_size_in", RECD_INT, 
> "65536", RECU_DYNAMIC, RR_NULL, RECC_STR, "^[0-9]+$", RECA_NULL}
>   ,
> -  {RECT_CONFIG, "proxy.config.spdy.verbose_in", RECD_INT, "0", RECU_DYNAMIC, 
> RR_NULL, RECC_INT, "[0-1]", RECA_NULL}
> -  ,
>   {RECT_CONFIG, "proxy.config.spdy.accept_no_activity_timeout", RECD_INT, 
> "30", RECU_DYNAMIC, RR_NULL, RECC_STR, "^[0-9]+$", RECA_NULL}
>   ,
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/d0f4d5c4/proxy/spdy/SpdyClientSession.cc
> --
> diff --git a/proxy/spdy/SpdyClientSession.cc b/proxy/spdy/SpdyClientSession.cc
> index 47f57c3..665510a 100644
> --- a/proxy/spdy/SpdyClientSession.cc
> +++ b/proxy/spdy/SpdyClientSession.cc
> @@ -389,7 +389,7 @@ spdy_read_fetch_body_callback(spdylay_session * 
> /*session*/, int32_t stream_id,
> 
>   Debug("spdy", "stream_id:%d, call:%d, length:%ld, already:%" PRId64,
> stream_id, g_call_cnt, length, already);
> -  if (SPDY_CFG.spdy.verbose)
> +  if (is_debug_tag_set("spdy"))
> MD5_Update(&req->recv_md5, buf, already);
> 
>   TSVIOReenable(sm->write_vio);
> @@ -404,7 +404,7 @@ spdy_read_fetch_body_callback(spdylay_session * 
> /*session*/, int32_t stream_id,
> req->url.c_str(), (end_time - req->start_time)/TS_HRTIME_MSECOND,
> req->fetch_data_len);
>   unsigned char digest[MD5_DIGEST_LENGTH];
> -  if (SPDY_CFG.spdy.verbose ) {
> +  if (is_debug_tag_set("spdy")) {
> MD5_Final(digest, &req->recv_md5);
> Debug("spdy", "recv md5sum: ");
> for (int i = 0; i < MD5_DIGEST_LENGTH; i++) {
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/d0f4d5c4/proxy/spdy/SpdyCommon.cc
> ---

Re: git commit: TS-2838 Add log fields for plugins on HttpSM and use for SPDY.

[James Peach]

Please remove the documentation for the 'cps' log field.
Please document the 'piid' log field.
I don't see how this is wired up to SPDY, could you spell that out for me?

On May 23, 2014, at 9:30 AM, a...@apache.org wrote:

> Repository: trafficserver
> Updated Branches:
>  refs/heads/master b6383928a -> aadb75226
> 
> 
> TS-2838 Add log fields for plugins on HttpSM and use for SPDY.
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/aadb7522
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/aadb7522
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/aadb7522
> 
> Branch: refs/heads/master
> Commit: aadb752267ba194a3a6f584c0874cda36e6ab859
> Parents: b638392
> Author: Alan M. Carroll 
> Authored: Fri May 23 09:30:03 2014 -0700
> Committer: Alan M. Carroll 
> Committed: Fri May 23 09:30:03 2014 -0700
> 
> --
> CHANGES   |  2 ++
> proxy/FetchSM.cc  | 17 ++---
> proxy/FetchSM.h   |  4 ++-
> proxy/InkAPI.cc   |  9 ++-
> proxy/Plugin.h| 29 ++
> proxy/PluginVC.cc | 12 -
> proxy/PluginVC.h  | 23 -
> proxy/api/ts/ts.h | 14 +++
> proxy/http/HttpClientSession.cc   |  8 ++
> proxy/http/HttpProxyServerMain.cc |  9 +++
> proxy/http/HttpSM.cc  |  1 +
> proxy/http/HttpSM.h   |  5 
> proxy/logging/Log.cc  | 30 +--
> proxy/logging/LogAccess.cc| 45 ++
> proxy/logging/LogAccess.h |  3 ++-
> proxy/logging/LogAccessHttp.cc| 23 -
> proxy/logging/LogAccessHttp.h |  3 ++-
> proxy/spdy/SpdyClientSession.cc   | 36 ---
> proxy/spdy/SpdyClientSession.h| 12 ++---
> proxy/spdy/SpdySessionAccept.cc   | 15 +++-
> proxy/spdy/SpdySessionAccept.h| 11 +++--
> 21 files changed, 198 insertions(+), 113 deletions(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/aadb7522/CHANGES
> --
> diff --git a/CHANGES b/CHANGES
> index 4f8d8af..fa34b15 100644
> --- a/CHANGES
> +++ b/CHANGES
> @@ -1,6 +1,8 @@
>  -*- coding: utf-8 -*-
> Changes with Apache Traffic Server 5.0.0
> 
> +  *) [TS-2838] Add logging fields for plugins to TS connect API. Use for 
> SPDY.
> +
>   *) [TS-2833] Remove REC_BUILD_STAND_ALONE, REC_BUILD_MGMT and
>REC_BUILD_STUB. Dead code.
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/aadb7522/proxy/FetchSM.cc
> --
> diff --git a/proxy/FetchSM.cc b/proxy/FetchSM.cc
> index c119aca..a0a4033 100644
> --- a/proxy/FetchSM.cc
> +++ b/proxy/FetchSM.cc
> @@ -61,23 +61,22 @@ FetchSM::cleanUp()
>   client_response_hdr.destroy();
>   ats_free(client_response);
>   cont_mutex.clear();
> -
> -  PluginVC *vc = (PluginVC *) http_vc;
> -
> -  vc->do_io_close();
> +  http_vc->do_io_close();
>   FetchSMAllocator.free(this);
> }
> 
> void
> FetchSM::httpConnect()
> {
> -  Debug(DEBUG_TAG, "[%s] calling httpconnect write", __FUNCTION__);
> -  http_vc = TSHttpConnect(&_addr.sa);
> +  PluginIdentity* pi = dynamic_cast(contp);
> +  char const* tag = pi ? pi->getPluginTag() : "fetchSM";
> +  int64_t id = pi ? pi->getPluginId() : 0;
> 
> -  PluginVC *vc = (PluginVC *) http_vc;
> +  Debug(DEBUG_TAG, "[%s] calling httpconnect write", __FUNCTION__);
> +  http_vc = reinterpret_cast(TSHttpConnectWithPluginId(&_addr.sa, 
> tag, id));
> 
> -  read_vio = vc->do_io_read(this, INT64_MAX, resp_buffer);
> -  write_vio = vc->do_io_write(this, getReqLen() + req_content_length, 
> req_reader);
> +  read_vio = http_vc->do_io_read(this, INT64_MAX, resp_buffer);
> +  write_vio = http_vc->do_io_write(this, getReqLen() + req_content_length, 
> req_reader);
> }
> 
> char* FetchSM::resp_get(int *length) {
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/aadb7522/proxy/FetchSM.h
> --
> diff --git a/proxy/FetchSM.h b/proxy/FetchSM.h
> index 0de5d96..c1b94bf 100644
> --- a/proxy/FetchSM.h
> +++ b/proxy/FetchSM.h
> @@ -35,6 +35,8 @@
> #include "HttpSM.h"
> #include "HttpTunnel.h"
> 
> +class PluginVC;
> +
> class FetchSM: public Continuation
> {
> public:
> @@ -139,7 +141,7 @@ private:
>   int dechunk_body();
> 
>   int recursion;
> -  TSVConn http_vc;
> +  PluginVC* http_vc;
>   VIO *read_vio;
>   VIO *write_vio;
>   MIOBuffer *req_buffer;
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/aadb7522/proxy/InkAPI.cc
> ---

Re: git commit: TS-2753: Add more SPDY and HTTPS statistics Reviewed: Bryan Call

[James Peach]

On May 27, 2014, at 9:14 AM, Sudheer Vinukonda  wrote:

> 
> 
> On 5/27/14, 8:47 AM, "James Peach"  wrote:
> 
>> On May 27, 2014, at 7:33 AM, Sudheer Vinukonda 
>> wrote:
>> 
>>> 
>>> 
>>> On 5/26/14, 6:00 PM, "James Peach"  wrote:
>>> 
>>>> Bryan,
>>>> 
>>>> How are you expecting proxy.process.ssl.total_handshake_time to be
>>>> used?
>>>> What units is it in? Unless I'm missing something, the aggregate
>>>> handshake time does not seem very interesting.
>>> 
>>> 
>>> From what I can see in the patch,
>>> proxy.process.ssl.total_handshake_time =
>>> proxy.process.ssl.total_*successful*_handshake_time and this is likely
>>> meant to be used with proxy.process.ssl.total_success_handshake_count,
>>> to
>>> determine the average *successful* handshake duration. This should be
>>> useful in measuring the ssl handshake latency and perhaps how much
>>> improvement is seen with features such as ssl session reuse or spdy.
>> 
>> This is a very shaky rationale for this metric and a very limited use
>> case. If you want to measure SSL handshake time, then you should measure
>> it directly. 
>> 
>> If you try to measure it with this metric, you will need to use the
>> connection_count metrics, which are also of dubious general utility since
>> they are counters, not gauges. Even when you do that, all you can do is
>> measure the average, which is not very interesting. You really want to
>> measure the average and the variance, or even generate a heat map. To do
>> this, you need per-request values.
>> 
>> I think that you should nuke this metric and write a plugin to gather the
>> data you actually need. Use or extend the transaction milestone API to
>> get the session accept timestamp. You can use the kernel TCP info socket
>> option to measure round trips, estimated latency, etc.
> 
> We have post processing tools that collect this metric pretty regularly
> and should be able to give the variance/average etc. Yes, we do plan to
> extend the transaction milestone API to get the session accept timestamp.

How can you get variance of a total?

J

Re: git commit: TS-2753: Add more SPDY and HTTPS statistics Reviewed: Bryan Call

[James Peach]

On May 27, 2014, at 7:33 AM, Sudheer Vinukonda  wrote:

> 
> 
> On 5/26/14, 6:00 PM, "James Peach"  wrote:
> 
>> Bryan,
>> 
>> How are you expecting proxy.process.ssl.total_handshake_time to be used?
>> What units is it in? Unless I'm missing something, the aggregate
>> handshake time does not seem very interesting.
> 
> 
> From what I can see in the patch, proxy.process.ssl.total_handshake_time =
> proxy.process.ssl.total_*successful*_handshake_time and this is likely
> meant to be used with proxy.process.ssl.total_success_handshake_count, to
> determine the average *successful* handshake duration. This should be
> useful in measuring the ssl handshake latency and perhaps how much
> improvement is seen with features such as ssl session reuse or spdy.

This is a very shaky rationale for this metric and a very limited use case. If 
you want to measure SSL handshake time, then you should measure it directly. 

If you try to measure it with this metric, you will need to use the 
connection_count metrics, which are also of dubious general utility since they 
are counters, not gauges. Even when you do that, all you can do is measure the 
average, which is not very interesting. You really want to measure the average 
and the variance, or even generate a heat map. To do this, you need per-request 
values.

I think that you should nuke this metric and write a plugin to gather the data 
you actually need. Use or extend the transaction milestone API to get the 
session accept timestamp. You can use the kernel TCP info socket option to 
measure round trips, estimated latency, etc.

>> What does proxy.process.spdy.total_time mean, and how would I use it?
>> 
>> Why does proxy.process.ssl.total_success_handshake_count only count
>> successes? What about failed handshakes?
>> 
>> Where is proxy.process.spdy.total_streams decremented?
>> 
>> Where is proxy.process.https.connection_count decremented? Why is it
>> persistent? If it's measuring the total number of https connections
>> accepted, where is proxy.process.http.connection_count?
>> 
>> SSL_INCREMENT_DYN_STAT_EX is not a great name, and it's only used once.
>> Does it really need a global macro?
>> 
>> I'm not sure that sslHandshakeBeginTime is needed;
>> UnixNetVConnection::submit_time seems to be there to mark the birth time
>> of a netvc.
>> 
>> On May 23, 2014, at 1:14 PM, bc...@apache.org wrote:
>> 
>>> Repository: trafficserver
>>> Updated Branches:
>>> refs/heads/master 33a477145 -> 12bde5b9c
>>> 
>>> 
>>> TS-2753: Add more SPDY and HTTPS statistics
>>> Reviewed: Bryan Call
>>> 
>>> 
>>> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
>>> Commit: 
>>> http://git-wip-us.apache.org/repos/asf/trafficserver/commit/12bde5b9
>>> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/12bde5b9
>>> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/12bde5b9
>>> 
>>> Branch: refs/heads/master
>>> Commit: 12bde5b9cfbc99a6bc9227166b0107cee35d8f9f
>>> Parents: 33a4771
>>> Author: Wei Sun 
>>> Authored: Fri May 23 13:14:31 2014 -0700
>>> Committer: Bryan Call 
>>> Committed: Fri May 23 13:14:31 2014 -0700
>>> 
>>> --
>>> CHANGES | 2 ++
>>> iocore/net/P_SSLNetVConnection.h| 1 +
>>> iocore/net/P_SSLUtils.h | 3 +++
>>> iocore/net/SSLNetVConnection.cc | 9 +
>>> iocore/net/SSLNextProtocolAccept.cc | 1 +
>>> iocore/net/SSLUtils.cc  | 8 
>>> proxy/http/HttpClientSession.cc | 4 
>>> proxy/http/HttpConfig.cc| 6 ++
>>> proxy/http/HttpConfig.h | 3 +++
>>> proxy/http/HttpTransact.cc  | 4 
>>> proxy/spdy/SpdyClientSession.cc | 2 ++
>>> proxy/spdy/SpdyCommon.cc| 4 
>>> proxy/spdy/SpdyCommon.h | 8 
>>> 13 files changed, 55 insertions(+)
>>> --
>>> 
>>> 
>>> 
>>> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/12bde5b9/CHANGE
>>> S
>>> --
>>> diff --git a/CHANGES b/CHANGES
>>> index c589668..8793946 100644
>>> --- a/CHANGES
>>> +++ b/CHANGES
>>> @@ -1,6 +1,8 @@
>>> 

Re: git commit: TS-2528 Use in the public mgmtapi.h interface, C99 standard.

[James Peach]

On May 26, 2014, at 5:56 PM, Leif Hedstrom  wrote:

> 
> 
>> On May 26, 2014, at 6:41 PM, James Peach  wrote:
>> 
>>> On May 23, 2014, at 5:47 PM, zw...@apache.org wrote:
>>> 
>>> Repository: trafficserver
>>> Updated Branches:
>>> refs/heads/master 639329e38 -> bc054cf8d
>>> 
>>> 
>>> TS-2528 Use  in the public mgmtapi.h interface, C99 standard.
>> 
>> So we now require C99 to build plugins? That sounds reasonable, but it 
>> should be 
> 
> No. We have no mgmt plugins / code, other than traffic_line. In fact, I don't 
> think it's documented at all ? I'll have a loksie though.

But AFAICT we do ship it as public API.

>> documented. Are you planning to change the ts.h APIs as well?
> 
> That was the plan, but I don't know if that will go in for 5.0.0.
> 
> -- Leif 
>> 
>>> 
>>> 
>>> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
>>> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/bc054cf8
>>> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/bc054cf8
>>> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/bc054cf8
>>> 
>>> Branch: refs/heads/master
>>> Commit: bc054cf8d8a7254061411f53b999c8b0cfa1052d
>>> Parents: 639329e
>>> Author: Leif Hedstrom 
>>> Authored: Fri May 23 18:46:13 2014 -0600
>>> Committer: Leif Hedstrom 
>>> Committed: Fri May 23 18:46:13 2014 -0600
>>> 
>>> --
>>> CHANGES|  3 +++
>>> mgmt/api/include/mgmtapi.h | 26 +-
>>> 2 files changed, 4 insertions(+), 25 deletions(-)
>>> --
>>> 
>>> 
>>> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/bc054cf8/CHANGES
>>> --
>>> diff --git a/CHANGES b/CHANGES
>>> index e50acad..a45df8a 100644
>>> --- a/CHANGES
>>> +++ b/CHANGES
>>> @@ -1,6 +1,9 @@
>>>-*- coding: utf-8 -*-
>>> Changes with Apache Traffic Server 5.0.0
>>> 
>>> +  *) [TS-2528] Use  in the public mgmtapi.h interface. This 
>>> follows
>>> +   the C99 standard, and we should move other public APIs to it.
>>> +
>>> *) [TS-2428] Move P_Freer.h continuations to run on ET_TASK (if available).
>>> 
>>> *) [TS-2344] 404 error was logged while url redirect request was processed
>>> 
>>> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/bc054cf8/mgmt/api/include/mgmtapi.h
>>> --
>>> diff --git a/mgmt/api/include/mgmtapi.h b/mgmt/api/include/mgmtapi.h
>>> index 19ff2ec..c9fc90d 100644
>>> --- a/mgmt/api/include/mgmtapi.h
>>> +++ b/mgmt/api/include/mgmtapi.h
>>> @@ -28,6 +28,7 @@
>>> #ifndef __TS_MGMT_API_H__
>>> #define __TS_MGMT_API_H__
>>> 
>>> +#include 
>>> #include 
>>> 
>>> /***
>>> @@ -35,34 +36,9 @@
>>> ***/
>>> 
>>> #define tsapi
>>> -
>>> #define inkexp
>>> #define inkimp
>>> 
>>> -#if !defined(linux)
>>> -#if defined (__SUNPRO_CC) || (defined (__GNUC__) || ! defined(__cplusplus))
>>> -#if !defined (bool)
>>> -#if !defined(darwin) && !defined(freebsd) && !defined(solaris)
>>> -/* XXX: What other platforms are there? */
>>> -#define bool int
>>> -#endif
>>> -#endif
>>> -
>>> -#if !defined (true)
>>> -#define true 1
>>> -#endif
>>> -
>>> -#if !defined (false)
>>> -#define false 0
>>> -#endif
>>> -
>>> -#endif
>>> -#endif  /* not linux */
>>> -
>>> -#if !defined (NULL)
>>> -#define NULL 0
>>> -#endif
>>> -
>>> #ifdef __cplusplus
>>> extern "C"
>>> {
>> 

Re: git commit: TS-2753: Add more SPDY and HTTPS statistics Reviewed: Bryan Call

[James Peach]

Bryan,

How are you expecting proxy.process.ssl.total_handshake_time to be used? What 
units is it in? Unless I'm missing something, the aggregate handshake time does 
not seem very interesting.

What does proxy.process.spdy.total_time mean, and how would I use it?

Why does proxy.process.ssl.total_success_handshake_count only count successes? 
What about failed handshakes?

Where is proxy.process.spdy.total_streams decremented?

Where is proxy.process.https.connection_count decremented? Why is it 
persistent? If it's measuring the total number of https connections accepted, 
where is proxy.process.http.connection_count?

SSL_INCREMENT_DYN_STAT_EX is not a great name, and it's only used once. Does it 
really need a global macro?

I'm not sure that sslHandshakeBeginTime is needed; 
UnixNetVConnection::submit_time seems to be there to mark the birth time of a 
netvc.

On May 23, 2014, at 1:14 PM, bc...@apache.org wrote:

> Repository: trafficserver
> Updated Branches:
>  refs/heads/master 33a477145 -> 12bde5b9c
> 
> 
> TS-2753: Add more SPDY and HTTPS statistics
> Reviewed: Bryan Call
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/12bde5b9
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/12bde5b9
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/12bde5b9
> 
> Branch: refs/heads/master
> Commit: 12bde5b9cfbc99a6bc9227166b0107cee35d8f9f
> Parents: 33a4771
> Author: Wei Sun 
> Authored: Fri May 23 13:14:31 2014 -0700
> Committer: Bryan Call 
> Committed: Fri May 23 13:14:31 2014 -0700
> 
> --
> CHANGES | 2 ++
> iocore/net/P_SSLNetVConnection.h| 1 +
> iocore/net/P_SSLUtils.h | 3 +++
> iocore/net/SSLNetVConnection.cc | 9 +
> iocore/net/SSLNextProtocolAccept.cc | 1 +
> iocore/net/SSLUtils.cc  | 8 
> proxy/http/HttpClientSession.cc | 4 
> proxy/http/HttpConfig.cc| 6 ++
> proxy/http/HttpConfig.h | 3 +++
> proxy/http/HttpTransact.cc  | 4 
> proxy/spdy/SpdyClientSession.cc | 2 ++
> proxy/spdy/SpdyCommon.cc| 4 
> proxy/spdy/SpdyCommon.h | 8 
> 13 files changed, 55 insertions(+)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/12bde5b9/CHANGES
> --
> diff --git a/CHANGES b/CHANGES
> index c589668..8793946 100644
> --- a/CHANGES
> +++ b/CHANGES
> @@ -1,6 +1,8 @@
>  -*- coding: utf-8 -*-
> Changes with Apache Traffic Server 5.0.0
> 
> +  *) [TS-2753] Add more SPDY and HTTPS statistics
> +
>   *) [TS-2677] Don't apply path / scheme URL changes in remap when method is
>CONNECT.
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/12bde5b9/iocore/net/P_SSLNetVConnection.h
> --
> diff --git a/iocore/net/P_SSLNetVConnection.h 
> b/iocore/net/P_SSLNetVConnection.h
> index 47861e4..c464e60 100644
> --- a/iocore/net/P_SSLNetVConnection.h
> +++ b/iocore/net/P_SSLNetVConnection.h
> @@ -101,6 +101,7 @@ public:
>   virtual ~SSLNetVConnection() { }
> 
>   SSL *ssl;
> +  ink_hrtime sslHandshakeBeginTime;
> 
>   static int advertise_next_protocol(SSL * ssl, const unsigned char ** out, 
> unsigned * outlen, void *);
>   static int select_next_protocol(SSL * ssl, const unsigned char ** out, 
> unsigned char * outlen, const unsigned char * in, unsigned inlen, void *);
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/12bde5b9/iocore/net/P_SSLUtils.h
> --
> diff --git a/iocore/net/P_SSLUtils.h b/iocore/net/P_SSLUtils.h
> index 7297eac..5145cb7 100644
> --- a/iocore/net/P_SSLUtils.h
> +++ b/iocore/net/P_SSLUtils.h
> @@ -62,6 +62,8 @@ enum SSL_Stats
>   ssl_user_agent_session_hit_stat,
>   ssl_user_agent_session_miss_stat,
>   ssl_user_agent_session_timeout_stat,
> +  ssl_total_handshake_time_stat,
> +  ssl_total_success_handshake_count_stat,
> 
>   ssl_cipher_stats_start = 100,
>   ssl_cipher_stats_end = 300,
> @@ -75,6 +77,7 @@ extern RecRawStatBlock *ssl_rsb;
> #define SSL_INCREMENT_DYN_STAT(x) RecIncrRawStat(ssl_rsb, NULL, (int) x, 1)
> #define SSL_DECREMENT_DYN_STAT(x) RecIncrRawStat(ssl_rsb, NULL, (int) x, -1)
> #define SSL_SET_COUNT_DYN_STAT(x,count) RecSetRawStatCount(ssl_rsb, x, count)
> +#define SSL_INCREMENT_DYN_STAT_EX(x, y) RecIncrRawStat(ssl_rsb, NULL, (int) 
> x, y)
> #define SSL_CLEAR_DYN_STAT(x) \
>   do { \
> RecSetRawStatSum(ssl_rsb, (x), 0); \
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/12bde5b9/iocore/net/SSLNetVConnection.cc
> ---

Re: git commit: TS-2808 Remove confusing and probably bogus comment in TSVConnCreate

[James Peach]

So how does one use he TSVConn that TSVConnCreate() returns?

On May 23, 2014, at 10:10 AM, zw...@apache.org wrote:

> Repository: trafficserver
> Updated Branches:
>  refs/heads/master 7fe24aefa -> 59fa19025
> 
> 
> TS-2808 Remove confusing and probably bogus comment in TSVConnCreate
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/59fa1902
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/59fa1902
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/59fa1902
> 
> Branch: refs/heads/master
> Commit: 59fa19025f0009065ee9217117b098749d31320b
> Parents: 7fe24ae
> Author: Leif Hedstrom 
> Authored: Fri May 23 11:10:40 2014 -0600
> Committer: Leif Hedstrom 
> Committed: Fri May 23 11:10:44 2014 -0600
> 
> --
> proxy/InkAPI.cc   | 20 
> proxy/api/ts/ts.h | 28 
> 2 files changed, 48 deletions(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/59fa1902/proxy/InkAPI.cc
> --
> diff --git a/proxy/InkAPI.cc b/proxy/InkAPI.cc
> index dceea6e..3811292 100644
> --- a/proxy/InkAPI.cc
> +++ b/proxy/InkAPI.cc
> @@ -6148,26 +6148,6 @@ TSActionDone(TSAction actionp)
> 
> /* Connections */
> 
> -/* Deprectated.
> -   Do not use this API.
> -   The reason is even if VConn is created using this API, it is still 
> useless.
> -   For example, if we do TSVConnRead, the read operation returns read_vio, if
> -   we do TSVIOReenable (read_vio), it actually calls:
> -   void VIO::reenable()
> -   {
> -   if (vc_server) vc_server->reenable(this);
> -   }
> -   vc_server->reenable calls:
> -   VConnection::reenable(VIO)
> -
> -   this function is virtual in VConnection.h. It is defined separately for
> -   UnixNet, NTNet and CacheVConnection.
> -
> -   Thus, unless VConn is either NetVConnection or CacheVConnection, it can't
> -   be instantiated for functions like reenable.
> -
> -   Meanwhile, this function has never been used.
> -   */
> TSVConn
> TSVConnCreate(TSEventFunc event_funcp, TSMutex mutexp)
> {
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/59fa1902/proxy/api/ts/ts.h
> --
> diff --git a/proxy/api/ts/ts.h b/proxy/api/ts/ts.h
> index 917d6b2..30f14ce 100644
> --- a/proxy/api/ts/ts.h
> +++ b/proxy/api/ts/ts.h
> @@ -2165,35 +2165,7 @@ extern "C"
>   */
>   tsapi TSReturnCode TSHttpTxnAborted(TSHttpTxn txnp);
> 
> -  /*
> -The reason is even if VConn is created using this API, it is
> -still useless. For example, if we do TSVConnRead(), the read
> -operation returns read_vio. If we do TSVIOReenable(read_vio),
> -it actually calls:
> -
> -@code
> -void VIO::reenable() {
> -if (vc_server) vc_server->reenable(this);
> -}
> -@endcode
> -
> -vc_server->reenable calls:
> -
> -@code
> -VConnection::reenable(VIO);
> -@endcode
> -
> -This function is virtual in VConnection.h. It is defined separately for
> -UnixNet, NTNet and CacheVConnection.
> -
> -Thus, unless VConn is either NetVConnection or CacheVConnection, it can't
> -be instantiated for functions like reenable.
> -
> -In addition, this function has never been used.
> -
> -  */
>   tsapi TSVConn TSVConnCreate(TSEventFunc event_funcp, TSMutex mutexp);
> -
>   tsapi TSVConn TSVConnFdCreate(int fd);
> 
>   /* api functions to access stats */
> 

Re: git commit: TS-2528 Use in the public mgmtapi.h interface, C99 standard.

[James Peach]

On May 23, 2014, at 5:47 PM, zw...@apache.org wrote:

> Repository: trafficserver
> Updated Branches:
>  refs/heads/master 639329e38 -> bc054cf8d
> 
> 
> TS-2528 Use  in the public mgmtapi.h interface, C99 standard.

So we now require C99 to build plugins? That sounds reasonable, but it should 
be documented. Are you planning to change the ts.h APIs as well?

> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/bc054cf8
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/bc054cf8
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/bc054cf8
> 
> Branch: refs/heads/master
> Commit: bc054cf8d8a7254061411f53b999c8b0cfa1052d
> Parents: 639329e
> Author: Leif Hedstrom 
> Authored: Fri May 23 18:46:13 2014 -0600
> Committer: Leif Hedstrom 
> Committed: Fri May 23 18:46:13 2014 -0600
> 
> --
> CHANGES|  3 +++
> mgmt/api/include/mgmtapi.h | 26 +-
> 2 files changed, 4 insertions(+), 25 deletions(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/bc054cf8/CHANGES
> --
> diff --git a/CHANGES b/CHANGES
> index e50acad..a45df8a 100644
> --- a/CHANGES
> +++ b/CHANGES
> @@ -1,6 +1,9 @@
>  -*- coding: utf-8 -*-
> Changes with Apache Traffic Server 5.0.0
> 
> +  *) [TS-2528] Use  in the public mgmtapi.h interface. This 
> follows
> +   the C99 standard, and we should move other public APIs to it.
> +
>   *) [TS-2428] Move P_Freer.h continuations to run on ET_TASK (if available).
> 
>   *) [TS-2344] 404 error was logged while url redirect request was processed
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/bc054cf8/mgmt/api/include/mgmtapi.h
> --
> diff --git a/mgmt/api/include/mgmtapi.h b/mgmt/api/include/mgmtapi.h
> index 19ff2ec..c9fc90d 100644
> --- a/mgmt/api/include/mgmtapi.h
> +++ b/mgmt/api/include/mgmtapi.h
> @@ -28,6 +28,7 @@
> #ifndef __TS_MGMT_API_H__
> #define __TS_MGMT_API_H__
> 
> +#include 
> #include 
> 
> /***
> @@ -35,34 +36,9 @@
>  ***/
> 
> #define tsapi
> -
> #define inkexp
> #define inkimp
> 
> -#if !defined(linux)
> -#if defined (__SUNPRO_CC) || (defined (__GNUC__) || ! defined(__cplusplus))
> -#if !defined (bool)
> -#if !defined(darwin) && !defined(freebsd) && !defined(solaris)
> -/* XXX: What other platforms are there? */
> -#define bool int
> -#endif
> -#endif
> -
> -#if !defined (true)
> -#define true 1
> -#endif
> -
> -#if !defined (false)
> -#define false 0
> -#endif
> -
> -#endif
> -#endif  /* not linux */
> -
> -#if !defined (NULL)
> -#define NULL 0
> -#endif
> -
> #ifdef __cplusplus
> extern "C"
> {
> 

Re: git commit: TS-2838 Add log fields for plugins on HttpSM and use for SPDY.

[James Peach]

This should have gone through API review. Can you please post a review email?

On May 23, 2014, at 9:30 AM, a...@apache.org wrote:

> Repository: trafficserver
> Updated Branches:
>  refs/heads/master b6383928a -> aadb75226
> 
> 
> TS-2838 Add log fields for plugins on HttpSM and use for SPDY.
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/aadb7522
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/aadb7522
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/aadb7522
> 
> Branch: refs/heads/master
> Commit: aadb752267ba194a3a6f584c0874cda36e6ab859
> Parents: b638392
> Author: Alan M. Carroll 
> Authored: Fri May 23 09:30:03 2014 -0700
> Committer: Alan M. Carroll 
> Committed: Fri May 23 09:30:03 2014 -0700
> 
> --
> CHANGES   |  2 ++
> proxy/FetchSM.cc  | 17 ++---
> proxy/FetchSM.h   |  4 ++-
> proxy/InkAPI.cc   |  9 ++-
> proxy/Plugin.h| 29 ++
> proxy/PluginVC.cc | 12 -
> proxy/PluginVC.h  | 23 -
> proxy/api/ts/ts.h | 14 +++
> proxy/http/HttpClientSession.cc   |  8 ++
> proxy/http/HttpProxyServerMain.cc |  9 +++
> proxy/http/HttpSM.cc  |  1 +
> proxy/http/HttpSM.h   |  5 
> proxy/logging/Log.cc  | 30 +--
> proxy/logging/LogAccess.cc| 45 ++
> proxy/logging/LogAccess.h |  3 ++-
> proxy/logging/LogAccessHttp.cc| 23 -
> proxy/logging/LogAccessHttp.h |  3 ++-
> proxy/spdy/SpdyClientSession.cc   | 36 ---
> proxy/spdy/SpdyClientSession.h| 12 ++---
> proxy/spdy/SpdySessionAccept.cc   | 15 +++-
> proxy/spdy/SpdySessionAccept.h| 11 +++--
> 21 files changed, 198 insertions(+), 113 deletions(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/aadb7522/CHANGES
> --
> diff --git a/CHANGES b/CHANGES
> index 4f8d8af..fa34b15 100644
> --- a/CHANGES
> +++ b/CHANGES
> @@ -1,6 +1,8 @@
>  -*- coding: utf-8 -*-
> Changes with Apache Traffic Server 5.0.0
> 
> +  *) [TS-2838] Add logging fields for plugins to TS connect API. Use for 
> SPDY.
> +
>   *) [TS-2833] Remove REC_BUILD_STAND_ALONE, REC_BUILD_MGMT and
>REC_BUILD_STUB. Dead code.
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/aadb7522/proxy/FetchSM.cc
> --
> diff --git a/proxy/FetchSM.cc b/proxy/FetchSM.cc
> index c119aca..a0a4033 100644
> --- a/proxy/FetchSM.cc
> +++ b/proxy/FetchSM.cc
> @@ -61,23 +61,22 @@ FetchSM::cleanUp()
>   client_response_hdr.destroy();
>   ats_free(client_response);
>   cont_mutex.clear();
> -
> -  PluginVC *vc = (PluginVC *) http_vc;
> -
> -  vc->do_io_close();
> +  http_vc->do_io_close();
>   FetchSMAllocator.free(this);
> }
> 
> void
> FetchSM::httpConnect()
> {
> -  Debug(DEBUG_TAG, "[%s] calling httpconnect write", __FUNCTION__);
> -  http_vc = TSHttpConnect(&_addr.sa);
> +  PluginIdentity* pi = dynamic_cast(contp);
> +  char const* tag = pi ? pi->getPluginTag() : "fetchSM";
> +  int64_t id = pi ? pi->getPluginId() : 0;
> 
> -  PluginVC *vc = (PluginVC *) http_vc;
> +  Debug(DEBUG_TAG, "[%s] calling httpconnect write", __FUNCTION__);
> +  http_vc = reinterpret_cast(TSHttpConnectWithPluginId(&_addr.sa, 
> tag, id));
> 
> -  read_vio = vc->do_io_read(this, INT64_MAX, resp_buffer);
> -  write_vio = vc->do_io_write(this, getReqLen() + req_content_length, 
> req_reader);
> +  read_vio = http_vc->do_io_read(this, INT64_MAX, resp_buffer);
> +  write_vio = http_vc->do_io_write(this, getReqLen() + req_content_length, 
> req_reader);
> }
> 
> char* FetchSM::resp_get(int *length) {
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/aadb7522/proxy/FetchSM.h
> --
> diff --git a/proxy/FetchSM.h b/proxy/FetchSM.h
> index 0de5d96..c1b94bf 100644
> --- a/proxy/FetchSM.h
> +++ b/proxy/FetchSM.h
> @@ -35,6 +35,8 @@
> #include "HttpSM.h"
> #include "HttpTunnel.h"
> 
> +class PluginVC;
> +
> class FetchSM: public Continuation
> {
> public:
> @@ -139,7 +141,7 @@ private:
>   int dechunk_body();
> 
>   int recursion;
> -  TSVConn http_vc;
> +  PluginVC* http_vc;
>   VIO *read_vio;
>   VIO *write_vio;
>   MIOBuffer *req_buffer;
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/aadb7522/proxy/InkAPI.cc
> --
> diff --git a/proxy/Ink

Re: git commit: TS-2381 Add SPDY stream count statistic.

[James Peach]

On May 22, 2014, at 9:05 AM, a...@apache.org wrote:

> Repository: trafficserver
> Updated Branches:
>  refs/heads/master 2dd94936e -> f230e610c
> 
> 
> TS-2381 Add SPDY stream count statistic.
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/f230e610
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/f230e610
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/f230e610
> 
> Branch: refs/heads/master
> Commit: f230e610cc144b1798229c004217d7bdff8d2dac
> Parents: 2dd9493
> Author: Alan M. Carroll 
> Authored: Wed May 21 19:21:28 2014 -0700
> Committer: Alan M. Carroll 
> Committed: Thu May 22 09:05:15 2014 -0700
> 
> --
> CHANGES | 2 ++
> proxy/Main.cc   | 4 
> proxy/spdy/SpdySessionAccept.cc | 3 ---
> 3 files changed, 6 insertions(+), 3 deletions(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/f230e610/CHANGES
> --
> diff --git a/CHANGES b/CHANGES
> index 0e81b9b..3fce1ff 100644
> --- a/CHANGES
> +++ b/CHANGES
> @@ -1,6 +1,8 @@
>  -*- coding: utf-8 -*-
> Changes with Apache Traffic Server 5.0.0
> 
> +  *) [TS-2831] Add SPDY stream count statistic.
> +
>   *) [TS-1665] Remove the old traffic_shell (R.I.P.).
> 
>   *) [TS-2830] Make SPDY configurable.
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/f230e610/proxy/Main.cc
> --
> diff --git a/proxy/Main.cc b/proxy/Main.cc
> index e6fd0cb..bd74d66 100644
> --- a/proxy/Main.cc
> +++ b/proxy/Main.cc
> @@ -1531,6 +1531,10 @@ main(int /* argc ATS_UNUSED */, char **argv)
> SplitDNSConfig::startup();
> #endif
> 
> +# if TS_HAS_SPDY
> +extern int spdy_config_load ();
> +spdy_config_load(); // must be before HttpProxyPort init.
> +# endif

This seems really fragile, is it going to stay, or do you have a plan to remove 
it?


> // Load HTTP port data. getNumSSLThreads depends on this.
> if (!HttpProxyPort::loadValue(http_accept_port_descriptor))
>   HttpProxyPort::loadConfig();
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/f230e610/proxy/spdy/SpdySessionAccept.cc
> --
> diff --git a/proxy/spdy/SpdySessionAccept.cc b/proxy/spdy/SpdySessionAccept.cc
> index 79b465d..6da5608 100644
> --- a/proxy/spdy/SpdySessionAccept.cc
> +++ b/proxy/spdy/SpdySessionAccept.cc
> @@ -31,9 +31,6 @@
> SpdySessionAccept::SpdySessionAccept(Continuation *ep)
> : SessionAccept(new_ProxyMutex()), endpoint(ep)
> {
> -#if TS_HAS_SPDY
> -  spdy_config_load();
> -#endif
>   SET_HANDLER(&SpdySessionAccept::mainEvent);
> }
> 
> 

Re: git commit: TS-2381 The actual "add SPDY stream counter" patch.

[James Peach]

Pretty sure TS-2831 is the bug you're looking for ...

On May 22, 2014, at 9:26 AM, a...@apache.org wrote:

> Repository: trafficserver
> Updated Branches:
>  refs/heads/master f230e610c -> 77dd7d282
> 
> 
> TS-2381 The actual "add SPDY stream counter" patch.
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/77dd7d28
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/77dd7d28
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/77dd7d28
> 
> Branch: refs/heads/master
> Commit: 77dd7d2828c40eed2baf554b8e75ecc58208f687
> Parents: f230e61
> Author: Alan M. Carroll 
> Authored: Wed May 21 17:11:24 2014 -0700
> Committer: Alan M. Carroll 
> Committed: Thu May 22 09:26:25 2014 -0700
> 
> --
> proxy/spdy/SpdyCommon.cc|  8 
> proxy/spdy/SpdyCommon.h | 15 +++
> proxy/spdy/SpdySessionAccept.cc |  1 +
> 3 files changed, 24 insertions(+)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/77dd7d28/proxy/spdy/SpdyCommon.cc
> --
> diff --git a/proxy/spdy/SpdyCommon.cc b/proxy/spdy/SpdyCommon.cc
> index 775923d..47358c2 100644
> --- a/proxy/spdy/SpdyCommon.cc
> +++ b/proxy/spdy/SpdyCommon.cc
> @@ -26,6 +26,10 @@
> 
> Config SPDY_CFG;
> 
> +// statistic names
> +// Total # of streams created.
> +static char const * const SPDY_STAT_TOTAL_STREAMS_NAME = 
> "proxy.process.spdy.total_streams";
> +
> string
> http_date(time_t t)
> {
> @@ -46,6 +50,10 @@ spdy_config_load()
> 
>   spdy_callbacks_init(&SPDY_CFG.spdy.callbacks);
> 
> +  // Get our statistics up
> +  SPDY_CFG.rsb = RecAllocateRawStatBlock(static_cast(Config::N_STATS));
> +  RecRegisterRawStat(SPDY_CFG.rsb, RECT_PROCESS, 
> SPDY_STAT_TOTAL_STREAMS_NAME, RECD_INT, RECP_NON_PERSISTENT, 
> static_cast(Config::STAT_TOTAL_STREAMS), RecRawStatSyncCount);
> +
>   return 0;
> }
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/77dd7d28/proxy/spdy/SpdyCommon.h
> --
> diff --git a/proxy/spdy/SpdyCommon.h b/proxy/spdy/SpdyCommon.h
> index cb32672..6a701c8 100644
> --- a/proxy/spdy/SpdyCommon.h
> +++ b/proxy/spdy/SpdyCommon.h
> @@ -66,6 +66,14 @@ struct Config {
>   SpdyConfig spdy;
>   int32_t accept_no_activity_timeout;
>   int32_t no_activity_timeout_in;
> +
> +  // Statistics
> +  /// This is the stat slot index for each statistic.
> +  enum StatIndex {
> +STAT_TOTAL_STREAMS,
> +N_STATS ///< Terminal counter, NOT A STAT INDEX.
> +  };
> +  RecRawStatBlock* rsb; ///< Container for statistics.
> };
> 
> // Spdy Name/Value pairs
> @@ -89,4 +97,11 @@ string http_date(time_t t);
> int spdy_config_load();
> 
> extern Config SPDY_CFG;
> +
> +// Stat helper functions
> +
> +inline void
> +SpdyStatIncrCount(Config::StatIndex idx, Continuation* contp) {
> +  RecIncrRawStatCount(SPDY_CFG.rsb, contp->mutex->thread_holding, idx, 1);
> +}
> #endif
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/77dd7d28/proxy/spdy/SpdySessionAccept.cc
> --
> diff --git a/proxy/spdy/SpdySessionAccept.cc b/proxy/spdy/SpdySessionAccept.cc
> index 6da5608..452d7c7 100644
> --- a/proxy/spdy/SpdySessionAccept.cc
> +++ b/proxy/spdy/SpdySessionAccept.cc
> @@ -42,6 +42,7 @@ SpdySessionAccept::mainEvent(int event, void * edata)
> 
> #if TS_HAS_SPDY
> spdy_sm_create(netvc, NULL, NULL);
> +SpdyStatIncrCount(Config::STAT_TOTAL_STREAMS, this);
> #else
> Error("accepted a SPDY session, but SPDY support is not available");
> netvc->do_io_close();
> 

Re: git commit: TS-2830 Make SPDY configurable.

[James Peach]

On May 21, 2014, at 6:24 PM, a...@apache.org wrote:

> Repository: trafficserver
> Updated Branches:
>  refs/heads/master 24d860c05 -> b7e69b0fa
> 
> 
> TS-2830 Make SPDY configurable.
[snip]
> cts only new SPDY connections, not existing connects.
> +
> +.. ts:cv:: CONFIG proxy.config.spdy.verbose_in INT 65536
>:reloadable:
> 
> -   Set the maximum number of concurrent streams per client connection.
> +   Set the verbose flag for SPDY streams on inbound connections.

Huh? Do we have to keep this?

Re: git commit: TS-2830 Make gcc 4.1.2 happy.

[James Peach]

On May 21, 2014, at 6:58 PM, a...@apache.org wrote:

> Repository: trafficserver
> Updated Branches:
> refs/heads/master ad6d30052 -> ada6b33e8
> 
> 
> TS-2830 Make gcc 4.1.2 happy.
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/ada6b33e
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/ada6b33e
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/ada6b33e
> 
> Branch: refs/heads/master
> Commit: ada6b33e8b55a16acbb8d9580530e4c11fc24862
> Parents: ad6d300
> Author: Alan M. Carroll 
> Authored: Wed May 21 18:58:24 2014 -0700
> Committer: Alan M. Carroll 
> Committed: Wed May 21 18:58:24 2014 -0700
> 
> --
> lib/records/I_RecHttp.h | 5 -
> 1 file changed, 4 insertions(+), 1 deletion(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ada6b33e/lib/records/I_RecHttp.h
> --
> diff --git a/lib/records/I_RecHttp.h b/lib/records/I_RecHttp.h
> index 83ffcbe..c121dfe 100644
> --- a/lib/records/I_RecHttp.h
> +++ b/lib/records/I_RecHttp.h
> @@ -45,7 +45,10 @@ class SessionProtocolSet {
>  /// Storage for the set - a bit vector.
>  uint32_t m_bits;
> public:
> -  static int const MAX = sizeof(m_bits) * CHAR_BIT;
> +  // The right way.
> +  //  static int const MAX = sizeof(m_bits) * CHAR_BIT;
> +  // The RHEL5/gcc 4.1.2 way
> +  static int const MAX = sizeof(uint32_t) * 8;

CHAR_BIT was already used as a compile-time constant and worked perfectly well. 
What wend wrong here?

Also, MAX is a terrible name, prone to collision with macros. We #define it 
ourselves in ink_defs.h. Can you please give MAX a descriptive name?

>  /// Default constructor.
>  /// Constructs and empty set.
>  SessionProtocolSet() : m_bits(0) { }
> 

Re: [1/2] git commit: [TS-2822] Crash in LogBufferIterator::next

[James Peach]

On May 21, 2014, at 9:50 AM, bri...@apache.org wrote:

> Repository: trafficserver
> Updated Branches:
>  refs/heads/master 2ba2baaa6 -> 374355c01
> 
> 
> [TS-2822] Crash in LogBufferIterator::next
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/7c314c5c
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/7c314c5c
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/7c314c5c
> 
> Branch: refs/heads/master
> Commit: 7c314c5c9bfa347fd583500698c2c586f668f7ea
> Parents: 2ba2baa
> Author: Brian Geffon 
> Authored: Wed May 21 09:49:13 2014 -0700
> Committer: Brian Geffon 
> Committed: Wed May 21 09:49:13 2014 -0700
> 
> --
> proxy/logstats.cc | 37 +++--
> 1 file changed, 31 insertions(+), 6 deletions(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/7c314c5c/proxy/logstats.cc
> --
> diff --git a/proxy/logstats.cc b/proxy/logstats.cc
> index 15aeaa2..c6386a6 100644
> --- a/proxy/logstats.cc
> +++ b/proxy/logstats.cc
> @@ -1672,6 +1672,7 @@ parse_log_buff(LogBufferHeader * buf_header, bool 
> summary = false)
> int
> process_file(int in_fd, off_t offset, unsigned max_age)
> {
> +  ink_release_assert(in_fd < FD_SETSIZE); // If you're exceeding this then 
> use CLOSEONEXEC with your opens before FORK/EXEC
>   char buffer[MAX_LOGBUFFER_SIZE];
>   int nread, buffer_bytes;
> 
> @@ -1730,7 +1731,7 @@ process_file(int in_fd, off_t offset, unsigned max_age)
> unsigned second_read_size = sizeof(LogBufferHeader) - first_read_size;
> nread = read(in_fd, &buffer[first_read_size], second_read_size);
> if (!nread || EOF == nread) {
> -  Debug("logstats", "Second read of header failed (attemped %d bytes at 
> offset %d, got nothing).", second_read_size, first_read_size);
> +  Debug("logstats", "Second read of header failed (attemped %d bytes at 
> offset %d, got nothing), errno=%d.", second_read_size, first_read_size, 
> errno);
>   return 1;
> }
> 
> @@ -1746,11 +1747,35 @@ process_file(int in_fd, off_t offset, unsigned 
> max_age)
>   return 1;
> }
> 
> -nread = read(in_fd, &buffer[sizeof(LogBufferHeader)], buffer_bytes);
> -if (!nread || EOF == nread) {
> -  Debug("logstats", "Failed to read buffer payload [%d bytes]", 
> buffer_bytes);
> -  return 1;
> -}
> +const int MAX_READ_TRIES = 5;
> +int total_read = 0;
> +int read_tries_remaining = MAX_READ_TRIES; // since the data will be old 
> anyway, let's only try a few times.
> +nread = 0;
> +do {
> +  nread = read(in_fd, &buffer[sizeof(LogBufferHeader) + total_read], 
> buffer_bytes - total_read);
> +  if (EOF == nread || !nread) { // just bail on error
> +Debug("logstats", "Read failed while reading log buffer, wanted %d 
> bytes, nread=%d, total_read=%d, errno=%d, tries_remaining=%d", buffer_bytes - 
> total_read, nread, total_read,
> +errno, read_tries_remaining);
> +return 1;
> +  } else {
> +total_read += nread;
> +  }
> +
> +  if (total_read < buffer_bytes) {
> +if (--read_tries_remaining <= 0) {
> +  Debug("logstats_failed_retries", "Unable to read after %d tries, 
> total_read=%d, buffer_bytes=%d", MAX_READ_TRIES, total_read, buffer_bytes);
> +  return 1;
> +}
> +// let's wait until we get more data on this file descriptor
> +Debug("logstats_partial_read", "Failed to read buffer payload [%d 
> bytes], total_read=%d, buffer_bytes=%d, tries_remaining=%d",
> +buffer_bytes - total_read, total_read, buffer_bytes, 
> read_tries_remaining);
> +fd_set fds;
> +struct timeval tv = {0, 50*1000}; // wait only up to 50ms
> +FD_ZERO(&fds);
> +FD_SET(in_fd, &fds);
> +select(in_fd + 1, &fds, NULL, NULL, &tv);  // we only need to select 
> up to in_fd + 1.

Can in_fd ever be a socket?


> +  }
> +} while (total_read < buffer_bytes);
> 
> // Possibly skip too old entries (the entire buffer is skipped)
> if (header->high_timestamp >= max_age) {
> 

Re: git commit: TS-2555: add more hook support for ts_lua plugin

[James Peach]

On May 20, 2014, at 9:59 PM, kic...@apache.org wrote:

> Repository: trafficserver
> Updated Branches:
>  refs/heads/master 90f44b522 -> 2ba2baaa6
> 
> 
> TS-2555: add more hook support for ts_lua plugin

I still think that declaring global functions for hooks is ugly and not 
Lua-like. Automatically re-enabling the transaction is also problem, since a 
very common pattern is to hook the transaction and reenable it later after 
doing some other work.

> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/2ba2baaa
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/2ba2baaa
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/2ba2baaa
> 
> Branch: refs/heads/master
> Commit: 2ba2baaa69c2bb99dfa95c783a79ec0165690c31
> Parents: 90f44b5
> Author: Kit Chan 
> Authored: Tue May 20 21:50:49 2014 -0700
> Committer: Kit Chan 
> Committed: Tue May 20 21:50:49 2014 -0700
> 
> --
> doc/reference/plugins/ts_lua.en.rst |  60 +++
> .../ts_lua/example/test_global_hook.lua |  88 
> .../ts_lua/example/test_txn_hook.lua| 100 +++
> plugins/experimental/ts_lua/ts_lua.c|  81 ++-
> plugins/experimental/ts_lua/ts_lua_common.h |  16 +++
> plugins/experimental/ts_lua/ts_lua_hook.c   |  64 
> plugins/experimental/ts_lua/ts_lua_util.c   |  84 
> 7 files changed, 472 insertions(+), 21 deletions(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/2ba2baaa/doc/reference/plugins/ts_lua.en.rst
> --
> diff --git a/doc/reference/plugins/ts_lua.en.rst 
> b/doc/reference/plugins/ts_lua.en.rst
> index ebcb88c..8049a53 100644
> --- a/doc/reference/plugins/ts_lua.en.rst
> +++ b/doc/reference/plugins/ts_lua.en.rst
> @@ -261,21 +261,34 @@ Description
> ===
> 
> This module embeds Lua, into Apache Traffic Server. This module acts as remap 
> plugin of Traffic Server. In this case we
> -should provide **'do_remap'** function in each lua script. We can write this 
> in remap.config:::
> +should provide **'do_remap'** function in each lua script. We can write this 
> in remap.config::
> 
> - map http://a.tbcdn.cn/ http://inner.tbcdn.cn/ 
> @plugin=/usr/lib64/trafficserver/plugins/tslua.so
> -@pparam=/etc/trafficserver/script/test_hdr.lua
> + map http://a.tbcdn.cn/ http://inner.tbcdn.cn/ 
> @plugin=/usr/lib64/trafficserver/plugins/tslua.so \
> +   @pparam=/etc/trafficserver/script/test_hdr.lua
> 
> Sometimes we want to receive parameters and process them in the script, we 
> should realize **'\__init__'** function in
> -the lua script(sethost.lua is a reference), and we can write this in 
> remap.config:::
> +the lua script(sethost.lua is a reference), and we can write this in 
> remap.config::
> 
> - map http://a.tbcdn.cn/ http://inner.tbcdn.cn/ 
> @plugin=/usr/lib64/trafficserver/plugins/tslua.so
> -@pparam=/etc/trafficserver/script/sethost.lua @pparam=img03.tbcdn.cn
> + map http://a.tbcdn.cn/ http://inner.tbcdn.cn/ 
> @plugin=/usr/lib64/trafficserver/plugins/tslua.so \
> +   @pparam=/etc/trafficserver/script/sethost.lua @pparam=img03.tbcdn.cn
> 
> -This module can also act as a global plugin of Traffic Server. In this case 
> we should provide one of these functions
> -(**'do_global_read_request'**, **'do_global_send_request'**, 
> **'do_global_read_response'**,
> -**'do_global_send_response'**, **'do_global_cache_lookup_complete'**) in 
> each lua script. We can write this in
> -plugin.config:::
> +This module can also act as a global plugin of Traffic Server. In this case 
> we should provide one of these functions in
> +each lua script:
> +
> +- **'do_global_txn_start'**
> +- **'do_global_txn_close'**
> +- **'do_global_os_dns'**
> +- **'do_global_pre_remap'**
> +- **'do_global_post_remap'**
> +- **'do_global_read_request'**
> +- **'do_global_send_request'**
> +- **'do_global_read_response'**
> +- **'do_global_send_response'**
> +- **'do_global_cache_lookup_complete'**
> +- **'do_global_read_cache'**
> +- **'do_global_select_alt'**
> +
> +We can write this in plugin.config::
> 
>  tslua.so /etc/trafficserver/script/test_global_hdr.lua
> 
> @@ -350,12 +363,19 @@ Hook point constants
> 
> **context**: do_remap/do_global_*/later
> 
> -TS_LUA_HOOK_CACHE_LOOKUP_COMPLETE
> -TS_LUA_HOOK_SEND_REQUEST_HDR
> -TS_LUA_HOOK_READ_RESPONSE_HDR
> -TS_LUA_HOOK_SEND_RESPONSE_HDR
> -TS_LUA_REQUEST_TRANSFORM
> -TS_LUA_RESPONSE_TRANSFORM
> +- TS_LUA_HOOK_OS_DNS
> +- TS_LUA_HOOK_PRE_REMAP
> +- TS_LUA_HOOK_POST_REMAP
> +- TS_LUA_HOOK_READ_CACHE_HDR
> +- TS_LUA_HOOK_SELECT_ALT
> +- TS_LUA_HOOK_TXN_CLOSE
> +- TS_LUA_HOOK_C

Re: git commit: TS-2821 Add to default records.config and tweak the name to be more consistent.

[James Peach]

On May 19, 2014, at 5:26 PM, a...@apache.org wrote:

> Repository: trafficserver
> Updated Branches:
>  refs/heads/master c25fb7541 -> ce8304309
> 
> 
> TS-2821 Add to default records.config and tweak the name to be more 
> consistent.
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/ce830430
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/ce830430
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/ce830430
> 
> Branch: refs/heads/master
> Commit: ce8304309e2c17b4b3efbe93f990f7106a28a7e1
> Parents: c25fb75
> Author: Alan M. Carroll 
> Authored: Mon May 19 17:25:29 2014 -0700
> Committer: Alan M. Carroll 
> Committed: Mon May 19 17:25:29 2014 -0700
> 
> --
> doc/reference/configuration/records.config.en.rst | 4 ++--
> mgmt/RecordsConfig.cc | 2 +-
> proxy/config/records.config.default.in| 6 ++
> proxy/spdy/SpdyCommon.cc  | 2 +-
> 4 files changed, 10 insertions(+), 4 deletions(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce830430/doc/reference/configuration/records.config.en.rst
> --
> diff --git a/doc/reference/configuration/records.config.en.rst 
> b/doc/reference/configuration/records.config.en.rst
> index 4ca5613..cef38fe 100644
> --- a/doc/reference/configuration/records.config.en.rst
> +++ b/doc/reference/configuration/records.config.en.rst
> @@ -2227,10 +2227,10 @@ ICP Configuration
> SPDY Configuration
> ==
> 
> -.. ts:cv:: CONFIG proxy.config.spdy.max_concurrent_streams INT 1000
> +.. ts:cv:: CONFIG proxy.config.spdy.client.max_concurrent_streams INT 1000
>:reloadable:

This should be proxy.config.spdy.SERVER.max_concurrent_streams. The terms 
"client" and "server" are well established. "proxy.config.ssl.client" refers to 
ATS making outbound SSL connections, and "proxy.config.ssl.server" and 
"proxy.config.http.server" refer to ATS accepting sessions from clients. This 
setting is all about ATS acting as a SPDY server.

The default value of 1000 is *huge*. 10 would be better IMHO.

We should not be adding more default entries to "records.config"; the goal is 
to reduce the number of entries that are duplicated in that file.

Finally, I think that it is worth documenting that which this is reloadable, 
only new SPDY sessions will get the new value. Since SPDY sessions are 
long-lasting, this may surprise people.

> 
> -   Set the maximum number of concurrent streams per client SPDY connection.
> +   Set the maximum number of concurrent streams per client connection.
> 
> Scheduled Update Configuration
> ==
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce830430/mgmt/RecordsConfig.cc
> --
> diff --git a/mgmt/RecordsConfig.cc b/mgmt/RecordsConfig.cc
> index 0551ec4..f46a5b8 100644
> --- a/mgmt/RecordsConfig.cc
> +++ b/mgmt/RecordsConfig.cc
> @@ -1934,7 +1934,7 @@ RecordElement RecordsConfig[] = {
>   //# SPDY global configuration.
>   //#
>   //
> -  {RECT_CONFIG, "proxy.config.spdy.max_concurrent_streams", RECD_INT, 
> "1000", RECU_DYNAMIC, RR_NULL, RECC_INT, NULL, RECA_NULL},
> +  {RECT_CONFIG, "proxy.config.spdy.client.max_concurrent_streams", RECD_INT, 
> "1000", RECU_DYNAMIC, RR_NULL, RECC_INT, NULL, RECA_NULL},
> 
>   //# Add LOCAL Records Here
>   {RECT_LOCAL, "proxy.local.incoming_ip_to_bind", RECD_STRING, NULL, 
> RECU_NULL, RR_NULL, RECC_NULL, NULL, RECA_NULL}
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce830430/proxy/config/records.config.default.in
> --
> diff --git a/proxy/config/records.config.default.in 
> b/proxy/config/records.config.default.in
> index 22a2587..3445a24 100644
> --- a/proxy/config/records.config.default.in
> +++ b/proxy/config/records.config.default.in
> @@ -541,6 +541,12 @@ CONFIG proxy.config.ssl.client.CA.cert.filename STRING 
> NULL
> CONFIG proxy.config.ssl.client.CA.cert.path STRING @rel_sysconfdir@
> ##
> #
> +# SPDY Configuration.
> +#
> +##
> +CONFIG proxy.config.spdy.client.max_concurrent_streams INT 1000
> +##
> +#
> # ICP Configuration.
> #
> ##
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce830430/proxy/spdy/SpdyCommon.cc
> --

Re: git commit: TS-2736: Add config option to set the max open files limit for the traffic_server process to some percentage of the fs.file-max proc value on Linux. The default is 90%.

[James Peach]

On Apr 24, 2014, at 12:50 PM, James Peach  wrote:

> On Apr 24, 2014, at 10:45 AM, sor...@apache.org wrote:
> 
>> Repository: trafficserver
>> Updated Branches:
>> refs/heads/master eb183eef9 -> 22ed6d8ed
>> 
>> 
>> TS-2736: Add config option to set the max open files limit for the 
>> traffic_server
>> process to some percentage of the fs.file-max proc value on Linux. The 
>> default is 90%.
>> 
>> 
>> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
>> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/22ed6d8e
>> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/22ed6d8e
>> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/22ed6d8e
>> 
>> Branch: refs/heads/master
>> Commit: 22ed6d8ed1f63b87e3c9dc419f8ce58555aa6001
>> Parents: eb183ee
>> Author: Phil Sorber 
>> Authored: Thu Apr 24 11:43:25 2014 -0600
>> Committer: Phil Sorber 
>> Committed: Thu Apr 24 11:43:45 2014 -0600
>> 
>> --
>> CHANGES |  3 +++
>> .../configuration/records.config.en.rst |  4 
>> lib/perl/lib/Apache/TS/AdminClient.pm   |  1 +
>> mgmt/Main.cc| 24 
>> mgmt/RecordsConfig.cc   |  3 +++
>> proxy/Main.cc   | 19 +++-
> 
> Are you planning to move the duplicate code into a single place in libts? 
> That would be pretty helpful.

One more thing ... this should not use syslog directly. Use the diagnostics API.

J

Re: [1/3] git commit: TS-2766: HdrHeap::coalesce_str_heaps doesn't properly calculate new heap size

[James Peach]

Brian,

Can you please add a test for this? That would help other devs understand how 
this is supposed to work and prevent regressions in the future

thanks!

On May 1, 2014, at 1:51 PM, bri...@apache.org wrote:

> Repository: trafficserver
> Updated Branches:
>  refs/heads/master 22bb4923e -> eaf5795e1
> 
> 
> TS-2766: HdrHeap::coalesce_str_heaps doesn't properly calculate new heap size
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/1630af7f
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/1630af7f
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/1630af7f
> 
> Branch: refs/heads/master
> Commit: 1630af7f76643c86751398838fbdf8a1363ce292
> Parents: 0df8e86
> Author: Brian Geffon 
> Authored: Thu May 1 13:50:55 2014 -0700
> Committer: Brian Geffon 
> Committed: Thu May 1 13:50:55 2014 -0700
> 
> --
> proxy/hdrs/HTTP.cc| 11 +++
> proxy/hdrs/HTTP.h |  1 +
> proxy/hdrs/HdrHeap.cc | 48 +-
> proxy/hdrs/HdrHeap.h  |  6 +++---
> proxy/hdrs/MIME.cc| 21 
> proxy/hdrs/MIME.h |  2 ++
> proxy/hdrs/URL.cc | 15 +++
> proxy/hdrs/URL.h  |  1 +
> 8 files changed, 93 insertions(+), 12 deletions(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/1630af7f/proxy/hdrs/HTTP.cc
> --
> diff --git a/proxy/hdrs/HTTP.cc b/proxy/hdrs/HTTP.cc
> index 4ce25c8..44cf881 100644
> --- a/proxy/hdrs/HTTP.cc
> +++ b/proxy/hdrs/HTTP.cc
> @@ -1687,6 +1687,17 @@ HTTPHdrImpl::move_strings(HdrStrHeap *new_heap)
>   }
> }
> 
> +size_t
> +HTTPHdrImpl::strings_length() {
> +  size_t ret = 0;
> +  if (m_polarity == HTTP_TYPE_REQUEST) {
> +   ret += u.req.m_len_method;
> +  } else if (m_polarity == HTTP_TYPE_RESPONSE) {
> +   ret += u.resp.m_len_reason;
> +  }
> +  return ret;
> +}
> +
> void
> HTTPHdrImpl::check_strings(HeapCheck *heaps, int num_heaps)
> {
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/1630af7f/proxy/hdrs/HTTP.h
> --
> diff --git a/proxy/hdrs/HTTP.h b/proxy/hdrs/HTTP.h
> index e6eabac..cddd2fe 100644
> --- a/proxy/hdrs/HTTP.h
> +++ b/proxy/hdrs/HTTP.h
> @@ -267,6 +267,7 @@ struct HTTPHdrImpl:public HdrHeapObjImpl
>   int marshal(MarshalXlate *ptr_xlate, int num_ptr, MarshalXlate *str_xlate, 
> int num_str);
>   void unmarshal(intptr_t offset);
>   void move_strings(HdrStrHeap *new_heap);
> +  size_t strings_length();

size_t strings_length() const;

> 
>   // Sanity Check Functions
>   void check_strings(HeapCheck *heaps, int num_heaps);
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/1630af7f/proxy/hdrs/HdrHeap.cc
> --
> diff --git a/proxy/hdrs/HdrHeap.cc b/proxy/hdrs/HdrHeap.cc
> index a70c0ab..296f766 100644
> --- a/proxy/hdrs/HdrHeap.cc
> +++ b/proxy/hdrs/HdrHeap.cc
> @@ -369,15 +369,7 @@ HdrHeap::coalesce_str_heaps(int incoming_size)
>   ink_assert(incoming_size >= 0);
>   ink_assert(m_writeable);
> 
> -  if (m_read_write_heap) {
> -new_heap_size += m_read_write_heap->m_heap_size;
> -  }
> -
> -  for (int i = 0; i < HDR_BUF_RONLY_HEAPS; i++) {
> -if (m_ronly_heap[i].m_heap_start != NULL) {
> -  new_heap_size += m_ronly_heap[i].m_heap_len;
> -}
> -  }
> +  new_heap_size += required_space_for_evacuation();
> 
>   HdrStrHeap *new_heap = new_HdrStrHeap(new_heap_size);
>   evacuate_from_str_heaps(new_heap);
> @@ -448,6 +440,44 @@ HdrHeap::evacuate_from_str_heaps(HdrStrHeap * new_heap)
>   }
> }
> 
> +size_t
> +HdrHeap::required_space_for_evacuation()

size_t HdrHeap::required_space_for_evacuation() const

> +{
> +  size_t ret = 0;
> +  HdrHeap *h = this;
> +  while (h) {
> +char *data = h->m_data_start;
> +
> +while (data < h->m_free_start) {
> +  HdrHeapObjImpl *obj = (HdrHeapObjImpl *) data;
> +
> +  switch (obj->m_type) {
> +  case HDR_HEAP_OBJ_URL:
> +ret += ((URLImpl *) obj)->strings_length();
> +break;
> +  case HDR_HEAP_OBJ_HTTP_HEADER:
> +ret += ((HTTPHdrImpl *) obj)->strings_length();
> +break;
> +  case HDR_HEAP_OBJ_MIME_HEADER:
> +ret += ((MIMEHdrImpl *) obj)->strings_length();
> +break;
> +  case HDR_HEAP_OBJ_FIELD_BLOCK:
> +ret += ((MIMEFieldBlockImpl *) obj)->strings_length();
> +break;
> +  case HDR_HEAP_OBJ_EMPTY:
> +  case HDR_HEAP_OBJ_RAW:
> +// Nothing to do
> +break;
> +  default:
> +ink_release_assert(0);
> +  }
> +  data = data + obj->m_length;
> +}
> +h = h->m_next;
> +  }
> +  return ret;
> +}
> +
> void
> HdrHeap::sanity_check_st

Re: git commit: TS-2555: updates to share lua context across hook invocations and only add global hooks if corresponding lua functions exist

[James Peach]

On May 4, 2014, at 5:58 PM, kic...@apache.org wrote:

> Repository: trafficserver
> Updated Branches:
>  refs/heads/master ea7849384 -> 5b1535e39
> 
> 
> TS-2555: updates to share lua context across hook invocations and only add 
> global hooks if corresponding lua functions exist
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/5b1535e3
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/5b1535e3
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/5b1535e3
> 
> Branch: refs/heads/master
> Commit: 5b1535e399aa450eeb1c75e43c2398042036c363
> Parents: ea78493
> Author: Kit Chan 
> Authored: Sun May 4 17:58:47 2014 -0700
> Committer: Kit Chan 
> Committed: Sun May 4 17:58:47 2014 -0700
> 
> --
> plugins/experimental/ts_lua/ts_lua.c | 111 +++---
> 1 file changed, 86 insertions(+), 25 deletions(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/5b1535e3/plugins/experimental/ts_lua/ts_lua.c
> --
> diff --git a/plugins/experimental/ts_lua/ts_lua.c 
> b/plugins/experimental/ts_lua/ts_lua.c
> index 32cb58c..6c72a98 100644
> --- a/plugins/experimental/ts_lua/ts_lua.c
> +++ b/plugins/experimental/ts_lua/ts_lua.c
> @@ -31,6 +31,8 @@ static volatile int32_t ts_lua_g_http_next_id = 0;
> ts_lua_main_ctx *ts_lua_main_ctx_array;
> ts_lua_main_ctx *ts_lua_g_main_ctx_array;

These can be static.

> 
> +TSCont global_contp;

This can be static.

> +
> TSReturnCode
> TSRemapInit(TSRemapInterface *api_info, char * errbuf ATS_UNUSED , int 
> errbuf_size ATS_UNUSED )
> {
> @@ -147,14 +149,11 @@ TSRemapDoRemap(void* ih, TSHttpTxn rh, 
> TSRemapRequestInfo *rri)
> return ret;
> }
> 
> -static int
> -globalHookHandler(TSCont contp, TSEvent event, void *edata) {
> +static int 
> +transactionStartHookHandler(TSCont contp, TSEvent event, void *edata) {
>   TSHttpTxn txnp = (TSHttpTxn) edata;
> 
> -  int ret = 0;
>   int64_t req_id;
> -
> -  lua_State *l;
>   TSCont txn_contp;
> 
>   ts_lua_main_ctx *main_ctx;
> @@ -165,12 +164,36 @@ globalHookHandler(TSCont contp, TSEvent event, void 
> *edata) {
>   req_id = (int64_t) ts_lua_atomic_increment((&ts_lua_g_http_next_id), 1);
>   main_ctx = &ts_lua_g_main_ctx_array[req_id%TS_LUA_MAX_STATE_COUNT];
> 
> +  TSDebug(TS_LUA_DEBUG_TAG, "[%s] req_id: %" PRId64, __FUNCTION__, req_id);  
>   TSMutexLock(main_ctx->mutexp);
> 
>   http_ctx = ts_lua_create_http_ctx(main_ctx, conf);
>   http_ctx->txnp = txnp;
>   http_ctx->remap = 0;

OK, so here you are hooking TS_HTTP_TXN_START_HOOK on every request as a 
side-effect of simply loading the plugin.

> +  txn_contp = TSContCreate(ts_lua_http_cont_handler, NULL);
> +  TSContDataSet(txn_contp, http_ctx);
> +  http_ctx->main_contp = txn_contp;
> +  TSHttpTxnHookAdd(txnp, TS_HTTP_TXN_CLOSE_HOOK, txn_contp);

And then you set up a TS_HTTP_TXN_CLOSE_HOOK so that you can nuke the Lua stat 
you just allocated.

> +
> +  TSContDataSet(global_contp, http_ctx);

How is it OK to reset the continuation data on a global continuation on each 
request? Other requests that are in flight are going to get their lua_State 
switched out from under them. What prevent multiple threads knocking on the 
same lua_State?

> +  TSMutexUnlock(main_ctx->mutexp);
> +
> +  TSHttpTxnReenable(txnp, TS_EVENT_HTTP_CONTINUE);
> +  return 0;
> +}
> +
> +static int
> +globalHookHandler(TSCont contp, TSEvent event, void *edata) {

The guts of this could be largely implemented by ts_lua_http_cont_handler() 
AFAICT.

> +  TSHttpTxn txnp = (TSHttpTxn) edata;
> +
> +  int ret = 0;
> +
> +  lua_State *l;
> +
> +  ts_lua_http_ctx *http_ctx = (ts_lua_http_ctx *) TSContDataGet(contp);
> +
>   TSMBuffer bufp;
>   TSMLoc hdr_loc;
>   TSMLoc url_loc;
> @@ -184,11 +207,10 @@ globalHookHandler(TSCont contp, TSEvent event, void 
> *edata) {
>   }
> 
>   if(!http_ctx->client_request_hdrp) {
> -TSMutexUnlock(main_ctx->mutexp);
> TSHttpTxnReenable(txnp,TS_EVENT_HTTP_CONTINUE);
> return 0;
>   }
> - 
> +
>   l = http_ctx->lua;
> 
>   switch (event) {
> @@ -213,22 +235,16 @@ globalHookHandler(TSCont contp, TSEvent event, void 
> *edata) {
> break;
> 
>   default:
> -TSMutexUnlock(main_ctx->mutexp);
> TSHttpTxnReenable(txnp, TS_EVENT_HTTP_CONTINUE);
> return 0;
> break;
>   }
> 
>   if (lua_type(l, -1) != LUA_TFUNCTION) {
> -  TSMutexUnlock(main_ctx->mutexp);
>   TSHttpTxnReenable(txnp, TS_EVENT_HTTP_CONTINUE);

Missing lua_pop.

>   return 0;
>   }
> 
> -  txn_contp = TSContCreate(ts_lua_http_cont_handler, NULL);
> -  TSContDataSet(txn_contp, http_ctx);
> -  http_ctx->main_contp = txn_contp;
> -
>   if (lua_pcall(l, 0, 1, 0) != 0) {
>   fprintf(stderr, "lua_pcall failed

Re: git commit: TS-2736: Add config option to set the max open files limit for the traffic_server process to some percentage of the fs.file-max proc value on Linux. The default is 90%.

[James Peach]

On Apr 24, 2014, at 10:45 AM, sor...@apache.org wrote:

> Repository: trafficserver
> Updated Branches:
>  refs/heads/master eb183eef9 -> 22ed6d8ed
> 
> 
> TS-2736: Add config option to set the max open files limit for the 
> traffic_server
> process to some percentage of the fs.file-max proc value on Linux. The 
> default is 90%.
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/22ed6d8e
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/22ed6d8e
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/22ed6d8e
> 
> Branch: refs/heads/master
> Commit: 22ed6d8ed1f63b87e3c9dc419f8ce58555aa6001
> Parents: eb183ee
> Author: Phil Sorber 
> Authored: Thu Apr 24 11:43:25 2014 -0600
> Committer: Phil Sorber 
> Committed: Thu Apr 24 11:43:45 2014 -0600
> 
> --
> CHANGES |  3 +++
> .../configuration/records.config.en.rst |  4 
> lib/perl/lib/Apache/TS/AdminClient.pm   |  1 +
> mgmt/Main.cc| 24 
> mgmt/RecordsConfig.cc   |  3 +++
> proxy/Main.cc   | 19 +++-

Are you planning to move the duplicate code into a single place in libts? That 
would be pretty helpful.

> proxy/config/records.config.default.in  |  1 +
> 7 files changed, 54 insertions(+), 1 deletion(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/22ed6d8e/CHANGES
> --
> diff --git a/CHANGES b/CHANGES
> index 9f55f55..31d869e 100644
> --- a/CHANGES
> +++ b/CHANGES
> @@ -1,6 +1,9 @@
>  -*- coding: utf-8 -*-
> Changes with Apache Traffic Server 5.0.0
> 
> +  *) [TS-2736] Add config option to set the max open files limit for the 
> traffic_server
> +process to some percentage of the fs.file-max proc value on Linux. The 
> default is 90%.
> +
>   *) [TS-2741] Add a server intercept example plugins and documentation.
> 
>   *) [TS-2616] Sanitize duplicate Transfer-Encoding: chunked headers.
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/22ed6d8e/doc/reference/configuration/records.config.en.rst
> --
> diff --git a/doc/reference/configuration/records.config.en.rst 
> b/doc/reference/configuration/records.config.en.rst
> index 556073c..95f547b 100644
> --- a/doc/reference/configuration/records.config.en.rst
> +++ b/doc/reference/configuration/records.config.en.rst
> @@ -257,6 +257,10 @@ Value Effect
> 
>This option only has an affect when Traffic Server has been compiled with 
> ``--enable-hwloc``.
> 
> +.. ts:cv:: CONFIG proxy.config.system.file_max_pct FLOAT 0.9
> +
> +   Set the maximum number of file handles for the traffic_server process as 
> a percentage of the the fs.file-max proc value in Linux. The default is 90%.

Documentation! Nice!

> +
> Network
> ===
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/22ed6d8e/lib/perl/lib/Apache/TS/AdminClient.pm
> --
> diff --git a/lib/perl/lib/Apache/TS/AdminClient.pm 
> b/lib/perl/lib/Apache/TS/AdminClient.pm
> index 61a0f6a..c7a6d1f 100644
> --- a/lib/perl/lib/Apache/TS/AdminClient.pm
> +++ b/lib/perl/lib/Apache/TS/AdminClient.pm
> @@ -718,6 +718,7 @@ The Apache Traffic Server Administration Manual will 
> explain what these strings
>  proxy.config.stats.snap_frequency
>  proxy.config.syslog_facility
>  proxy.config.system.mmap_max
> + proxy.config.system.file_max_pct
>  proxy.config.thread.default.stacksize
>  proxy.config.udp.free_cancelled_pkts_sec
>  proxy.config.udp.periodic_cleanup
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/22ed6d8e/mgmt/Main.cc
> --
> diff --git a/mgmt/Main.cc b/mgmt/Main.cc
> index ac23d21..8e7cfad 100644
> --- a/mgmt/Main.cc
> +++ b/mgmt/Main.cc
> @@ -311,6 +311,30 @@ set_process_limits(int fds_throttle)
>   ink_max_out_rlimit(RLIMIT_RSS, true, true);
> #endif
> 
> +#if defined(linux)

This doesn't need to be #ifdef(linux); just let it fail on systems that don't 
have the file you need.

I was going to suggest you use sysctl(3), but that doesn't really look more 
portable. Can you wrap the non-portable stuff in a function in 
lib/ts/ink_sys_control.cc, then call it from where you need it?

> +  float file_max_pct = 0.9;
> +  FILE *fd;
> +
> +  if ((fd = fopen("/proc/sys/fs/file-max","r"))) {
> +fscanf(fd, "%lu", &lim.rlim_max);
> +fclose(fd);
> +REC_ReadConfigFloat(file_max_pct, "proxy.config.system.file_max_pct");
> +lim.rl

Re: git commit: TS-2735: Align all memory during freelist allocations

[James Peach]

On Apr 21, 2014, at 12:58 PM, sor...@apache.org wrote:

> Repository: trafficserver
> Updated Branches:
> refs/heads/master 4f2dc70f0 -> 32f71df3a
> 
> 
> TS-2735: Align all memory during freelist allocations
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/32f71df3
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/32f71df3
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/32f71df3
> 
> Branch: refs/heads/master
> Commit: 32f71df3a021dbeaac3aff7bf0b23c9ec5fd64f4
> Parents: 4f2dc70
> Author: Phil Sorber 
> Authored: Mon Apr 21 13:57:22 2014 -0600
> Committer: Phil Sorber 
> Committed: Mon Apr 21 13:57:22 2014 -0600
> 
> --
> CHANGES | 2 ++
> lib/ts/ink_queue.cc | 3 ++-
> 2 files changed, 4 insertions(+), 1 deletion(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/32f71df3/CHANGES
> --
> diff --git a/CHANGES b/CHANGES
> index d8ef4a0..c5e21f9 100644
> --- a/CHANGES
> +++ b/CHANGES
> @@ -1,6 +1,8 @@
> -*- coding: utf-8 -*-
> Changes with Apache Traffic Server 5.0.0
> 
> +  *) [TS-2735] Align all memory during freelist allocations.
> +
>  *) [TS-2120] remove stale_while_revalidate plugin.
> 
>  *) [TS-2732] Add url_sign (Signed URL's) plugin.
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/32f71df3/lib/ts/ink_queue.cc
> --
> diff --git a/lib/ts/ink_queue.cc b/lib/ts/ink_queue.cc
> index c94e32d..7f5c172 100644
> --- a/lib/ts/ink_queue.cc
> +++ b/lib/ts/ink_queue.cc
> @@ -103,7 +103,8 @@ ink_freelist_init(InkFreeList **fl, const char *name, 
> uint32_t type_size,
>  ink_assert(!(alignment & (alignment - 1)));
>  f->alignment = alignment;
>  f->chunk_size = chunk_size;
> -  f->type_size = type_size;
> +  // Make sure we align *all* the objects in the allocation, not just the 
> first one
> +  f->type_size = (type_size + (alignment - 1)) & ~(alignment - 1);

INK_ALIGN?

>  SET_FREELIST_POINTER_VERSION(f->head, FROM_PTR(0), 0);
> 
>  f->used = 0;
> 

Re: git commit: TS-2732: Add url_sig plugin

[James Peach]

On Apr 18, 2014, at 12:56 PM, sor...@apache.org wrote:

> Repository: trafficserver
> Updated Branches:
>  refs/heads/master bcfd36abf -> 88f18501d
> 
> 
> TS-2732: Add url_sig plugin
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/88f18501
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/88f18501
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/88f18501
> 
> Branch: refs/heads/master
> Commit: 88f18501d110bdff6a7347f307d94eb3629dad79
> Parents: bcfd36a
> Author: Phil Sorber 
> Authored: Fri Apr 18 13:49:12 2014 -0600
> Committer: Phil Sorber 
> Committed: Fri Apr 18 13:55:42 2014 -0600
> 
> --
> CHANGES|   2 +
> NOTICE |   4 +-
> configure.ac   |   1 +
> plugins/experimental/Makefile.am   |   1 +
> plugins/experimental/url_sig/Makefile.am   |  21 ++
> plugins/experimental/url_sig/Makefile.tsxs |  26 ++
> plugins/experimental/url_sig/README| 183 ++
> plugins/experimental/url_sig/genkeys.pl|  29 ++
> plugins/experimental/url_sig/sign.pl   | 101 ++
> plugins/experimental/url_sig/url_sig.c | 456 
> plugins/experimental/url_sig/url_sig.h |  52 +++
> 11 files changed, 874 insertions(+), 2 deletions(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/88f18501/CHANGES
> --
> diff --git a/CHANGES b/CHANGES
> index 49c2227..d30d399 100644
> --- a/CHANGES
> +++ b/CHANGES
> @@ -1,6 +1,8 @@
>  -*- coding: utf-8 -*-
> Changes with Apache Traffic Server 5.0.0
> 
> +  *) [TS-2732] Add url_sign (Signed URL's) plugin.
> +
>   *) [TS-2650] Redirect handling enhancements in ATS
> 
>   *) [TS-1125] POST's with Expect: 100-continue are slowed by delayed 100 
> response
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/88f18501/NOTICE
> --
> diff --git a/NOTICE b/NOTICE
> index 00196cb..dacd4a7 100644
> --- a/NOTICE
> +++ b/NOTICE
> @@ -54,8 +54,8 @@ Copyright (c) 2013 LinkedIn
> 
> ~~~
> 
> -remap_stats plugin developed by Comcast.
> -Copyright (C) 2013 Comcast
> +remap_stats and url_sig plugins developed by Comcast.
> +Copyright (C) 2014 Comcast
> 
> ~~~
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/88f18501/configure.ac
> --
> diff --git a/configure.ac b/configure.ac
> index bd3ddcf..7467a2d 100644
> --- a/configure.ac
> +++ b/configure.ac
> @@ -1946,6 +1946,7 @@ AC_CONFIG_FILES([
>   plugins/experimental/s3_auth/Makefile
>   plugins/experimental/spdy/Makefile
>   plugins/experimental/ts_lua/Makefile
> +  plugins/experimental/url_sig/Makefile
>   plugins/experimental/xdebug/Makefile
>   proxy/Makefile
>   proxy/api/ts/Makefile
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/88f18501/plugins/experimental/Makefile.am
> --
> diff --git a/plugins/experimental/Makefile.am 
> b/plugins/experimental/Makefile.am
> index 4b811c1..b8bddce 100644
> --- a/plugins/experimental/Makefile.am
> +++ b/plugins/experimental/Makefile.am
> @@ -34,6 +34,7 @@ SUBDIRS = \
>  s3_auth \
>  spdy \
>  ts_lua \
> + url_sig \
>  xdebug
> 
> endif
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/88f18501/plugins/experimental/url_sig/Makefile.am
> --
> diff --git a/plugins/experimental/url_sig/Makefile.am 
> b/plugins/experimental/url_sig/Makefile.am
> new file mode 100644
> index 000..a9011ff
> --- /dev/null
> +++ b/plugins/experimental/url_sig/Makefile.am
> @@ -0,0 +1,21 @@
> +#  Licensed to the Apache Software Foundation (ASF) under one
> +#  or more contributor license agreements.  See the NOTICE file
> +#  distributed with this work for additional information
> +#  regarding copyright ownership.  The ASF licenses this file
> +#  to you under the Apache License, Version 2.0 (the
> +#  "License"); you may not use this file except in compliance
> +#  with the License.  You may obtain a copy of the License at
> +#
> +#  http://www.apache.org/licenses/LICENSE-2.0
> +#
> +#  Unless required by applicable law or agreed to in writing, software
> +#  distributed under the License is distributed on an "AS IS" BASIS,
> +#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> +#  See the License for the specific language governing permissions and
> +#  limitations under the License.
> +
> +include $(top_srcdir)/build/plugins.mk
> +
> +pk

Re: git commit: TS-1125: POST's with Expect: 100-continue are slowed by delayed 100 response Additional Authors: Feifei Cai Sudheer Vinukonda

[James Peach]

This crashes traffic_server and traffic_manager on startup; please revert.

In addition, the actual implementation seems very ugly ... were you planning to 
revisit that? Since we have a good place for docs, now, we should document 
proxy.config.http.send_100_continue_response.

On Apr 17, 2014, at 4:14 PM, bc...@apache.org wrote:

> Repository: trafficserver
> Updated Branches:
> refs/heads/master 700875f18 -> 9da123014
> 
> 
> TS-1125: POST's with Expect: 100-continue are slowed by delayed 100 response
> Additional Authors:
> Feifei Cai 
> Sudheer Vinukonda 
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
> Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/9da12301
> Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/9da12301
> Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/9da12301
> 
> Branch: refs/heads/master
> Commit: 9da123014382c00bccb1869782a4f2502229a459
> Parents: 700875f
> Author: Bryan Call 
> Authored: Thu Apr 17 16:10:40 2014 -0700
> Committer: Bryan Call 
> Committed: Thu Apr 17 16:10:40 2014 -0700
> 
> --
> iocore/net/P_SSLNetVConnection.h  |  1 +
> iocore/net/SSLNetVConnection.cc   |  2 +-
> mgmt/RecordsConfig.cc |  2 ++
> proxy/hdrs/HTTP.cc|  4 
> proxy/hdrs/HTTP.h |  2 ++
> proxy/http/HttpConfig.cc  |  3 +++
> proxy/http/HttpConfig.h   |  3 +++
> proxy/http/HttpSM.cc  | 24 
> proxy/http/HttpTransact.cc|  5 +
> proxy/http/HttpTransactHeaders.cc | 12 
> proxy/http/HttpTransactHeaders.h  |  1 +
> 11 files changed, 58 insertions(+), 1 deletion(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/9da12301/iocore/net/P_SSLNetVConnection.h
> --
> diff --git a/iocore/net/P_SSLNetVConnection.h 
> b/iocore/net/P_SSLNetVConnection.h
> index 47861e4..577995b 100644
> --- a/iocore/net/P_SSLNetVConnection.h
> +++ b/iocore/net/P_SSLNetVConnection.h
> @@ -133,5 +133,6 @@ private:
> typedef int (SSLNetVConnection::*SSLNetVConnHandler) (int, void *);
> 
> extern ClassAllocator sslNetVCAllocator;
> +extern int do_SSL_write(SSL *ssl, void *buf, int size);
> 
> #endif /* _SSLNetVConnection_h_ */
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/9da12301/iocore/net/SSLNetVConnection.cc
> --
> diff --git a/iocore/net/SSLNetVConnection.cc b/iocore/net/SSLNetVConnection.cc
> index 02f3f4a..c62b1c1 100644
> --- a/iocore/net/SSLNetVConnection.cc
> +++ b/iocore/net/SSLNetVConnection.cc
> @@ -80,7 +80,7 @@ debug_certificate_name(const char * msg, X509_NAME * name)
>  BIO_free(bio);
> }
> 
> -static inline int
> +int
> do_SSL_write(SSL * ssl, void *buf, int size)
> {
>  int r = 0;
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/9da12301/mgmt/RecordsConfig.cc
> --
> diff --git a/mgmt/RecordsConfig.cc b/mgmt/RecordsConfig.cc
> index 9e45111..1516d1a 100644
> --- a/mgmt/RecordsConfig.cc
> +++ b/mgmt/RecordsConfig.cc
> @@ -426,6 +426,8 @@ RecordElement RecordsConfig[] = {
>  //   #
>  {RECT_CONFIG, "proxy.config.http.send_http11_requests", RECD_INT, "1", 
> RECU_DYNAMIC, RR_NULL, RECC_NULL, NULL, RECA_NULL}
>  ,
> +  {RECT_CONFIG, "proxy.config.http.send_100_continue_response", RECD_INT, 
> "0", RECU_DYNAMIC, RR_NULL, RECC_NULL, NULL, RECA_NULL}
> +  ,
>  {RECT_CONFIG, "proxy.config.http.share_server_sessions", RECD_INT, "2", 
> RECU_RESTART_TS, RR_NULL, RECC_NULL, NULL, RECA_NULL}
>  ,
>  {RECT_CONFIG, "proxy.config.http.server_session_sharing.match", RECD_STRING, 
> "both", RECU_RESTART_TS, RR_NULL, RECC_NULL, NULL, RECA_NULL}
> 
> http://git-wip-us.apache.org/repos/asf/trafficserver/blob/9da12301/proxy/hdrs/HTTP.cc
> --
> diff --git a/proxy/hdrs/HTTP.cc b/proxy/hdrs/HTTP.cc
> index 4ce25c8..20f702a 100644
> --- a/proxy/hdrs/HTTP.cc
> +++ b/proxy/hdrs/HTTP.cc
> @@ -109,6 +109,7 @@ const char *HTTP_VALUE_PROXY_REVALIDATE;
> const char *HTTP_VALUE_PUBLIC;
> const char *HTTP_VALUE_S_MAXAGE;
> const char *HTTP_VALUE_NEED_REVALIDATE_ONCE;
> +const char *HTTP_VALUE_100_CONTINUE;
> // Cache-control: extension "need-revalidate-once" is used internally by T.S.
> // to invalidate a document, and it is not returned/forwarded.
> // If a cached document has this extension set (ie, is invalidated),
> @@ -143,6 +144,7 @@ int HTTP_LEN_PROXY_REVALIDATE;
> int HTTP_LEN_PUBLIC;
> int HTTP_LEN_S_MAXAGE;
> int HTTP_LEN_NEED_REVALIDATE_ONCE;
> +int HTTP_LEN_100_CONTINUE;
> 
> Arena* const HTTPHdr::USE_HDR_HEAP_MAGIC = reinterpret_cast(1);
> 
> @@ -257,6 +259,7 @@ http_init()
>HTTP_VALUE_