[
https://issues.apache.org/jira/browse/TS-4331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15349883#comment-15349883
]
ASF GitHub Bot commented on TS-4331:
------------------------------------
Github user jpeach commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/653#discussion_r68495828
--- Diff: iocore/hostdb/HostDB.cc ---
@@ -2882,3 +2605,101 @@ ParseHostFile(char const *path, unsigned int
hostdb_hostfile_check_interval)
// Mark this one as completed, so we can allow another update to happen
HostDBFileUpdateActive = 0;
}
+
+//
+// Regression tests
+//
+// Take a started hostDB and fill it up and make sure it doesn't explode
+#ifdef TS_HAS_TESTS
+struct HostDBRegressionContinuation;
+typedef int (HostDBRegressionContinuation::*HostDBRegContHandler)(int,
void *);
+
+struct HostDBRegressionContinuation : public Continuation {
+ int hosts;
+ const char **hostnames;
+ RegressionTest *test;
+ int type;
+ int *status;
+
+ int success;
+ int failure;
+ int outstanding;
+ int i;
+
+ int
+ mainEvent(int event, HostDBInfo *r)
+ {
+ (void)event;
+
+ if (event == EVENT_INTERVAL) {
+ rprintf(test, "hosts=%d success=%d failure=%d outstanding=%d
i=%d\n", hosts, success, failure, outstanding, i);
+ }
+ if (event == EVENT_HOST_DB_LOOKUP) {
+ --outstanding;
+ // since this is a lookup done, data is either hostdbInfo or NULL
+ if (r) {
+ rprintf(test, "hostdbinfo r=%x\n", r);
+ rprintf(test, "hostdbinfo hostname=%s\n", r->perm_hostname());
+ rprintf(test, "hostdbinfo rr %x\n", r->rr());
+ // If RR, print all of the enclosed records
+ if (r->rr()) {
+ rprintf(test, "hostdbinfo good=%d\n", r->rr()->good);
+ for (int x = 0; x < r->rr()->good; x++) {
+ ip_port_text_buffer ip_buf;
+ ats_ip_ntop(r->rr()->info(x).ip(), ip_buf, sizeof(ip_buf));
+ rprintf(test, "hostdbinfo RR%d ip=%s\n", x, ip_buf);
+ }
+ } else { // Otherwise, just the one will do
+ ip_port_text_buffer ip_buf;
+ ats_ip_ntop(r->ip(), ip_buf, sizeof(ip_buf));
+ rprintf(test, "hostdbinfo A ip=%s\n", ip_buf);
+ }
+ ++success;
+ } else {
+ ++failure;
+ }
+ }
+
+ if (i < hosts) {
+ hostDBProcessor.getbyname_re(this, hostnames[i++], 0);
+ return EVENT_CONT;
+ } else {
+ rprintf(test, "HostDBTestRR: %d outstanding %d succcess %d
failure\n", outstanding, success, failure);
+ if (success == hosts) {
+ *status = REGRESSION_TEST_PASSED;
+ } else {
+ *status = REGRESSION_TEST_FAILED;
+ }
+ return EVENT_DONE;
+ }
+ }
+
+ HostDBRegressionContinuation(int ahosts, const char **ahostnames,
RegressionTest *t, int atype, int *astatus)
+ : Continuation(new_ProxyMutex()),
+ hosts(ahosts),
+ hostnames(ahostnames),
+ test(t),
+ type(atype),
+ status(astatus),
+ success(0),
+ failure(0),
+ i(0)
+ {
+ outstanding = ahosts;
+
SET_HANDLER((HostDBRegContHandler)&HostDBRegressionContinuation::mainEvent);
--- End diff --
Remove unneeded cast.
> Hostdb consistency problems due to MultiCache
> ---------------------------------------------
>
> Key: TS-4331
> URL: https://issues.apache.org/jira/browse/TS-4331
> Project: Traffic Server
> Issue Type: Bug
> Components: HostDB
> Reporter: Thomas Jackson
> Assignee: Thomas Jackson
> Fix For: 7.0.0
>
>
> This ticket is for the correct long term fix to TS-4207
> pulled from a comment, which wraps up the issue
> {quote}
> Leif Hedstrom I have spent a decent amount of time on this while I was OOO on
> vacation the last couple of weeks. It seems that the root cause of this issue
> has always existed, and that the addition of always doing hostname storing
> (https://github.com/apache/trafficserver/commit/0e703e1e) we are just causing
> the issue to happen all the time.
> To understand the issue I'll give a little background in how hostdb is
> currently working. Basically hostdb is just a wrapper around this templated
> struct called MultiCache. MultiCache is "multi" not because it is templated,
> but because it has two types of storage (static-- blocks and dynamic--
> alloc). The static side of the cache can hold N HostDBInfo structs (the
> results of DNS queries). The dynamic side is used to store the round robin
> records and various strings associated with the record. The size of this
> dynamic space is defined as (N x [estimated_heap_bytes_per_entry. The basic
> problem we are running into is that we are putting too much preassure on the
> dynamic heap-- such that the heap is getting re-used while people still have
> references to items in that space.
> So, I've actually been working on re-writing MultiCache to allocate the
> entire required block at once (so we don't have this problem where the parent
> exists but not the children), but I'm not certain if we want such a change to
> go into the 6.x branch (I'm willing to discuss if we want). If we aren't
> comfortable with such a large change I suggest just accounting for the
> hostname size in the estimated_heap_bytes_per_entry as a stopgap solution.
> The maximum allowable size is 253 (so 254 with null terminator), but we could
> pick a smaller number (~120 or so seems to be more reasonable). Alternatively
> you can increase the number of records in hostdb (and the size accordingly)
> to increase the dynamic heap size.
> TLDR; almost done with the long term solution, but I'm not sure if we want to
> merge that into 6.x-- alternatively we can do a simple workaround in 6.x
> (https://github.com/apache/trafficserver/pull/553)
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
