[jira] [Work logged] (TS-4915) Crash from hostdb in PriorityQueueLess

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-4915?focusedWorklogId=30603=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-30603
 ]

ASF GitHub Bot logged work on TS-4915:
--

Author: ASF GitHub Bot
Created on: 13/Oct/16 19:03
Start Date: 13/Oct/16 19:03
Worklog Time Spent: 10m 
  Work Description: Github user shinrich closed the pull request at:

https://github.com/apache/trafficserver/pull/1088


Issue Time Tracking
---

Worklog Id: (was: 30603)
Time Spent: 6h 10m  (was: 6h)

> Crash from hostdb in PriorityQueueLess
> --
>
> Key: TS-4915
> URL: https://issues.apache.org/jira/browse/TS-4915
> Project: Traffic Server
>  Issue Type: Bug
>  Components: HostDB
>Reporter: Susan Hinrichs
>Priority: Blocker
> Fix For: 7.1.0
>
> Attachments: ts-4915.diff, ts-4915.diff
>
>  Time Spent: 6h 10m
>  Remaining Estimate: 0h
>
> Saw this while testing fix for TS-4813 with debug enabled.
> {code}
> (gdb) bt full
> #0  0x00547bfe in RefCountCacheHashEntry::operator< (this=0x1cc0880, 
> v2=...) at ../iocore/hostdb/P_RefCountCache.h:94
> No locals.
> #1  0x0054988d in 
> PriorityQueueLess::operator() (this=0x2b78a9a2587b, 
> a=@0x2b78f402af68, b=@0x2b78f402aa28)
> at ../lib/ts/PriorityQueue.h:41
> No locals.
> #2  0x00549785 in PriorityQueue PriorityQueueLess >::_bubble_up (this=0x1cb2990, 
> index=2) at ../lib/ts/PriorityQueue.h:191
> comp = {}
> parent = 0
> #3  0x006ecfcc in PriorityQueue PriorityQueueLess >::push (this=0x1cb2990, 
> entry=0x2b78f402af60) at ../../lib/ts/PriorityQueue.h:91
> len = 2
> #4  0x006ec206 in RefCountCachePartition::put 
> (this=0x1cb2900, key=6912554662447498853, item=0x2b78aee04f00, size=96, 
> expire_time=1475202356) at ./P_RefCountCache.h:210
> expiry_entry = 0x2b78f402af60
> __func__ = "put"
> val = 0x1cc0880
> #5  0x006eb3de in RefCountCache::put (this=0x18051e0, 
> key=6912554662447498853, item=0x2b78aee04f00, size=16, 
> expiry_time=1475202356) at ./P_RefCountCache.h:462
> No locals.
> #6  0x006e2d8e in HostDBContinuation::dnsEvent (this=0x2b7938020f00, 
> event=600, e=0x2b78ac009440) at HostDB.cc:1422
> is_rr = false
> old_rr_data = 0x0
> first_record = 0x2b78ac0094f8
> m = 0x1
> failed = false
> old_r = {m_ptr = 0x0}
> af = 2 '\002'
> s_size = 16
> rrsize = 0
> allocSize = 16
> r = 0x2b78aee04f00
> old_info = { = { = {_vptr.ForceVFPTToTop 
> = 0x7f3630}, m_refcount = 0}, iobuffer_index = 0, 
>   key = 47797242059264, app = {allotment = {application1 = 5326300, 
> application2 = 0}, http_data = {http_version = 4, 
>   pipeline_max = 59, keepalive_timeout = 17, fail_count = 81, 
> unused1 = 0, last_failure = 0}, rr = {offset = 5326300}}, data = {
> ip = {sa = {sa_family = 54488, sa_data = 
> "^\000\000\000\000\000\020\034$\274x+\000"}, sin = {sin_family = 54488, 
> sin_port = 94, 
> sin_addr = {s_addr = 0}, sin_zero = "\020\034$\274x+\000"}, 
> sin6 = {sin6_family = 54488, sin6_port = 94, sin6_flowinfo = 0, 
> sin6_addr = {__in6_u = {__u6_addr8 = 
> "\020\034$\274x+\000\000\030\036$\274\375\b\000", __u6_addr16 = {7184, 48164, 
> 11128, 
>   0, 7704, 48164, 2301, 0}, __u6_addr32 = {3156483088, 
> 11128, 3156483608, 2301}}}, sin6_scope_id = 3156478176}}, 
> hostname_offset = 6214872, srv = {srv_offset = 54488, srv_weight 
> = 94, srv_priority = 0, srv_port = 0, key = 3156483088}}, 
>   hostname_offset = 11128, ip_timestamp = 2845989456, 
> ip_timeout_interval = 11128, is_srv = 0, reverse_dns = 0, round_robin = 1, 
>   round_robin_elt = 0}
> valid_records = 0
> tip = {_family = 2, _addr = {_ip4 = 540420056, _ip6 = {__in6_u = 
> {__u6_addr8 = "\330'6 x+\000\000\360L\020\250x+\000", 
> __u6_addr16 = {10200, 8246, 11128, 0, 19696, 43024, 11128, 
> 0}, __u6_addr32 = {540420056, 11128, 2819640560, 11128}}}, 
> _byte = "\330'6 x+\000\000\360L\020\250x+\000", _u32 = 
> {540420056, 11128, 2819640560, 11128}, _u64 = {47794936489944, 
>   47797215710448}}}
> ttl_seconds = 132
> aname = 0x2b7938021000 "fbmm1.zenfs.com"
> offset = 96
> thread = 0x2b78a8101010
> __func__ = "dnsEvent"
> #7  0x005145dc in Continuation::handleEvent (this=0x2b7938020f00, 
> event=600, data=0x2b78ac009440)
> at ../iocore/eventsystem/I_Continuation.h:153
> No locals.
> #8  

[jira] [Work logged] (TS-4915) Crash from hostdb in PriorityQueueLess

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-4915?focusedWorklogId=30580=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-30580
 ]

ASF GitHub Bot logged work on TS-4915:
--

Author: ASF GitHub Bot
Created on: 13/Oct/16 16:02
Start Date: 13/Oct/16 16:02
Worklog Time Spent: 10m 
  Work Description: Github user atsci commented on the issue:

https://github.com/apache/trafficserver/pull/1088
  
FreeBSD build *successful*! See 
https://ci.trafficserver.apache.org/job/Github-FreeBSD/996/ for details.
 



Issue Time Tracking
---

Worklog Id: (was: 30580)
Time Spent: 6h  (was: 5h 50m)

> Crash from hostdb in PriorityQueueLess
> --
>
> Key: TS-4915
> URL: https://issues.apache.org/jira/browse/TS-4915
> Project: Traffic Server
>  Issue Type: Bug
>  Components: HostDB
>Reporter: Susan Hinrichs
>Priority: Blocker
> Fix For: 7.1.0
>
> Attachments: ts-4915.diff, ts-4915.diff
>
>  Time Spent: 6h
>  Remaining Estimate: 0h
>
> Saw this while testing fix for TS-4813 with debug enabled.
> {code}
> (gdb) bt full
> #0  0x00547bfe in RefCountCacheHashEntry::operator< (this=0x1cc0880, 
> v2=...) at ../iocore/hostdb/P_RefCountCache.h:94
> No locals.
> #1  0x0054988d in 
> PriorityQueueLess::operator() (this=0x2b78a9a2587b, 
> a=@0x2b78f402af68, b=@0x2b78f402aa28)
> at ../lib/ts/PriorityQueue.h:41
> No locals.
> #2  0x00549785 in PriorityQueue PriorityQueueLess >::_bubble_up (this=0x1cb2990, 
> index=2) at ../lib/ts/PriorityQueue.h:191
> comp = {}
> parent = 0
> #3  0x006ecfcc in PriorityQueue PriorityQueueLess >::push (this=0x1cb2990, 
> entry=0x2b78f402af60) at ../../lib/ts/PriorityQueue.h:91
> len = 2
> #4  0x006ec206 in RefCountCachePartition::put 
> (this=0x1cb2900, key=6912554662447498853, item=0x2b78aee04f00, size=96, 
> expire_time=1475202356) at ./P_RefCountCache.h:210
> expiry_entry = 0x2b78f402af60
> __func__ = "put"
> val = 0x1cc0880
> #5  0x006eb3de in RefCountCache::put (this=0x18051e0, 
> key=6912554662447498853, item=0x2b78aee04f00, size=16, 
> expiry_time=1475202356) at ./P_RefCountCache.h:462
> No locals.
> #6  0x006e2d8e in HostDBContinuation::dnsEvent (this=0x2b7938020f00, 
> event=600, e=0x2b78ac009440) at HostDB.cc:1422
> is_rr = false
> old_rr_data = 0x0
> first_record = 0x2b78ac0094f8
> m = 0x1
> failed = false
> old_r = {m_ptr = 0x0}
> af = 2 '\002'
> s_size = 16
> rrsize = 0
> allocSize = 16
> r = 0x2b78aee04f00
> old_info = { = { = {_vptr.ForceVFPTToTop 
> = 0x7f3630}, m_refcount = 0}, iobuffer_index = 0, 
>   key = 47797242059264, app = {allotment = {application1 = 5326300, 
> application2 = 0}, http_data = {http_version = 4, 
>   pipeline_max = 59, keepalive_timeout = 17, fail_count = 81, 
> unused1 = 0, last_failure = 0}, rr = {offset = 5326300}}, data = {
> ip = {sa = {sa_family = 54488, sa_data = 
> "^\000\000\000\000\000\020\034$\274x+\000"}, sin = {sin_family = 54488, 
> sin_port = 94, 
> sin_addr = {s_addr = 0}, sin_zero = "\020\034$\274x+\000"}, 
> sin6 = {sin6_family = 54488, sin6_port = 94, sin6_flowinfo = 0, 
> sin6_addr = {__in6_u = {__u6_addr8 = 
> "\020\034$\274x+\000\000\030\036$\274\375\b\000", __u6_addr16 = {7184, 48164, 
> 11128, 
>   0, 7704, 48164, 2301, 0}, __u6_addr32 = {3156483088, 
> 11128, 3156483608, 2301}}}, sin6_scope_id = 3156478176}}, 
> hostname_offset = 6214872, srv = {srv_offset = 54488, srv_weight 
> = 94, srv_priority = 0, srv_port = 0, key = 3156483088}}, 
>   hostname_offset = 11128, ip_timestamp = 2845989456, 
> ip_timeout_interval = 11128, is_srv = 0, reverse_dns = 0, round_robin = 1, 
>   round_robin_elt = 0}
> valid_records = 0
> tip = {_family = 2, _addr = {_ip4 = 540420056, _ip6 = {__in6_u = 
> {__u6_addr8 = "\330'6 x+\000\000\360L\020\250x+\000", 
> __u6_addr16 = {10200, 8246, 11128, 0, 19696, 43024, 11128, 
> 0}, __u6_addr32 = {540420056, 11128, 2819640560, 11128}}}, 
> _byte = "\330'6 x+\000\000\360L\020\250x+\000", _u32 = 
> {540420056, 11128, 2819640560, 11128}, _u64 = {47794936489944, 
>   47797215710448}}}
> ttl_seconds = 132
> aname = 0x2b7938021000 "fbmm1.zenfs.com"
> offset = 96
> thread = 0x2b78a8101010
> __func__ = "dnsEvent"
> #7  0x005145dc in Continuation::handleEvent (this=0x2b7938020f00, 
> 

[jira] [Work logged] (TS-4915) Crash from hostdb in PriorityQueueLess

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-4915?focusedWorklogId=30579=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-30579
 ]

ASF GitHub Bot logged work on TS-4915:
--

Author: ASF GitHub Bot
Created on: 13/Oct/16 15:59
Start Date: 13/Oct/16 15:59
Worklog Time Spent: 10m 
  Work Description: Github user atsci commented on the issue:

https://github.com/apache/trafficserver/pull/1088
  
Linux build *successful*! See 
https://ci.trafficserver.apache.org/job/Github-Linux/888/ for details.
 



Issue Time Tracking
---

Worklog Id: (was: 30579)
Time Spent: 5h 50m  (was: 5h 40m)

> Crash from hostdb in PriorityQueueLess
> --
>
> Key: TS-4915
> URL: https://issues.apache.org/jira/browse/TS-4915
> Project: Traffic Server
>  Issue Type: Bug
>  Components: HostDB
>Reporter: Susan Hinrichs
>Priority: Blocker
> Fix For: 7.1.0
>
> Attachments: ts-4915.diff, ts-4915.diff
>
>  Time Spent: 5h 50m
>  Remaining Estimate: 0h
>
> Saw this while testing fix for TS-4813 with debug enabled.
> {code}
> (gdb) bt full
> #0  0x00547bfe in RefCountCacheHashEntry::operator< (this=0x1cc0880, 
> v2=...) at ../iocore/hostdb/P_RefCountCache.h:94
> No locals.
> #1  0x0054988d in 
> PriorityQueueLess::operator() (this=0x2b78a9a2587b, 
> a=@0x2b78f402af68, b=@0x2b78f402aa28)
> at ../lib/ts/PriorityQueue.h:41
> No locals.
> #2  0x00549785 in PriorityQueue PriorityQueueLess >::_bubble_up (this=0x1cb2990, 
> index=2) at ../lib/ts/PriorityQueue.h:191
> comp = {}
> parent = 0
> #3  0x006ecfcc in PriorityQueue PriorityQueueLess >::push (this=0x1cb2990, 
> entry=0x2b78f402af60) at ../../lib/ts/PriorityQueue.h:91
> len = 2
> #4  0x006ec206 in RefCountCachePartition::put 
> (this=0x1cb2900, key=6912554662447498853, item=0x2b78aee04f00, size=96, 
> expire_time=1475202356) at ./P_RefCountCache.h:210
> expiry_entry = 0x2b78f402af60
> __func__ = "put"
> val = 0x1cc0880
> #5  0x006eb3de in RefCountCache::put (this=0x18051e0, 
> key=6912554662447498853, item=0x2b78aee04f00, size=16, 
> expiry_time=1475202356) at ./P_RefCountCache.h:462
> No locals.
> #6  0x006e2d8e in HostDBContinuation::dnsEvent (this=0x2b7938020f00, 
> event=600, e=0x2b78ac009440) at HostDB.cc:1422
> is_rr = false
> old_rr_data = 0x0
> first_record = 0x2b78ac0094f8
> m = 0x1
> failed = false
> old_r = {m_ptr = 0x0}
> af = 2 '\002'
> s_size = 16
> rrsize = 0
> allocSize = 16
> r = 0x2b78aee04f00
> old_info = { = { = {_vptr.ForceVFPTToTop 
> = 0x7f3630}, m_refcount = 0}, iobuffer_index = 0, 
>   key = 47797242059264, app = {allotment = {application1 = 5326300, 
> application2 = 0}, http_data = {http_version = 4, 
>   pipeline_max = 59, keepalive_timeout = 17, fail_count = 81, 
> unused1 = 0, last_failure = 0}, rr = {offset = 5326300}}, data = {
> ip = {sa = {sa_family = 54488, sa_data = 
> "^\000\000\000\000\000\020\034$\274x+\000"}, sin = {sin_family = 54488, 
> sin_port = 94, 
> sin_addr = {s_addr = 0}, sin_zero = "\020\034$\274x+\000"}, 
> sin6 = {sin6_family = 54488, sin6_port = 94, sin6_flowinfo = 0, 
> sin6_addr = {__in6_u = {__u6_addr8 = 
> "\020\034$\274x+\000\000\030\036$\274\375\b\000", __u6_addr16 = {7184, 48164, 
> 11128, 
>   0, 7704, 48164, 2301, 0}, __u6_addr32 = {3156483088, 
> 11128, 3156483608, 2301}}}, sin6_scope_id = 3156478176}}, 
> hostname_offset = 6214872, srv = {srv_offset = 54488, srv_weight 
> = 94, srv_priority = 0, srv_port = 0, key = 3156483088}}, 
>   hostname_offset = 11128, ip_timestamp = 2845989456, 
> ip_timeout_interval = 11128, is_srv = 0, reverse_dns = 0, round_robin = 1, 
>   round_robin_elt = 0}
> valid_records = 0
> tip = {_family = 2, _addr = {_ip4 = 540420056, _ip6 = {__in6_u = 
> {__u6_addr8 = "\330'6 x+\000\000\360L\020\250x+\000", 
> __u6_addr16 = {10200, 8246, 11128, 0, 19696, 43024, 11128, 
> 0}, __u6_addr32 = {540420056, 11128, 2819640560, 11128}}}, 
> _byte = "\330'6 x+\000\000\360L\020\250x+\000", _u32 = 
> {540420056, 11128, 2819640560, 11128}, _u64 = {47794936489944, 
>   47797215710448}}}
> ttl_seconds = 132
> aname = 0x2b7938021000 "fbmm1.zenfs.com"
> offset = 96
> thread = 0x2b78a8101010
> __func__ = "dnsEvent"
> #7  0x005145dc in Continuation::handleEvent 

[jira] [Work logged] (TS-4915) Crash from hostdb in PriorityQueueLess

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-4915?focusedWorklogId=30574=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-30574
 ]

ASF GitHub Bot logged work on TS-4915:
--

Author: ASF GitHub Bot
Created on: 13/Oct/16 15:48
Start Date: 13/Oct/16 15:48
Worklog Time Spent: 10m 
  Work Description: Github user shinrich commented on the issue:

https://github.com/apache/trafficserver/pull/1088
  
Removed the extra !empty() check in ::erase.  The tests still pass.  I'm 
running it in production again without incident.  Will let it bake some more 
and then will merge it if all looks good.


Issue Time Tracking
---

Worklog Id: (was: 30574)
Time Spent: 5h 40m  (was: 5.5h)

> Crash from hostdb in PriorityQueueLess
> --
>
> Key: TS-4915
> URL: https://issues.apache.org/jira/browse/TS-4915
> Project: Traffic Server
>  Issue Type: Bug
>  Components: HostDB
>Reporter: Susan Hinrichs
>Priority: Blocker
> Fix For: 7.1.0
>
> Attachments: ts-4915.diff, ts-4915.diff
>
>  Time Spent: 5h 40m
>  Remaining Estimate: 0h
>
> Saw this while testing fix for TS-4813 with debug enabled.
> {code}
> (gdb) bt full
> #0  0x00547bfe in RefCountCacheHashEntry::operator< (this=0x1cc0880, 
> v2=...) at ../iocore/hostdb/P_RefCountCache.h:94
> No locals.
> #1  0x0054988d in 
> PriorityQueueLess::operator() (this=0x2b78a9a2587b, 
> a=@0x2b78f402af68, b=@0x2b78f402aa28)
> at ../lib/ts/PriorityQueue.h:41
> No locals.
> #2  0x00549785 in PriorityQueue PriorityQueueLess >::_bubble_up (this=0x1cb2990, 
> index=2) at ../lib/ts/PriorityQueue.h:191
> comp = {}
> parent = 0
> #3  0x006ecfcc in PriorityQueue PriorityQueueLess >::push (this=0x1cb2990, 
> entry=0x2b78f402af60) at ../../lib/ts/PriorityQueue.h:91
> len = 2
> #4  0x006ec206 in RefCountCachePartition::put 
> (this=0x1cb2900, key=6912554662447498853, item=0x2b78aee04f00, size=96, 
> expire_time=1475202356) at ./P_RefCountCache.h:210
> expiry_entry = 0x2b78f402af60
> __func__ = "put"
> val = 0x1cc0880
> #5  0x006eb3de in RefCountCache::put (this=0x18051e0, 
> key=6912554662447498853, item=0x2b78aee04f00, size=16, 
> expiry_time=1475202356) at ./P_RefCountCache.h:462
> No locals.
> #6  0x006e2d8e in HostDBContinuation::dnsEvent (this=0x2b7938020f00, 
> event=600, e=0x2b78ac009440) at HostDB.cc:1422
> is_rr = false
> old_rr_data = 0x0
> first_record = 0x2b78ac0094f8
> m = 0x1
> failed = false
> old_r = {m_ptr = 0x0}
> af = 2 '\002'
> s_size = 16
> rrsize = 0
> allocSize = 16
> r = 0x2b78aee04f00
> old_info = { = { = {_vptr.ForceVFPTToTop 
> = 0x7f3630}, m_refcount = 0}, iobuffer_index = 0, 
>   key = 47797242059264, app = {allotment = {application1 = 5326300, 
> application2 = 0}, http_data = {http_version = 4, 
>   pipeline_max = 59, keepalive_timeout = 17, fail_count = 81, 
> unused1 = 0, last_failure = 0}, rr = {offset = 5326300}}, data = {
> ip = {sa = {sa_family = 54488, sa_data = 
> "^\000\000\000\000\000\020\034$\274x+\000"}, sin = {sin_family = 54488, 
> sin_port = 94, 
> sin_addr = {s_addr = 0}, sin_zero = "\020\034$\274x+\000"}, 
> sin6 = {sin6_family = 54488, sin6_port = 94, sin6_flowinfo = 0, 
> sin6_addr = {__in6_u = {__u6_addr8 = 
> "\020\034$\274x+\000\000\030\036$\274\375\b\000", __u6_addr16 = {7184, 48164, 
> 11128, 
>   0, 7704, 48164, 2301, 0}, __u6_addr32 = {3156483088, 
> 11128, 3156483608, 2301}}}, sin6_scope_id = 3156478176}}, 
> hostname_offset = 6214872, srv = {srv_offset = 54488, srv_weight 
> = 94, srv_priority = 0, srv_port = 0, key = 3156483088}}, 
>   hostname_offset = 11128, ip_timestamp = 2845989456, 
> ip_timeout_interval = 11128, is_srv = 0, reverse_dns = 0, round_robin = 1, 
>   round_robin_elt = 0}
> valid_records = 0
> tip = {_family = 2, _addr = {_ip4 = 540420056, _ip6 = {__in6_u = 
> {__u6_addr8 = "\330'6 x+\000\000\360L\020\250x+\000", 
> __u6_addr16 = {10200, 8246, 11128, 0, 19696, 43024, 11128, 
> 0}, __u6_addr32 = {540420056, 11128, 2819640560, 11128}}}, 
> _byte = "\330'6 x+\000\000\360L\020\250x+\000", _u32 = 
> {540420056, 11128, 2819640560, 11128}, _u64 = {47794936489944, 
>   47797215710448}}}
> ttl_seconds = 132
> aname = 0x2b7938021000 "fbmm1.zenfs.com"
> offset = 96
> thread = 0x2b78a8101010
> __func__ = 

[jira] [Work logged] (TS-4915) Crash from hostdb in PriorityQueueLess

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-4915?focusedWorklogId=30561=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-30561
 ]

ASF GitHub Bot logged work on TS-4915:
--

Author: ASF GitHub Bot
Created on: 13/Oct/16 02:19
Start Date: 13/Oct/16 02:19
Worklog Time Spent: 10m 
  Work Description: Github user bryancall commented on the issue:

https://github.com/apache/trafficserver/pull/1088
  
I have been running this in production without an issue.  However, I still 
have a comment in the code about the !empt() conditional and the bubble_up 
function call.


Issue Time Tracking
---

Worklog Id: (was: 30561)
Time Spent: 5.5h  (was: 5h 20m)

> Crash from hostdb in PriorityQueueLess
> --
>
> Key: TS-4915
> URL: https://issues.apache.org/jira/browse/TS-4915
> Project: Traffic Server
>  Issue Type: Bug
>  Components: HostDB
>Reporter: Susan Hinrichs
>Priority: Blocker
> Fix For: 7.1.0
>
> Attachments: ts-4915.diff, ts-4915.diff
>
>  Time Spent: 5.5h
>  Remaining Estimate: 0h
>
> Saw this while testing fix for TS-4813 with debug enabled.
> {code}
> (gdb) bt full
> #0  0x00547bfe in RefCountCacheHashEntry::operator< (this=0x1cc0880, 
> v2=...) at ../iocore/hostdb/P_RefCountCache.h:94
> No locals.
> #1  0x0054988d in 
> PriorityQueueLess::operator() (this=0x2b78a9a2587b, 
> a=@0x2b78f402af68, b=@0x2b78f402aa28)
> at ../lib/ts/PriorityQueue.h:41
> No locals.
> #2  0x00549785 in PriorityQueue PriorityQueueLess >::_bubble_up (this=0x1cb2990, 
> index=2) at ../lib/ts/PriorityQueue.h:191
> comp = {}
> parent = 0
> #3  0x006ecfcc in PriorityQueue PriorityQueueLess >::push (this=0x1cb2990, 
> entry=0x2b78f402af60) at ../../lib/ts/PriorityQueue.h:91
> len = 2
> #4  0x006ec206 in RefCountCachePartition::put 
> (this=0x1cb2900, key=6912554662447498853, item=0x2b78aee04f00, size=96, 
> expire_time=1475202356) at ./P_RefCountCache.h:210
> expiry_entry = 0x2b78f402af60
> __func__ = "put"
> val = 0x1cc0880
> #5  0x006eb3de in RefCountCache::put (this=0x18051e0, 
> key=6912554662447498853, item=0x2b78aee04f00, size=16, 
> expiry_time=1475202356) at ./P_RefCountCache.h:462
> No locals.
> #6  0x006e2d8e in HostDBContinuation::dnsEvent (this=0x2b7938020f00, 
> event=600, e=0x2b78ac009440) at HostDB.cc:1422
> is_rr = false
> old_rr_data = 0x0
> first_record = 0x2b78ac0094f8
> m = 0x1
> failed = false
> old_r = {m_ptr = 0x0}
> af = 2 '\002'
> s_size = 16
> rrsize = 0
> allocSize = 16
> r = 0x2b78aee04f00
> old_info = { = { = {_vptr.ForceVFPTToTop 
> = 0x7f3630}, m_refcount = 0}, iobuffer_index = 0, 
>   key = 47797242059264, app = {allotment = {application1 = 5326300, 
> application2 = 0}, http_data = {http_version = 4, 
>   pipeline_max = 59, keepalive_timeout = 17, fail_count = 81, 
> unused1 = 0, last_failure = 0}, rr = {offset = 5326300}}, data = {
> ip = {sa = {sa_family = 54488, sa_data = 
> "^\000\000\000\000\000\020\034$\274x+\000"}, sin = {sin_family = 54488, 
> sin_port = 94, 
> sin_addr = {s_addr = 0}, sin_zero = "\020\034$\274x+\000"}, 
> sin6 = {sin6_family = 54488, sin6_port = 94, sin6_flowinfo = 0, 
> sin6_addr = {__in6_u = {__u6_addr8 = 
> "\020\034$\274x+\000\000\030\036$\274\375\b\000", __u6_addr16 = {7184, 48164, 
> 11128, 
>   0, 7704, 48164, 2301, 0}, __u6_addr32 = {3156483088, 
> 11128, 3156483608, 2301}}}, sin6_scope_id = 3156478176}}, 
> hostname_offset = 6214872, srv = {srv_offset = 54488, srv_weight 
> = 94, srv_priority = 0, srv_port = 0, key = 3156483088}}, 
>   hostname_offset = 11128, ip_timestamp = 2845989456, 
> ip_timeout_interval = 11128, is_srv = 0, reverse_dns = 0, round_robin = 1, 
>   round_robin_elt = 0}
> valid_records = 0
> tip = {_family = 2, _addr = {_ip4 = 540420056, _ip6 = {__in6_u = 
> {__u6_addr8 = "\330'6 x+\000\000\360L\020\250x+\000", 
> __u6_addr16 = {10200, 8246, 11128, 0, 19696, 43024, 11128, 
> 0}, __u6_addr32 = {540420056, 11128, 2819640560, 11128}}}, 
> _byte = "\330'6 x+\000\000\360L\020\250x+\000", _u32 = 
> {540420056, 11128, 2819640560, 11128}, _u64 = {47794936489944, 
>   47797215710448}}}
> ttl_seconds = 132
> aname = 0x2b7938021000 "fbmm1.zenfs.com"
> offset = 96
> thread = 0x2b78a8101010
> __func__ = "dnsEvent"
> #7  

[jira] [Work logged] (TS-4915) Crash from hostdb in PriorityQueueLess

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-4915?focusedWorklogId=30560=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-30560
 ]

ASF GitHub Bot logged work on TS-4915:
--

Author: ASF GitHub Bot
Created on: 13/Oct/16 01:51
Start Date: 13/Oct/16 01:51
Worklog Time Spent: 10m 
  Work Description: Github user atsci commented on the issue:

https://github.com/apache/trafficserver/pull/1088
  
Linux build *successful*! See 
https://ci.trafficserver.apache.org/job/Github-Linux/883/ for details.
 



Issue Time Tracking
---

Worklog Id: (was: 30560)
Time Spent: 5h 20m  (was: 5h 10m)

> Crash from hostdb in PriorityQueueLess
> --
>
> Key: TS-4915
> URL: https://issues.apache.org/jira/browse/TS-4915
> Project: Traffic Server
>  Issue Type: Bug
>  Components: HostDB
>Reporter: Susan Hinrichs
>Priority: Blocker
> Fix For: 7.1.0
>
> Attachments: ts-4915.diff, ts-4915.diff
>
>  Time Spent: 5h 20m
>  Remaining Estimate: 0h
>
> Saw this while testing fix for TS-4813 with debug enabled.
> {code}
> (gdb) bt full
> #0  0x00547bfe in RefCountCacheHashEntry::operator< (this=0x1cc0880, 
> v2=...) at ../iocore/hostdb/P_RefCountCache.h:94
> No locals.
> #1  0x0054988d in 
> PriorityQueueLess::operator() (this=0x2b78a9a2587b, 
> a=@0x2b78f402af68, b=@0x2b78f402aa28)
> at ../lib/ts/PriorityQueue.h:41
> No locals.
> #2  0x00549785 in PriorityQueue PriorityQueueLess >::_bubble_up (this=0x1cb2990, 
> index=2) at ../lib/ts/PriorityQueue.h:191
> comp = {}
> parent = 0
> #3  0x006ecfcc in PriorityQueue PriorityQueueLess >::push (this=0x1cb2990, 
> entry=0x2b78f402af60) at ../../lib/ts/PriorityQueue.h:91
> len = 2
> #4  0x006ec206 in RefCountCachePartition::put 
> (this=0x1cb2900, key=6912554662447498853, item=0x2b78aee04f00, size=96, 
> expire_time=1475202356) at ./P_RefCountCache.h:210
> expiry_entry = 0x2b78f402af60
> __func__ = "put"
> val = 0x1cc0880
> #5  0x006eb3de in RefCountCache::put (this=0x18051e0, 
> key=6912554662447498853, item=0x2b78aee04f00, size=16, 
> expiry_time=1475202356) at ./P_RefCountCache.h:462
> No locals.
> #6  0x006e2d8e in HostDBContinuation::dnsEvent (this=0x2b7938020f00, 
> event=600, e=0x2b78ac009440) at HostDB.cc:1422
> is_rr = false
> old_rr_data = 0x0
> first_record = 0x2b78ac0094f8
> m = 0x1
> failed = false
> old_r = {m_ptr = 0x0}
> af = 2 '\002'
> s_size = 16
> rrsize = 0
> allocSize = 16
> r = 0x2b78aee04f00
> old_info = { = { = {_vptr.ForceVFPTToTop 
> = 0x7f3630}, m_refcount = 0}, iobuffer_index = 0, 
>   key = 47797242059264, app = {allotment = {application1 = 5326300, 
> application2 = 0}, http_data = {http_version = 4, 
>   pipeline_max = 59, keepalive_timeout = 17, fail_count = 81, 
> unused1 = 0, last_failure = 0}, rr = {offset = 5326300}}, data = {
> ip = {sa = {sa_family = 54488, sa_data = 
> "^\000\000\000\000\000\020\034$\274x+\000"}, sin = {sin_family = 54488, 
> sin_port = 94, 
> sin_addr = {s_addr = 0}, sin_zero = "\020\034$\274x+\000"}, 
> sin6 = {sin6_family = 54488, sin6_port = 94, sin6_flowinfo = 0, 
> sin6_addr = {__in6_u = {__u6_addr8 = 
> "\020\034$\274x+\000\000\030\036$\274\375\b\000", __u6_addr16 = {7184, 48164, 
> 11128, 
>   0, 7704, 48164, 2301, 0}, __u6_addr32 = {3156483088, 
> 11128, 3156483608, 2301}}}, sin6_scope_id = 3156478176}}, 
> hostname_offset = 6214872, srv = {srv_offset = 54488, srv_weight 
> = 94, srv_priority = 0, srv_port = 0, key = 3156483088}}, 
>   hostname_offset = 11128, ip_timestamp = 2845989456, 
> ip_timeout_interval = 11128, is_srv = 0, reverse_dns = 0, round_robin = 1, 
>   round_robin_elt = 0}
> valid_records = 0
> tip = {_family = 2, _addr = {_ip4 = 540420056, _ip6 = {__in6_u = 
> {__u6_addr8 = "\330'6 x+\000\000\360L\020\250x+\000", 
> __u6_addr16 = {10200, 8246, 11128, 0, 19696, 43024, 11128, 
> 0}, __u6_addr32 = {540420056, 11128, 2819640560, 11128}}}, 
> _byte = "\330'6 x+\000\000\360L\020\250x+\000", _u32 = 
> {540420056, 11128, 2819640560, 11128}, _u64 = {47794936489944, 
>   47797215710448}}}
> ttl_seconds = 132
> aname = 0x2b7938021000 "fbmm1.zenfs.com"
> offset = 96
> thread = 0x2b78a8101010
> __func__ = "dnsEvent"
> #7  0x005145dc in Continuation::handleEvent 

[jira] [Work logged] (TS-4915) Crash from hostdb in PriorityQueueLess

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-4915?focusedWorklogId=30559=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-30559
 ]

ASF GitHub Bot logged work on TS-4915:
--

Author: ASF GitHub Bot
Created on: 13/Oct/16 01:46
Start Date: 13/Oct/16 01:46
Worklog Time Spent: 10m 
  Work Description: Github user atsci commented on the issue:

https://github.com/apache/trafficserver/pull/1088
  
FreeBSD build *successful*! See 
https://ci.trafficserver.apache.org/job/Github-FreeBSD/991/ for details.
 



Issue Time Tracking
---

Worklog Id: (was: 30559)
Time Spent: 5h 10m  (was: 5h)

> Crash from hostdb in PriorityQueueLess
> --
>
> Key: TS-4915
> URL: https://issues.apache.org/jira/browse/TS-4915
> Project: Traffic Server
>  Issue Type: Bug
>  Components: HostDB
>Reporter: Susan Hinrichs
>Priority: Blocker
> Fix For: 7.1.0
>
> Attachments: ts-4915.diff, ts-4915.diff
>
>  Time Spent: 5h 10m
>  Remaining Estimate: 0h
>
> Saw this while testing fix for TS-4813 with debug enabled.
> {code}
> (gdb) bt full
> #0  0x00547bfe in RefCountCacheHashEntry::operator< (this=0x1cc0880, 
> v2=...) at ../iocore/hostdb/P_RefCountCache.h:94
> No locals.
> #1  0x0054988d in 
> PriorityQueueLess::operator() (this=0x2b78a9a2587b, 
> a=@0x2b78f402af68, b=@0x2b78f402aa28)
> at ../lib/ts/PriorityQueue.h:41
> No locals.
> #2  0x00549785 in PriorityQueue PriorityQueueLess >::_bubble_up (this=0x1cb2990, 
> index=2) at ../lib/ts/PriorityQueue.h:191
> comp = {}
> parent = 0
> #3  0x006ecfcc in PriorityQueue PriorityQueueLess >::push (this=0x1cb2990, 
> entry=0x2b78f402af60) at ../../lib/ts/PriorityQueue.h:91
> len = 2
> #4  0x006ec206 in RefCountCachePartition::put 
> (this=0x1cb2900, key=6912554662447498853, item=0x2b78aee04f00, size=96, 
> expire_time=1475202356) at ./P_RefCountCache.h:210
> expiry_entry = 0x2b78f402af60
> __func__ = "put"
> val = 0x1cc0880
> #5  0x006eb3de in RefCountCache::put (this=0x18051e0, 
> key=6912554662447498853, item=0x2b78aee04f00, size=16, 
> expiry_time=1475202356) at ./P_RefCountCache.h:462
> No locals.
> #6  0x006e2d8e in HostDBContinuation::dnsEvent (this=0x2b7938020f00, 
> event=600, e=0x2b78ac009440) at HostDB.cc:1422
> is_rr = false
> old_rr_data = 0x0
> first_record = 0x2b78ac0094f8
> m = 0x1
> failed = false
> old_r = {m_ptr = 0x0}
> af = 2 '\002'
> s_size = 16
> rrsize = 0
> allocSize = 16
> r = 0x2b78aee04f00
> old_info = { = { = {_vptr.ForceVFPTToTop 
> = 0x7f3630}, m_refcount = 0}, iobuffer_index = 0, 
>   key = 47797242059264, app = {allotment = {application1 = 5326300, 
> application2 = 0}, http_data = {http_version = 4, 
>   pipeline_max = 59, keepalive_timeout = 17, fail_count = 81, 
> unused1 = 0, last_failure = 0}, rr = {offset = 5326300}}, data = {
> ip = {sa = {sa_family = 54488, sa_data = 
> "^\000\000\000\000\000\020\034$\274x+\000"}, sin = {sin_family = 54488, 
> sin_port = 94, 
> sin_addr = {s_addr = 0}, sin_zero = "\020\034$\274x+\000"}, 
> sin6 = {sin6_family = 54488, sin6_port = 94, sin6_flowinfo = 0, 
> sin6_addr = {__in6_u = {__u6_addr8 = 
> "\020\034$\274x+\000\000\030\036$\274\375\b\000", __u6_addr16 = {7184, 48164, 
> 11128, 
>   0, 7704, 48164, 2301, 0}, __u6_addr32 = {3156483088, 
> 11128, 3156483608, 2301}}}, sin6_scope_id = 3156478176}}, 
> hostname_offset = 6214872, srv = {srv_offset = 54488, srv_weight 
> = 94, srv_priority = 0, srv_port = 0, key = 3156483088}}, 
>   hostname_offset = 11128, ip_timestamp = 2845989456, 
> ip_timeout_interval = 11128, is_srv = 0, reverse_dns = 0, round_robin = 1, 
>   round_robin_elt = 0}
> valid_records = 0
> tip = {_family = 2, _addr = {_ip4 = 540420056, _ip6 = {__in6_u = 
> {__u6_addr8 = "\330'6 x+\000\000\360L\020\250x+\000", 
> __u6_addr16 = {10200, 8246, 11128, 0, 19696, 43024, 11128, 
> 0}, __u6_addr32 = {540420056, 11128, 2819640560, 11128}}}, 
> _byte = "\330'6 x+\000\000\360L\020\250x+\000", _u32 = 
> {540420056, 11128, 2819640560, 11128}, _u64 = {47794936489944, 
>   47797215710448}}}
> ttl_seconds = 132
> aname = 0x2b7938021000 "fbmm1.zenfs.com"
> offset = 96
> thread = 0x2b78a8101010
> __func__ = "dnsEvent"
> #7  0x005145dc in Continuation::handleEvent 

[jira] [Work logged] (TS-4915) Crash from hostdb in PriorityQueueLess

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-4915?focusedWorklogId=30558=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-30558
 ]

ASF GitHub Bot logged work on TS-4915:
--

Author: ASF GitHub Bot
Created on: 13/Oct/16 01:30
Start Date: 13/Oct/16 01:30
Worklog Time Spent: 10m 
  Work Description: Github user shinrich commented on the issue:

https://github.com/apache/trafficserver/pull/1088
  
Squashed and rebased.   Assuming all looks good with merge in the morning.


Issue Time Tracking
---

Worklog Id: (was: 30558)
Time Spent: 5h  (was: 4h 50m)

> Crash from hostdb in PriorityQueueLess
> --
>
> Key: TS-4915
> URL: https://issues.apache.org/jira/browse/TS-4915
> Project: Traffic Server
>  Issue Type: Bug
>  Components: HostDB
>Reporter: Susan Hinrichs
>Priority: Blocker
> Fix For: 7.1.0
>
> Attachments: ts-4915.diff, ts-4915.diff
>
>  Time Spent: 5h
>  Remaining Estimate: 0h
>
> Saw this while testing fix for TS-4813 with debug enabled.
> {code}
> (gdb) bt full
> #0  0x00547bfe in RefCountCacheHashEntry::operator< (this=0x1cc0880, 
> v2=...) at ../iocore/hostdb/P_RefCountCache.h:94
> No locals.
> #1  0x0054988d in 
> PriorityQueueLess::operator() (this=0x2b78a9a2587b, 
> a=@0x2b78f402af68, b=@0x2b78f402aa28)
> at ../lib/ts/PriorityQueue.h:41
> No locals.
> #2  0x00549785 in PriorityQueue PriorityQueueLess >::_bubble_up (this=0x1cb2990, 
> index=2) at ../lib/ts/PriorityQueue.h:191
> comp = {}
> parent = 0
> #3  0x006ecfcc in PriorityQueue PriorityQueueLess >::push (this=0x1cb2990, 
> entry=0x2b78f402af60) at ../../lib/ts/PriorityQueue.h:91
> len = 2
> #4  0x006ec206 in RefCountCachePartition::put 
> (this=0x1cb2900, key=6912554662447498853, item=0x2b78aee04f00, size=96, 
> expire_time=1475202356) at ./P_RefCountCache.h:210
> expiry_entry = 0x2b78f402af60
> __func__ = "put"
> val = 0x1cc0880
> #5  0x006eb3de in RefCountCache::put (this=0x18051e0, 
> key=6912554662447498853, item=0x2b78aee04f00, size=16, 
> expiry_time=1475202356) at ./P_RefCountCache.h:462
> No locals.
> #6  0x006e2d8e in HostDBContinuation::dnsEvent (this=0x2b7938020f00, 
> event=600, e=0x2b78ac009440) at HostDB.cc:1422
> is_rr = false
> old_rr_data = 0x0
> first_record = 0x2b78ac0094f8
> m = 0x1
> failed = false
> old_r = {m_ptr = 0x0}
> af = 2 '\002'
> s_size = 16
> rrsize = 0
> allocSize = 16
> r = 0x2b78aee04f00
> old_info = { = { = {_vptr.ForceVFPTToTop 
> = 0x7f3630}, m_refcount = 0}, iobuffer_index = 0, 
>   key = 47797242059264, app = {allotment = {application1 = 5326300, 
> application2 = 0}, http_data = {http_version = 4, 
>   pipeline_max = 59, keepalive_timeout = 17, fail_count = 81, 
> unused1 = 0, last_failure = 0}, rr = {offset = 5326300}}, data = {
> ip = {sa = {sa_family = 54488, sa_data = 
> "^\000\000\000\000\000\020\034$\274x+\000"}, sin = {sin_family = 54488, 
> sin_port = 94, 
> sin_addr = {s_addr = 0}, sin_zero = "\020\034$\274x+\000"}, 
> sin6 = {sin6_family = 54488, sin6_port = 94, sin6_flowinfo = 0, 
> sin6_addr = {__in6_u = {__u6_addr8 = 
> "\020\034$\274x+\000\000\030\036$\274\375\b\000", __u6_addr16 = {7184, 48164, 
> 11128, 
>   0, 7704, 48164, 2301, 0}, __u6_addr32 = {3156483088, 
> 11128, 3156483608, 2301}}}, sin6_scope_id = 3156478176}}, 
> hostname_offset = 6214872, srv = {srv_offset = 54488, srv_weight 
> = 94, srv_priority = 0, srv_port = 0, key = 3156483088}}, 
>   hostname_offset = 11128, ip_timestamp = 2845989456, 
> ip_timeout_interval = 11128, is_srv = 0, reverse_dns = 0, round_robin = 1, 
>   round_robin_elt = 0}
> valid_records = 0
> tip = {_family = 2, _addr = {_ip4 = 540420056, _ip6 = {__in6_u = 
> {__u6_addr8 = "\330'6 x+\000\000\360L\020\250x+\000", 
> __u6_addr16 = {10200, 8246, 11128, 0, 19696, 43024, 11128, 
> 0}, __u6_addr32 = {540420056, 11128, 2819640560, 11128}}}, 
> _byte = "\330'6 x+\000\000\360L\020\250x+\000", _u32 = 
> {540420056, 11128, 2819640560, 11128}, _u64 = {47794936489944, 
>   47797215710448}}}
> ttl_seconds = 132
> aname = 0x2b7938021000 "fbmm1.zenfs.com"
> offset = 96
> thread = 0x2b78a8101010
> __func__ = "dnsEvent"
> #7  0x005145dc in Continuation::handleEvent (this=0x2b7938020f00, 
> event=600, data=0x2b78ac009440)
>  

[jira] [Work logged] (TS-4915) Crash from hostdb in PriorityQueueLess

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-4915?focusedWorklogId=30523=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-30523
 ]

ASF GitHub Bot logged work on TS-4915:
--

Author: ASF GitHub Bot
Created on: 12/Oct/16 20:14
Start Date: 12/Oct/16 20:14
Worklog Time Spent: 10m 
  Work Description: Github user atsci commented on the issue:

https://github.com/apache/trafficserver/pull/1088
  
Linux build *successful*! See 
https://ci.trafficserver.apache.org/job/Github-Linux/874/ for details.
 



Issue Time Tracking
---

Worklog Id: (was: 30523)
Time Spent: 4h 50m  (was: 4h 40m)

> Crash from hostdb in PriorityQueueLess
> --
>
> Key: TS-4915
> URL: https://issues.apache.org/jira/browse/TS-4915
> Project: Traffic Server
>  Issue Type: Bug
>  Components: HostDB
>Reporter: Susan Hinrichs
>Priority: Blocker
> Fix For: 7.1.0
>
> Attachments: ts-4915.diff, ts-4915.diff
>
>  Time Spent: 4h 50m
>  Remaining Estimate: 0h
>
> Saw this while testing fix for TS-4813 with debug enabled.
> {code}
> (gdb) bt full
> #0  0x00547bfe in RefCountCacheHashEntry::operator< (this=0x1cc0880, 
> v2=...) at ../iocore/hostdb/P_RefCountCache.h:94
> No locals.
> #1  0x0054988d in 
> PriorityQueueLess::operator() (this=0x2b78a9a2587b, 
> a=@0x2b78f402af68, b=@0x2b78f402aa28)
> at ../lib/ts/PriorityQueue.h:41
> No locals.
> #2  0x00549785 in PriorityQueue PriorityQueueLess >::_bubble_up (this=0x1cb2990, 
> index=2) at ../lib/ts/PriorityQueue.h:191
> comp = {}
> parent = 0
> #3  0x006ecfcc in PriorityQueue PriorityQueueLess >::push (this=0x1cb2990, 
> entry=0x2b78f402af60) at ../../lib/ts/PriorityQueue.h:91
> len = 2
> #4  0x006ec206 in RefCountCachePartition::put 
> (this=0x1cb2900, key=6912554662447498853, item=0x2b78aee04f00, size=96, 
> expire_time=1475202356) at ./P_RefCountCache.h:210
> expiry_entry = 0x2b78f402af60
> __func__ = "put"
> val = 0x1cc0880
> #5  0x006eb3de in RefCountCache::put (this=0x18051e0, 
> key=6912554662447498853, item=0x2b78aee04f00, size=16, 
> expiry_time=1475202356) at ./P_RefCountCache.h:462
> No locals.
> #6  0x006e2d8e in HostDBContinuation::dnsEvent (this=0x2b7938020f00, 
> event=600, e=0x2b78ac009440) at HostDB.cc:1422
> is_rr = false
> old_rr_data = 0x0
> first_record = 0x2b78ac0094f8
> m = 0x1
> failed = false
> old_r = {m_ptr = 0x0}
> af = 2 '\002'
> s_size = 16
> rrsize = 0
> allocSize = 16
> r = 0x2b78aee04f00
> old_info = { = { = {_vptr.ForceVFPTToTop 
> = 0x7f3630}, m_refcount = 0}, iobuffer_index = 0, 
>   key = 47797242059264, app = {allotment = {application1 = 5326300, 
> application2 = 0}, http_data = {http_version = 4, 
>   pipeline_max = 59, keepalive_timeout = 17, fail_count = 81, 
> unused1 = 0, last_failure = 0}, rr = {offset = 5326300}}, data = {
> ip = {sa = {sa_family = 54488, sa_data = 
> "^\000\000\000\000\000\020\034$\274x+\000"}, sin = {sin_family = 54488, 
> sin_port = 94, 
> sin_addr = {s_addr = 0}, sin_zero = "\020\034$\274x+\000"}, 
> sin6 = {sin6_family = 54488, sin6_port = 94, sin6_flowinfo = 0, 
> sin6_addr = {__in6_u = {__u6_addr8 = 
> "\020\034$\274x+\000\000\030\036$\274\375\b\000", __u6_addr16 = {7184, 48164, 
> 11128, 
>   0, 7704, 48164, 2301, 0}, __u6_addr32 = {3156483088, 
> 11128, 3156483608, 2301}}}, sin6_scope_id = 3156478176}}, 
> hostname_offset = 6214872, srv = {srv_offset = 54488, srv_weight 
> = 94, srv_priority = 0, srv_port = 0, key = 3156483088}}, 
>   hostname_offset = 11128, ip_timestamp = 2845989456, 
> ip_timeout_interval = 11128, is_srv = 0, reverse_dns = 0, round_robin = 1, 
>   round_robin_elt = 0}
> valid_records = 0
> tip = {_family = 2, _addr = {_ip4 = 540420056, _ip6 = {__in6_u = 
> {__u6_addr8 = "\330'6 x+\000\000\360L\020\250x+\000", 
> __u6_addr16 = {10200, 8246, 11128, 0, 19696, 43024, 11128, 
> 0}, __u6_addr32 = {540420056, 11128, 2819640560, 11128}}}, 
> _byte = "\330'6 x+\000\000\360L\020\250x+\000", _u32 = 
> {540420056, 11128, 2819640560, 11128}, _u64 = {47794936489944, 
>   47797215710448}}}
> ttl_seconds = 132
> aname = 0x2b7938021000 "fbmm1.zenfs.com"
> offset = 96
> thread = 0x2b78a8101010
> __func__ = "dnsEvent"
> #7  0x005145dc in Continuation::handleEvent 

[jira] [Work logged] (TS-4915) Crash from hostdb in PriorityQueueLess

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-4915?focusedWorklogId=30521=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-30521
 ]

ASF GitHub Bot logged work on TS-4915:
--

Author: ASF GitHub Bot
Created on: 12/Oct/16 20:12
Start Date: 12/Oct/16 20:12
Worklog Time Spent: 10m 
  Work Description: Github user atsci commented on the issue:

https://github.com/apache/trafficserver/pull/1088
  
FreeBSD build *successful*! See 
https://ci.trafficserver.apache.org/job/Github-FreeBSD/982/ for details.
 



Issue Time Tracking
---

Worklog Id: (was: 30521)
Time Spent: 4h 40m  (was: 4.5h)

> Crash from hostdb in PriorityQueueLess
> --
>
> Key: TS-4915
> URL: https://issues.apache.org/jira/browse/TS-4915
> Project: Traffic Server
>  Issue Type: Bug
>  Components: HostDB
>Reporter: Susan Hinrichs
>Priority: Blocker
> Fix For: 7.1.0
>
> Attachments: ts-4915.diff, ts-4915.diff
>
>  Time Spent: 4h 40m
>  Remaining Estimate: 0h
>
> Saw this while testing fix for TS-4813 with debug enabled.
> {code}
> (gdb) bt full
> #0  0x00547bfe in RefCountCacheHashEntry::operator< (this=0x1cc0880, 
> v2=...) at ../iocore/hostdb/P_RefCountCache.h:94
> No locals.
> #1  0x0054988d in 
> PriorityQueueLess::operator() (this=0x2b78a9a2587b, 
> a=@0x2b78f402af68, b=@0x2b78f402aa28)
> at ../lib/ts/PriorityQueue.h:41
> No locals.
> #2  0x00549785 in PriorityQueue PriorityQueueLess >::_bubble_up (this=0x1cb2990, 
> index=2) at ../lib/ts/PriorityQueue.h:191
> comp = {}
> parent = 0
> #3  0x006ecfcc in PriorityQueue PriorityQueueLess >::push (this=0x1cb2990, 
> entry=0x2b78f402af60) at ../../lib/ts/PriorityQueue.h:91
> len = 2
> #4  0x006ec206 in RefCountCachePartition::put 
> (this=0x1cb2900, key=6912554662447498853, item=0x2b78aee04f00, size=96, 
> expire_time=1475202356) at ./P_RefCountCache.h:210
> expiry_entry = 0x2b78f402af60
> __func__ = "put"
> val = 0x1cc0880
> #5  0x006eb3de in RefCountCache::put (this=0x18051e0, 
> key=6912554662447498853, item=0x2b78aee04f00, size=16, 
> expiry_time=1475202356) at ./P_RefCountCache.h:462
> No locals.
> #6  0x006e2d8e in HostDBContinuation::dnsEvent (this=0x2b7938020f00, 
> event=600, e=0x2b78ac009440) at HostDB.cc:1422
> is_rr = false
> old_rr_data = 0x0
> first_record = 0x2b78ac0094f8
> m = 0x1
> failed = false
> old_r = {m_ptr = 0x0}
> af = 2 '\002'
> s_size = 16
> rrsize = 0
> allocSize = 16
> r = 0x2b78aee04f00
> old_info = { = { = {_vptr.ForceVFPTToTop 
> = 0x7f3630}, m_refcount = 0}, iobuffer_index = 0, 
>   key = 47797242059264, app = {allotment = {application1 = 5326300, 
> application2 = 0}, http_data = {http_version = 4, 
>   pipeline_max = 59, keepalive_timeout = 17, fail_count = 81, 
> unused1 = 0, last_failure = 0}, rr = {offset = 5326300}}, data = {
> ip = {sa = {sa_family = 54488, sa_data = 
> "^\000\000\000\000\000\020\034$\274x+\000"}, sin = {sin_family = 54488, 
> sin_port = 94, 
> sin_addr = {s_addr = 0}, sin_zero = "\020\034$\274x+\000"}, 
> sin6 = {sin6_family = 54488, sin6_port = 94, sin6_flowinfo = 0, 
> sin6_addr = {__in6_u = {__u6_addr8 = 
> "\020\034$\274x+\000\000\030\036$\274\375\b\000", __u6_addr16 = {7184, 48164, 
> 11128, 
>   0, 7704, 48164, 2301, 0}, __u6_addr32 = {3156483088, 
> 11128, 3156483608, 2301}}}, sin6_scope_id = 3156478176}}, 
> hostname_offset = 6214872, srv = {srv_offset = 54488, srv_weight 
> = 94, srv_priority = 0, srv_port = 0, key = 3156483088}}, 
>   hostname_offset = 11128, ip_timestamp = 2845989456, 
> ip_timeout_interval = 11128, is_srv = 0, reverse_dns = 0, round_robin = 1, 
>   round_robin_elt = 0}
> valid_records = 0
> tip = {_family = 2, _addr = {_ip4 = 540420056, _ip6 = {__in6_u = 
> {__u6_addr8 = "\330'6 x+\000\000\360L\020\250x+\000", 
> __u6_addr16 = {10200, 8246, 11128, 0, 19696, 43024, 11128, 
> 0}, __u6_addr32 = {540420056, 11128, 2819640560, 11128}}}, 
> _byte = "\330'6 x+\000\000\360L\020\250x+\000", _u32 = 
> {540420056, 11128, 2819640560, 11128}, _u64 = {47794936489944, 
>   47797215710448}}}
> ttl_seconds = 132
> aname = 0x2b7938021000 "fbmm1.zenfs.com"
> offset = 96
> thread = 0x2b78a8101010
> __func__ = "dnsEvent"
> #7  0x005145dc in Continuation::handleEvent 

[jira] [Work logged] (TS-4915) Crash from hostdb in PriorityQueueLess

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-4915?focusedWorklogId=30519=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-30519
 ]

ASF GitHub Bot logged work on TS-4915:
--

Author: ASF GitHub Bot
Created on: 12/Oct/16 20:02
Start Date: 12/Oct/16 20:02
Worklog Time Spent: 10m 
  Work Description: Github user shinrich commented on a diff in the pull 
request:

https://github.com/apache/trafficserver/pull/1088#discussion_r83086541
  
--- Diff: lib/ts/PriorityQueue.h ---
@@ -110,6 +110,12 @@ PriorityQueue::pop()
 return;
   }
 
+  // SKH - I suspect this assignment is not preserving entry indices 
correctly.
--- End diff --

Ok, with a short enough Vec I could exercise the problem.  Definitely an 
edge case for pop(), but we should go ahead and fix it.  New commit with the 
fix and a test.


Issue Time Tracking
---

Worklog Id: (was: 30519)
Time Spent: 4.5h  (was: 4h 20m)

> Crash from hostdb in PriorityQueueLess
> --
>
> Key: TS-4915
> URL: https://issues.apache.org/jira/browse/TS-4915
> Project: Traffic Server
>  Issue Type: Bug
>  Components: HostDB
>Reporter: Susan Hinrichs
>Priority: Blocker
> Fix For: 7.1.0
>
> Attachments: ts-4915.diff, ts-4915.diff
>
>  Time Spent: 4.5h
>  Remaining Estimate: 0h
>
> Saw this while testing fix for TS-4813 with debug enabled.
> {code}
> (gdb) bt full
> #0  0x00547bfe in RefCountCacheHashEntry::operator< (this=0x1cc0880, 
> v2=...) at ../iocore/hostdb/P_RefCountCache.h:94
> No locals.
> #1  0x0054988d in 
> PriorityQueueLess::operator() (this=0x2b78a9a2587b, 
> a=@0x2b78f402af68, b=@0x2b78f402aa28)
> at ../lib/ts/PriorityQueue.h:41
> No locals.
> #2  0x00549785 in PriorityQueue PriorityQueueLess >::_bubble_up (this=0x1cb2990, 
> index=2) at ../lib/ts/PriorityQueue.h:191
> comp = {}
> parent = 0
> #3  0x006ecfcc in PriorityQueue PriorityQueueLess >::push (this=0x1cb2990, 
> entry=0x2b78f402af60) at ../../lib/ts/PriorityQueue.h:91
> len = 2
> #4  0x006ec206 in RefCountCachePartition::put 
> (this=0x1cb2900, key=6912554662447498853, item=0x2b78aee04f00, size=96, 
> expire_time=1475202356) at ./P_RefCountCache.h:210
> expiry_entry = 0x2b78f402af60
> __func__ = "put"
> val = 0x1cc0880
> #5  0x006eb3de in RefCountCache::put (this=0x18051e0, 
> key=6912554662447498853, item=0x2b78aee04f00, size=16, 
> expiry_time=1475202356) at ./P_RefCountCache.h:462
> No locals.
> #6  0x006e2d8e in HostDBContinuation::dnsEvent (this=0x2b7938020f00, 
> event=600, e=0x2b78ac009440) at HostDB.cc:1422
> is_rr = false
> old_rr_data = 0x0
> first_record = 0x2b78ac0094f8
> m = 0x1
> failed = false
> old_r = {m_ptr = 0x0}
> af = 2 '\002'
> s_size = 16
> rrsize = 0
> allocSize = 16
> r = 0x2b78aee04f00
> old_info = { = { = {_vptr.ForceVFPTToTop 
> = 0x7f3630}, m_refcount = 0}, iobuffer_index = 0, 
>   key = 47797242059264, app = {allotment = {application1 = 5326300, 
> application2 = 0}, http_data = {http_version = 4, 
>   pipeline_max = 59, keepalive_timeout = 17, fail_count = 81, 
> unused1 = 0, last_failure = 0}, rr = {offset = 5326300}}, data = {
> ip = {sa = {sa_family = 54488, sa_data = 
> "^\000\000\000\000\000\020\034$\274x+\000"}, sin = {sin_family = 54488, 
> sin_port = 94, 
> sin_addr = {s_addr = 0}, sin_zero = "\020\034$\274x+\000"}, 
> sin6 = {sin6_family = 54488, sin6_port = 94, sin6_flowinfo = 0, 
> sin6_addr = {__in6_u = {__u6_addr8 = 
> "\020\034$\274x+\000\000\030\036$\274\375\b\000", __u6_addr16 = {7184, 48164, 
> 11128, 
>   0, 7704, 48164, 2301, 0}, __u6_addr32 = {3156483088, 
> 11128, 3156483608, 2301}}}, sin6_scope_id = 3156478176}}, 
> hostname_offset = 6214872, srv = {srv_offset = 54488, srv_weight 
> = 94, srv_priority = 0, srv_port = 0, key = 3156483088}}, 
>   hostname_offset = 11128, ip_timestamp = 2845989456, 
> ip_timeout_interval = 11128, is_srv = 0, reverse_dns = 0, round_robin = 1, 
>   round_robin_elt = 0}
> valid_records = 0
> tip = {_family = 2, _addr = {_ip4 = 540420056, _ip6 = {__in6_u = 
> {__u6_addr8 = "\330'6 x+\000\000\360L\020\250x+\000", 
> __u6_addr16 = {10200, 8246, 11128, 0, 19696, 43024, 11128, 
> 0}, __u6_addr32 = {540420056, 11128, 2819640560, 11128}}}, 
> _byte = "\330'6 x+\000\000\360L\020\250x+\000", _u32 = 
> 

[jira] [Work logged] (TS-4915) Crash from hostdb in PriorityQueueLess

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-4915?focusedWorklogId=30515=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-30515
 ]

ASF GitHub Bot logged work on TS-4915:
--

Author: ASF GitHub Bot
Created on: 12/Oct/16 19:50
Start Date: 12/Oct/16 19:50
Worklog Time Spent: 10m 
  Work Description: Github user shinrich commented on a diff in the pull 
request:

https://github.com/apache/trafficserver/pull/1088#discussion_r83084441
  
--- Diff: lib/ts/PriorityQueue.h ---
@@ -110,6 +110,12 @@ PriorityQueue::pop()
 return;
   }
 
+  // SKH - I suspect this assignment is not preserving entry indices 
correctly.
--- End diff --

Actually, I think that pop() is almost always safe since the last entry is 
largest and the swaps in bubble_down will fix up the indices.


Issue Time Tracking
---

Worklog Id: (was: 30515)
Time Spent: 4h 20m  (was: 4h 10m)

> Crash from hostdb in PriorityQueueLess
> --
>
> Key: TS-4915
> URL: https://issues.apache.org/jira/browse/TS-4915
> Project: Traffic Server
>  Issue Type: Bug
>  Components: HostDB
>Reporter: Susan Hinrichs
>Priority: Blocker
> Fix For: 7.1.0
>
> Attachments: ts-4915.diff, ts-4915.diff
>
>  Time Spent: 4h 20m
>  Remaining Estimate: 0h
>
> Saw this while testing fix for TS-4813 with debug enabled.
> {code}
> (gdb) bt full
> #0  0x00547bfe in RefCountCacheHashEntry::operator< (this=0x1cc0880, 
> v2=...) at ../iocore/hostdb/P_RefCountCache.h:94
> No locals.
> #1  0x0054988d in 
> PriorityQueueLess::operator() (this=0x2b78a9a2587b, 
> a=@0x2b78f402af68, b=@0x2b78f402aa28)
> at ../lib/ts/PriorityQueue.h:41
> No locals.
> #2  0x00549785 in PriorityQueue PriorityQueueLess >::_bubble_up (this=0x1cb2990, 
> index=2) at ../lib/ts/PriorityQueue.h:191
> comp = {}
> parent = 0
> #3  0x006ecfcc in PriorityQueue PriorityQueueLess >::push (this=0x1cb2990, 
> entry=0x2b78f402af60) at ../../lib/ts/PriorityQueue.h:91
> len = 2
> #4  0x006ec206 in RefCountCachePartition::put 
> (this=0x1cb2900, key=6912554662447498853, item=0x2b78aee04f00, size=96, 
> expire_time=1475202356) at ./P_RefCountCache.h:210
> expiry_entry = 0x2b78f402af60
> __func__ = "put"
> val = 0x1cc0880
> #5  0x006eb3de in RefCountCache::put (this=0x18051e0, 
> key=6912554662447498853, item=0x2b78aee04f00, size=16, 
> expiry_time=1475202356) at ./P_RefCountCache.h:462
> No locals.
> #6  0x006e2d8e in HostDBContinuation::dnsEvent (this=0x2b7938020f00, 
> event=600, e=0x2b78ac009440) at HostDB.cc:1422
> is_rr = false
> old_rr_data = 0x0
> first_record = 0x2b78ac0094f8
> m = 0x1
> failed = false
> old_r = {m_ptr = 0x0}
> af = 2 '\002'
> s_size = 16
> rrsize = 0
> allocSize = 16
> r = 0x2b78aee04f00
> old_info = { = { = {_vptr.ForceVFPTToTop 
> = 0x7f3630}, m_refcount = 0}, iobuffer_index = 0, 
>   key = 47797242059264, app = {allotment = {application1 = 5326300, 
> application2 = 0}, http_data = {http_version = 4, 
>   pipeline_max = 59, keepalive_timeout = 17, fail_count = 81, 
> unused1 = 0, last_failure = 0}, rr = {offset = 5326300}}, data = {
> ip = {sa = {sa_family = 54488, sa_data = 
> "^\000\000\000\000\000\020\034$\274x+\000"}, sin = {sin_family = 54488, 
> sin_port = 94, 
> sin_addr = {s_addr = 0}, sin_zero = "\020\034$\274x+\000"}, 
> sin6 = {sin6_family = 54488, sin6_port = 94, sin6_flowinfo = 0, 
> sin6_addr = {__in6_u = {__u6_addr8 = 
> "\020\034$\274x+\000\000\030\036$\274\375\b\000", __u6_addr16 = {7184, 48164, 
> 11128, 
>   0, 7704, 48164, 2301, 0}, __u6_addr32 = {3156483088, 
> 11128, 3156483608, 2301}}}, sin6_scope_id = 3156478176}}, 
> hostname_offset = 6214872, srv = {srv_offset = 54488, srv_weight 
> = 94, srv_priority = 0, srv_port = 0, key = 3156483088}}, 
>   hostname_offset = 11128, ip_timestamp = 2845989456, 
> ip_timeout_interval = 11128, is_srv = 0, reverse_dns = 0, round_robin = 1, 
>   round_robin_elt = 0}
> valid_records = 0
> tip = {_family = 2, _addr = {_ip4 = 540420056, _ip6 = {__in6_u = 
> {__u6_addr8 = "\330'6 x+\000\000\360L\020\250x+\000", 
> __u6_addr16 = {10200, 8246, 11128, 0, 19696, 43024, 11128, 
> 0}, __u6_addr32 = {540420056, 11128, 2819640560, 11128}}}, 
> _byte = "\330'6 x+\000\000\360L\020\250x+\000", _u32 = 
> {540420056, 11128, 

[jira] [Work logged] (TS-4915) Crash from hostdb in PriorityQueueLess

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-4915?focusedWorklogId=30504=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-30504
 ]

ASF GitHub Bot logged work on TS-4915:
--

Author: ASF GitHub Bot
Created on: 12/Oct/16 16:49
Start Date: 12/Oct/16 16:49
Worklog Time Spent: 10m 
  Work Description: Github user jpeach commented on a diff in the pull 
request:

https://github.com/apache/trafficserver/pull/1088#discussion_r83048953
  
--- Diff: lib/ts/PriorityQueue.h ---
@@ -110,6 +110,12 @@ PriorityQueue::pop()
 return;
   }
 
+  // SKH - I suspect this assignment is not preserving entry indices 
correctly.
--- End diff --

Can we construct a test case that checks this invariant?


Issue Time Tracking
---

Worklog Id: (was: 30504)
Time Spent: 4h 10m  (was: 4h)

> Crash from hostdb in PriorityQueueLess
> --
>
> Key: TS-4915
> URL: https://issues.apache.org/jira/browse/TS-4915
> Project: Traffic Server
>  Issue Type: Bug
>  Components: HostDB
>Reporter: Susan Hinrichs
>Priority: Blocker
> Fix For: 7.1.0
>
> Attachments: ts-4915.diff, ts-4915.diff
>
>  Time Spent: 4h 10m
>  Remaining Estimate: 0h
>
> Saw this while testing fix for TS-4813 with debug enabled.
> {code}
> (gdb) bt full
> #0  0x00547bfe in RefCountCacheHashEntry::operator< (this=0x1cc0880, 
> v2=...) at ../iocore/hostdb/P_RefCountCache.h:94
> No locals.
> #1  0x0054988d in 
> PriorityQueueLess::operator() (this=0x2b78a9a2587b, 
> a=@0x2b78f402af68, b=@0x2b78f402aa28)
> at ../lib/ts/PriorityQueue.h:41
> No locals.
> #2  0x00549785 in PriorityQueue PriorityQueueLess >::_bubble_up (this=0x1cb2990, 
> index=2) at ../lib/ts/PriorityQueue.h:191
> comp = {}
> parent = 0
> #3  0x006ecfcc in PriorityQueue PriorityQueueLess >::push (this=0x1cb2990, 
> entry=0x2b78f402af60) at ../../lib/ts/PriorityQueue.h:91
> len = 2
> #4  0x006ec206 in RefCountCachePartition::put 
> (this=0x1cb2900, key=6912554662447498853, item=0x2b78aee04f00, size=96, 
> expire_time=1475202356) at ./P_RefCountCache.h:210
> expiry_entry = 0x2b78f402af60
> __func__ = "put"
> val = 0x1cc0880
> #5  0x006eb3de in RefCountCache::put (this=0x18051e0, 
> key=6912554662447498853, item=0x2b78aee04f00, size=16, 
> expiry_time=1475202356) at ./P_RefCountCache.h:462
> No locals.
> #6  0x006e2d8e in HostDBContinuation::dnsEvent (this=0x2b7938020f00, 
> event=600, e=0x2b78ac009440) at HostDB.cc:1422
> is_rr = false
> old_rr_data = 0x0
> first_record = 0x2b78ac0094f8
> m = 0x1
> failed = false
> old_r = {m_ptr = 0x0}
> af = 2 '\002'
> s_size = 16
> rrsize = 0
> allocSize = 16
> r = 0x2b78aee04f00
> old_info = { = { = {_vptr.ForceVFPTToTop 
> = 0x7f3630}, m_refcount = 0}, iobuffer_index = 0, 
>   key = 47797242059264, app = {allotment = {application1 = 5326300, 
> application2 = 0}, http_data = {http_version = 4, 
>   pipeline_max = 59, keepalive_timeout = 17, fail_count = 81, 
> unused1 = 0, last_failure = 0}, rr = {offset = 5326300}}, data = {
> ip = {sa = {sa_family = 54488, sa_data = 
> "^\000\000\000\000\000\020\034$\274x+\000"}, sin = {sin_family = 54488, 
> sin_port = 94, 
> sin_addr = {s_addr = 0}, sin_zero = "\020\034$\274x+\000"}, 
> sin6 = {sin6_family = 54488, sin6_port = 94, sin6_flowinfo = 0, 
> sin6_addr = {__in6_u = {__u6_addr8 = 
> "\020\034$\274x+\000\000\030\036$\274\375\b\000", __u6_addr16 = {7184, 48164, 
> 11128, 
>   0, 7704, 48164, 2301, 0}, __u6_addr32 = {3156483088, 
> 11128, 3156483608, 2301}}}, sin6_scope_id = 3156478176}}, 
> hostname_offset = 6214872, srv = {srv_offset = 54488, srv_weight 
> = 94, srv_priority = 0, srv_port = 0, key = 3156483088}}, 
>   hostname_offset = 11128, ip_timestamp = 2845989456, 
> ip_timeout_interval = 11128, is_srv = 0, reverse_dns = 0, round_robin = 1, 
>   round_robin_elt = 0}
> valid_records = 0
> tip = {_family = 2, _addr = {_ip4 = 540420056, _ip6 = {__in6_u = 
> {__u6_addr8 = "\330'6 x+\000\000\360L\020\250x+\000", 
> __u6_addr16 = {10200, 8246, 11128, 0, 19696, 43024, 11128, 
> 0}, __u6_addr32 = {540420056, 11128, 2819640560, 11128}}}, 
> _byte = "\330'6 x+\000\000\360L\020\250x+\000", _u32 = 
> {540420056, 11128, 2819640560, 11128}, _u64 = {47794936489944, 
>   47797215710448}}}
> 

[jira] [Work logged] (TS-4915) Crash from hostdb in PriorityQueueLess

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-4915?focusedWorklogId=30499=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-30499
 ]

ASF GitHub Bot logged work on TS-4915:
--

Author: ASF GitHub Bot
Created on: 12/Oct/16 14:34
Start Date: 12/Oct/16 14:34
Worklog Time Spent: 10m 
  Work Description: Github user atsci commented on the issue:

https://github.com/apache/trafficserver/pull/1088
  
Linux build *successful*! See 
https://ci.trafficserver.apache.org/job/Github-Linux/871/ for details.
 



Issue Time Tracking
---

Worklog Id: (was: 30499)
Time Spent: 4h  (was: 3h 50m)

> Crash from hostdb in PriorityQueueLess
> --
>
> Key: TS-4915
> URL: https://issues.apache.org/jira/browse/TS-4915
> Project: Traffic Server
>  Issue Type: Bug
>  Components: HostDB
>Reporter: Susan Hinrichs
>Priority: Blocker
> Fix For: 7.1.0
>
> Attachments: ts-4915.diff, ts-4915.diff
>
>  Time Spent: 4h
>  Remaining Estimate: 0h
>
> Saw this while testing fix for TS-4813 with debug enabled.
> {code}
> (gdb) bt full
> #0  0x00547bfe in RefCountCacheHashEntry::operator< (this=0x1cc0880, 
> v2=...) at ../iocore/hostdb/P_RefCountCache.h:94
> No locals.
> #1  0x0054988d in 
> PriorityQueueLess::operator() (this=0x2b78a9a2587b, 
> a=@0x2b78f402af68, b=@0x2b78f402aa28)
> at ../lib/ts/PriorityQueue.h:41
> No locals.
> #2  0x00549785 in PriorityQueue PriorityQueueLess >::_bubble_up (this=0x1cb2990, 
> index=2) at ../lib/ts/PriorityQueue.h:191
> comp = {}
> parent = 0
> #3  0x006ecfcc in PriorityQueue PriorityQueueLess >::push (this=0x1cb2990, 
> entry=0x2b78f402af60) at ../../lib/ts/PriorityQueue.h:91
> len = 2
> #4  0x006ec206 in RefCountCachePartition::put 
> (this=0x1cb2900, key=6912554662447498853, item=0x2b78aee04f00, size=96, 
> expire_time=1475202356) at ./P_RefCountCache.h:210
> expiry_entry = 0x2b78f402af60
> __func__ = "put"
> val = 0x1cc0880
> #5  0x006eb3de in RefCountCache::put (this=0x18051e0, 
> key=6912554662447498853, item=0x2b78aee04f00, size=16, 
> expiry_time=1475202356) at ./P_RefCountCache.h:462
> No locals.
> #6  0x006e2d8e in HostDBContinuation::dnsEvent (this=0x2b7938020f00, 
> event=600, e=0x2b78ac009440) at HostDB.cc:1422
> is_rr = false
> old_rr_data = 0x0
> first_record = 0x2b78ac0094f8
> m = 0x1
> failed = false
> old_r = {m_ptr = 0x0}
> af = 2 '\002'
> s_size = 16
> rrsize = 0
> allocSize = 16
> r = 0x2b78aee04f00
> old_info = { = { = {_vptr.ForceVFPTToTop 
> = 0x7f3630}, m_refcount = 0}, iobuffer_index = 0, 
>   key = 47797242059264, app = {allotment = {application1 = 5326300, 
> application2 = 0}, http_data = {http_version = 4, 
>   pipeline_max = 59, keepalive_timeout = 17, fail_count = 81, 
> unused1 = 0, last_failure = 0}, rr = {offset = 5326300}}, data = {
> ip = {sa = {sa_family = 54488, sa_data = 
> "^\000\000\000\000\000\020\034$\274x+\000"}, sin = {sin_family = 54488, 
> sin_port = 94, 
> sin_addr = {s_addr = 0}, sin_zero = "\020\034$\274x+\000"}, 
> sin6 = {sin6_family = 54488, sin6_port = 94, sin6_flowinfo = 0, 
> sin6_addr = {__in6_u = {__u6_addr8 = 
> "\020\034$\274x+\000\000\030\036$\274\375\b\000", __u6_addr16 = {7184, 48164, 
> 11128, 
>   0, 7704, 48164, 2301, 0}, __u6_addr32 = {3156483088, 
> 11128, 3156483608, 2301}}}, sin6_scope_id = 3156478176}}, 
> hostname_offset = 6214872, srv = {srv_offset = 54488, srv_weight 
> = 94, srv_priority = 0, srv_port = 0, key = 3156483088}}, 
>   hostname_offset = 11128, ip_timestamp = 2845989456, 
> ip_timeout_interval = 11128, is_srv = 0, reverse_dns = 0, round_robin = 1, 
>   round_robin_elt = 0}
> valid_records = 0
> tip = {_family = 2, _addr = {_ip4 = 540420056, _ip6 = {__in6_u = 
> {__u6_addr8 = "\330'6 x+\000\000\360L\020\250x+\000", 
> __u6_addr16 = {10200, 8246, 11128, 0, 19696, 43024, 11128, 
> 0}, __u6_addr32 = {540420056, 11128, 2819640560, 11128}}}, 
> _byte = "\330'6 x+\000\000\360L\020\250x+\000", _u32 = 
> {540420056, 11128, 2819640560, 11128}, _u64 = {47794936489944, 
>   47797215710448}}}
> ttl_seconds = 132
> aname = 0x2b7938021000 "fbmm1.zenfs.com"
> offset = 96
> thread = 0x2b78a8101010
> __func__ = "dnsEvent"
> #7  0x005145dc in Continuation::handleEvent (this=0x2b7938020f00, 
> 

[jira] [Work logged] (TS-4915) Crash from hostdb in PriorityQueueLess

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-4915?focusedWorklogId=30498=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-30498
 ]

ASF GitHub Bot logged work on TS-4915:
--

Author: ASF GitHub Bot
Created on: 12/Oct/16 14:30
Start Date: 12/Oct/16 14:30
Worklog Time Spent: 10m 
  Work Description: Github user atsci commented on the issue:

https://github.com/apache/trafficserver/pull/1088
  
FreeBSD build *successful*! See 
https://ci.trafficserver.apache.org/job/Github-FreeBSD/979/ for details.
 



Issue Time Tracking
---

Worklog Id: (was: 30498)
Time Spent: 3h 50m  (was: 3h 40m)

> Crash from hostdb in PriorityQueueLess
> --
>
> Key: TS-4915
> URL: https://issues.apache.org/jira/browse/TS-4915
> Project: Traffic Server
>  Issue Type: Bug
>  Components: HostDB
>Reporter: Susan Hinrichs
>Priority: Blocker
> Fix For: 7.1.0
>
> Attachments: ts-4915.diff, ts-4915.diff
>
>  Time Spent: 3h 50m
>  Remaining Estimate: 0h
>
> Saw this while testing fix for TS-4813 with debug enabled.
> {code}
> (gdb) bt full
> #0  0x00547bfe in RefCountCacheHashEntry::operator< (this=0x1cc0880, 
> v2=...) at ../iocore/hostdb/P_RefCountCache.h:94
> No locals.
> #1  0x0054988d in 
> PriorityQueueLess::operator() (this=0x2b78a9a2587b, 
> a=@0x2b78f402af68, b=@0x2b78f402aa28)
> at ../lib/ts/PriorityQueue.h:41
> No locals.
> #2  0x00549785 in PriorityQueue PriorityQueueLess >::_bubble_up (this=0x1cb2990, 
> index=2) at ../lib/ts/PriorityQueue.h:191
> comp = {}
> parent = 0
> #3  0x006ecfcc in PriorityQueue PriorityQueueLess >::push (this=0x1cb2990, 
> entry=0x2b78f402af60) at ../../lib/ts/PriorityQueue.h:91
> len = 2
> #4  0x006ec206 in RefCountCachePartition::put 
> (this=0x1cb2900, key=6912554662447498853, item=0x2b78aee04f00, size=96, 
> expire_time=1475202356) at ./P_RefCountCache.h:210
> expiry_entry = 0x2b78f402af60
> __func__ = "put"
> val = 0x1cc0880
> #5  0x006eb3de in RefCountCache::put (this=0x18051e0, 
> key=6912554662447498853, item=0x2b78aee04f00, size=16, 
> expiry_time=1475202356) at ./P_RefCountCache.h:462
> No locals.
> #6  0x006e2d8e in HostDBContinuation::dnsEvent (this=0x2b7938020f00, 
> event=600, e=0x2b78ac009440) at HostDB.cc:1422
> is_rr = false
> old_rr_data = 0x0
> first_record = 0x2b78ac0094f8
> m = 0x1
> failed = false
> old_r = {m_ptr = 0x0}
> af = 2 '\002'
> s_size = 16
> rrsize = 0
> allocSize = 16
> r = 0x2b78aee04f00
> old_info = { = { = {_vptr.ForceVFPTToTop 
> = 0x7f3630}, m_refcount = 0}, iobuffer_index = 0, 
>   key = 47797242059264, app = {allotment = {application1 = 5326300, 
> application2 = 0}, http_data = {http_version = 4, 
>   pipeline_max = 59, keepalive_timeout = 17, fail_count = 81, 
> unused1 = 0, last_failure = 0}, rr = {offset = 5326300}}, data = {
> ip = {sa = {sa_family = 54488, sa_data = 
> "^\000\000\000\000\000\020\034$\274x+\000"}, sin = {sin_family = 54488, 
> sin_port = 94, 
> sin_addr = {s_addr = 0}, sin_zero = "\020\034$\274x+\000"}, 
> sin6 = {sin6_family = 54488, sin6_port = 94, sin6_flowinfo = 0, 
> sin6_addr = {__in6_u = {__u6_addr8 = 
> "\020\034$\274x+\000\000\030\036$\274\375\b\000", __u6_addr16 = {7184, 48164, 
> 11128, 
>   0, 7704, 48164, 2301, 0}, __u6_addr32 = {3156483088, 
> 11128, 3156483608, 2301}}}, sin6_scope_id = 3156478176}}, 
> hostname_offset = 6214872, srv = {srv_offset = 54488, srv_weight 
> = 94, srv_priority = 0, srv_port = 0, key = 3156483088}}, 
>   hostname_offset = 11128, ip_timestamp = 2845989456, 
> ip_timeout_interval = 11128, is_srv = 0, reverse_dns = 0, round_robin = 1, 
>   round_robin_elt = 0}
> valid_records = 0
> tip = {_family = 2, _addr = {_ip4 = 540420056, _ip6 = {__in6_u = 
> {__u6_addr8 = "\330'6 x+\000\000\360L\020\250x+\000", 
> __u6_addr16 = {10200, 8246, 11128, 0, 19696, 43024, 11128, 
> 0}, __u6_addr32 = {540420056, 11128, 2819640560, 11128}}}, 
> _byte = "\330'6 x+\000\000\360L\020\250x+\000", _u32 = 
> {540420056, 11128, 2819640560, 11128}, _u64 = {47794936489944, 
>   47797215710448}}}
> ttl_seconds = 132
> aname = 0x2b7938021000 "fbmm1.zenfs.com"
> offset = 96
> thread = 0x2b78a8101010
> __func__ = "dnsEvent"
> #7  0x005145dc in Continuation::handleEvent 

[jira] [Work logged] (TS-4915) Crash from hostdb in PriorityQueueLess

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-4915?focusedWorklogId=30497=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-30497
 ]

ASF GitHub Bot logged work on TS-4915:
--

Author: ASF GitHub Bot
Created on: 12/Oct/16 14:22
Start Date: 12/Oct/16 14:22
Worklog Time Spent: 10m 
  Work Description: Github user shinrich commented on the issue:

https://github.com/apache/trafficserver/pull/1088
  
@masaori335 ok, that makes sense.  I reverted my hack in the _bubble_down.  
Hopefully that fixes that transient TSConfigSyntax build failure that occurred 
on the Linux build.

If all are agreed, I think we should commit this fix.  My build was running 
without crash in production for over 24 hours until it re-installed last night.


Issue Time Tracking
---

Worklog Id: (was: 30497)
Time Spent: 3h 40m  (was: 3.5h)

> Crash from hostdb in PriorityQueueLess
> --
>
> Key: TS-4915
> URL: https://issues.apache.org/jira/browse/TS-4915
> Project: Traffic Server
>  Issue Type: Bug
>  Components: HostDB
>Reporter: Susan Hinrichs
>Priority: Blocker
> Fix For: 7.1.0
>
> Attachments: ts-4915.diff, ts-4915.diff
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> Saw this while testing fix for TS-4813 with debug enabled.
> {code}
> (gdb) bt full
> #0  0x00547bfe in RefCountCacheHashEntry::operator< (this=0x1cc0880, 
> v2=...) at ../iocore/hostdb/P_RefCountCache.h:94
> No locals.
> #1  0x0054988d in 
> PriorityQueueLess::operator() (this=0x2b78a9a2587b, 
> a=@0x2b78f402af68, b=@0x2b78f402aa28)
> at ../lib/ts/PriorityQueue.h:41
> No locals.
> #2  0x00549785 in PriorityQueue PriorityQueueLess >::_bubble_up (this=0x1cb2990, 
> index=2) at ../lib/ts/PriorityQueue.h:191
> comp = {}
> parent = 0
> #3  0x006ecfcc in PriorityQueue PriorityQueueLess >::push (this=0x1cb2990, 
> entry=0x2b78f402af60) at ../../lib/ts/PriorityQueue.h:91
> len = 2
> #4  0x006ec206 in RefCountCachePartition::put 
> (this=0x1cb2900, key=6912554662447498853, item=0x2b78aee04f00, size=96, 
> expire_time=1475202356) at ./P_RefCountCache.h:210
> expiry_entry = 0x2b78f402af60
> __func__ = "put"
> val = 0x1cc0880
> #5  0x006eb3de in RefCountCache::put (this=0x18051e0, 
> key=6912554662447498853, item=0x2b78aee04f00, size=16, 
> expiry_time=1475202356) at ./P_RefCountCache.h:462
> No locals.
> #6  0x006e2d8e in HostDBContinuation::dnsEvent (this=0x2b7938020f00, 
> event=600, e=0x2b78ac009440) at HostDB.cc:1422
> is_rr = false
> old_rr_data = 0x0
> first_record = 0x2b78ac0094f8
> m = 0x1
> failed = false
> old_r = {m_ptr = 0x0}
> af = 2 '\002'
> s_size = 16
> rrsize = 0
> allocSize = 16
> r = 0x2b78aee04f00
> old_info = { = { = {_vptr.ForceVFPTToTop 
> = 0x7f3630}, m_refcount = 0}, iobuffer_index = 0, 
>   key = 47797242059264, app = {allotment = {application1 = 5326300, 
> application2 = 0}, http_data = {http_version = 4, 
>   pipeline_max = 59, keepalive_timeout = 17, fail_count = 81, 
> unused1 = 0, last_failure = 0}, rr = {offset = 5326300}}, data = {
> ip = {sa = {sa_family = 54488, sa_data = 
> "^\000\000\000\000\000\020\034$\274x+\000"}, sin = {sin_family = 54488, 
> sin_port = 94, 
> sin_addr = {s_addr = 0}, sin_zero = "\020\034$\274x+\000"}, 
> sin6 = {sin6_family = 54488, sin6_port = 94, sin6_flowinfo = 0, 
> sin6_addr = {__in6_u = {__u6_addr8 = 
> "\020\034$\274x+\000\000\030\036$\274\375\b\000", __u6_addr16 = {7184, 48164, 
> 11128, 
>   0, 7704, 48164, 2301, 0}, __u6_addr32 = {3156483088, 
> 11128, 3156483608, 2301}}}, sin6_scope_id = 3156478176}}, 
> hostname_offset = 6214872, srv = {srv_offset = 54488, srv_weight 
> = 94, srv_priority = 0, srv_port = 0, key = 3156483088}}, 
>   hostname_offset = 11128, ip_timestamp = 2845989456, 
> ip_timeout_interval = 11128, is_srv = 0, reverse_dns = 0, round_robin = 1, 
>   round_robin_elt = 0}
> valid_records = 0
> tip = {_family = 2, _addr = {_ip4 = 540420056, _ip6 = {__in6_u = 
> {__u6_addr8 = "\330'6 x+\000\000\360L\020\250x+\000", 
> __u6_addr16 = {10200, 8246, 11128, 0, 19696, 43024, 11128, 
> 0}, __u6_addr32 = {540420056, 11128, 2819640560, 11128}}}, 
> _byte = "\330'6 x+\000\000\360L\020\250x+\000", _u32 = 
> {540420056, 11128, 2819640560, 11128}, _u64 = {47794936489944, 
>   47797215710448}}}
>  

[jira] [Work logged] (TS-4915) Crash from hostdb in PriorityQueueLess

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-4915?focusedWorklogId=30480=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-30480
 ]

ASF GitHub Bot logged work on TS-4915:
--

Author: ASF GitHub Bot
Created on: 12/Oct/16 05:32
Start Date: 12/Oct/16 05:32
Worklog Time Spent: 10m 
  Work Description: Github user masaori335 commented on the issue:

https://github.com/apache/trafficserver/pull/1088
  
@shinrich It looks working correctly. What is problem with the vector 
`W(10), Z(40), and Y(30)` in that case?
This PriorityQueue is implemented as BinaryHeap in an array. So left child 
node(`Z(40)`) could be larger than right child node(`Y(30)`). And we don't need 
to sort the array.


Issue Time Tracking
---

Worklog Id: (was: 30480)
Time Spent: 3.5h  (was: 3h 20m)

> Crash from hostdb in PriorityQueueLess
> --
>
> Key: TS-4915
> URL: https://issues.apache.org/jira/browse/TS-4915
> Project: Traffic Server
>  Issue Type: Bug
>  Components: HostDB
>Reporter: Susan Hinrichs
>Priority: Blocker
> Fix For: 7.1.0
>
> Attachments: ts-4915.diff, ts-4915.diff
>
>  Time Spent: 3.5h
>  Remaining Estimate: 0h
>
> Saw this while testing fix for TS-4813 with debug enabled.
> {code}
> (gdb) bt full
> #0  0x00547bfe in RefCountCacheHashEntry::operator< (this=0x1cc0880, 
> v2=...) at ../iocore/hostdb/P_RefCountCache.h:94
> No locals.
> #1  0x0054988d in 
> PriorityQueueLess::operator() (this=0x2b78a9a2587b, 
> a=@0x2b78f402af68, b=@0x2b78f402aa28)
> at ../lib/ts/PriorityQueue.h:41
> No locals.
> #2  0x00549785 in PriorityQueue PriorityQueueLess >::_bubble_up (this=0x1cb2990, 
> index=2) at ../lib/ts/PriorityQueue.h:191
> comp = {}
> parent = 0
> #3  0x006ecfcc in PriorityQueue PriorityQueueLess >::push (this=0x1cb2990, 
> entry=0x2b78f402af60) at ../../lib/ts/PriorityQueue.h:91
> len = 2
> #4  0x006ec206 in RefCountCachePartition::put 
> (this=0x1cb2900, key=6912554662447498853, item=0x2b78aee04f00, size=96, 
> expire_time=1475202356) at ./P_RefCountCache.h:210
> expiry_entry = 0x2b78f402af60
> __func__ = "put"
> val = 0x1cc0880
> #5  0x006eb3de in RefCountCache::put (this=0x18051e0, 
> key=6912554662447498853, item=0x2b78aee04f00, size=16, 
> expiry_time=1475202356) at ./P_RefCountCache.h:462
> No locals.
> #6  0x006e2d8e in HostDBContinuation::dnsEvent (this=0x2b7938020f00, 
> event=600, e=0x2b78ac009440) at HostDB.cc:1422
> is_rr = false
> old_rr_data = 0x0
> first_record = 0x2b78ac0094f8
> m = 0x1
> failed = false
> old_r = {m_ptr = 0x0}
> af = 2 '\002'
> s_size = 16
> rrsize = 0
> allocSize = 16
> r = 0x2b78aee04f00
> old_info = { = { = {_vptr.ForceVFPTToTop 
> = 0x7f3630}, m_refcount = 0}, iobuffer_index = 0, 
>   key = 47797242059264, app = {allotment = {application1 = 5326300, 
> application2 = 0}, http_data = {http_version = 4, 
>   pipeline_max = 59, keepalive_timeout = 17, fail_count = 81, 
> unused1 = 0, last_failure = 0}, rr = {offset = 5326300}}, data = {
> ip = {sa = {sa_family = 54488, sa_data = 
> "^\000\000\000\000\000\020\034$\274x+\000"}, sin = {sin_family = 54488, 
> sin_port = 94, 
> sin_addr = {s_addr = 0}, sin_zero = "\020\034$\274x+\000"}, 
> sin6 = {sin6_family = 54488, sin6_port = 94, sin6_flowinfo = 0, 
> sin6_addr = {__in6_u = {__u6_addr8 = 
> "\020\034$\274x+\000\000\030\036$\274\375\b\000", __u6_addr16 = {7184, 48164, 
> 11128, 
>   0, 7704, 48164, 2301, 0}, __u6_addr32 = {3156483088, 
> 11128, 3156483608, 2301}}}, sin6_scope_id = 3156478176}}, 
> hostname_offset = 6214872, srv = {srv_offset = 54488, srv_weight 
> = 94, srv_priority = 0, srv_port = 0, key = 3156483088}}, 
>   hostname_offset = 11128, ip_timestamp = 2845989456, 
> ip_timeout_interval = 11128, is_srv = 0, reverse_dns = 0, round_robin = 1, 
>   round_robin_elt = 0}
> valid_records = 0
> tip = {_family = 2, _addr = {_ip4 = 540420056, _ip6 = {__in6_u = 
> {__u6_addr8 = "\330'6 x+\000\000\360L\020\250x+\000", 
> __u6_addr16 = {10200, 8246, 11128, 0, 19696, 43024, 11128, 
> 0}, __u6_addr32 = {540420056, 11128, 2819640560, 11128}}}, 
> _byte = "\330'6 x+\000\000\360L\020\250x+\000", _u32 = 
> {540420056, 11128, 2819640560, 11128}, _u64 = {47794936489944, 
>   47797215710448}}}
> ttl_seconds = 132
> aname = 

[jira] [Work logged] (TS-4915) Crash from hostdb in PriorityQueueLess

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-4915?focusedWorklogId=30446=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-30446
 ]

ASF GitHub Bot logged work on TS-4915:
--

Author: ASF GitHub Bot
Created on: 11/Oct/16 22:24
Start Date: 11/Oct/16 22:24
Worklog Time Spent: 10m 
  Work Description: Github user atsci commented on the issue:

https://github.com/apache/trafficserver/pull/1088
  
FreeBSD build *successful*! See 
https://ci.trafficserver.apache.org/job/Github-FreeBSD/973/ for details.
 



Issue Time Tracking
---

Worklog Id: (was: 30446)
Time Spent: 3h 20m  (was: 3h 10m)

> Crash from hostdb in PriorityQueueLess
> --
>
> Key: TS-4915
> URL: https://issues.apache.org/jira/browse/TS-4915
> Project: Traffic Server
>  Issue Type: Bug
>  Components: HostDB
>Reporter: Susan Hinrichs
>Priority: Blocker
> Fix For: 7.1.0
>
> Attachments: ts-4915.diff, ts-4915.diff
>
>  Time Spent: 3h 20m
>  Remaining Estimate: 0h
>
> Saw this while testing fix for TS-4813 with debug enabled.
> {code}
> (gdb) bt full
> #0  0x00547bfe in RefCountCacheHashEntry::operator< (this=0x1cc0880, 
> v2=...) at ../iocore/hostdb/P_RefCountCache.h:94
> No locals.
> #1  0x0054988d in 
> PriorityQueueLess::operator() (this=0x2b78a9a2587b, 
> a=@0x2b78f402af68, b=@0x2b78f402aa28)
> at ../lib/ts/PriorityQueue.h:41
> No locals.
> #2  0x00549785 in PriorityQueue PriorityQueueLess >::_bubble_up (this=0x1cb2990, 
> index=2) at ../lib/ts/PriorityQueue.h:191
> comp = {}
> parent = 0
> #3  0x006ecfcc in PriorityQueue PriorityQueueLess >::push (this=0x1cb2990, 
> entry=0x2b78f402af60) at ../../lib/ts/PriorityQueue.h:91
> len = 2
> #4  0x006ec206 in RefCountCachePartition::put 
> (this=0x1cb2900, key=6912554662447498853, item=0x2b78aee04f00, size=96, 
> expire_time=1475202356) at ./P_RefCountCache.h:210
> expiry_entry = 0x2b78f402af60
> __func__ = "put"
> val = 0x1cc0880
> #5  0x006eb3de in RefCountCache::put (this=0x18051e0, 
> key=6912554662447498853, item=0x2b78aee04f00, size=16, 
> expiry_time=1475202356) at ./P_RefCountCache.h:462
> No locals.
> #6  0x006e2d8e in HostDBContinuation::dnsEvent (this=0x2b7938020f00, 
> event=600, e=0x2b78ac009440) at HostDB.cc:1422
> is_rr = false
> old_rr_data = 0x0
> first_record = 0x2b78ac0094f8
> m = 0x1
> failed = false
> old_r = {m_ptr = 0x0}
> af = 2 '\002'
> s_size = 16
> rrsize = 0
> allocSize = 16
> r = 0x2b78aee04f00
> old_info = { = { = {_vptr.ForceVFPTToTop 
> = 0x7f3630}, m_refcount = 0}, iobuffer_index = 0, 
>   key = 47797242059264, app = {allotment = {application1 = 5326300, 
> application2 = 0}, http_data = {http_version = 4, 
>   pipeline_max = 59, keepalive_timeout = 17, fail_count = 81, 
> unused1 = 0, last_failure = 0}, rr = {offset = 5326300}}, data = {
> ip = {sa = {sa_family = 54488, sa_data = 
> "^\000\000\000\000\000\020\034$\274x+\000"}, sin = {sin_family = 54488, 
> sin_port = 94, 
> sin_addr = {s_addr = 0}, sin_zero = "\020\034$\274x+\000"}, 
> sin6 = {sin6_family = 54488, sin6_port = 94, sin6_flowinfo = 0, 
> sin6_addr = {__in6_u = {__u6_addr8 = 
> "\020\034$\274x+\000\000\030\036$\274\375\b\000", __u6_addr16 = {7184, 48164, 
> 11128, 
>   0, 7704, 48164, 2301, 0}, __u6_addr32 = {3156483088, 
> 11128, 3156483608, 2301}}}, sin6_scope_id = 3156478176}}, 
> hostname_offset = 6214872, srv = {srv_offset = 54488, srv_weight 
> = 94, srv_priority = 0, srv_port = 0, key = 3156483088}}, 
>   hostname_offset = 11128, ip_timestamp = 2845989456, 
> ip_timeout_interval = 11128, is_srv = 0, reverse_dns = 0, round_robin = 1, 
>   round_robin_elt = 0}
> valid_records = 0
> tip = {_family = 2, _addr = {_ip4 = 540420056, _ip6 = {__in6_u = 
> {__u6_addr8 = "\330'6 x+\000\000\360L\020\250x+\000", 
> __u6_addr16 = {10200, 8246, 11128, 0, 19696, 43024, 11128, 
> 0}, __u6_addr32 = {540420056, 11128, 2819640560, 11128}}}, 
> _byte = "\330'6 x+\000\000\360L\020\250x+\000", _u32 = 
> {540420056, 11128, 2819640560, 11128}, _u64 = {47794936489944, 
>   47797215710448}}}
> ttl_seconds = 132
> aname = 0x2b7938021000 "fbmm1.zenfs.com"
> offset = 96
> thread = 0x2b78a8101010
> __func__ = "dnsEvent"
> #7  0x005145dc in Continuation::handleEvent 

[jira] [Work logged] (TS-4915) Crash from hostdb in PriorityQueueLess

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-4915?focusedWorklogId=30443=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-30443
 ]

ASF GitHub Bot logged work on TS-4915:
--

Author: ASF GitHub Bot
Created on: 11/Oct/16 22:14
Start Date: 11/Oct/16 22:14
Worklog Time Spent: 10m 
  Work Description: Github user atsci commented on the issue:

https://github.com/apache/trafficserver/pull/1088
  
Linux build *failed*! See 
https://ci.trafficserver.apache.org/job/Github-Linux/865/ for details.
 



Issue Time Tracking
---

Worklog Id: (was: 30443)
Time Spent: 3h 10m  (was: 3h)

> Crash from hostdb in PriorityQueueLess
> --
>
> Key: TS-4915
> URL: https://issues.apache.org/jira/browse/TS-4915
> Project: Traffic Server
>  Issue Type: Bug
>  Components: HostDB
>Reporter: Susan Hinrichs
>Priority: Blocker
> Fix For: 7.1.0
>
> Attachments: ts-4915.diff, ts-4915.diff
>
>  Time Spent: 3h 10m
>  Remaining Estimate: 0h
>
> Saw this while testing fix for TS-4813 with debug enabled.
> {code}
> (gdb) bt full
> #0  0x00547bfe in RefCountCacheHashEntry::operator< (this=0x1cc0880, 
> v2=...) at ../iocore/hostdb/P_RefCountCache.h:94
> No locals.
> #1  0x0054988d in 
> PriorityQueueLess::operator() (this=0x2b78a9a2587b, 
> a=@0x2b78f402af68, b=@0x2b78f402aa28)
> at ../lib/ts/PriorityQueue.h:41
> No locals.
> #2  0x00549785 in PriorityQueue PriorityQueueLess >::_bubble_up (this=0x1cb2990, 
> index=2) at ../lib/ts/PriorityQueue.h:191
> comp = {}
> parent = 0
> #3  0x006ecfcc in PriorityQueue PriorityQueueLess >::push (this=0x1cb2990, 
> entry=0x2b78f402af60) at ../../lib/ts/PriorityQueue.h:91
> len = 2
> #4  0x006ec206 in RefCountCachePartition::put 
> (this=0x1cb2900, key=6912554662447498853, item=0x2b78aee04f00, size=96, 
> expire_time=1475202356) at ./P_RefCountCache.h:210
> expiry_entry = 0x2b78f402af60
> __func__ = "put"
> val = 0x1cc0880
> #5  0x006eb3de in RefCountCache::put (this=0x18051e0, 
> key=6912554662447498853, item=0x2b78aee04f00, size=16, 
> expiry_time=1475202356) at ./P_RefCountCache.h:462
> No locals.
> #6  0x006e2d8e in HostDBContinuation::dnsEvent (this=0x2b7938020f00, 
> event=600, e=0x2b78ac009440) at HostDB.cc:1422
> is_rr = false
> old_rr_data = 0x0
> first_record = 0x2b78ac0094f8
> m = 0x1
> failed = false
> old_r = {m_ptr = 0x0}
> af = 2 '\002'
> s_size = 16
> rrsize = 0
> allocSize = 16
> r = 0x2b78aee04f00
> old_info = { = { = {_vptr.ForceVFPTToTop 
> = 0x7f3630}, m_refcount = 0}, iobuffer_index = 0, 
>   key = 47797242059264, app = {allotment = {application1 = 5326300, 
> application2 = 0}, http_data = {http_version = 4, 
>   pipeline_max = 59, keepalive_timeout = 17, fail_count = 81, 
> unused1 = 0, last_failure = 0}, rr = {offset = 5326300}}, data = {
> ip = {sa = {sa_family = 54488, sa_data = 
> "^\000\000\000\000\000\020\034$\274x+\000"}, sin = {sin_family = 54488, 
> sin_port = 94, 
> sin_addr = {s_addr = 0}, sin_zero = "\020\034$\274x+\000"}, 
> sin6 = {sin6_family = 54488, sin6_port = 94, sin6_flowinfo = 0, 
> sin6_addr = {__in6_u = {__u6_addr8 = 
> "\020\034$\274x+\000\000\030\036$\274\375\b\000", __u6_addr16 = {7184, 48164, 
> 11128, 
>   0, 7704, 48164, 2301, 0}, __u6_addr32 = {3156483088, 
> 11128, 3156483608, 2301}}}, sin6_scope_id = 3156478176}}, 
> hostname_offset = 6214872, srv = {srv_offset = 54488, srv_weight 
> = 94, srv_priority = 0, srv_port = 0, key = 3156483088}}, 
>   hostname_offset = 11128, ip_timestamp = 2845989456, 
> ip_timeout_interval = 11128, is_srv = 0, reverse_dns = 0, round_robin = 1, 
>   round_robin_elt = 0}
> valid_records = 0
> tip = {_family = 2, _addr = {_ip4 = 540420056, _ip6 = {__in6_u = 
> {__u6_addr8 = "\330'6 x+\000\000\360L\020\250x+\000", 
> __u6_addr16 = {10200, 8246, 11128, 0, 19696, 43024, 11128, 
> 0}, __u6_addr32 = {540420056, 11128, 2819640560, 11128}}}, 
> _byte = "\330'6 x+\000\000\360L\020\250x+\000", _u32 = 
> {540420056, 11128, 2819640560, 11128}, _u64 = {47794936489944, 
>   47797215710448}}}
> ttl_seconds = 132
> aname = 0x2b7938021000 "fbmm1.zenfs.com"
> offset = 96
> thread = 0x2b78a8101010
> __func__ = "dnsEvent"
> #7  0x005145dc in Continuation::handleEvent (this=0x2b7938020f00, 
> 

[jira] [Work logged] (TS-4915) Crash from hostdb in PriorityQueueLess

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-4915?focusedWorklogId=30440=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-30440
 ]

ASF GitHub Bot logged work on TS-4915:
--

Author: ASF GitHub Bot
Created on: 11/Oct/16 22:10
Start Date: 11/Oct/16 22:10
Worklog Time Spent: 10m 
  Work Description: Github user shinrich commented on the issue:

https://github.com/apache/trafficserver/pull/1088
  
Bleah.  Clang-formatted.  The fix has been running without crash for nearly 
24 hours on my prod box.


Issue Time Tracking
---

Worklog Id: (was: 30440)
Time Spent: 3h  (was: 2h 50m)

> Crash from hostdb in PriorityQueueLess
> --
>
> Key: TS-4915
> URL: https://issues.apache.org/jira/browse/TS-4915
> Project: Traffic Server
>  Issue Type: Bug
>  Components: HostDB
>Reporter: Susan Hinrichs
>Priority: Blocker
> Fix For: 7.1.0
>
> Attachments: ts-4915.diff, ts-4915.diff
>
>  Time Spent: 3h
>  Remaining Estimate: 0h
>
> Saw this while testing fix for TS-4813 with debug enabled.
> {code}
> (gdb) bt full
> #0  0x00547bfe in RefCountCacheHashEntry::operator< (this=0x1cc0880, 
> v2=...) at ../iocore/hostdb/P_RefCountCache.h:94
> No locals.
> #1  0x0054988d in 
> PriorityQueueLess::operator() (this=0x2b78a9a2587b, 
> a=@0x2b78f402af68, b=@0x2b78f402aa28)
> at ../lib/ts/PriorityQueue.h:41
> No locals.
> #2  0x00549785 in PriorityQueue PriorityQueueLess >::_bubble_up (this=0x1cb2990, 
> index=2) at ../lib/ts/PriorityQueue.h:191
> comp = {}
> parent = 0
> #3  0x006ecfcc in PriorityQueue PriorityQueueLess >::push (this=0x1cb2990, 
> entry=0x2b78f402af60) at ../../lib/ts/PriorityQueue.h:91
> len = 2
> #4  0x006ec206 in RefCountCachePartition::put 
> (this=0x1cb2900, key=6912554662447498853, item=0x2b78aee04f00, size=96, 
> expire_time=1475202356) at ./P_RefCountCache.h:210
> expiry_entry = 0x2b78f402af60
> __func__ = "put"
> val = 0x1cc0880
> #5  0x006eb3de in RefCountCache::put (this=0x18051e0, 
> key=6912554662447498853, item=0x2b78aee04f00, size=16, 
> expiry_time=1475202356) at ./P_RefCountCache.h:462
> No locals.
> #6  0x006e2d8e in HostDBContinuation::dnsEvent (this=0x2b7938020f00, 
> event=600, e=0x2b78ac009440) at HostDB.cc:1422
> is_rr = false
> old_rr_data = 0x0
> first_record = 0x2b78ac0094f8
> m = 0x1
> failed = false
> old_r = {m_ptr = 0x0}
> af = 2 '\002'
> s_size = 16
> rrsize = 0
> allocSize = 16
> r = 0x2b78aee04f00
> old_info = { = { = {_vptr.ForceVFPTToTop 
> = 0x7f3630}, m_refcount = 0}, iobuffer_index = 0, 
>   key = 47797242059264, app = {allotment = {application1 = 5326300, 
> application2 = 0}, http_data = {http_version = 4, 
>   pipeline_max = 59, keepalive_timeout = 17, fail_count = 81, 
> unused1 = 0, last_failure = 0}, rr = {offset = 5326300}}, data = {
> ip = {sa = {sa_family = 54488, sa_data = 
> "^\000\000\000\000\000\020\034$\274x+\000"}, sin = {sin_family = 54488, 
> sin_port = 94, 
> sin_addr = {s_addr = 0}, sin_zero = "\020\034$\274x+\000"}, 
> sin6 = {sin6_family = 54488, sin6_port = 94, sin6_flowinfo = 0, 
> sin6_addr = {__in6_u = {__u6_addr8 = 
> "\020\034$\274x+\000\000\030\036$\274\375\b\000", __u6_addr16 = {7184, 48164, 
> 11128, 
>   0, 7704, 48164, 2301, 0}, __u6_addr32 = {3156483088, 
> 11128, 3156483608, 2301}}}, sin6_scope_id = 3156478176}}, 
> hostname_offset = 6214872, srv = {srv_offset = 54488, srv_weight 
> = 94, srv_priority = 0, srv_port = 0, key = 3156483088}}, 
>   hostname_offset = 11128, ip_timestamp = 2845989456, 
> ip_timeout_interval = 11128, is_srv = 0, reverse_dns = 0, round_robin = 1, 
>   round_robin_elt = 0}
> valid_records = 0
> tip = {_family = 2, _addr = {_ip4 = 540420056, _ip6 = {__in6_u = 
> {__u6_addr8 = "\330'6 x+\000\000\360L\020\250x+\000", 
> __u6_addr16 = {10200, 8246, 11128, 0, 19696, 43024, 11128, 
> 0}, __u6_addr32 = {540420056, 11128, 2819640560, 11128}}}, 
> _byte = "\330'6 x+\000\000\360L\020\250x+\000", _u32 = 
> {540420056, 11128, 2819640560, 11128}, _u64 = {47794936489944, 
>   47797215710448}}}
> ttl_seconds = 132
> aname = 0x2b7938021000 "fbmm1.zenfs.com"
> offset = 96
> thread = 0x2b78a8101010
> __func__ = "dnsEvent"
> #7  0x005145dc in Continuation::handleEvent (this=0x2b7938020f00, 
> 

[jira] [Work logged] (TS-4915) Crash from hostdb in PriorityQueueLess

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-4915?focusedWorklogId=30432=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-30432
 ]

ASF GitHub Bot logged work on TS-4915:
--

Author: ASF GitHub Bot
Created on: 11/Oct/16 20:47
Start Date: 11/Oct/16 20:47
Worklog Time Spent: 10m 
  Work Description: Github user zwoop commented on the issue:

https://github.com/apache/trafficserver/pull/1088
  
Clang-format ...


Issue Time Tracking
---

Worklog Id: (was: 30432)
Time Spent: 2h 50m  (was: 2h 40m)

> Crash from hostdb in PriorityQueueLess
> --
>
> Key: TS-4915
> URL: https://issues.apache.org/jira/browse/TS-4915
> Project: Traffic Server
>  Issue Type: Bug
>  Components: HostDB
>Reporter: Susan Hinrichs
>Priority: Blocker
> Fix For: 7.1.0
>
> Attachments: ts-4915.diff, ts-4915.diff
>
>  Time Spent: 2h 50m
>  Remaining Estimate: 0h
>
> Saw this while testing fix for TS-4813 with debug enabled.
> {code}
> (gdb) bt full
> #0  0x00547bfe in RefCountCacheHashEntry::operator< (this=0x1cc0880, 
> v2=...) at ../iocore/hostdb/P_RefCountCache.h:94
> No locals.
> #1  0x0054988d in 
> PriorityQueueLess::operator() (this=0x2b78a9a2587b, 
> a=@0x2b78f402af68, b=@0x2b78f402aa28)
> at ../lib/ts/PriorityQueue.h:41
> No locals.
> #2  0x00549785 in PriorityQueue PriorityQueueLess >::_bubble_up (this=0x1cb2990, 
> index=2) at ../lib/ts/PriorityQueue.h:191
> comp = {}
> parent = 0
> #3  0x006ecfcc in PriorityQueue PriorityQueueLess >::push (this=0x1cb2990, 
> entry=0x2b78f402af60) at ../../lib/ts/PriorityQueue.h:91
> len = 2
> #4  0x006ec206 in RefCountCachePartition::put 
> (this=0x1cb2900, key=6912554662447498853, item=0x2b78aee04f00, size=96, 
> expire_time=1475202356) at ./P_RefCountCache.h:210
> expiry_entry = 0x2b78f402af60
> __func__ = "put"
> val = 0x1cc0880
> #5  0x006eb3de in RefCountCache::put (this=0x18051e0, 
> key=6912554662447498853, item=0x2b78aee04f00, size=16, 
> expiry_time=1475202356) at ./P_RefCountCache.h:462
> No locals.
> #6  0x006e2d8e in HostDBContinuation::dnsEvent (this=0x2b7938020f00, 
> event=600, e=0x2b78ac009440) at HostDB.cc:1422
> is_rr = false
> old_rr_data = 0x0
> first_record = 0x2b78ac0094f8
> m = 0x1
> failed = false
> old_r = {m_ptr = 0x0}
> af = 2 '\002'
> s_size = 16
> rrsize = 0
> allocSize = 16
> r = 0x2b78aee04f00
> old_info = { = { = {_vptr.ForceVFPTToTop 
> = 0x7f3630}, m_refcount = 0}, iobuffer_index = 0, 
>   key = 47797242059264, app = {allotment = {application1 = 5326300, 
> application2 = 0}, http_data = {http_version = 4, 
>   pipeline_max = 59, keepalive_timeout = 17, fail_count = 81, 
> unused1 = 0, last_failure = 0}, rr = {offset = 5326300}}, data = {
> ip = {sa = {sa_family = 54488, sa_data = 
> "^\000\000\000\000\000\020\034$\274x+\000"}, sin = {sin_family = 54488, 
> sin_port = 94, 
> sin_addr = {s_addr = 0}, sin_zero = "\020\034$\274x+\000"}, 
> sin6 = {sin6_family = 54488, sin6_port = 94, sin6_flowinfo = 0, 
> sin6_addr = {__in6_u = {__u6_addr8 = 
> "\020\034$\274x+\000\000\030\036$\274\375\b\000", __u6_addr16 = {7184, 48164, 
> 11128, 
>   0, 7704, 48164, 2301, 0}, __u6_addr32 = {3156483088, 
> 11128, 3156483608, 2301}}}, sin6_scope_id = 3156478176}}, 
> hostname_offset = 6214872, srv = {srv_offset = 54488, srv_weight 
> = 94, srv_priority = 0, srv_port = 0, key = 3156483088}}, 
>   hostname_offset = 11128, ip_timestamp = 2845989456, 
> ip_timeout_interval = 11128, is_srv = 0, reverse_dns = 0, round_robin = 1, 
>   round_robin_elt = 0}
> valid_records = 0
> tip = {_family = 2, _addr = {_ip4 = 540420056, _ip6 = {__in6_u = 
> {__u6_addr8 = "\330'6 x+\000\000\360L\020\250x+\000", 
> __u6_addr16 = {10200, 8246, 11128, 0, 19696, 43024, 11128, 
> 0}, __u6_addr32 = {540420056, 11128, 2819640560, 11128}}}, 
> _byte = "\330'6 x+\000\000\360L\020\250x+\000", _u32 = 
> {540420056, 11128, 2819640560, 11128}, _u64 = {47794936489944, 
>   47797215710448}}}
> ttl_seconds = 132
> aname = 0x2b7938021000 "fbmm1.zenfs.com"
> offset = 96
> thread = 0x2b78a8101010
> __func__ = "dnsEvent"
> #7  0x005145dc in Continuation::handleEvent (this=0x2b7938020f00, 
> event=600, data=0x2b78ac009440)
> at ../iocore/eventsystem/I_Continuation.h:153
> 

[jira] [Work logged] (TS-4915) Crash from hostdb in PriorityQueueLess

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-4915?focusedWorklogId=30430=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-30430
 ]

ASF GitHub Bot logged work on TS-4915:
--

Author: ASF GitHub Bot
Created on: 11/Oct/16 20:22
Start Date: 11/Oct/16 20:22
Worklog Time Spent: 10m 
  Work Description: Github user atsci commented on the issue:

https://github.com/apache/trafficserver/pull/1088
  
FreeBSD build *successful*! See 
https://ci.trafficserver.apache.org/job/Github-FreeBSD/969/ for details.
 



Issue Time Tracking
---

Worklog Id: (was: 30430)
Time Spent: 2h 40m  (was: 2.5h)

> Crash from hostdb in PriorityQueueLess
> --
>
> Key: TS-4915
> URL: https://issues.apache.org/jira/browse/TS-4915
> Project: Traffic Server
>  Issue Type: Bug
>  Components: HostDB
>Reporter: Susan Hinrichs
>Priority: Blocker
> Fix For: 7.1.0
>
> Attachments: ts-4915.diff, ts-4915.diff
>
>  Time Spent: 2h 40m
>  Remaining Estimate: 0h
>
> Saw this while testing fix for TS-4813 with debug enabled.
> {code}
> (gdb) bt full
> #0  0x00547bfe in RefCountCacheHashEntry::operator< (this=0x1cc0880, 
> v2=...) at ../iocore/hostdb/P_RefCountCache.h:94
> No locals.
> #1  0x0054988d in 
> PriorityQueueLess::operator() (this=0x2b78a9a2587b, 
> a=@0x2b78f402af68, b=@0x2b78f402aa28)
> at ../lib/ts/PriorityQueue.h:41
> No locals.
> #2  0x00549785 in PriorityQueue PriorityQueueLess >::_bubble_up (this=0x1cb2990, 
> index=2) at ../lib/ts/PriorityQueue.h:191
> comp = {}
> parent = 0
> #3  0x006ecfcc in PriorityQueue PriorityQueueLess >::push (this=0x1cb2990, 
> entry=0x2b78f402af60) at ../../lib/ts/PriorityQueue.h:91
> len = 2
> #4  0x006ec206 in RefCountCachePartition::put 
> (this=0x1cb2900, key=6912554662447498853, item=0x2b78aee04f00, size=96, 
> expire_time=1475202356) at ./P_RefCountCache.h:210
> expiry_entry = 0x2b78f402af60
> __func__ = "put"
> val = 0x1cc0880
> #5  0x006eb3de in RefCountCache::put (this=0x18051e0, 
> key=6912554662447498853, item=0x2b78aee04f00, size=16, 
> expiry_time=1475202356) at ./P_RefCountCache.h:462
> No locals.
> #6  0x006e2d8e in HostDBContinuation::dnsEvent (this=0x2b7938020f00, 
> event=600, e=0x2b78ac009440) at HostDB.cc:1422
> is_rr = false
> old_rr_data = 0x0
> first_record = 0x2b78ac0094f8
> m = 0x1
> failed = false
> old_r = {m_ptr = 0x0}
> af = 2 '\002'
> s_size = 16
> rrsize = 0
> allocSize = 16
> r = 0x2b78aee04f00
> old_info = { = { = {_vptr.ForceVFPTToTop 
> = 0x7f3630}, m_refcount = 0}, iobuffer_index = 0, 
>   key = 47797242059264, app = {allotment = {application1 = 5326300, 
> application2 = 0}, http_data = {http_version = 4, 
>   pipeline_max = 59, keepalive_timeout = 17, fail_count = 81, 
> unused1 = 0, last_failure = 0}, rr = {offset = 5326300}}, data = {
> ip = {sa = {sa_family = 54488, sa_data = 
> "^\000\000\000\000\000\020\034$\274x+\000"}, sin = {sin_family = 54488, 
> sin_port = 94, 
> sin_addr = {s_addr = 0}, sin_zero = "\020\034$\274x+\000"}, 
> sin6 = {sin6_family = 54488, sin6_port = 94, sin6_flowinfo = 0, 
> sin6_addr = {__in6_u = {__u6_addr8 = 
> "\020\034$\274x+\000\000\030\036$\274\375\b\000", __u6_addr16 = {7184, 48164, 
> 11128, 
>   0, 7704, 48164, 2301, 0}, __u6_addr32 = {3156483088, 
> 11128, 3156483608, 2301}}}, sin6_scope_id = 3156478176}}, 
> hostname_offset = 6214872, srv = {srv_offset = 54488, srv_weight 
> = 94, srv_priority = 0, srv_port = 0, key = 3156483088}}, 
>   hostname_offset = 11128, ip_timestamp = 2845989456, 
> ip_timeout_interval = 11128, is_srv = 0, reverse_dns = 0, round_robin = 1, 
>   round_robin_elt = 0}
> valid_records = 0
> tip = {_family = 2, _addr = {_ip4 = 540420056, _ip6 = {__in6_u = 
> {__u6_addr8 = "\330'6 x+\000\000\360L\020\250x+\000", 
> __u6_addr16 = {10200, 8246, 11128, 0, 19696, 43024, 11128, 
> 0}, __u6_addr32 = {540420056, 11128, 2819640560, 11128}}}, 
> _byte = "\330'6 x+\000\000\360L\020\250x+\000", _u32 = 
> {540420056, 11128, 2819640560, 11128}, _u64 = {47794936489944, 
>   47797215710448}}}
> ttl_seconds = 132
> aname = 0x2b7938021000 "fbmm1.zenfs.com"
> offset = 96
> thread = 0x2b78a8101010
> __func__ = "dnsEvent"
> #7  0x005145dc in Continuation::handleEvent 

[jira] [Work logged] (TS-4915) Crash from hostdb in PriorityQueueLess

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-4915?focusedWorklogId=30428=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-30428
 ]

ASF GitHub Bot logged work on TS-4915:
--

Author: ASF GitHub Bot
Created on: 11/Oct/16 20:13
Start Date: 11/Oct/16 20:13
Worklog Time Spent: 10m 
  Work Description: Github user atsci commented on the issue:

https://github.com/apache/trafficserver/pull/1088
  
Linux build *failed*! See 
https://ci.trafficserver.apache.org/job/Github-Linux/861/ for details.
 



Issue Time Tracking
---

Worklog Id: (was: 30428)
Time Spent: 2.5h  (was: 2h 20m)

> Crash from hostdb in PriorityQueueLess
> --
>
> Key: TS-4915
> URL: https://issues.apache.org/jira/browse/TS-4915
> Project: Traffic Server
>  Issue Type: Bug
>  Components: HostDB
>Reporter: Susan Hinrichs
>Priority: Blocker
> Fix For: 7.1.0
>
> Attachments: ts-4915.diff, ts-4915.diff
>
>  Time Spent: 2.5h
>  Remaining Estimate: 0h
>
> Saw this while testing fix for TS-4813 with debug enabled.
> {code}
> (gdb) bt full
> #0  0x00547bfe in RefCountCacheHashEntry::operator< (this=0x1cc0880, 
> v2=...) at ../iocore/hostdb/P_RefCountCache.h:94
> No locals.
> #1  0x0054988d in 
> PriorityQueueLess::operator() (this=0x2b78a9a2587b, 
> a=@0x2b78f402af68, b=@0x2b78f402aa28)
> at ../lib/ts/PriorityQueue.h:41
> No locals.
> #2  0x00549785 in PriorityQueue PriorityQueueLess >::_bubble_up (this=0x1cb2990, 
> index=2) at ../lib/ts/PriorityQueue.h:191
> comp = {}
> parent = 0
> #3  0x006ecfcc in PriorityQueue PriorityQueueLess >::push (this=0x1cb2990, 
> entry=0x2b78f402af60) at ../../lib/ts/PriorityQueue.h:91
> len = 2
> #4  0x006ec206 in RefCountCachePartition::put 
> (this=0x1cb2900, key=6912554662447498853, item=0x2b78aee04f00, size=96, 
> expire_time=1475202356) at ./P_RefCountCache.h:210
> expiry_entry = 0x2b78f402af60
> __func__ = "put"
> val = 0x1cc0880
> #5  0x006eb3de in RefCountCache::put (this=0x18051e0, 
> key=6912554662447498853, item=0x2b78aee04f00, size=16, 
> expiry_time=1475202356) at ./P_RefCountCache.h:462
> No locals.
> #6  0x006e2d8e in HostDBContinuation::dnsEvent (this=0x2b7938020f00, 
> event=600, e=0x2b78ac009440) at HostDB.cc:1422
> is_rr = false
> old_rr_data = 0x0
> first_record = 0x2b78ac0094f8
> m = 0x1
> failed = false
> old_r = {m_ptr = 0x0}
> af = 2 '\002'
> s_size = 16
> rrsize = 0
> allocSize = 16
> r = 0x2b78aee04f00
> old_info = { = { = {_vptr.ForceVFPTToTop 
> = 0x7f3630}, m_refcount = 0}, iobuffer_index = 0, 
>   key = 47797242059264, app = {allotment = {application1 = 5326300, 
> application2 = 0}, http_data = {http_version = 4, 
>   pipeline_max = 59, keepalive_timeout = 17, fail_count = 81, 
> unused1 = 0, last_failure = 0}, rr = {offset = 5326300}}, data = {
> ip = {sa = {sa_family = 54488, sa_data = 
> "^\000\000\000\000\000\020\034$\274x+\000"}, sin = {sin_family = 54488, 
> sin_port = 94, 
> sin_addr = {s_addr = 0}, sin_zero = "\020\034$\274x+\000"}, 
> sin6 = {sin6_family = 54488, sin6_port = 94, sin6_flowinfo = 0, 
> sin6_addr = {__in6_u = {__u6_addr8 = 
> "\020\034$\274x+\000\000\030\036$\274\375\b\000", __u6_addr16 = {7184, 48164, 
> 11128, 
>   0, 7704, 48164, 2301, 0}, __u6_addr32 = {3156483088, 
> 11128, 3156483608, 2301}}}, sin6_scope_id = 3156478176}}, 
> hostname_offset = 6214872, srv = {srv_offset = 54488, srv_weight 
> = 94, srv_priority = 0, srv_port = 0, key = 3156483088}}, 
>   hostname_offset = 11128, ip_timestamp = 2845989456, 
> ip_timeout_interval = 11128, is_srv = 0, reverse_dns = 0, round_robin = 1, 
>   round_robin_elt = 0}
> valid_records = 0
> tip = {_family = 2, _addr = {_ip4 = 540420056, _ip6 = {__in6_u = 
> {__u6_addr8 = "\330'6 x+\000\000\360L\020\250x+\000", 
> __u6_addr16 = {10200, 8246, 11128, 0, 19696, 43024, 11128, 
> 0}, __u6_addr32 = {540420056, 11128, 2819640560, 11128}}}, 
> _byte = "\330'6 x+\000\000\360L\020\250x+\000", _u32 = 
> {540420056, 11128, 2819640560, 11128}, _u64 = {47794936489944, 
>   47797215710448}}}
> ttl_seconds = 132
> aname = 0x2b7938021000 "fbmm1.zenfs.com"
> offset = 96
> thread = 0x2b78a8101010
> __func__ = "dnsEvent"
> #7  0x005145dc in Continuation::handleEvent (this=0x2b7938020f00, 
> 

[jira] [Work logged] (TS-4915) Crash from hostdb in PriorityQueueLess

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-4915?focusedWorklogId=30424=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-30424
 ]

ASF GitHub Bot logged work on TS-4915:
--

Author: ASF GitHub Bot
Created on: 11/Oct/16 20:09
Start Date: 11/Oct/16 20:09
Worklog Time Spent: 10m 
  Work Description: Github user shinrich commented on the issue:

https://github.com/apache/trafficserver/pull/1088
  
Added a potential fix to _bubble_down to just do a simple bubble sort from 
the last index to the end of the array to catch any overshooting left index 
calculations.


Issue Time Tracking
---

Worklog Id: (was: 30424)
Time Spent: 2h 20m  (was: 2h 10m)

> Crash from hostdb in PriorityQueueLess
> --
>
> Key: TS-4915
> URL: https://issues.apache.org/jira/browse/TS-4915
> Project: Traffic Server
>  Issue Type: Bug
>  Components: HostDB
>Reporter: Susan Hinrichs
>Priority: Blocker
> Fix For: 7.1.0
>
> Attachments: ts-4915.diff, ts-4915.diff
>
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> Saw this while testing fix for TS-4813 with debug enabled.
> {code}
> (gdb) bt full
> #0  0x00547bfe in RefCountCacheHashEntry::operator< (this=0x1cc0880, 
> v2=...) at ../iocore/hostdb/P_RefCountCache.h:94
> No locals.
> #1  0x0054988d in 
> PriorityQueueLess::operator() (this=0x2b78a9a2587b, 
> a=@0x2b78f402af68, b=@0x2b78f402aa28)
> at ../lib/ts/PriorityQueue.h:41
> No locals.
> #2  0x00549785 in PriorityQueue PriorityQueueLess >::_bubble_up (this=0x1cb2990, 
> index=2) at ../lib/ts/PriorityQueue.h:191
> comp = {}
> parent = 0
> #3  0x006ecfcc in PriorityQueue PriorityQueueLess >::push (this=0x1cb2990, 
> entry=0x2b78f402af60) at ../../lib/ts/PriorityQueue.h:91
> len = 2
> #4  0x006ec206 in RefCountCachePartition::put 
> (this=0x1cb2900, key=6912554662447498853, item=0x2b78aee04f00, size=96, 
> expire_time=1475202356) at ./P_RefCountCache.h:210
> expiry_entry = 0x2b78f402af60
> __func__ = "put"
> val = 0x1cc0880
> #5  0x006eb3de in RefCountCache::put (this=0x18051e0, 
> key=6912554662447498853, item=0x2b78aee04f00, size=16, 
> expiry_time=1475202356) at ./P_RefCountCache.h:462
> No locals.
> #6  0x006e2d8e in HostDBContinuation::dnsEvent (this=0x2b7938020f00, 
> event=600, e=0x2b78ac009440) at HostDB.cc:1422
> is_rr = false
> old_rr_data = 0x0
> first_record = 0x2b78ac0094f8
> m = 0x1
> failed = false
> old_r = {m_ptr = 0x0}
> af = 2 '\002'
> s_size = 16
> rrsize = 0
> allocSize = 16
> r = 0x2b78aee04f00
> old_info = { = { = {_vptr.ForceVFPTToTop 
> = 0x7f3630}, m_refcount = 0}, iobuffer_index = 0, 
>   key = 47797242059264, app = {allotment = {application1 = 5326300, 
> application2 = 0}, http_data = {http_version = 4, 
>   pipeline_max = 59, keepalive_timeout = 17, fail_count = 81, 
> unused1 = 0, last_failure = 0}, rr = {offset = 5326300}}, data = {
> ip = {sa = {sa_family = 54488, sa_data = 
> "^\000\000\000\000\000\020\034$\274x+\000"}, sin = {sin_family = 54488, 
> sin_port = 94, 
> sin_addr = {s_addr = 0}, sin_zero = "\020\034$\274x+\000"}, 
> sin6 = {sin6_family = 54488, sin6_port = 94, sin6_flowinfo = 0, 
> sin6_addr = {__in6_u = {__u6_addr8 = 
> "\020\034$\274x+\000\000\030\036$\274\375\b\000", __u6_addr16 = {7184, 48164, 
> 11128, 
>   0, 7704, 48164, 2301, 0}, __u6_addr32 = {3156483088, 
> 11128, 3156483608, 2301}}}, sin6_scope_id = 3156478176}}, 
> hostname_offset = 6214872, srv = {srv_offset = 54488, srv_weight 
> = 94, srv_priority = 0, srv_port = 0, key = 3156483088}}, 
>   hostname_offset = 11128, ip_timestamp = 2845989456, 
> ip_timeout_interval = 11128, is_srv = 0, reverse_dns = 0, round_robin = 1, 
>   round_robin_elt = 0}
> valid_records = 0
> tip = {_family = 2, _addr = {_ip4 = 540420056, _ip6 = {__in6_u = 
> {__u6_addr8 = "\330'6 x+\000\000\360L\020\250x+\000", 
> __u6_addr16 = {10200, 8246, 11128, 0, 19696, 43024, 11128, 
> 0}, __u6_addr32 = {540420056, 11128, 2819640560, 11128}}}, 
> _byte = "\330'6 x+\000\000\360L\020\250x+\000", _u32 = 
> {540420056, 11128, 2819640560, 11128}, _u64 = {47794936489944, 
>   47797215710448}}}
> ttl_seconds = 132
> aname = 0x2b7938021000 "fbmm1.zenfs.com"
> offset = 96
> thread = 0x2b78a8101010
> __func__ = "dnsEvent"
> #7  

[jira] [Work logged] (TS-4915) Crash from hostdb in PriorityQueueLess

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-4915?focusedWorklogId=30423=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-30423
 ]

ASF GitHub Bot logged work on TS-4915:
--

Author: ASF GitHub Bot
Created on: 11/Oct/16 20:05
Start Date: 11/Oct/16 20:05
Worklog Time Spent: 10m 
  Work Description: Github user atsci commented on the issue:

https://github.com/apache/trafficserver/pull/1088
  
Linux build *successful*! See 
https://ci.trafficserver.apache.org/job/Github-Linux/859/ for details.
 



Issue Time Tracking
---

Worklog Id: (was: 30423)
Time Spent: 2h 10m  (was: 2h)

> Crash from hostdb in PriorityQueueLess
> --
>
> Key: TS-4915
> URL: https://issues.apache.org/jira/browse/TS-4915
> Project: Traffic Server
>  Issue Type: Bug
>  Components: HostDB
>Reporter: Susan Hinrichs
>Priority: Blocker
> Fix For: 7.1.0
>
> Attachments: ts-4915.diff, ts-4915.diff
>
>  Time Spent: 2h 10m
>  Remaining Estimate: 0h
>
> Saw this while testing fix for TS-4813 with debug enabled.
> {code}
> (gdb) bt full
> #0  0x00547bfe in RefCountCacheHashEntry::operator< (this=0x1cc0880, 
> v2=...) at ../iocore/hostdb/P_RefCountCache.h:94
> No locals.
> #1  0x0054988d in 
> PriorityQueueLess::operator() (this=0x2b78a9a2587b, 
> a=@0x2b78f402af68, b=@0x2b78f402aa28)
> at ../lib/ts/PriorityQueue.h:41
> No locals.
> #2  0x00549785 in PriorityQueue PriorityQueueLess >::_bubble_up (this=0x1cb2990, 
> index=2) at ../lib/ts/PriorityQueue.h:191
> comp = {}
> parent = 0
> #3  0x006ecfcc in PriorityQueue PriorityQueueLess >::push (this=0x1cb2990, 
> entry=0x2b78f402af60) at ../../lib/ts/PriorityQueue.h:91
> len = 2
> #4  0x006ec206 in RefCountCachePartition::put 
> (this=0x1cb2900, key=6912554662447498853, item=0x2b78aee04f00, size=96, 
> expire_time=1475202356) at ./P_RefCountCache.h:210
> expiry_entry = 0x2b78f402af60
> __func__ = "put"
> val = 0x1cc0880
> #5  0x006eb3de in RefCountCache::put (this=0x18051e0, 
> key=6912554662447498853, item=0x2b78aee04f00, size=16, 
> expiry_time=1475202356) at ./P_RefCountCache.h:462
> No locals.
> #6  0x006e2d8e in HostDBContinuation::dnsEvent (this=0x2b7938020f00, 
> event=600, e=0x2b78ac009440) at HostDB.cc:1422
> is_rr = false
> old_rr_data = 0x0
> first_record = 0x2b78ac0094f8
> m = 0x1
> failed = false
> old_r = {m_ptr = 0x0}
> af = 2 '\002'
> s_size = 16
> rrsize = 0
> allocSize = 16
> r = 0x2b78aee04f00
> old_info = { = { = {_vptr.ForceVFPTToTop 
> = 0x7f3630}, m_refcount = 0}, iobuffer_index = 0, 
>   key = 47797242059264, app = {allotment = {application1 = 5326300, 
> application2 = 0}, http_data = {http_version = 4, 
>   pipeline_max = 59, keepalive_timeout = 17, fail_count = 81, 
> unused1 = 0, last_failure = 0}, rr = {offset = 5326300}}, data = {
> ip = {sa = {sa_family = 54488, sa_data = 
> "^\000\000\000\000\000\020\034$\274x+\000"}, sin = {sin_family = 54488, 
> sin_port = 94, 
> sin_addr = {s_addr = 0}, sin_zero = "\020\034$\274x+\000"}, 
> sin6 = {sin6_family = 54488, sin6_port = 94, sin6_flowinfo = 0, 
> sin6_addr = {__in6_u = {__u6_addr8 = 
> "\020\034$\274x+\000\000\030\036$\274\375\b\000", __u6_addr16 = {7184, 48164, 
> 11128, 
>   0, 7704, 48164, 2301, 0}, __u6_addr32 = {3156483088, 
> 11128, 3156483608, 2301}}}, sin6_scope_id = 3156478176}}, 
> hostname_offset = 6214872, srv = {srv_offset = 54488, srv_weight 
> = 94, srv_priority = 0, srv_port = 0, key = 3156483088}}, 
>   hostname_offset = 11128, ip_timestamp = 2845989456, 
> ip_timeout_interval = 11128, is_srv = 0, reverse_dns = 0, round_robin = 1, 
>   round_robin_elt = 0}
> valid_records = 0
> tip = {_family = 2, _addr = {_ip4 = 540420056, _ip6 = {__in6_u = 
> {__u6_addr8 = "\330'6 x+\000\000\360L\020\250x+\000", 
> __u6_addr16 = {10200, 8246, 11128, 0, 19696, 43024, 11128, 
> 0}, __u6_addr32 = {540420056, 11128, 2819640560, 11128}}}, 
> _byte = "\330'6 x+\000\000\360L\020\250x+\000", _u32 = 
> {540420056, 11128, 2819640560, 11128}, _u64 = {47794936489944, 
>   47797215710448}}}
> ttl_seconds = 132
> aname = 0x2b7938021000 "fbmm1.zenfs.com"
> offset = 96
> thread = 0x2b78a8101010
> __func__ = "dnsEvent"
> #7  0x005145dc in Continuation::handleEvent (this=0x2b7938020f00, 
> 

[jira] [Work logged] (TS-4915) Crash from hostdb in PriorityQueueLess

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-4915?focusedWorklogId=30422=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-30422
 ]

ASF GitHub Bot logged work on TS-4915:
--

Author: ASF GitHub Bot
Created on: 11/Oct/16 20:03
Start Date: 11/Oct/16 20:03
Worklog Time Spent: 10m 
  Work Description: Github user atsci commented on the issue:

https://github.com/apache/trafficserver/pull/1088
  
FreeBSD build *successful*! See 
https://ci.trafficserver.apache.org/job/Github-FreeBSD/967/ for details.
 



Issue Time Tracking
---

Worklog Id: (was: 30422)
Time Spent: 2h  (was: 1h 50m)

> Crash from hostdb in PriorityQueueLess
> --
>
> Key: TS-4915
> URL: https://issues.apache.org/jira/browse/TS-4915
> Project: Traffic Server
>  Issue Type: Bug
>  Components: HostDB
>Reporter: Susan Hinrichs
>Priority: Blocker
> Fix For: 7.1.0
>
> Attachments: ts-4915.diff, ts-4915.diff
>
>  Time Spent: 2h
>  Remaining Estimate: 0h
>
> Saw this while testing fix for TS-4813 with debug enabled.
> {code}
> (gdb) bt full
> #0  0x00547bfe in RefCountCacheHashEntry::operator< (this=0x1cc0880, 
> v2=...) at ../iocore/hostdb/P_RefCountCache.h:94
> No locals.
> #1  0x0054988d in 
> PriorityQueueLess::operator() (this=0x2b78a9a2587b, 
> a=@0x2b78f402af68, b=@0x2b78f402aa28)
> at ../lib/ts/PriorityQueue.h:41
> No locals.
> #2  0x00549785 in PriorityQueue PriorityQueueLess >::_bubble_up (this=0x1cb2990, 
> index=2) at ../lib/ts/PriorityQueue.h:191
> comp = {}
> parent = 0
> #3  0x006ecfcc in PriorityQueue PriorityQueueLess >::push (this=0x1cb2990, 
> entry=0x2b78f402af60) at ../../lib/ts/PriorityQueue.h:91
> len = 2
> #4  0x006ec206 in RefCountCachePartition::put 
> (this=0x1cb2900, key=6912554662447498853, item=0x2b78aee04f00, size=96, 
> expire_time=1475202356) at ./P_RefCountCache.h:210
> expiry_entry = 0x2b78f402af60
> __func__ = "put"
> val = 0x1cc0880
> #5  0x006eb3de in RefCountCache::put (this=0x18051e0, 
> key=6912554662447498853, item=0x2b78aee04f00, size=16, 
> expiry_time=1475202356) at ./P_RefCountCache.h:462
> No locals.
> #6  0x006e2d8e in HostDBContinuation::dnsEvent (this=0x2b7938020f00, 
> event=600, e=0x2b78ac009440) at HostDB.cc:1422
> is_rr = false
> old_rr_data = 0x0
> first_record = 0x2b78ac0094f8
> m = 0x1
> failed = false
> old_r = {m_ptr = 0x0}
> af = 2 '\002'
> s_size = 16
> rrsize = 0
> allocSize = 16
> r = 0x2b78aee04f00
> old_info = { = { = {_vptr.ForceVFPTToTop 
> = 0x7f3630}, m_refcount = 0}, iobuffer_index = 0, 
>   key = 47797242059264, app = {allotment = {application1 = 5326300, 
> application2 = 0}, http_data = {http_version = 4, 
>   pipeline_max = 59, keepalive_timeout = 17, fail_count = 81, 
> unused1 = 0, last_failure = 0}, rr = {offset = 5326300}}, data = {
> ip = {sa = {sa_family = 54488, sa_data = 
> "^\000\000\000\000\000\020\034$\274x+\000"}, sin = {sin_family = 54488, 
> sin_port = 94, 
> sin_addr = {s_addr = 0}, sin_zero = "\020\034$\274x+\000"}, 
> sin6 = {sin6_family = 54488, sin6_port = 94, sin6_flowinfo = 0, 
> sin6_addr = {__in6_u = {__u6_addr8 = 
> "\020\034$\274x+\000\000\030\036$\274\375\b\000", __u6_addr16 = {7184, 48164, 
> 11128, 
>   0, 7704, 48164, 2301, 0}, __u6_addr32 = {3156483088, 
> 11128, 3156483608, 2301}}}, sin6_scope_id = 3156478176}}, 
> hostname_offset = 6214872, srv = {srv_offset = 54488, srv_weight 
> = 94, srv_priority = 0, srv_port = 0, key = 3156483088}}, 
>   hostname_offset = 11128, ip_timestamp = 2845989456, 
> ip_timeout_interval = 11128, is_srv = 0, reverse_dns = 0, round_robin = 1, 
>   round_robin_elt = 0}
> valid_records = 0
> tip = {_family = 2, _addr = {_ip4 = 540420056, _ip6 = {__in6_u = 
> {__u6_addr8 = "\330'6 x+\000\000\360L\020\250x+\000", 
> __u6_addr16 = {10200, 8246, 11128, 0, 19696, 43024, 11128, 
> 0}, __u6_addr32 = {540420056, 11128, 2819640560, 11128}}}, 
> _byte = "\330'6 x+\000\000\360L\020\250x+\000", _u32 = 
> {540420056, 11128, 2819640560, 11128}, _u64 = {47794936489944, 
>   47797215710448}}}
> ttl_seconds = 132
> aname = 0x2b7938021000 "fbmm1.zenfs.com"
> offset = 96
> thread = 0x2b78a8101010
> __func__ = "dnsEvent"
> #7  0x005145dc in Continuation::handleEvent (this=0x2b7938020f00, 
> 

[jira] [Work logged] (TS-4915) Crash from hostdb in PriorityQueueLess

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-4915?focusedWorklogId=30421=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-30421
 ]

ASF GitHub Bot logged work on TS-4915:
--

Author: ASF GitHub Bot
Created on: 11/Oct/16 19:59
Start Date: 11/Oct/16 19:59
Worklog Time Spent: 10m 
  Work Description: Github user atsci commented on the issue:

https://github.com/apache/trafficserver/pull/1088
  
Linux build *successful*! See 
https://ci.trafficserver.apache.org/job/Github-Linux/858/ for details.
 



Issue Time Tracking
---

Worklog Id: (was: 30421)
Time Spent: 1h 50m  (was: 1h 40m)

> Crash from hostdb in PriorityQueueLess
> --
>
> Key: TS-4915
> URL: https://issues.apache.org/jira/browse/TS-4915
> Project: Traffic Server
>  Issue Type: Bug
>  Components: HostDB
>Reporter: Susan Hinrichs
>Priority: Blocker
> Fix For: 7.1.0
>
> Attachments: ts-4915.diff, ts-4915.diff
>
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> Saw this while testing fix for TS-4813 with debug enabled.
> {code}
> (gdb) bt full
> #0  0x00547bfe in RefCountCacheHashEntry::operator< (this=0x1cc0880, 
> v2=...) at ../iocore/hostdb/P_RefCountCache.h:94
> No locals.
> #1  0x0054988d in 
> PriorityQueueLess::operator() (this=0x2b78a9a2587b, 
> a=@0x2b78f402af68, b=@0x2b78f402aa28)
> at ../lib/ts/PriorityQueue.h:41
> No locals.
> #2  0x00549785 in PriorityQueue PriorityQueueLess >::_bubble_up (this=0x1cb2990, 
> index=2) at ../lib/ts/PriorityQueue.h:191
> comp = {}
> parent = 0
> #3  0x006ecfcc in PriorityQueue PriorityQueueLess >::push (this=0x1cb2990, 
> entry=0x2b78f402af60) at ../../lib/ts/PriorityQueue.h:91
> len = 2
> #4  0x006ec206 in RefCountCachePartition::put 
> (this=0x1cb2900, key=6912554662447498853, item=0x2b78aee04f00, size=96, 
> expire_time=1475202356) at ./P_RefCountCache.h:210
> expiry_entry = 0x2b78f402af60
> __func__ = "put"
> val = 0x1cc0880
> #5  0x006eb3de in RefCountCache::put (this=0x18051e0, 
> key=6912554662447498853, item=0x2b78aee04f00, size=16, 
> expiry_time=1475202356) at ./P_RefCountCache.h:462
> No locals.
> #6  0x006e2d8e in HostDBContinuation::dnsEvent (this=0x2b7938020f00, 
> event=600, e=0x2b78ac009440) at HostDB.cc:1422
> is_rr = false
> old_rr_data = 0x0
> first_record = 0x2b78ac0094f8
> m = 0x1
> failed = false
> old_r = {m_ptr = 0x0}
> af = 2 '\002'
> s_size = 16
> rrsize = 0
> allocSize = 16
> r = 0x2b78aee04f00
> old_info = { = { = {_vptr.ForceVFPTToTop 
> = 0x7f3630}, m_refcount = 0}, iobuffer_index = 0, 
>   key = 47797242059264, app = {allotment = {application1 = 5326300, 
> application2 = 0}, http_data = {http_version = 4, 
>   pipeline_max = 59, keepalive_timeout = 17, fail_count = 81, 
> unused1 = 0, last_failure = 0}, rr = {offset = 5326300}}, data = {
> ip = {sa = {sa_family = 54488, sa_data = 
> "^\000\000\000\000\000\020\034$\274x+\000"}, sin = {sin_family = 54488, 
> sin_port = 94, 
> sin_addr = {s_addr = 0}, sin_zero = "\020\034$\274x+\000"}, 
> sin6 = {sin6_family = 54488, sin6_port = 94, sin6_flowinfo = 0, 
> sin6_addr = {__in6_u = {__u6_addr8 = 
> "\020\034$\274x+\000\000\030\036$\274\375\b\000", __u6_addr16 = {7184, 48164, 
> 11128, 
>   0, 7704, 48164, 2301, 0}, __u6_addr32 = {3156483088, 
> 11128, 3156483608, 2301}}}, sin6_scope_id = 3156478176}}, 
> hostname_offset = 6214872, srv = {srv_offset = 54488, srv_weight 
> = 94, srv_priority = 0, srv_port = 0, key = 3156483088}}, 
>   hostname_offset = 11128, ip_timestamp = 2845989456, 
> ip_timeout_interval = 11128, is_srv = 0, reverse_dns = 0, round_robin = 1, 
>   round_robin_elt = 0}
> valid_records = 0
> tip = {_family = 2, _addr = {_ip4 = 540420056, _ip6 = {__in6_u = 
> {__u6_addr8 = "\330'6 x+\000\000\360L\020\250x+\000", 
> __u6_addr16 = {10200, 8246, 11128, 0, 19696, 43024, 11128, 
> 0}, __u6_addr32 = {540420056, 11128, 2819640560, 11128}}}, 
> _byte = "\330'6 x+\000\000\360L\020\250x+\000", _u32 = 
> {540420056, 11128, 2819640560, 11128}, _u64 = {47794936489944, 
>   47797215710448}}}
> ttl_seconds = 132
> aname = 0x2b7938021000 "fbmm1.zenfs.com"
> offset = 96
> thread = 0x2b78a8101010
> __func__ = "dnsEvent"
> #7  0x005145dc in Continuation::handleEvent 

[jira] [Work logged] (TS-4915) Crash from hostdb in PriorityQueueLess

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-4915?focusedWorklogId=30419=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-30419
 ]

ASF GitHub Bot logged work on TS-4915:
--

Author: ASF GitHub Bot
Created on: 11/Oct/16 19:58
Start Date: 11/Oct/16 19:58
Worklog Time Spent: 10m 
  Work Description: Github user shinrich commented on the issue:

https://github.com/apache/trafficserver/pull/1088
  
The problem appears to be in _bubble_down.  In the test, I had 4 items, w, 
x, y, z with weights 10, 20, 30, 40 respectively.  I pushed in reverse order, 
but the overall order afterwards was correct.

I deleted item X (entry index 1).  In the original code, this swaps Z (with 
index 3) into slot 1.  Then we call bubble_up, which doesn't do anything 
because Z's weight is greater than entry 0.  Then it calls bubble_down with 
index 1.  It computes left as 2*index + 1 or 3 in this case.  This is greater 
than or equal to the current length of 3, so it gives up.  It leaves the vector 
with with W(10), Z(40), and Y(30).  If the vector were longer, the bubble_down 
logic would work, but with the short vector it gives up too soon.  @masaori335 
any ideas?


Issue Time Tracking
---

Worklog Id: (was: 30419)
Time Spent: 1h 40m  (was: 1.5h)

> Crash from hostdb in PriorityQueueLess
> --
>
> Key: TS-4915
> URL: https://issues.apache.org/jira/browse/TS-4915
> Project: Traffic Server
>  Issue Type: Bug
>  Components: HostDB
>Reporter: Susan Hinrichs
>Priority: Blocker
> Fix For: 7.1.0
>
> Attachments: ts-4915.diff, ts-4915.diff
>
>  Time Spent: 1h 40m
>  Remaining Estimate: 0h
>
> Saw this while testing fix for TS-4813 with debug enabled.
> {code}
> (gdb) bt full
> #0  0x00547bfe in RefCountCacheHashEntry::operator< (this=0x1cc0880, 
> v2=...) at ../iocore/hostdb/P_RefCountCache.h:94
> No locals.
> #1  0x0054988d in 
> PriorityQueueLess::operator() (this=0x2b78a9a2587b, 
> a=@0x2b78f402af68, b=@0x2b78f402aa28)
> at ../lib/ts/PriorityQueue.h:41
> No locals.
> #2  0x00549785 in PriorityQueue PriorityQueueLess >::_bubble_up (this=0x1cb2990, 
> index=2) at ../lib/ts/PriorityQueue.h:191
> comp = {}
> parent = 0
> #3  0x006ecfcc in PriorityQueue PriorityQueueLess >::push (this=0x1cb2990, 
> entry=0x2b78f402af60) at ../../lib/ts/PriorityQueue.h:91
> len = 2
> #4  0x006ec206 in RefCountCachePartition::put 
> (this=0x1cb2900, key=6912554662447498853, item=0x2b78aee04f00, size=96, 
> expire_time=1475202356) at ./P_RefCountCache.h:210
> expiry_entry = 0x2b78f402af60
> __func__ = "put"
> val = 0x1cc0880
> #5  0x006eb3de in RefCountCache::put (this=0x18051e0, 
> key=6912554662447498853, item=0x2b78aee04f00, size=16, 
> expiry_time=1475202356) at ./P_RefCountCache.h:462
> No locals.
> #6  0x006e2d8e in HostDBContinuation::dnsEvent (this=0x2b7938020f00, 
> event=600, e=0x2b78ac009440) at HostDB.cc:1422
> is_rr = false
> old_rr_data = 0x0
> first_record = 0x2b78ac0094f8
> m = 0x1
> failed = false
> old_r = {m_ptr = 0x0}
> af = 2 '\002'
> s_size = 16
> rrsize = 0
> allocSize = 16
> r = 0x2b78aee04f00
> old_info = { = { = {_vptr.ForceVFPTToTop 
> = 0x7f3630}, m_refcount = 0}, iobuffer_index = 0, 
>   key = 47797242059264, app = {allotment = {application1 = 5326300, 
> application2 = 0}, http_data = {http_version = 4, 
>   pipeline_max = 59, keepalive_timeout = 17, fail_count = 81, 
> unused1 = 0, last_failure = 0}, rr = {offset = 5326300}}, data = {
> ip = {sa = {sa_family = 54488, sa_data = 
> "^\000\000\000\000\000\020\034$\274x+\000"}, sin = {sin_family = 54488, 
> sin_port = 94, 
> sin_addr = {s_addr = 0}, sin_zero = "\020\034$\274x+\000"}, 
> sin6 = {sin6_family = 54488, sin6_port = 94, sin6_flowinfo = 0, 
> sin6_addr = {__in6_u = {__u6_addr8 = 
> "\020\034$\274x+\000\000\030\036$\274\375\b\000", __u6_addr16 = {7184, 48164, 
> 11128, 
>   0, 7704, 48164, 2301, 0}, __u6_addr32 = {3156483088, 
> 11128, 3156483608, 2301}}}, sin6_scope_id = 3156478176}}, 
> hostname_offset = 6214872, srv = {srv_offset = 54488, srv_weight 
> = 94, srv_priority = 0, srv_port = 0, key = 3156483088}}, 
>   hostname_offset = 11128, ip_timestamp = 2845989456, 
> ip_timeout_interval = 11128, is_srv = 0, reverse_dns = 0, round_robin = 1, 
>   round_robin_elt = 0}
> valid_records = 0
> tip = 

[jira] [Work logged] (TS-4915) Crash from hostdb in PriorityQueueLess

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-4915?focusedWorklogId=30418=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-30418
 ]

ASF GitHub Bot logged work on TS-4915:
--

Author: ASF GitHub Bot
Created on: 11/Oct/16 19:57
Start Date: 11/Oct/16 19:57
Worklog Time Spent: 10m 
  Work Description: Github user atsci commented on the issue:

https://github.com/apache/trafficserver/pull/1088
  
FreeBSD build *successful*! See 
https://ci.trafficserver.apache.org/job/Github-FreeBSD/966/ for details.
 



Issue Time Tracking
---

Worklog Id: (was: 30418)
Time Spent: 1.5h  (was: 1h 20m)

> Crash from hostdb in PriorityQueueLess
> --
>
> Key: TS-4915
> URL: https://issues.apache.org/jira/browse/TS-4915
> Project: Traffic Server
>  Issue Type: Bug
>  Components: HostDB
>Reporter: Susan Hinrichs
>Priority: Blocker
> Fix For: 7.1.0
>
> Attachments: ts-4915.diff, ts-4915.diff
>
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> Saw this while testing fix for TS-4813 with debug enabled.
> {code}
> (gdb) bt full
> #0  0x00547bfe in RefCountCacheHashEntry::operator< (this=0x1cc0880, 
> v2=...) at ../iocore/hostdb/P_RefCountCache.h:94
> No locals.
> #1  0x0054988d in 
> PriorityQueueLess::operator() (this=0x2b78a9a2587b, 
> a=@0x2b78f402af68, b=@0x2b78f402aa28)
> at ../lib/ts/PriorityQueue.h:41
> No locals.
> #2  0x00549785 in PriorityQueue PriorityQueueLess >::_bubble_up (this=0x1cb2990, 
> index=2) at ../lib/ts/PriorityQueue.h:191
> comp = {}
> parent = 0
> #3  0x006ecfcc in PriorityQueue PriorityQueueLess >::push (this=0x1cb2990, 
> entry=0x2b78f402af60) at ../../lib/ts/PriorityQueue.h:91
> len = 2
> #4  0x006ec206 in RefCountCachePartition::put 
> (this=0x1cb2900, key=6912554662447498853, item=0x2b78aee04f00, size=96, 
> expire_time=1475202356) at ./P_RefCountCache.h:210
> expiry_entry = 0x2b78f402af60
> __func__ = "put"
> val = 0x1cc0880
> #5  0x006eb3de in RefCountCache::put (this=0x18051e0, 
> key=6912554662447498853, item=0x2b78aee04f00, size=16, 
> expiry_time=1475202356) at ./P_RefCountCache.h:462
> No locals.
> #6  0x006e2d8e in HostDBContinuation::dnsEvent (this=0x2b7938020f00, 
> event=600, e=0x2b78ac009440) at HostDB.cc:1422
> is_rr = false
> old_rr_data = 0x0
> first_record = 0x2b78ac0094f8
> m = 0x1
> failed = false
> old_r = {m_ptr = 0x0}
> af = 2 '\002'
> s_size = 16
> rrsize = 0
> allocSize = 16
> r = 0x2b78aee04f00
> old_info = { = { = {_vptr.ForceVFPTToTop 
> = 0x7f3630}, m_refcount = 0}, iobuffer_index = 0, 
>   key = 47797242059264, app = {allotment = {application1 = 5326300, 
> application2 = 0}, http_data = {http_version = 4, 
>   pipeline_max = 59, keepalive_timeout = 17, fail_count = 81, 
> unused1 = 0, last_failure = 0}, rr = {offset = 5326300}}, data = {
> ip = {sa = {sa_family = 54488, sa_data = 
> "^\000\000\000\000\000\020\034$\274x+\000"}, sin = {sin_family = 54488, 
> sin_port = 94, 
> sin_addr = {s_addr = 0}, sin_zero = "\020\034$\274x+\000"}, 
> sin6 = {sin6_family = 54488, sin6_port = 94, sin6_flowinfo = 0, 
> sin6_addr = {__in6_u = {__u6_addr8 = 
> "\020\034$\274x+\000\000\030\036$\274\375\b\000", __u6_addr16 = {7184, 48164, 
> 11128, 
>   0, 7704, 48164, 2301, 0}, __u6_addr32 = {3156483088, 
> 11128, 3156483608, 2301}}}, sin6_scope_id = 3156478176}}, 
> hostname_offset = 6214872, srv = {srv_offset = 54488, srv_weight 
> = 94, srv_priority = 0, srv_port = 0, key = 3156483088}}, 
>   hostname_offset = 11128, ip_timestamp = 2845989456, 
> ip_timeout_interval = 11128, is_srv = 0, reverse_dns = 0, round_robin = 1, 
>   round_robin_elt = 0}
> valid_records = 0
> tip = {_family = 2, _addr = {_ip4 = 540420056, _ip6 = {__in6_u = 
> {__u6_addr8 = "\330'6 x+\000\000\360L\020\250x+\000", 
> __u6_addr16 = {10200, 8246, 11128, 0, 19696, 43024, 11128, 
> 0}, __u6_addr32 = {540420056, 11128, 2819640560, 11128}}}, 
> _byte = "\330'6 x+\000\000\360L\020\250x+\000", _u32 = 
> {540420056, 11128, 2819640560, 11128}, _u64 = {47794936489944, 
>   47797215710448}}}
> ttl_seconds = 132
> aname = 0x2b7938021000 "fbmm1.zenfs.com"
> offset = 96
> thread = 0x2b78a8101010
> __func__ = "dnsEvent"
> #7  0x005145dc in Continuation::handleEvent 

[jira] [Work logged] (TS-4915) Crash from hostdb in PriorityQueueLess

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-4915?focusedWorklogId=30414=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-30414
 ]

ASF GitHub Bot logged work on TS-4915:
--

Author: ASF GitHub Bot
Created on: 11/Oct/16 19:26
Start Date: 11/Oct/16 19:26
Worklog Time Spent: 10m 
  Work Description: Github user shinrich commented on the issue:

https://github.com/apache/trafficserver/pull/1088
  
While writing the test, I see that the bubble_up/down isn't work (or at 
least isn't working as I expect).  Was able to make a failing test without the 
erase fix.


Issue Time Tracking
---

Worklog Id: (was: 30414)
Time Spent: 1h 20m  (was: 1h 10m)

> Crash from hostdb in PriorityQueueLess
> --
>
> Key: TS-4915
> URL: https://issues.apache.org/jira/browse/TS-4915
> Project: Traffic Server
>  Issue Type: Bug
>  Components: HostDB
>Reporter: Susan Hinrichs
>Priority: Blocker
> Fix For: 7.1.0
>
> Attachments: ts-4915.diff, ts-4915.diff
>
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> Saw this while testing fix for TS-4813 with debug enabled.
> {code}
> (gdb) bt full
> #0  0x00547bfe in RefCountCacheHashEntry::operator< (this=0x1cc0880, 
> v2=...) at ../iocore/hostdb/P_RefCountCache.h:94
> No locals.
> #1  0x0054988d in 
> PriorityQueueLess::operator() (this=0x2b78a9a2587b, 
> a=@0x2b78f402af68, b=@0x2b78f402aa28)
> at ../lib/ts/PriorityQueue.h:41
> No locals.
> #2  0x00549785 in PriorityQueue PriorityQueueLess >::_bubble_up (this=0x1cb2990, 
> index=2) at ../lib/ts/PriorityQueue.h:191
> comp = {}
> parent = 0
> #3  0x006ecfcc in PriorityQueue PriorityQueueLess >::push (this=0x1cb2990, 
> entry=0x2b78f402af60) at ../../lib/ts/PriorityQueue.h:91
> len = 2
> #4  0x006ec206 in RefCountCachePartition::put 
> (this=0x1cb2900, key=6912554662447498853, item=0x2b78aee04f00, size=96, 
> expire_time=1475202356) at ./P_RefCountCache.h:210
> expiry_entry = 0x2b78f402af60
> __func__ = "put"
> val = 0x1cc0880
> #5  0x006eb3de in RefCountCache::put (this=0x18051e0, 
> key=6912554662447498853, item=0x2b78aee04f00, size=16, 
> expiry_time=1475202356) at ./P_RefCountCache.h:462
> No locals.
> #6  0x006e2d8e in HostDBContinuation::dnsEvent (this=0x2b7938020f00, 
> event=600, e=0x2b78ac009440) at HostDB.cc:1422
> is_rr = false
> old_rr_data = 0x0
> first_record = 0x2b78ac0094f8
> m = 0x1
> failed = false
> old_r = {m_ptr = 0x0}
> af = 2 '\002'
> s_size = 16
> rrsize = 0
> allocSize = 16
> r = 0x2b78aee04f00
> old_info = { = { = {_vptr.ForceVFPTToTop 
> = 0x7f3630}, m_refcount = 0}, iobuffer_index = 0, 
>   key = 47797242059264, app = {allotment = {application1 = 5326300, 
> application2 = 0}, http_data = {http_version = 4, 
>   pipeline_max = 59, keepalive_timeout = 17, fail_count = 81, 
> unused1 = 0, last_failure = 0}, rr = {offset = 5326300}}, data = {
> ip = {sa = {sa_family = 54488, sa_data = 
> "^\000\000\000\000\000\020\034$\274x+\000"}, sin = {sin_family = 54488, 
> sin_port = 94, 
> sin_addr = {s_addr = 0}, sin_zero = "\020\034$\274x+\000"}, 
> sin6 = {sin6_family = 54488, sin6_port = 94, sin6_flowinfo = 0, 
> sin6_addr = {__in6_u = {__u6_addr8 = 
> "\020\034$\274x+\000\000\030\036$\274\375\b\000", __u6_addr16 = {7184, 48164, 
> 11128, 
>   0, 7704, 48164, 2301, 0}, __u6_addr32 = {3156483088, 
> 11128, 3156483608, 2301}}}, sin6_scope_id = 3156478176}}, 
> hostname_offset = 6214872, srv = {srv_offset = 54488, srv_weight 
> = 94, srv_priority = 0, srv_port = 0, key = 3156483088}}, 
>   hostname_offset = 11128, ip_timestamp = 2845989456, 
> ip_timeout_interval = 11128, is_srv = 0, reverse_dns = 0, round_robin = 1, 
>   round_robin_elt = 0}
> valid_records = 0
> tip = {_family = 2, _addr = {_ip4 = 540420056, _ip6 = {__in6_u = 
> {__u6_addr8 = "\330'6 x+\000\000\360L\020\250x+\000", 
> __u6_addr16 = {10200, 8246, 11128, 0, 19696, 43024, 11128, 
> 0}, __u6_addr32 = {540420056, 11128, 2819640560, 11128}}}, 
> _byte = "\330'6 x+\000\000\360L\020\250x+\000", _u32 = 
> {540420056, 11128, 2819640560, 11128}, _u64 = {47794936489944, 
>   47797215710448}}}
> ttl_seconds = 132
> aname = 0x2b7938021000 "fbmm1.zenfs.com"
> offset = 96
> thread = 0x2b78a8101010
> __func__ = "dnsEvent"
> #7  

[jira] [Work logged] (TS-4915) Crash from hostdb in PriorityQueueLess

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-4915?focusedWorklogId=30410=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-30410
 ]

ASF GitHub Bot logged work on TS-4915:
--

Author: ASF GitHub Bot
Created on: 11/Oct/16 18:54
Start Date: 11/Oct/16 18:54
Worklog Time Spent: 10m 
  Work Description: Github user shinrich commented on a diff in the pull 
request:

https://github.com/apache/trafficserver/pull/1088#discussion_r82863899
  
--- Diff: lib/ts/PriorityQueue.h ---
@@ -110,6 +110,12 @@ PriorityQueue::pop()
 return;
   }
 
+  // SKH - I suspect this assignment is not preserving entry indices 
correctly.
--- End diff --

Ok.  Hopefully a transient comment anyway.  @masaori335 should be able to 
take a look and tell me whether this is a concern or not.


Issue Time Tracking
---

Worklog Id: (was: 30410)
Time Spent: 1h 10m  (was: 1h)

> Crash from hostdb in PriorityQueueLess
> --
>
> Key: TS-4915
> URL: https://issues.apache.org/jira/browse/TS-4915
> Project: Traffic Server
>  Issue Type: Bug
>  Components: HostDB
>Reporter: Susan Hinrichs
>Priority: Blocker
> Fix For: 7.1.0
>
> Attachments: ts-4915.diff, ts-4915.diff
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> Saw this while testing fix for TS-4813 with debug enabled.
> {code}
> (gdb) bt full
> #0  0x00547bfe in RefCountCacheHashEntry::operator< (this=0x1cc0880, 
> v2=...) at ../iocore/hostdb/P_RefCountCache.h:94
> No locals.
> #1  0x0054988d in 
> PriorityQueueLess::operator() (this=0x2b78a9a2587b, 
> a=@0x2b78f402af68, b=@0x2b78f402aa28)
> at ../lib/ts/PriorityQueue.h:41
> No locals.
> #2  0x00549785 in PriorityQueue PriorityQueueLess >::_bubble_up (this=0x1cb2990, 
> index=2) at ../lib/ts/PriorityQueue.h:191
> comp = {}
> parent = 0
> #3  0x006ecfcc in PriorityQueue PriorityQueueLess >::push (this=0x1cb2990, 
> entry=0x2b78f402af60) at ../../lib/ts/PriorityQueue.h:91
> len = 2
> #4  0x006ec206 in RefCountCachePartition::put 
> (this=0x1cb2900, key=6912554662447498853, item=0x2b78aee04f00, size=96, 
> expire_time=1475202356) at ./P_RefCountCache.h:210
> expiry_entry = 0x2b78f402af60
> __func__ = "put"
> val = 0x1cc0880
> #5  0x006eb3de in RefCountCache::put (this=0x18051e0, 
> key=6912554662447498853, item=0x2b78aee04f00, size=16, 
> expiry_time=1475202356) at ./P_RefCountCache.h:462
> No locals.
> #6  0x006e2d8e in HostDBContinuation::dnsEvent (this=0x2b7938020f00, 
> event=600, e=0x2b78ac009440) at HostDB.cc:1422
> is_rr = false
> old_rr_data = 0x0
> first_record = 0x2b78ac0094f8
> m = 0x1
> failed = false
> old_r = {m_ptr = 0x0}
> af = 2 '\002'
> s_size = 16
> rrsize = 0
> allocSize = 16
> r = 0x2b78aee04f00
> old_info = { = { = {_vptr.ForceVFPTToTop 
> = 0x7f3630}, m_refcount = 0}, iobuffer_index = 0, 
>   key = 47797242059264, app = {allotment = {application1 = 5326300, 
> application2 = 0}, http_data = {http_version = 4, 
>   pipeline_max = 59, keepalive_timeout = 17, fail_count = 81, 
> unused1 = 0, last_failure = 0}, rr = {offset = 5326300}}, data = {
> ip = {sa = {sa_family = 54488, sa_data = 
> "^\000\000\000\000\000\020\034$\274x+\000"}, sin = {sin_family = 54488, 
> sin_port = 94, 
> sin_addr = {s_addr = 0}, sin_zero = "\020\034$\274x+\000"}, 
> sin6 = {sin6_family = 54488, sin6_port = 94, sin6_flowinfo = 0, 
> sin6_addr = {__in6_u = {__u6_addr8 = 
> "\020\034$\274x+\000\000\030\036$\274\375\b\000", __u6_addr16 = {7184, 48164, 
> 11128, 
>   0, 7704, 48164, 2301, 0}, __u6_addr32 = {3156483088, 
> 11128, 3156483608, 2301}}}, sin6_scope_id = 3156478176}}, 
> hostname_offset = 6214872, srv = {srv_offset = 54488, srv_weight 
> = 94, srv_priority = 0, srv_port = 0, key = 3156483088}}, 
>   hostname_offset = 11128, ip_timestamp = 2845989456, 
> ip_timeout_interval = 11128, is_srv = 0, reverse_dns = 0, round_robin = 1, 
>   round_robin_elt = 0}
> valid_records = 0
> tip = {_family = 2, _addr = {_ip4 = 540420056, _ip6 = {__in6_u = 
> {__u6_addr8 = "\330'6 x+\000\000\360L\020\250x+\000", 
> __u6_addr16 = {10200, 8246, 11128, 0, 19696, 43024, 11128, 
> 0}, __u6_addr32 = {540420056, 11128, 2819640560, 11128}}}, 
> _byte = "\330'6 x+\000\000\360L\020\250x+\000", _u32 = 
> {540420056, 11128, 2819640560, 11128}, 

[jira] [Work logged] (TS-4915) Crash from hostdb in PriorityQueueLess

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-4915?focusedWorklogId=30384=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-30384
 ]

ASF GitHub Bot logged work on TS-4915:
--

Author: ASF GitHub Bot
Created on: 11/Oct/16 17:58
Start Date: 11/Oct/16 17:58
Worklog Time Spent: 10m 
  Work Description: Github user PSUdaemon commented on a diff in the pull 
request:

https://github.com/apache/trafficserver/pull/1088#discussion_r82852063
  
--- Diff: lib/ts/PriorityQueue.h ---
@@ -110,6 +110,12 @@ PriorityQueue::pop()
 return;
   }
 
+  // SKH - I suspect this assignment is not preserving entry indices 
correctly.
--- End diff --

No need to put your initials in the comment as `git blame` will tell us who 
added this if someone is curious.


Issue Time Tracking
---

Worklog Id: (was: 30384)
Time Spent: 1h  (was: 50m)

> Crash from hostdb in PriorityQueueLess
> --
>
> Key: TS-4915
> URL: https://issues.apache.org/jira/browse/TS-4915
> Project: Traffic Server
>  Issue Type: Bug
>  Components: HostDB
>Reporter: Susan Hinrichs
>Priority: Blocker
> Fix For: 7.1.0
>
> Attachments: ts-4915.diff, ts-4915.diff
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> Saw this while testing fix for TS-4813 with debug enabled.
> {code}
> (gdb) bt full
> #0  0x00547bfe in RefCountCacheHashEntry::operator< (this=0x1cc0880, 
> v2=...) at ../iocore/hostdb/P_RefCountCache.h:94
> No locals.
> #1  0x0054988d in 
> PriorityQueueLess::operator() (this=0x2b78a9a2587b, 
> a=@0x2b78f402af68, b=@0x2b78f402aa28)
> at ../lib/ts/PriorityQueue.h:41
> No locals.
> #2  0x00549785 in PriorityQueue PriorityQueueLess >::_bubble_up (this=0x1cb2990, 
> index=2) at ../lib/ts/PriorityQueue.h:191
> comp = {}
> parent = 0
> #3  0x006ecfcc in PriorityQueue PriorityQueueLess >::push (this=0x1cb2990, 
> entry=0x2b78f402af60) at ../../lib/ts/PriorityQueue.h:91
> len = 2
> #4  0x006ec206 in RefCountCachePartition::put 
> (this=0x1cb2900, key=6912554662447498853, item=0x2b78aee04f00, size=96, 
> expire_time=1475202356) at ./P_RefCountCache.h:210
> expiry_entry = 0x2b78f402af60
> __func__ = "put"
> val = 0x1cc0880
> #5  0x006eb3de in RefCountCache::put (this=0x18051e0, 
> key=6912554662447498853, item=0x2b78aee04f00, size=16, 
> expiry_time=1475202356) at ./P_RefCountCache.h:462
> No locals.
> #6  0x006e2d8e in HostDBContinuation::dnsEvent (this=0x2b7938020f00, 
> event=600, e=0x2b78ac009440) at HostDB.cc:1422
> is_rr = false
> old_rr_data = 0x0
> first_record = 0x2b78ac0094f8
> m = 0x1
> failed = false
> old_r = {m_ptr = 0x0}
> af = 2 '\002'
> s_size = 16
> rrsize = 0
> allocSize = 16
> r = 0x2b78aee04f00
> old_info = { = { = {_vptr.ForceVFPTToTop 
> = 0x7f3630}, m_refcount = 0}, iobuffer_index = 0, 
>   key = 47797242059264, app = {allotment = {application1 = 5326300, 
> application2 = 0}, http_data = {http_version = 4, 
>   pipeline_max = 59, keepalive_timeout = 17, fail_count = 81, 
> unused1 = 0, last_failure = 0}, rr = {offset = 5326300}}, data = {
> ip = {sa = {sa_family = 54488, sa_data = 
> "^\000\000\000\000\000\020\034$\274x+\000"}, sin = {sin_family = 54488, 
> sin_port = 94, 
> sin_addr = {s_addr = 0}, sin_zero = "\020\034$\274x+\000"}, 
> sin6 = {sin6_family = 54488, sin6_port = 94, sin6_flowinfo = 0, 
> sin6_addr = {__in6_u = {__u6_addr8 = 
> "\020\034$\274x+\000\000\030\036$\274\375\b\000", __u6_addr16 = {7184, 48164, 
> 11128, 
>   0, 7704, 48164, 2301, 0}, __u6_addr32 = {3156483088, 
> 11128, 3156483608, 2301}}}, sin6_scope_id = 3156478176}}, 
> hostname_offset = 6214872, srv = {srv_offset = 54488, srv_weight 
> = 94, srv_priority = 0, srv_port = 0, key = 3156483088}}, 
>   hostname_offset = 11128, ip_timestamp = 2845989456, 
> ip_timeout_interval = 11128, is_srv = 0, reverse_dns = 0, round_robin = 1, 
>   round_robin_elt = 0}
> valid_records = 0
> tip = {_family = 2, _addr = {_ip4 = 540420056, _ip6 = {__in6_u = 
> {__u6_addr8 = "\330'6 x+\000\000\360L\020\250x+\000", 
> __u6_addr16 = {10200, 8246, 11128, 0, 19696, 43024, 11128, 
> 0}, __u6_addr32 = {540420056, 11128, 2819640560, 11128}}}, 
> _byte = "\330'6 x+\000\000\360L\020\250x+\000", _u32 = 
> {540420056, 11128, 2819640560, 11128}, _u64 = {47794936489944, 
>  

[jira] [Work logged] (TS-4915) Crash from hostdb in PriorityQueueLess

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-4915?focusedWorklogId=30380=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-30380
 ]

ASF GitHub Bot logged work on TS-4915:
--

Author: ASF GitHub Bot
Created on: 11/Oct/16 16:43
Start Date: 11/Oct/16 16:43
Worklog Time Spent: 10m 
  Work Description: Github user bryancall commented on a diff in the pull 
request:

https://github.com/apache/trafficserver/pull/1088#discussion_r82836137
  
--- Diff: lib/ts/PriorityQueue.h ---
@@ -123,11 +129,18 @@ PriorityQueue::erase(PriorityQueueEntry 
*entry)
 return;
   }
 
-  _v[entry->index] = _v[_v.length() - 1];
-  _v.pop();
-  _bubble_down(entry->index);
-  if (!empty()) {
-_bubble_up(entry->index);
+  ink_release_assert(entry->index < _v.length());
+  unsigned int original_index = entry->index;
+  if (original_index != (_v.length() - 1)) {
+// Move the erased item to the end to be popped off
+_swap(original_index, _v.length() - 1);
+_v.pop();
+_bubble_down(original_index);
+if (!empty()) {
--- End diff --

Wouldn't this always be true and why are we doing a bubble up it is is in 
sorted order.


Issue Time Tracking
---

Worklog Id: (was: 30380)
Time Spent: 50m  (was: 40m)

> Crash from hostdb in PriorityQueueLess
> --
>
> Key: TS-4915
> URL: https://issues.apache.org/jira/browse/TS-4915
> Project: Traffic Server
>  Issue Type: Bug
>  Components: HostDB
>Reporter: Susan Hinrichs
>Priority: Blocker
> Fix For: 7.1.0
>
> Attachments: ts-4915.diff, ts-4915.diff
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Saw this while testing fix for TS-4813 with debug enabled.
> {code}
> (gdb) bt full
> #0  0x00547bfe in RefCountCacheHashEntry::operator< (this=0x1cc0880, 
> v2=...) at ../iocore/hostdb/P_RefCountCache.h:94
> No locals.
> #1  0x0054988d in 
> PriorityQueueLess::operator() (this=0x2b78a9a2587b, 
> a=@0x2b78f402af68, b=@0x2b78f402aa28)
> at ../lib/ts/PriorityQueue.h:41
> No locals.
> #2  0x00549785 in PriorityQueue PriorityQueueLess >::_bubble_up (this=0x1cb2990, 
> index=2) at ../lib/ts/PriorityQueue.h:191
> comp = {}
> parent = 0
> #3  0x006ecfcc in PriorityQueue PriorityQueueLess >::push (this=0x1cb2990, 
> entry=0x2b78f402af60) at ../../lib/ts/PriorityQueue.h:91
> len = 2
> #4  0x006ec206 in RefCountCachePartition::put 
> (this=0x1cb2900, key=6912554662447498853, item=0x2b78aee04f00, size=96, 
> expire_time=1475202356) at ./P_RefCountCache.h:210
> expiry_entry = 0x2b78f402af60
> __func__ = "put"
> val = 0x1cc0880
> #5  0x006eb3de in RefCountCache::put (this=0x18051e0, 
> key=6912554662447498853, item=0x2b78aee04f00, size=16, 
> expiry_time=1475202356) at ./P_RefCountCache.h:462
> No locals.
> #6  0x006e2d8e in HostDBContinuation::dnsEvent (this=0x2b7938020f00, 
> event=600, e=0x2b78ac009440) at HostDB.cc:1422
> is_rr = false
> old_rr_data = 0x0
> first_record = 0x2b78ac0094f8
> m = 0x1
> failed = false
> old_r = {m_ptr = 0x0}
> af = 2 '\002'
> s_size = 16
> rrsize = 0
> allocSize = 16
> r = 0x2b78aee04f00
> old_info = { = { = {_vptr.ForceVFPTToTop 
> = 0x7f3630}, m_refcount = 0}, iobuffer_index = 0, 
>   key = 47797242059264, app = {allotment = {application1 = 5326300, 
> application2 = 0}, http_data = {http_version = 4, 
>   pipeline_max = 59, keepalive_timeout = 17, fail_count = 81, 
> unused1 = 0, last_failure = 0}, rr = {offset = 5326300}}, data = {
> ip = {sa = {sa_family = 54488, sa_data = 
> "^\000\000\000\000\000\020\034$\274x+\000"}, sin = {sin_family = 54488, 
> sin_port = 94, 
> sin_addr = {s_addr = 0}, sin_zero = "\020\034$\274x+\000"}, 
> sin6 = {sin6_family = 54488, sin6_port = 94, sin6_flowinfo = 0, 
> sin6_addr = {__in6_u = {__u6_addr8 = 
> "\020\034$\274x+\000\000\030\036$\274\375\b\000", __u6_addr16 = {7184, 48164, 
> 11128, 
>   0, 7704, 48164, 2301, 0}, __u6_addr32 = {3156483088, 
> 11128, 3156483608, 2301}}}, sin6_scope_id = 3156478176}}, 
> hostname_offset = 6214872, srv = {srv_offset = 54488, srv_weight 
> = 94, srv_priority = 0, srv_port = 0, key = 3156483088}}, 
>   hostname_offset = 11128, ip_timestamp = 2845989456, 
> ip_timeout_interval = 11128, is_srv = 0, reverse_dns = 0, round_robin = 1, 
>   round_robin_elt = 0}
> 

[jira] [Work logged] (TS-4915) Crash from hostdb in PriorityQueueLess

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-4915?focusedWorklogId=30381=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-30381
 ]

ASF GitHub Bot logged work on TS-4915:
--

Author: ASF GitHub Bot
Created on: 11/Oct/16 16:43
Start Date: 11/Oct/16 16:43
Worklog Time Spent: 10m 
  Work Description: Github user bryancall commented on a diff in the pull 
request:

https://github.com/apache/trafficserver/pull/1088#discussion_r82832753
  
--- Diff: lib/ts/PriorityQueue.h ---
@@ -123,11 +129,18 @@ PriorityQueue::erase(PriorityQueueEntry 
*entry)
 return;
   }
 
-  _v[entry->index] = _v[_v.length() - 1];
-  _v.pop();
-  _bubble_down(entry->index);
-  if (!empty()) {
-_bubble_up(entry->index);
+  ink_release_assert(entry->index < _v.length());
+  unsigned int original_index = entry->index;
--- End diff --

Should be a const uint32_t


Issue Time Tracking
---

Worklog Id: (was: 30381)
Time Spent: 50m  (was: 40m)

> Crash from hostdb in PriorityQueueLess
> --
>
> Key: TS-4915
> URL: https://issues.apache.org/jira/browse/TS-4915
> Project: Traffic Server
>  Issue Type: Bug
>  Components: HostDB
>Reporter: Susan Hinrichs
>Priority: Blocker
> Fix For: 7.1.0
>
> Attachments: ts-4915.diff, ts-4915.diff
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Saw this while testing fix for TS-4813 with debug enabled.
> {code}
> (gdb) bt full
> #0  0x00547bfe in RefCountCacheHashEntry::operator< (this=0x1cc0880, 
> v2=...) at ../iocore/hostdb/P_RefCountCache.h:94
> No locals.
> #1  0x0054988d in 
> PriorityQueueLess::operator() (this=0x2b78a9a2587b, 
> a=@0x2b78f402af68, b=@0x2b78f402aa28)
> at ../lib/ts/PriorityQueue.h:41
> No locals.
> #2  0x00549785 in PriorityQueue PriorityQueueLess >::_bubble_up (this=0x1cb2990, 
> index=2) at ../lib/ts/PriorityQueue.h:191
> comp = {}
> parent = 0
> #3  0x006ecfcc in PriorityQueue PriorityQueueLess >::push (this=0x1cb2990, 
> entry=0x2b78f402af60) at ../../lib/ts/PriorityQueue.h:91
> len = 2
> #4  0x006ec206 in RefCountCachePartition::put 
> (this=0x1cb2900, key=6912554662447498853, item=0x2b78aee04f00, size=96, 
> expire_time=1475202356) at ./P_RefCountCache.h:210
> expiry_entry = 0x2b78f402af60
> __func__ = "put"
> val = 0x1cc0880
> #5  0x006eb3de in RefCountCache::put (this=0x18051e0, 
> key=6912554662447498853, item=0x2b78aee04f00, size=16, 
> expiry_time=1475202356) at ./P_RefCountCache.h:462
> No locals.
> #6  0x006e2d8e in HostDBContinuation::dnsEvent (this=0x2b7938020f00, 
> event=600, e=0x2b78ac009440) at HostDB.cc:1422
> is_rr = false
> old_rr_data = 0x0
> first_record = 0x2b78ac0094f8
> m = 0x1
> failed = false
> old_r = {m_ptr = 0x0}
> af = 2 '\002'
> s_size = 16
> rrsize = 0
> allocSize = 16
> r = 0x2b78aee04f00
> old_info = { = { = {_vptr.ForceVFPTToTop 
> = 0x7f3630}, m_refcount = 0}, iobuffer_index = 0, 
>   key = 47797242059264, app = {allotment = {application1 = 5326300, 
> application2 = 0}, http_data = {http_version = 4, 
>   pipeline_max = 59, keepalive_timeout = 17, fail_count = 81, 
> unused1 = 0, last_failure = 0}, rr = {offset = 5326300}}, data = {
> ip = {sa = {sa_family = 54488, sa_data = 
> "^\000\000\000\000\000\020\034$\274x+\000"}, sin = {sin_family = 54488, 
> sin_port = 94, 
> sin_addr = {s_addr = 0}, sin_zero = "\020\034$\274x+\000"}, 
> sin6 = {sin6_family = 54488, sin6_port = 94, sin6_flowinfo = 0, 
> sin6_addr = {__in6_u = {__u6_addr8 = 
> "\020\034$\274x+\000\000\030\036$\274\375\b\000", __u6_addr16 = {7184, 48164, 
> 11128, 
>   0, 7704, 48164, 2301, 0}, __u6_addr32 = {3156483088, 
> 11128, 3156483608, 2301}}}, sin6_scope_id = 3156478176}}, 
> hostname_offset = 6214872, srv = {srv_offset = 54488, srv_weight 
> = 94, srv_priority = 0, srv_port = 0, key = 3156483088}}, 
>   hostname_offset = 11128, ip_timestamp = 2845989456, 
> ip_timeout_interval = 11128, is_srv = 0, reverse_dns = 0, round_robin = 1, 
>   round_robin_elt = 0}
> valid_records = 0
> tip = {_family = 2, _addr = {_ip4 = 540420056, _ip6 = {__in6_u = 
> {__u6_addr8 = "\330'6 x+\000\000\360L\020\250x+\000", 
> __u6_addr16 = {10200, 8246, 11128, 0, 19696, 43024, 11128, 
> 0}, __u6_addr32 = {540420056, 11128, 2819640560, 11128}}}, 
> 

[jira] [Work logged] (TS-4915) Crash from hostdb in PriorityQueueLess

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-4915?focusedWorklogId=30377=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-30377
 ]

ASF GitHub Bot logged work on TS-4915:
--

Author: ASF GitHub Bot
Created on: 11/Oct/16 16:22
Start Date: 11/Oct/16 16:22
Worklog Time Spent: 10m 
  Work Description: Github user jacksontj commented on the issue:

https://github.com/apache/trafficserver/pull/1088
  
Re-pasting comments from IRC-- for easier retrieval.

The pop you remove does seem extraneous-- and there are test cases for some 
of the refcountcache stuff 
(https://github.com/apache/trafficserver/blob/master/iocore/hostdb/test_RefCountCache.cc)
 should be easy enough to make a test case for evictions).

As for the priority queue, I ran into a couple issues of it not doing what 
I wanted either. Thankfully that has some decent tests-- so assuming you can 
figure out the sequence of events that causes the problem-- adding a test case 
is quite simple 
(https://github.com/apache/trafficserver/blob/master/lib/ts/test_PriorityQueue.cc)


Issue Time Tracking
---

Worklog Id: (was: 30377)
Time Spent: 40m  (was: 0.5h)

> Crash from hostdb in PriorityQueueLess
> --
>
> Key: TS-4915
> URL: https://issues.apache.org/jira/browse/TS-4915
> Project: Traffic Server
>  Issue Type: Bug
>  Components: HostDB
>Reporter: Susan Hinrichs
>Priority: Blocker
> Fix For: 7.1.0
>
> Attachments: ts-4915.diff, ts-4915.diff
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> Saw this while testing fix for TS-4813 with debug enabled.
> {code}
> (gdb) bt full
> #0  0x00547bfe in RefCountCacheHashEntry::operator< (this=0x1cc0880, 
> v2=...) at ../iocore/hostdb/P_RefCountCache.h:94
> No locals.
> #1  0x0054988d in 
> PriorityQueueLess::operator() (this=0x2b78a9a2587b, 
> a=@0x2b78f402af68, b=@0x2b78f402aa28)
> at ../lib/ts/PriorityQueue.h:41
> No locals.
> #2  0x00549785 in PriorityQueue PriorityQueueLess >::_bubble_up (this=0x1cb2990, 
> index=2) at ../lib/ts/PriorityQueue.h:191
> comp = {}
> parent = 0
> #3  0x006ecfcc in PriorityQueue PriorityQueueLess >::push (this=0x1cb2990, 
> entry=0x2b78f402af60) at ../../lib/ts/PriorityQueue.h:91
> len = 2
> #4  0x006ec206 in RefCountCachePartition::put 
> (this=0x1cb2900, key=6912554662447498853, item=0x2b78aee04f00, size=96, 
> expire_time=1475202356) at ./P_RefCountCache.h:210
> expiry_entry = 0x2b78f402af60
> __func__ = "put"
> val = 0x1cc0880
> #5  0x006eb3de in RefCountCache::put (this=0x18051e0, 
> key=6912554662447498853, item=0x2b78aee04f00, size=16, 
> expiry_time=1475202356) at ./P_RefCountCache.h:462
> No locals.
> #6  0x006e2d8e in HostDBContinuation::dnsEvent (this=0x2b7938020f00, 
> event=600, e=0x2b78ac009440) at HostDB.cc:1422
> is_rr = false
> old_rr_data = 0x0
> first_record = 0x2b78ac0094f8
> m = 0x1
> failed = false
> old_r = {m_ptr = 0x0}
> af = 2 '\002'
> s_size = 16
> rrsize = 0
> allocSize = 16
> r = 0x2b78aee04f00
> old_info = { = { = {_vptr.ForceVFPTToTop 
> = 0x7f3630}, m_refcount = 0}, iobuffer_index = 0, 
>   key = 47797242059264, app = {allotment = {application1 = 5326300, 
> application2 = 0}, http_data = {http_version = 4, 
>   pipeline_max = 59, keepalive_timeout = 17, fail_count = 81, 
> unused1 = 0, last_failure = 0}, rr = {offset = 5326300}}, data = {
> ip = {sa = {sa_family = 54488, sa_data = 
> "^\000\000\000\000\000\020\034$\274x+\000"}, sin = {sin_family = 54488, 
> sin_port = 94, 
> sin_addr = {s_addr = 0}, sin_zero = "\020\034$\274x+\000"}, 
> sin6 = {sin6_family = 54488, sin6_port = 94, sin6_flowinfo = 0, 
> sin6_addr = {__in6_u = {__u6_addr8 = 
> "\020\034$\274x+\000\000\030\036$\274\375\b\000", __u6_addr16 = {7184, 48164, 
> 11128, 
>   0, 7704, 48164, 2301, 0}, __u6_addr32 = {3156483088, 
> 11128, 3156483608, 2301}}}, sin6_scope_id = 3156478176}}, 
> hostname_offset = 6214872, srv = {srv_offset = 54488, srv_weight 
> = 94, srv_priority = 0, srv_port = 0, key = 3156483088}}, 
>   hostname_offset = 11128, ip_timestamp = 2845989456, 
> ip_timeout_interval = 11128, is_srv = 0, reverse_dns = 0, round_robin = 1, 
>   round_robin_elt = 0}
> valid_records = 0
> tip = {_family = 2, _addr = {_ip4 = 540420056, _ip6 = {__in6_u = 
> {__u6_addr8 = "\330'6 x+\000\000\360L\020\250x+\000", 
> 

[jira] [Work logged] (TS-4915) Crash from hostdb in PriorityQueueLess

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-4915?focusedWorklogId=30357=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-30357
 ]

ASF GitHub Bot logged work on TS-4915:
--

Author: ASF GitHub Bot
Created on: 11/Oct/16 09:44
Start Date: 11/Oct/16 09:44
Worklog Time Spent: 10m 
  Work Description: Github user atsci commented on the issue:

https://github.com/apache/trafficserver/pull/1088
  
Linux build *successful*! See 
https://ci.trafficserver.apache.org/job/Github-Linux/849/ for details.
 



Issue Time Tracking
---

Worklog Id: (was: 30357)
Time Spent: 0.5h  (was: 20m)

> Crash from hostdb in PriorityQueueLess
> --
>
> Key: TS-4915
> URL: https://issues.apache.org/jira/browse/TS-4915
> Project: Traffic Server
>  Issue Type: Bug
>  Components: HostDB
>Reporter: Susan Hinrichs
>Priority: Blocker
> Fix For: 7.1.0
>
> Attachments: ts-4915.diff, ts-4915.diff
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> Saw this while testing fix for TS-4813 with debug enabled.
> {code}
> (gdb) bt full
> #0  0x00547bfe in RefCountCacheHashEntry::operator< (this=0x1cc0880, 
> v2=...) at ../iocore/hostdb/P_RefCountCache.h:94
> No locals.
> #1  0x0054988d in 
> PriorityQueueLess::operator() (this=0x2b78a9a2587b, 
> a=@0x2b78f402af68, b=@0x2b78f402aa28)
> at ../lib/ts/PriorityQueue.h:41
> No locals.
> #2  0x00549785 in PriorityQueue PriorityQueueLess >::_bubble_up (this=0x1cb2990, 
> index=2) at ../lib/ts/PriorityQueue.h:191
> comp = {}
> parent = 0
> #3  0x006ecfcc in PriorityQueue PriorityQueueLess >::push (this=0x1cb2990, 
> entry=0x2b78f402af60) at ../../lib/ts/PriorityQueue.h:91
> len = 2
> #4  0x006ec206 in RefCountCachePartition::put 
> (this=0x1cb2900, key=6912554662447498853, item=0x2b78aee04f00, size=96, 
> expire_time=1475202356) at ./P_RefCountCache.h:210
> expiry_entry = 0x2b78f402af60
> __func__ = "put"
> val = 0x1cc0880
> #5  0x006eb3de in RefCountCache::put (this=0x18051e0, 
> key=6912554662447498853, item=0x2b78aee04f00, size=16, 
> expiry_time=1475202356) at ./P_RefCountCache.h:462
> No locals.
> #6  0x006e2d8e in HostDBContinuation::dnsEvent (this=0x2b7938020f00, 
> event=600, e=0x2b78ac009440) at HostDB.cc:1422
> is_rr = false
> old_rr_data = 0x0
> first_record = 0x2b78ac0094f8
> m = 0x1
> failed = false
> old_r = {m_ptr = 0x0}
> af = 2 '\002'
> s_size = 16
> rrsize = 0
> allocSize = 16
> r = 0x2b78aee04f00
> old_info = { = { = {_vptr.ForceVFPTToTop 
> = 0x7f3630}, m_refcount = 0}, iobuffer_index = 0, 
>   key = 47797242059264, app = {allotment = {application1 = 5326300, 
> application2 = 0}, http_data = {http_version = 4, 
>   pipeline_max = 59, keepalive_timeout = 17, fail_count = 81, 
> unused1 = 0, last_failure = 0}, rr = {offset = 5326300}}, data = {
> ip = {sa = {sa_family = 54488, sa_data = 
> "^\000\000\000\000\000\020\034$\274x+\000"}, sin = {sin_family = 54488, 
> sin_port = 94, 
> sin_addr = {s_addr = 0}, sin_zero = "\020\034$\274x+\000"}, 
> sin6 = {sin6_family = 54488, sin6_port = 94, sin6_flowinfo = 0, 
> sin6_addr = {__in6_u = {__u6_addr8 = 
> "\020\034$\274x+\000\000\030\036$\274\375\b\000", __u6_addr16 = {7184, 48164, 
> 11128, 
>   0, 7704, 48164, 2301, 0}, __u6_addr32 = {3156483088, 
> 11128, 3156483608, 2301}}}, sin6_scope_id = 3156478176}}, 
> hostname_offset = 6214872, srv = {srv_offset = 54488, srv_weight 
> = 94, srv_priority = 0, srv_port = 0, key = 3156483088}}, 
>   hostname_offset = 11128, ip_timestamp = 2845989456, 
> ip_timeout_interval = 11128, is_srv = 0, reverse_dns = 0, round_robin = 1, 
>   round_robin_elt = 0}
> valid_records = 0
> tip = {_family = 2, _addr = {_ip4 = 540420056, _ip6 = {__in6_u = 
> {__u6_addr8 = "\330'6 x+\000\000\360L\020\250x+\000", 
> __u6_addr16 = {10200, 8246, 11128, 0, 19696, 43024, 11128, 
> 0}, __u6_addr32 = {540420056, 11128, 2819640560, 11128}}}, 
> _byte = "\330'6 x+\000\000\360L\020\250x+\000", _u32 = 
> {540420056, 11128, 2819640560, 11128}, _u64 = {47794936489944, 
>   47797215710448}}}
> ttl_seconds = 132
> aname = 0x2b7938021000 "fbmm1.zenfs.com"
> offset = 96
> thread = 0x2b78a8101010
> __func__ = "dnsEvent"
> #7  0x005145dc in Continuation::handleEvent (this=0x2b7938020f00, 
> 

[jira] [Work logged] (TS-4915) Crash from hostdb in PriorityQueueLess

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-4915?focusedWorklogId=30356=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-30356
 ]

ASF GitHub Bot logged work on TS-4915:
--

Author: ASF GitHub Bot
Created on: 11/Oct/16 09:42
Start Date: 11/Oct/16 09:42
Worklog Time Spent: 10m 
  Work Description: Github user atsci commented on the issue:

https://github.com/apache/trafficserver/pull/1088
  
FreeBSD build *successful*! See 
https://ci.trafficserver.apache.org/job/Github-FreeBSD/957/ for details.
 



Issue Time Tracking
---

Worklog Id: (was: 30356)
Time Spent: 20m  (was: 10m)

> Crash from hostdb in PriorityQueueLess
> --
>
> Key: TS-4915
> URL: https://issues.apache.org/jira/browse/TS-4915
> Project: Traffic Server
>  Issue Type: Bug
>  Components: HostDB
>Reporter: Susan Hinrichs
>Priority: Blocker
> Fix For: 7.1.0
>
> Attachments: ts-4915.diff, ts-4915.diff
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Saw this while testing fix for TS-4813 with debug enabled.
> {code}
> (gdb) bt full
> #0  0x00547bfe in RefCountCacheHashEntry::operator< (this=0x1cc0880, 
> v2=...) at ../iocore/hostdb/P_RefCountCache.h:94
> No locals.
> #1  0x0054988d in 
> PriorityQueueLess::operator() (this=0x2b78a9a2587b, 
> a=@0x2b78f402af68, b=@0x2b78f402aa28)
> at ../lib/ts/PriorityQueue.h:41
> No locals.
> #2  0x00549785 in PriorityQueue PriorityQueueLess >::_bubble_up (this=0x1cb2990, 
> index=2) at ../lib/ts/PriorityQueue.h:191
> comp = {}
> parent = 0
> #3  0x006ecfcc in PriorityQueue PriorityQueueLess >::push (this=0x1cb2990, 
> entry=0x2b78f402af60) at ../../lib/ts/PriorityQueue.h:91
> len = 2
> #4  0x006ec206 in RefCountCachePartition::put 
> (this=0x1cb2900, key=6912554662447498853, item=0x2b78aee04f00, size=96, 
> expire_time=1475202356) at ./P_RefCountCache.h:210
> expiry_entry = 0x2b78f402af60
> __func__ = "put"
> val = 0x1cc0880
> #5  0x006eb3de in RefCountCache::put (this=0x18051e0, 
> key=6912554662447498853, item=0x2b78aee04f00, size=16, 
> expiry_time=1475202356) at ./P_RefCountCache.h:462
> No locals.
> #6  0x006e2d8e in HostDBContinuation::dnsEvent (this=0x2b7938020f00, 
> event=600, e=0x2b78ac009440) at HostDB.cc:1422
> is_rr = false
> old_rr_data = 0x0
> first_record = 0x2b78ac0094f8
> m = 0x1
> failed = false
> old_r = {m_ptr = 0x0}
> af = 2 '\002'
> s_size = 16
> rrsize = 0
> allocSize = 16
> r = 0x2b78aee04f00
> old_info = { = { = {_vptr.ForceVFPTToTop 
> = 0x7f3630}, m_refcount = 0}, iobuffer_index = 0, 
>   key = 47797242059264, app = {allotment = {application1 = 5326300, 
> application2 = 0}, http_data = {http_version = 4, 
>   pipeline_max = 59, keepalive_timeout = 17, fail_count = 81, 
> unused1 = 0, last_failure = 0}, rr = {offset = 5326300}}, data = {
> ip = {sa = {sa_family = 54488, sa_data = 
> "^\000\000\000\000\000\020\034$\274x+\000"}, sin = {sin_family = 54488, 
> sin_port = 94, 
> sin_addr = {s_addr = 0}, sin_zero = "\020\034$\274x+\000"}, 
> sin6 = {sin6_family = 54488, sin6_port = 94, sin6_flowinfo = 0, 
> sin6_addr = {__in6_u = {__u6_addr8 = 
> "\020\034$\274x+\000\000\030\036$\274\375\b\000", __u6_addr16 = {7184, 48164, 
> 11128, 
>   0, 7704, 48164, 2301, 0}, __u6_addr32 = {3156483088, 
> 11128, 3156483608, 2301}}}, sin6_scope_id = 3156478176}}, 
> hostname_offset = 6214872, srv = {srv_offset = 54488, srv_weight 
> = 94, srv_priority = 0, srv_port = 0, key = 3156483088}}, 
>   hostname_offset = 11128, ip_timestamp = 2845989456, 
> ip_timeout_interval = 11128, is_srv = 0, reverse_dns = 0, round_robin = 1, 
>   round_robin_elt = 0}
> valid_records = 0
> tip = {_family = 2, _addr = {_ip4 = 540420056, _ip6 = {__in6_u = 
> {__u6_addr8 = "\330'6 x+\000\000\360L\020\250x+\000", 
> __u6_addr16 = {10200, 8246, 11128, 0, 19696, 43024, 11128, 
> 0}, __u6_addr32 = {540420056, 11128, 2819640560, 11128}}}, 
> _byte = "\330'6 x+\000\000\360L\020\250x+\000", _u32 = 
> {540420056, 11128, 2819640560, 11128}, _u64 = {47794936489944, 
>   47797215710448}}}
> ttl_seconds = 132
> aname = 0x2b7938021000 "fbmm1.zenfs.com"
> offset = 96
> thread = 0x2b78a8101010
> __func__ = "dnsEvent"
> #7  0x005145dc in Continuation::handleEvent (this=0x2b7938020f00, 
> 

[jira] [Work logged] (TS-4915) Crash from hostdb in PriorityQueueLess

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-4915?focusedWorklogId=30355=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-30355
 ]

ASF GitHub Bot logged work on TS-4915:
--

Author: ASF GitHub Bot
Created on: 11/Oct/16 09:27
Start Date: 11/Oct/16 09:27
Worklog Time Spent: 10m 
  Work Description: GitHub user shinrich opened a pull request:

https://github.com/apache/trafficserver/pull/1088

TS-4915: Crash from hostdb in PriorityQueueLess

These changes have been running on my production box since leaving work 
Monday night.  Will keep an eye on it.  Lower traffic overnight might not be 
stressing it sufficiently.

The main change was in PriorityQueueLess<>::erase.  The assignment of the 
end item to the erase point was not preserving the entry index.  So the 
assumption that entry->index is less than _v.length() was made invalid the next 
time around.  I think breaking this entry->index == _v index assignment can 
also harm the bubble_sorting logic.  I think PriorityQueueLess<>::pop also has 
a problem, but my work load was not triggering that function, so I didn't dive 
in there.

The other change was in RefCountCachePartition::make_space_for.  There 
was an extra pop which I believe was doubly removing an entry already removed 
in PriorityQueueLess::erase (called from RefCountCachePartition::erase).

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/shinrich/trafficserver ts-4915-2

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/trafficserver/pull/1088.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1088


commit 0898a59bc33d63d18997a66437c808acd2e7e073
Author: Susan Hinrichs 
Date:   2016-10-11T09:20:11Z

TS-4915: Crash from hostdb in PriorityQueueLess




Issue Time Tracking
---

Worklog Id: (was: 30355)
Time Spent: 10m
Remaining Estimate: 0h

> Crash from hostdb in PriorityQueueLess
> --
>
> Key: TS-4915
> URL: https://issues.apache.org/jira/browse/TS-4915
> Project: Traffic Server
>  Issue Type: Bug
>  Components: HostDB
>Reporter: Susan Hinrichs
>Priority: Blocker
> Fix For: 7.1.0
>
> Attachments: ts-4915.diff, ts-4915.diff
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Saw this while testing fix for TS-4813 with debug enabled.
> {code}
> (gdb) bt full
> #0  0x00547bfe in RefCountCacheHashEntry::operator< (this=0x1cc0880, 
> v2=...) at ../iocore/hostdb/P_RefCountCache.h:94
> No locals.
> #1  0x0054988d in 
> PriorityQueueLess::operator() (this=0x2b78a9a2587b, 
> a=@0x2b78f402af68, b=@0x2b78f402aa28)
> at ../lib/ts/PriorityQueue.h:41
> No locals.
> #2  0x00549785 in PriorityQueue PriorityQueueLess >::_bubble_up (this=0x1cb2990, 
> index=2) at ../lib/ts/PriorityQueue.h:191
> comp = {}
> parent = 0
> #3  0x006ecfcc in PriorityQueue PriorityQueueLess >::push (this=0x1cb2990, 
> entry=0x2b78f402af60) at ../../lib/ts/PriorityQueue.h:91
> len = 2
> #4  0x006ec206 in RefCountCachePartition::put 
> (this=0x1cb2900, key=6912554662447498853, item=0x2b78aee04f00, size=96, 
> expire_time=1475202356) at ./P_RefCountCache.h:210
> expiry_entry = 0x2b78f402af60
> __func__ = "put"
> val = 0x1cc0880
> #5  0x006eb3de in RefCountCache::put (this=0x18051e0, 
> key=6912554662447498853, item=0x2b78aee04f00, size=16, 
> expiry_time=1475202356) at ./P_RefCountCache.h:462
> No locals.
> #6  0x006e2d8e in HostDBContinuation::dnsEvent (this=0x2b7938020f00, 
> event=600, e=0x2b78ac009440) at HostDB.cc:1422
> is_rr = false
> old_rr_data = 0x0
> first_record = 0x2b78ac0094f8
> m = 0x1
> failed = false
> old_r = {m_ptr = 0x0}
> af = 2 '\002'
> s_size = 16
> rrsize = 0
> allocSize = 16
> r = 0x2b78aee04f00
> old_info = { = { = {_vptr.ForceVFPTToTop 
> = 0x7f3630}, m_refcount = 0}, iobuffer_index = 0, 
>   key = 47797242059264, app = {allotment = {application1 = 5326300, 
> application2 = 0}, http_data = {http_version = 4, 
>   pipeline_max = 59, keepalive_timeout = 17, fail_count = 81, 
> unused1 = 0, last_failure = 0}, rr = {offset = 5326300}}, data = {
> ip = {sa = {sa_family = 54488, sa_data = 
> "^\000\000\000\000\000\020\034$\274x+\000"}, sin = {sin_family = 54488, 
> sin_port =