[jira] [Commented] (TS-2743) Crash with SPDY on production environment in SpdyNV::SpdyNV

2014-04-27 Thread ASF subversion and git services (JIRA) 

[ 
https://issues.apache.org/jira/browse/TS-2743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13982203#comment-13982203
 ] 

ASF subversion and git services commented on TS-2743:
-

Commit 9b72a7d1c6df9a4dfab449a2fb235035b7100b61 in trafficserver's branch 
refs/heads/master from [~yunkai]
[ https://git-wip-us.apache.org/repos/asf?p=trafficserver.git;h=9b72a7d ]

TS-2743: Ignore invalid HTTP headers in SpdyNV carefully

Any HTTP headers with empty value are invalid, SpdyNV should
ignore them carefully.

Signed-off-by: Yunkai Zhang qiushu@taobao.com


 Crash with SPDY on production environment in SpdyNV::SpdyNV
 ---

 Key: TS-2743
 URL: https://issues.apache.org/jira/browse/TS-2743
 Project: Traffic Server
  Issue Type: Bug
  Components: Core, SPDY
Reporter: Sudheer Vinukonda
Assignee: Yunkai Zhang
  Labels: SPDY_ATS
 Attachments: 
 0001-TS-2743-Ignore-invalid-HTTP-headers-in-SpdyNV-carefu.patch


 After patching a fix (by removing the apparent incorrect assert failure on 
 non-ipv4 address in TsFetchCreate - jira ts-2742), noticed a new crash, this 
 time in SpdyNV::SpdyNV with the below stack trace. Will update more upon 
 investigation.
 FATAL: SpdyCommon.cc:133: failed assert `value  value_len`
 FATAL: SpdyCommon.cc:133: failed assert `value  value_len`
 /home/y/bin/traffic_server - STACK TRACE:
 /home/y/bin/traffic_server - STACK TRACE:
 /home/y/lib64/libtsutil.so.5(+0x1f5c8)[0x2b7371f6a5c8]
 /home/y/lib64/libtsutil.so.5(+0x1f5c8)[0x2b7371f6a5c8]
 /home/y/lib64/libtsutil.so.5(+0x1da8f)[0x2b7371f68a8f]
 /home/y/lib64/libtsutil.so.5(+0x1da8f)[0x2b7371f68a8f]
 /home/y/bin/traffic_server[0x4af739]
 /home/y/bin/traffic_server[0x5cf7ea]
 /home/y/bin/traffic_server[0x5d0a6a]
 /home/y/bin/traffic_server(_ZN7FetchSM15InvokePluginExtEi+0x33a)[0x4c7a5a]
 /home/y/bin/traffic_server[0x4af739]
 /home/y/bin/traffic_server(_ZN7FetchSM18process_fetch_readEi+0x187)[0x4c7f37]
 /home/y/bin/traffic_server[0x5cf7ea]
 /home/y/bin/traffic_server[0x5d0a6a]
 /home/y/bin/traffic_server(_ZN7FetchSM15InvokePluginExtEi+0x33a)[0x4c7a5a]
 /home/y/bin/traffic_server(_ZN7FetchSM18process_fetch_readEi+0x187)[0x4c7f37]
 /home/y/bin/traffic_server(_ZN7FetchSM13fetch_handlerEiPv+0x8b)[0x4c80eb]
 /home/y/bin/traffic_server(_ZN8PluginVC17process_read_sideEb+0x375)[0x4dd0d5]
 /home/y/bin/traffic_server(_ZN8PluginVC18process_write_sideEb+0x57a)[0x4dda2a]
 /home/y/bin/traffic_server(_ZN7FetchSM13fetch_handlerEiPv+0x8b)[0x4c80eb]
 /home/y/bin/traffic_server(_ZN8PluginVC12main_handlerEiPv+0x372)[0x4de272]
 /home/y/bin/traffic_server(_ZN8PluginVC17process_read_sideEb+0x375)[0x4dd0d5]
 /home/y/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x8f)[0x712f7f]
 /home/y/bin/traffic_server(_ZN8PluginVC18process_write_sideEb+0x57a)[0x4dda2a]
 /home/y/bin/traffic_server(_ZN7EThread7executeEv+0x61b)[0x713aab]
 /home/y/bin/traffic_server(_ZN8PluginVC12main_handlerEiPv+0x372)[0x4de272]
 /home/y/bin/traffic_server[0x71232a]
 /home/y/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x8f)[0x712f7f]
 /lib64/libpthread.so.0(+0x3297207851)[0x2b7372601851]
 /home/y/bin/traffic_server(_ZN7EThread7executeEv+0x61b)[0x713aab]
 /lib64/libc.so.6(clone+0x6d)[0x3296ee890d]
 /home/y/bin/traffic_server[0x71232a]
 /lib64/libpthread.so.0(+0x3297207851)[0x2b7372601851]



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (TS-2743) Crash with SPDY on production environment in SpdyNV::SpdyNV

2014-04-27 Thread ASF subversion and git services (JIRA) 

[ 
https://issues.apache.org/jira/browse/TS-2743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13982205#comment-13982205
 ] 

ASF subversion and git services commented on TS-2743:
-

Commit ecf3e5fbd98a43c114e757a207aa303769424dbb in trafficserver's branch 
refs/heads/master from [~yunkai]
[ https://git-wip-us.apache.org/repos/asf?p=trafficserver.git;h=ecf3e5f ]

Forgot to update CHANGES with TS-2743


 Crash with SPDY on production environment in SpdyNV::SpdyNV
 ---

 Key: TS-2743
 URL: https://issues.apache.org/jira/browse/TS-2743
 Project: Traffic Server
  Issue Type: Bug
  Components: Core, SPDY
Reporter: Sudheer Vinukonda
Assignee: Yunkai Zhang
  Labels: SPDY_ATS
 Attachments: 
 0001-TS-2743-Ignore-invalid-HTTP-headers-in-SpdyNV-carefu.patch


 After patching a fix (by removing the apparent incorrect assert failure on 
 non-ipv4 address in TsFetchCreate - jira ts-2742), noticed a new crash, this 
 time in SpdyNV::SpdyNV with the below stack trace. Will update more upon 
 investigation.
 FATAL: SpdyCommon.cc:133: failed assert `value  value_len`
 FATAL: SpdyCommon.cc:133: failed assert `value  value_len`
 /home/y/bin/traffic_server - STACK TRACE:
 /home/y/bin/traffic_server - STACK TRACE:
 /home/y/lib64/libtsutil.so.5(+0x1f5c8)[0x2b7371f6a5c8]
 /home/y/lib64/libtsutil.so.5(+0x1f5c8)[0x2b7371f6a5c8]
 /home/y/lib64/libtsutil.so.5(+0x1da8f)[0x2b7371f68a8f]
 /home/y/lib64/libtsutil.so.5(+0x1da8f)[0x2b7371f68a8f]
 /home/y/bin/traffic_server[0x4af739]
 /home/y/bin/traffic_server[0x5cf7ea]
 /home/y/bin/traffic_server[0x5d0a6a]
 /home/y/bin/traffic_server(_ZN7FetchSM15InvokePluginExtEi+0x33a)[0x4c7a5a]
 /home/y/bin/traffic_server[0x4af739]
 /home/y/bin/traffic_server(_ZN7FetchSM18process_fetch_readEi+0x187)[0x4c7f37]
 /home/y/bin/traffic_server[0x5cf7ea]
 /home/y/bin/traffic_server[0x5d0a6a]
 /home/y/bin/traffic_server(_ZN7FetchSM15InvokePluginExtEi+0x33a)[0x4c7a5a]
 /home/y/bin/traffic_server(_ZN7FetchSM18process_fetch_readEi+0x187)[0x4c7f37]
 /home/y/bin/traffic_server(_ZN7FetchSM13fetch_handlerEiPv+0x8b)[0x4c80eb]
 /home/y/bin/traffic_server(_ZN8PluginVC17process_read_sideEb+0x375)[0x4dd0d5]
 /home/y/bin/traffic_server(_ZN8PluginVC18process_write_sideEb+0x57a)[0x4dda2a]
 /home/y/bin/traffic_server(_ZN7FetchSM13fetch_handlerEiPv+0x8b)[0x4c80eb]
 /home/y/bin/traffic_server(_ZN8PluginVC12main_handlerEiPv+0x372)[0x4de272]
 /home/y/bin/traffic_server(_ZN8PluginVC17process_read_sideEb+0x375)[0x4dd0d5]
 /home/y/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x8f)[0x712f7f]
 /home/y/bin/traffic_server(_ZN8PluginVC18process_write_sideEb+0x57a)[0x4dda2a]
 /home/y/bin/traffic_server(_ZN7EThread7executeEv+0x61b)[0x713aab]
 /home/y/bin/traffic_server(_ZN8PluginVC12main_handlerEiPv+0x372)[0x4de272]
 /home/y/bin/traffic_server[0x71232a]
 /home/y/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x8f)[0x712f7f]
 /lib64/libpthread.so.0(+0x3297207851)[0x2b7372601851]
 /home/y/bin/traffic_server(_ZN7EThread7executeEv+0x61b)[0x713aab]
 /lib64/libc.so.6(clone+0x6d)[0x3296ee890d]
 /home/y/bin/traffic_server[0x71232a]
 /lib64/libpthread.so.0(+0x3297207851)[0x2b7372601851]



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (TS-2743) Crash with SPDY on production environment in SpdyNV::SpdyNV

2014-04-27 Thread Sudheer Vinukonda (JIRA) 

[ 
https://issues.apache.org/jira/browse/TS-2743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13982349#comment-13982349
 ] 

Sudheer Vinukonda commented on TS-2743:
---

As I said, the API TSMimeHdrFieldValueDateGet() already does special handling 
for Date value being NULL. The temp fix I am using already ignores NULL values 
for other headers, so, I don't see how not adding special handling for Date 
makes any difference. 

Also, I am not sure how TS-2750 is a side affect of TS-2743. If any, I would 
think it's the other way around. Anyway, I do have core dumps enabled, will 
post the backtrace on TS-2750. 



 Crash with SPDY on production environment in SpdyNV::SpdyNV
 ---

 Key: TS-2743
 URL: https://issues.apache.org/jira/browse/TS-2743
 Project: Traffic Server
  Issue Type: Bug
  Components: Core, SPDY
Reporter: Sudheer Vinukonda
Assignee: Yunkai Zhang
  Labels: SPDY_ATS
 Fix For: 5.0.0

 Attachments: 
 0001-TS-2743-Ignore-invalid-HTTP-headers-in-SpdyNV-carefu.patch


 After patching a fix (by removing the apparent incorrect assert failure on 
 non-ipv4 address in TsFetchCreate - jira ts-2742), noticed a new crash, this 
 time in SpdyNV::SpdyNV with the below stack trace. Will update more upon 
 investigation.
 FATAL: SpdyCommon.cc:133: failed assert `value  value_len`
 FATAL: SpdyCommon.cc:133: failed assert `value  value_len`
 /home/y/bin/traffic_server - STACK TRACE:
 /home/y/bin/traffic_server - STACK TRACE:
 /home/y/lib64/libtsutil.so.5(+0x1f5c8)[0x2b7371f6a5c8]
 /home/y/lib64/libtsutil.so.5(+0x1f5c8)[0x2b7371f6a5c8]
 /home/y/lib64/libtsutil.so.5(+0x1da8f)[0x2b7371f68a8f]
 /home/y/lib64/libtsutil.so.5(+0x1da8f)[0x2b7371f68a8f]
 /home/y/bin/traffic_server[0x4af739]
 /home/y/bin/traffic_server[0x5cf7ea]
 /home/y/bin/traffic_server[0x5d0a6a]
 /home/y/bin/traffic_server(_ZN7FetchSM15InvokePluginExtEi+0x33a)[0x4c7a5a]
 /home/y/bin/traffic_server[0x4af739]
 /home/y/bin/traffic_server(_ZN7FetchSM18process_fetch_readEi+0x187)[0x4c7f37]
 /home/y/bin/traffic_server[0x5cf7ea]
 /home/y/bin/traffic_server[0x5d0a6a]
 /home/y/bin/traffic_server(_ZN7FetchSM15InvokePluginExtEi+0x33a)[0x4c7a5a]
 /home/y/bin/traffic_server(_ZN7FetchSM18process_fetch_readEi+0x187)[0x4c7f37]
 /home/y/bin/traffic_server(_ZN7FetchSM13fetch_handlerEiPv+0x8b)[0x4c80eb]
 /home/y/bin/traffic_server(_ZN8PluginVC17process_read_sideEb+0x375)[0x4dd0d5]
 /home/y/bin/traffic_server(_ZN8PluginVC18process_write_sideEb+0x57a)[0x4dda2a]
 /home/y/bin/traffic_server(_ZN7FetchSM13fetch_handlerEiPv+0x8b)[0x4c80eb]
 /home/y/bin/traffic_server(_ZN8PluginVC12main_handlerEiPv+0x372)[0x4de272]
 /home/y/bin/traffic_server(_ZN8PluginVC17process_read_sideEb+0x375)[0x4dd0d5]
 /home/y/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x8f)[0x712f7f]
 /home/y/bin/traffic_server(_ZN8PluginVC18process_write_sideEb+0x57a)[0x4dda2a]
 /home/y/bin/traffic_server(_ZN7EThread7executeEv+0x61b)[0x713aab]
 /home/y/bin/traffic_server(_ZN8PluginVC12main_handlerEiPv+0x372)[0x4de272]
 /home/y/bin/traffic_server[0x71232a]
 /home/y/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x8f)[0x712f7f]
 /lib64/libpthread.so.0(+0x3297207851)[0x2b7372601851]
 /home/y/bin/traffic_server(_ZN7EThread7executeEv+0x61b)[0x713aab]
 /lib64/libc.so.6(clone+0x6d)[0x3296ee890d]
 /home/y/bin/traffic_server[0x71232a]
 /lib64/libpthread.so.0(+0x3297207851)[0x2b7372601851]



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (TS-2743) Crash with SPDY on production environment in SpdyNV::SpdyNV

2014-04-27 Thread Yunkai Zhang (JIRA) 

[ 
https://issues.apache.org/jira/browse/TS-2743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13982359#comment-13982359
 ] 

Yunkai Zhang commented on TS-2743:
--

For SpdyNV::SpdyNV(), what it does is to convert HTTP headers to *string* 
key-value pairs, it does not care the type of each header.

Using TSMimeHdrFieldValueDateGet() here is bad! It will call mime_parse_date() 
to parse string, which will obviously waste CPU time.



 Crash with SPDY on production environment in SpdyNV::SpdyNV
 ---

 Key: TS-2743
 URL: https://issues.apache.org/jira/browse/TS-2743
 Project: Traffic Server
  Issue Type: Bug
  Components: Core, SPDY
Reporter: Sudheer Vinukonda
Assignee: Yunkai Zhang
  Labels: SPDY_ATS
 Fix For: 5.0.0

 Attachments: 
 0001-TS-2743-Ignore-invalid-HTTP-headers-in-SpdyNV-carefu.patch


 After patching a fix (by removing the apparent incorrect assert failure on 
 non-ipv4 address in TsFetchCreate - jira ts-2742), noticed a new crash, this 
 time in SpdyNV::SpdyNV with the below stack trace. Will update more upon 
 investigation.
 FATAL: SpdyCommon.cc:133: failed assert `value  value_len`
 FATAL: SpdyCommon.cc:133: failed assert `value  value_len`
 /home/y/bin/traffic_server - STACK TRACE:
 /home/y/bin/traffic_server - STACK TRACE:
 /home/y/lib64/libtsutil.so.5(+0x1f5c8)[0x2b7371f6a5c8]
 /home/y/lib64/libtsutil.so.5(+0x1f5c8)[0x2b7371f6a5c8]
 /home/y/lib64/libtsutil.so.5(+0x1da8f)[0x2b7371f68a8f]
 /home/y/lib64/libtsutil.so.5(+0x1da8f)[0x2b7371f68a8f]
 /home/y/bin/traffic_server[0x4af739]
 /home/y/bin/traffic_server[0x5cf7ea]
 /home/y/bin/traffic_server[0x5d0a6a]
 /home/y/bin/traffic_server(_ZN7FetchSM15InvokePluginExtEi+0x33a)[0x4c7a5a]
 /home/y/bin/traffic_server[0x4af739]
 /home/y/bin/traffic_server(_ZN7FetchSM18process_fetch_readEi+0x187)[0x4c7f37]
 /home/y/bin/traffic_server[0x5cf7ea]
 /home/y/bin/traffic_server[0x5d0a6a]
 /home/y/bin/traffic_server(_ZN7FetchSM15InvokePluginExtEi+0x33a)[0x4c7a5a]
 /home/y/bin/traffic_server(_ZN7FetchSM18process_fetch_readEi+0x187)[0x4c7f37]
 /home/y/bin/traffic_server(_ZN7FetchSM13fetch_handlerEiPv+0x8b)[0x4c80eb]
 /home/y/bin/traffic_server(_ZN8PluginVC17process_read_sideEb+0x375)[0x4dd0d5]
 /home/y/bin/traffic_server(_ZN8PluginVC18process_write_sideEb+0x57a)[0x4dda2a]
 /home/y/bin/traffic_server(_ZN7FetchSM13fetch_handlerEiPv+0x8b)[0x4c80eb]
 /home/y/bin/traffic_server(_ZN8PluginVC12main_handlerEiPv+0x372)[0x4de272]
 /home/y/bin/traffic_server(_ZN8PluginVC17process_read_sideEb+0x375)[0x4dd0d5]
 /home/y/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x8f)[0x712f7f]
 /home/y/bin/traffic_server(_ZN8PluginVC18process_write_sideEb+0x57a)[0x4dda2a]
 /home/y/bin/traffic_server(_ZN7EThread7executeEv+0x61b)[0x713aab]
 /home/y/bin/traffic_server(_ZN8PluginVC12main_handlerEiPv+0x372)[0x4de272]
 /home/y/bin/traffic_server[0x71232a]
 /home/y/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x8f)[0x712f7f]
 /lib64/libpthread.so.0(+0x3297207851)[0x2b7372601851]
 /home/y/bin/traffic_server(_ZN7EThread7executeEv+0x61b)[0x713aab]
 /lib64/libc.so.6(clone+0x6d)[0x3296ee890d]
 /home/y/bin/traffic_server[0x71232a]
 /lib64/libpthread.so.0(+0x3297207851)[0x2b7372601851]



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (TS-2743) Crash with SPDY on production environment in SpdyNV::SpdyNV

2014-04-27 Thread Sudheer Vinukonda (JIRA) 

[ 
https://issues.apache.org/jira/browse/TS-2743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13982362#comment-13982362
 ] 

Sudheer Vinukonda commented on TS-2743:
---

I did not use TSMimeHdrFieldValueDateGet() - what I am saying is I used a 
solution inline with what TSMimeHdrFieldValueDateGet() was doing - i.e when 
TSMimeHdrFieldValueStringGet() returns NULL, I use time(0) to set the value of 
Date header, just like TSMimeHdrFieldValueDateGet() - You may want to take a 
look at TSMimeHdrFieldValueDateGet(). Anyway, like I said above, this is a very 
minor point and does not make a difference to the solution. To me, the real 
question is why does the HTTP header value become NULL - I don't believe that 
the Origin is sending bad values - my bet is on the ATS doing something wrong 
to mess up the memory. 

 Crash with SPDY on production environment in SpdyNV::SpdyNV
 ---

 Key: TS-2743
 URL: https://issues.apache.org/jira/browse/TS-2743
 Project: Traffic Server
  Issue Type: Bug
  Components: Core, SPDY
Reporter: Sudheer Vinukonda
Assignee: Yunkai Zhang
  Labels: SPDY_ATS
 Fix For: 5.0.0

 Attachments: 
 0001-TS-2743-Ignore-invalid-HTTP-headers-in-SpdyNV-carefu.patch


 After patching a fix (by removing the apparent incorrect assert failure on 
 non-ipv4 address in TsFetchCreate - jira ts-2742), noticed a new crash, this 
 time in SpdyNV::SpdyNV with the below stack trace. Will update more upon 
 investigation.
 FATAL: SpdyCommon.cc:133: failed assert `value  value_len`
 FATAL: SpdyCommon.cc:133: failed assert `value  value_len`
 /home/y/bin/traffic_server - STACK TRACE:
 /home/y/bin/traffic_server - STACK TRACE:
 /home/y/lib64/libtsutil.so.5(+0x1f5c8)[0x2b7371f6a5c8]
 /home/y/lib64/libtsutil.so.5(+0x1f5c8)[0x2b7371f6a5c8]
 /home/y/lib64/libtsutil.so.5(+0x1da8f)[0x2b7371f68a8f]
 /home/y/lib64/libtsutil.so.5(+0x1da8f)[0x2b7371f68a8f]
 /home/y/bin/traffic_server[0x4af739]
 /home/y/bin/traffic_server[0x5cf7ea]
 /home/y/bin/traffic_server[0x5d0a6a]
 /home/y/bin/traffic_server(_ZN7FetchSM15InvokePluginExtEi+0x33a)[0x4c7a5a]
 /home/y/bin/traffic_server[0x4af739]
 /home/y/bin/traffic_server(_ZN7FetchSM18process_fetch_readEi+0x187)[0x4c7f37]
 /home/y/bin/traffic_server[0x5cf7ea]
 /home/y/bin/traffic_server[0x5d0a6a]
 /home/y/bin/traffic_server(_ZN7FetchSM15InvokePluginExtEi+0x33a)[0x4c7a5a]
 /home/y/bin/traffic_server(_ZN7FetchSM18process_fetch_readEi+0x187)[0x4c7f37]
 /home/y/bin/traffic_server(_ZN7FetchSM13fetch_handlerEiPv+0x8b)[0x4c80eb]
 /home/y/bin/traffic_server(_ZN8PluginVC17process_read_sideEb+0x375)[0x4dd0d5]
 /home/y/bin/traffic_server(_ZN8PluginVC18process_write_sideEb+0x57a)[0x4dda2a]
 /home/y/bin/traffic_server(_ZN7FetchSM13fetch_handlerEiPv+0x8b)[0x4c80eb]
 /home/y/bin/traffic_server(_ZN8PluginVC12main_handlerEiPv+0x372)[0x4de272]
 /home/y/bin/traffic_server(_ZN8PluginVC17process_read_sideEb+0x375)[0x4dd0d5]
 /home/y/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x8f)[0x712f7f]
 /home/y/bin/traffic_server(_ZN8PluginVC18process_write_sideEb+0x57a)[0x4dda2a]
 /home/y/bin/traffic_server(_ZN7EThread7executeEv+0x61b)[0x713aab]
 /home/y/bin/traffic_server(_ZN8PluginVC12main_handlerEiPv+0x372)[0x4de272]
 /home/y/bin/traffic_server[0x71232a]
 /home/y/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x8f)[0x712f7f]
 /lib64/libpthread.so.0(+0x3297207851)[0x2b7372601851]
 /home/y/bin/traffic_server(_ZN7EThread7executeEv+0x61b)[0x713aab]
 /lib64/libc.so.6(clone+0x6d)[0x3296ee890d]
 /home/y/bin/traffic_server[0x71232a]
 /lib64/libpthread.so.0(+0x3297207851)[0x2b7372601851]



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (TS-2743) Crash with SPDY on production environment in SpdyNV::SpdyNV

2014-04-27 Thread Yunkai Zhang (JIRA) 

[ 
https://issues.apache.org/jira/browse/TS-2743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13982365#comment-13982365
 ] 

Yunkai Zhang commented on TS-2743:
--

my bet is on the ATS doing something wrong to mess up the memory.

If that is true, I think we can continue get crashed after applying the newest 
patches, just waiting for it.

Note: I have committed several patches about SPDY to master today.


 Crash with SPDY on production environment in SpdyNV::SpdyNV
 ---

 Key: TS-2743
 URL: https://issues.apache.org/jira/browse/TS-2743
 Project: Traffic Server
  Issue Type: Bug
  Components: Core, SPDY
Reporter: Sudheer Vinukonda
Assignee: Yunkai Zhang
  Labels: SPDY_ATS
 Fix For: 5.0.0

 Attachments: 
 0001-TS-2743-Ignore-invalid-HTTP-headers-in-SpdyNV-carefu.patch


 After patching a fix (by removing the apparent incorrect assert failure on 
 non-ipv4 address in TsFetchCreate - jira ts-2742), noticed a new crash, this 
 time in SpdyNV::SpdyNV with the below stack trace. Will update more upon 
 investigation.
 FATAL: SpdyCommon.cc:133: failed assert `value  value_len`
 FATAL: SpdyCommon.cc:133: failed assert `value  value_len`
 /home/y/bin/traffic_server - STACK TRACE:
 /home/y/bin/traffic_server - STACK TRACE:
 /home/y/lib64/libtsutil.so.5(+0x1f5c8)[0x2b7371f6a5c8]
 /home/y/lib64/libtsutil.so.5(+0x1f5c8)[0x2b7371f6a5c8]
 /home/y/lib64/libtsutil.so.5(+0x1da8f)[0x2b7371f68a8f]
 /home/y/lib64/libtsutil.so.5(+0x1da8f)[0x2b7371f68a8f]
 /home/y/bin/traffic_server[0x4af739]
 /home/y/bin/traffic_server[0x5cf7ea]
 /home/y/bin/traffic_server[0x5d0a6a]
 /home/y/bin/traffic_server(_ZN7FetchSM15InvokePluginExtEi+0x33a)[0x4c7a5a]
 /home/y/bin/traffic_server[0x4af739]
 /home/y/bin/traffic_server(_ZN7FetchSM18process_fetch_readEi+0x187)[0x4c7f37]
 /home/y/bin/traffic_server[0x5cf7ea]
 /home/y/bin/traffic_server[0x5d0a6a]
 /home/y/bin/traffic_server(_ZN7FetchSM15InvokePluginExtEi+0x33a)[0x4c7a5a]
 /home/y/bin/traffic_server(_ZN7FetchSM18process_fetch_readEi+0x187)[0x4c7f37]
 /home/y/bin/traffic_server(_ZN7FetchSM13fetch_handlerEiPv+0x8b)[0x4c80eb]
 /home/y/bin/traffic_server(_ZN8PluginVC17process_read_sideEb+0x375)[0x4dd0d5]
 /home/y/bin/traffic_server(_ZN8PluginVC18process_write_sideEb+0x57a)[0x4dda2a]
 /home/y/bin/traffic_server(_ZN7FetchSM13fetch_handlerEiPv+0x8b)[0x4c80eb]
 /home/y/bin/traffic_server(_ZN8PluginVC12main_handlerEiPv+0x372)[0x4de272]
 /home/y/bin/traffic_server(_ZN8PluginVC17process_read_sideEb+0x375)[0x4dd0d5]
 /home/y/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x8f)[0x712f7f]
 /home/y/bin/traffic_server(_ZN8PluginVC18process_write_sideEb+0x57a)[0x4dda2a]
 /home/y/bin/traffic_server(_ZN7EThread7executeEv+0x61b)[0x713aab]
 /home/y/bin/traffic_server(_ZN8PluginVC12main_handlerEiPv+0x372)[0x4de272]
 /home/y/bin/traffic_server[0x71232a]
 /home/y/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x8f)[0x712f7f]
 /lib64/libpthread.so.0(+0x3297207851)[0x2b7372601851]
 /home/y/bin/traffic_server(_ZN7EThread7executeEv+0x61b)[0x713aab]
 /lib64/libc.so.6(clone+0x6d)[0x3296ee890d]
 /home/y/bin/traffic_server[0x71232a]
 /lib64/libpthread.so.0(+0x3297207851)[0x2b7372601851]



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (TS-2743) Crash with SPDY on production environment in SpdyNV::SpdyNV

2014-04-27 Thread Yunkai Zhang (JIRA) 

[ 
https://issues.apache.org/jira/browse/TS-2743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13982369#comment-13982369
 ] 

Yunkai Zhang commented on TS-2743:
--

To verify your bet, you can also add some code to make ATS generate core dump 
when the value of header is NULL.

FYI.

 Crash with SPDY on production environment in SpdyNV::SpdyNV
 ---

 Key: TS-2743
 URL: https://issues.apache.org/jira/browse/TS-2743
 Project: Traffic Server
  Issue Type: Bug
  Components: Core, SPDY
Reporter: Sudheer Vinukonda
Assignee: Yunkai Zhang
  Labels: SPDY_ATS
 Fix For: 5.0.0

 Attachments: 
 0001-TS-2743-Ignore-invalid-HTTP-headers-in-SpdyNV-carefu.patch


 After patching a fix (by removing the apparent incorrect assert failure on 
 non-ipv4 address in TsFetchCreate - jira ts-2742), noticed a new crash, this 
 time in SpdyNV::SpdyNV with the below stack trace. Will update more upon 
 investigation.
 FATAL: SpdyCommon.cc:133: failed assert `value  value_len`
 FATAL: SpdyCommon.cc:133: failed assert `value  value_len`
 /home/y/bin/traffic_server - STACK TRACE:
 /home/y/bin/traffic_server - STACK TRACE:
 /home/y/lib64/libtsutil.so.5(+0x1f5c8)[0x2b7371f6a5c8]
 /home/y/lib64/libtsutil.so.5(+0x1f5c8)[0x2b7371f6a5c8]
 /home/y/lib64/libtsutil.so.5(+0x1da8f)[0x2b7371f68a8f]
 /home/y/lib64/libtsutil.so.5(+0x1da8f)[0x2b7371f68a8f]
 /home/y/bin/traffic_server[0x4af739]
 /home/y/bin/traffic_server[0x5cf7ea]
 /home/y/bin/traffic_server[0x5d0a6a]
 /home/y/bin/traffic_server(_ZN7FetchSM15InvokePluginExtEi+0x33a)[0x4c7a5a]
 /home/y/bin/traffic_server[0x4af739]
 /home/y/bin/traffic_server(_ZN7FetchSM18process_fetch_readEi+0x187)[0x4c7f37]
 /home/y/bin/traffic_server[0x5cf7ea]
 /home/y/bin/traffic_server[0x5d0a6a]
 /home/y/bin/traffic_server(_ZN7FetchSM15InvokePluginExtEi+0x33a)[0x4c7a5a]
 /home/y/bin/traffic_server(_ZN7FetchSM18process_fetch_readEi+0x187)[0x4c7f37]
 /home/y/bin/traffic_server(_ZN7FetchSM13fetch_handlerEiPv+0x8b)[0x4c80eb]
 /home/y/bin/traffic_server(_ZN8PluginVC17process_read_sideEb+0x375)[0x4dd0d5]
 /home/y/bin/traffic_server(_ZN8PluginVC18process_write_sideEb+0x57a)[0x4dda2a]
 /home/y/bin/traffic_server(_ZN7FetchSM13fetch_handlerEiPv+0x8b)[0x4c80eb]
 /home/y/bin/traffic_server(_ZN8PluginVC12main_handlerEiPv+0x372)[0x4de272]
 /home/y/bin/traffic_server(_ZN8PluginVC17process_read_sideEb+0x375)[0x4dd0d5]
 /home/y/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x8f)[0x712f7f]
 /home/y/bin/traffic_server(_ZN8PluginVC18process_write_sideEb+0x57a)[0x4dda2a]
 /home/y/bin/traffic_server(_ZN7EThread7executeEv+0x61b)[0x713aab]
 /home/y/bin/traffic_server(_ZN8PluginVC12main_handlerEiPv+0x372)[0x4de272]
 /home/y/bin/traffic_server[0x71232a]
 /home/y/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x8f)[0x712f7f]
 /lib64/libpthread.so.0(+0x3297207851)[0x2b7372601851]
 /home/y/bin/traffic_server(_ZN7EThread7executeEv+0x61b)[0x713aab]
 /lib64/libc.so.6(clone+0x6d)[0x3296ee890d]
 /home/y/bin/traffic_server[0x71232a]
 /lib64/libpthread.so.0(+0x3297207851)[0x2b7372601851]



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (TS-2743) Crash with SPDY on production environment in SpdyNV::SpdyNV

2014-04-27 Thread Sudheer Vinukonda (JIRA) 

[ 
https://issues.apache.org/jira/browse/TS-2743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13982380#comment-13982380
 ] 

Sudheer Vinukonda commented on TS-2743:
---

Not sure to understand - Isn't this the same as the assert that was generating 
core on NULL value?

Agree about monitoring for new issues after applying the fixes - as a matter of 
fact, that is precisely what I've been doing. The temp fixes I put in are 
almost identical in behavior to the fixes that are committed here.

 Crash with SPDY on production environment in SpdyNV::SpdyNV
 ---

 Key: TS-2743
 URL: https://issues.apache.org/jira/browse/TS-2743
 Project: Traffic Server
  Issue Type: Bug
  Components: Core, SPDY
Reporter: Sudheer Vinukonda
Assignee: Yunkai Zhang
  Labels: SPDY_ATS
 Fix For: 5.0.0

 Attachments: 
 0001-TS-2743-Ignore-invalid-HTTP-headers-in-SpdyNV-carefu.patch


 After patching a fix (by removing the apparent incorrect assert failure on 
 non-ipv4 address in TsFetchCreate - jira ts-2742), noticed a new crash, this 
 time in SpdyNV::SpdyNV with the below stack trace. Will update more upon 
 investigation.
 FATAL: SpdyCommon.cc:133: failed assert `value  value_len`
 FATAL: SpdyCommon.cc:133: failed assert `value  value_len`
 /home/y/bin/traffic_server - STACK TRACE:
 /home/y/bin/traffic_server - STACK TRACE:
 /home/y/lib64/libtsutil.so.5(+0x1f5c8)[0x2b7371f6a5c8]
 /home/y/lib64/libtsutil.so.5(+0x1f5c8)[0x2b7371f6a5c8]
 /home/y/lib64/libtsutil.so.5(+0x1da8f)[0x2b7371f68a8f]
 /home/y/lib64/libtsutil.so.5(+0x1da8f)[0x2b7371f68a8f]
 /home/y/bin/traffic_server[0x4af739]
 /home/y/bin/traffic_server[0x5cf7ea]
 /home/y/bin/traffic_server[0x5d0a6a]
 /home/y/bin/traffic_server(_ZN7FetchSM15InvokePluginExtEi+0x33a)[0x4c7a5a]
 /home/y/bin/traffic_server[0x4af739]
 /home/y/bin/traffic_server(_ZN7FetchSM18process_fetch_readEi+0x187)[0x4c7f37]
 /home/y/bin/traffic_server[0x5cf7ea]
 /home/y/bin/traffic_server[0x5d0a6a]
 /home/y/bin/traffic_server(_ZN7FetchSM15InvokePluginExtEi+0x33a)[0x4c7a5a]
 /home/y/bin/traffic_server(_ZN7FetchSM18process_fetch_readEi+0x187)[0x4c7f37]
 /home/y/bin/traffic_server(_ZN7FetchSM13fetch_handlerEiPv+0x8b)[0x4c80eb]
 /home/y/bin/traffic_server(_ZN8PluginVC17process_read_sideEb+0x375)[0x4dd0d5]
 /home/y/bin/traffic_server(_ZN8PluginVC18process_write_sideEb+0x57a)[0x4dda2a]
 /home/y/bin/traffic_server(_ZN7FetchSM13fetch_handlerEiPv+0x8b)[0x4c80eb]
 /home/y/bin/traffic_server(_ZN8PluginVC12main_handlerEiPv+0x372)[0x4de272]
 /home/y/bin/traffic_server(_ZN8PluginVC17process_read_sideEb+0x375)[0x4dd0d5]
 /home/y/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x8f)[0x712f7f]
 /home/y/bin/traffic_server(_ZN8PluginVC18process_write_sideEb+0x57a)[0x4dda2a]
 /home/y/bin/traffic_server(_ZN7EThread7executeEv+0x61b)[0x713aab]
 /home/y/bin/traffic_server(_ZN8PluginVC12main_handlerEiPv+0x372)[0x4de272]
 /home/y/bin/traffic_server[0x71232a]
 /home/y/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x8f)[0x712f7f]
 /lib64/libpthread.so.0(+0x3297207851)[0x2b7372601851]
 /home/y/bin/traffic_server(_ZN7EThread7executeEv+0x61b)[0x713aab]
 /lib64/libc.so.6(clone+0x6d)[0x3296ee890d]
 /home/y/bin/traffic_server[0x71232a]
 /lib64/libpthread.so.0(+0x3297207851)[0x2b7372601851]



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (TS-2743) Crash with SPDY on production environment in SpdyNV::SpdyNV

2014-04-27 Thread Yunkai Zhang (JIRA) 

[ 
https://issues.apache.org/jira/browse/TS-2743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13982383#comment-13982383
 ] 

Yunkai Zhang commented on TS-2743:
--

Sorry, I forgot ink_release_assert() will generate core as well:)

If you have got core, why not show more context of the core to us? With the 
core, by printing the whole HTTP headers, it should be easy to know whether the 
memory was messed up.

 Crash with SPDY on production environment in SpdyNV::SpdyNV
 ---

 Key: TS-2743
 URL: https://issues.apache.org/jira/browse/TS-2743
 Project: Traffic Server
  Issue Type: Bug
  Components: Core, SPDY
Reporter: Sudheer Vinukonda
Assignee: Yunkai Zhang
  Labels: SPDY_ATS
 Fix For: 5.0.0

 Attachments: 
 0001-TS-2743-Ignore-invalid-HTTP-headers-in-SpdyNV-carefu.patch


 After patching a fix (by removing the apparent incorrect assert failure on 
 non-ipv4 address in TsFetchCreate - jira ts-2742), noticed a new crash, this 
 time in SpdyNV::SpdyNV with the below stack trace. Will update more upon 
 investigation.
 FATAL: SpdyCommon.cc:133: failed assert `value  value_len`
 FATAL: SpdyCommon.cc:133: failed assert `value  value_len`
 /home/y/bin/traffic_server - STACK TRACE:
 /home/y/bin/traffic_server - STACK TRACE:
 /home/y/lib64/libtsutil.so.5(+0x1f5c8)[0x2b7371f6a5c8]
 /home/y/lib64/libtsutil.so.5(+0x1f5c8)[0x2b7371f6a5c8]
 /home/y/lib64/libtsutil.so.5(+0x1da8f)[0x2b7371f68a8f]
 /home/y/lib64/libtsutil.so.5(+0x1da8f)[0x2b7371f68a8f]
 /home/y/bin/traffic_server[0x4af739]
 /home/y/bin/traffic_server[0x5cf7ea]
 /home/y/bin/traffic_server[0x5d0a6a]
 /home/y/bin/traffic_server(_ZN7FetchSM15InvokePluginExtEi+0x33a)[0x4c7a5a]
 /home/y/bin/traffic_server[0x4af739]
 /home/y/bin/traffic_server(_ZN7FetchSM18process_fetch_readEi+0x187)[0x4c7f37]
 /home/y/bin/traffic_server[0x5cf7ea]
 /home/y/bin/traffic_server[0x5d0a6a]
 /home/y/bin/traffic_server(_ZN7FetchSM15InvokePluginExtEi+0x33a)[0x4c7a5a]
 /home/y/bin/traffic_server(_ZN7FetchSM18process_fetch_readEi+0x187)[0x4c7f37]
 /home/y/bin/traffic_server(_ZN7FetchSM13fetch_handlerEiPv+0x8b)[0x4c80eb]
 /home/y/bin/traffic_server(_ZN8PluginVC17process_read_sideEb+0x375)[0x4dd0d5]
 /home/y/bin/traffic_server(_ZN8PluginVC18process_write_sideEb+0x57a)[0x4dda2a]
 /home/y/bin/traffic_server(_ZN7FetchSM13fetch_handlerEiPv+0x8b)[0x4c80eb]
 /home/y/bin/traffic_server(_ZN8PluginVC12main_handlerEiPv+0x372)[0x4de272]
 /home/y/bin/traffic_server(_ZN8PluginVC17process_read_sideEb+0x375)[0x4dd0d5]
 /home/y/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x8f)[0x712f7f]
 /home/y/bin/traffic_server(_ZN8PluginVC18process_write_sideEb+0x57a)[0x4dda2a]
 /home/y/bin/traffic_server(_ZN7EThread7executeEv+0x61b)[0x713aab]
 /home/y/bin/traffic_server(_ZN8PluginVC12main_handlerEiPv+0x372)[0x4de272]
 /home/y/bin/traffic_server[0x71232a]
 /home/y/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x8f)[0x712f7f]
 /lib64/libpthread.so.0(+0x3297207851)[0x2b7372601851]
 /home/y/bin/traffic_server(_ZN7EThread7executeEv+0x61b)[0x713aab]
 /lib64/libc.so.6(clone+0x6d)[0x3296ee890d]
 /home/y/bin/traffic_server[0x71232a]
 /lib64/libpthread.so.0(+0x3297207851)[0x2b7372601851]



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (TS-2743) Crash with SPDY on production environment in SpdyNV::SpdyNV

2014-04-26 Thread Yunkai Zhang (JIRA) 

[ 
https://issues.apache.org/jira/browse/TS-2743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13982032#comment-13982032
 ] 

Yunkai Zhang commented on TS-2743:
--

[~sudheerv]:

I have tested SPDY with 'Expires' and 'Content-Type' headers, all is ok(to see 
the SYN_REPLY frame showed as flow):
{code}
[root@test79 ~]# spdycat -3 -v --no-tls http://localhost/a.txt
[  0.000] Handshake complete
...
[  0.002] recv SYN_REPLY frame version=3, flags=0, length=153
  (stream_id=1)
  :status: 200 OK
  :version: HTTP/1.1
  accept-ranges: bytes
  age: 1974
  cache-control: max-age=86400
  content-length: 14
  content-type: text/plain
  date: Sat, 26 Apr 2014 15:20:26 GMT
  etag: 52bfa00f-e
  expires: Sun, 27 Apr 2014 15:20:26 GMT
  last-modified: Sun, 29 Dec 2013 04:07:43 GMT
  server: ATS/5.0.0
Hello, World!
...
{code}

According rfc2616/rfc1123/rfc822:
{code}
The format is an absolute date and time as defined by HTTP-date in section 
3.3.1; it MUST be in RFC 1123 date format:

  Expires = Expires : HTTP-date
An example of its use is

  Expires: Thu, 01 Dec 1994 16:00:00 GMT
  Note: if a response includes a Cache-Control field with the max-
  age directive (see section 14.9.3), that directive overrides the
  Expires field.
{code}

IIUC, the value of 'Expires' header, as well as 'Content-Type',  must be string 
and can't be NULL.

It seems that your original server returns invalid value of 'Expires' and 
'Content-Type' ?

 Crash with SPDY on production environment in SpdyNV::SpdyNV
 ---

 Key: TS-2743
 URL: https://issues.apache.org/jira/browse/TS-2743
 Project: Traffic Server
  Issue Type: Bug
  Components: Core, SPDY
Reporter: Sudheer Vinukonda
Assignee: Yunkai Zhang
  Labels: SPDY_ATS

 After patching a fix (by removing the apparent incorrect assert failure on 
 non-ipv4 address in TsFetchCreate - jira ts-2742), noticed a new crash, this 
 time in SpdyNV::SpdyNV with the below stack trace. Will update more upon 
 investigation.
 FATAL: SpdyCommon.cc:133: failed assert `value  value_len`
 FATAL: SpdyCommon.cc:133: failed assert `value  value_len`
 /home/y/bin/traffic_server - STACK TRACE:
 /home/y/bin/traffic_server - STACK TRACE:
 /home/y/lib64/libtsutil.so.5(+0x1f5c8)[0x2b7371f6a5c8]
 /home/y/lib64/libtsutil.so.5(+0x1f5c8)[0x2b7371f6a5c8]
 /home/y/lib64/libtsutil.so.5(+0x1da8f)[0x2b7371f68a8f]
 /home/y/lib64/libtsutil.so.5(+0x1da8f)[0x2b7371f68a8f]
 /home/y/bin/traffic_server[0x4af739]
 /home/y/bin/traffic_server[0x5cf7ea]
 /home/y/bin/traffic_server[0x5d0a6a]
 /home/y/bin/traffic_server(_ZN7FetchSM15InvokePluginExtEi+0x33a)[0x4c7a5a]
 /home/y/bin/traffic_server[0x4af739]
 /home/y/bin/traffic_server(_ZN7FetchSM18process_fetch_readEi+0x187)[0x4c7f37]
 /home/y/bin/traffic_server[0x5cf7ea]
 /home/y/bin/traffic_server[0x5d0a6a]
 /home/y/bin/traffic_server(_ZN7FetchSM15InvokePluginExtEi+0x33a)[0x4c7a5a]
 /home/y/bin/traffic_server(_ZN7FetchSM18process_fetch_readEi+0x187)[0x4c7f37]
 /home/y/bin/traffic_server(_ZN7FetchSM13fetch_handlerEiPv+0x8b)[0x4c80eb]
 /home/y/bin/traffic_server(_ZN8PluginVC17process_read_sideEb+0x375)[0x4dd0d5]
 /home/y/bin/traffic_server(_ZN8PluginVC18process_write_sideEb+0x57a)[0x4dda2a]
 /home/y/bin/traffic_server(_ZN7FetchSM13fetch_handlerEiPv+0x8b)[0x4c80eb]
 /home/y/bin/traffic_server(_ZN8PluginVC12main_handlerEiPv+0x372)[0x4de272]
 /home/y/bin/traffic_server(_ZN8PluginVC17process_read_sideEb+0x375)[0x4dd0d5]
 /home/y/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x8f)[0x712f7f]
 /home/y/bin/traffic_server(_ZN8PluginVC18process_write_sideEb+0x57a)[0x4dda2a]
 /home/y/bin/traffic_server(_ZN7EThread7executeEv+0x61b)[0x713aab]
 /home/y/bin/traffic_server(_ZN8PluginVC12main_handlerEiPv+0x372)[0x4de272]
 /home/y/bin/traffic_server[0x71232a]
 /home/y/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x8f)[0x712f7f]
 /lib64/libpthread.so.0(+0x3297207851)[0x2b7372601851]
 /home/y/bin/traffic_server(_ZN7EThread7executeEv+0x61b)[0x713aab]
 /lib64/libc.so.6(clone+0x6d)[0x3296ee890d]
 /home/y/bin/traffic_server[0x71232a]
 /lib64/libpthread.so.0(+0x3297207851)[0x2b7372601851]



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (TS-2743) Crash with SPDY on production environment in SpdyNV::SpdyNV

2014-04-26 Thread Sudheer Vinukonda (JIRA) 

[ 
https://issues.apache.org/jira/browse/TS-2743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13982049#comment-13982049
 ] 

Sudheer Vinukonda commented on TS-2743:
---

Thanks for looking into this - please note that the crash is happening in a 
production environment with traffic (couldn't reproduce it on a functional 
scenario). Given T-2750 as well, I suspect there might be some sort of 
incorrect memory access, causing these issues.

Specifically, as far as Date format is concerned, the 
TSMimeHdrFieldValueDateGet API does seem to handle NULL value (not clear 
how/when this could happen and the same can't be said for 'Content-Type' header 
as well).

time_t
TSMimeHdrFieldValueDateGet(TSMBuffer bufp, TSMLoc hdr, TSMLoc field)
{
  sdk_assert(sdk_sanity_check_mbuffer(bufp) == TS_SUCCESS);
  sdk_assert((sdk_sanity_check_mime_hdr_handle(hdr) == TS_SUCCESS) ||
 (sdk_sanity_check_http_hdr_handle(hdr) == TS_SUCCESS));
  sdk_assert(sdk_sanity_check_field_handle(field, hdr) == TS_SUCCESS);

  int value_len;
  const char *value_str = TSMimeFieldValueGet(bufp, field, -1, value_len);

  if (value_str == NULL)
return (time_t) 0;

  return mime_parse_date(value_str, value_str + value_len);
}




 Crash with SPDY on production environment in SpdyNV::SpdyNV
 ---

 Key: TS-2743
 URL: https://issues.apache.org/jira/browse/TS-2743
 Project: Traffic Server
  Issue Type: Bug
  Components: Core, SPDY
Reporter: Sudheer Vinukonda
Assignee: Yunkai Zhang
  Labels: SPDY_ATS

 After patching a fix (by removing the apparent incorrect assert failure on 
 non-ipv4 address in TsFetchCreate - jira ts-2742), noticed a new crash, this 
 time in SpdyNV::SpdyNV with the below stack trace. Will update more upon 
 investigation.
 FATAL: SpdyCommon.cc:133: failed assert `value  value_len`
 FATAL: SpdyCommon.cc:133: failed assert `value  value_len`
 /home/y/bin/traffic_server - STACK TRACE:
 /home/y/bin/traffic_server - STACK TRACE:
 /home/y/lib64/libtsutil.so.5(+0x1f5c8)[0x2b7371f6a5c8]
 /home/y/lib64/libtsutil.so.5(+0x1f5c8)[0x2b7371f6a5c8]
 /home/y/lib64/libtsutil.so.5(+0x1da8f)[0x2b7371f68a8f]
 /home/y/lib64/libtsutil.so.5(+0x1da8f)[0x2b7371f68a8f]
 /home/y/bin/traffic_server[0x4af739]
 /home/y/bin/traffic_server[0x5cf7ea]
 /home/y/bin/traffic_server[0x5d0a6a]
 /home/y/bin/traffic_server(_ZN7FetchSM15InvokePluginExtEi+0x33a)[0x4c7a5a]
 /home/y/bin/traffic_server[0x4af739]
 /home/y/bin/traffic_server(_ZN7FetchSM18process_fetch_readEi+0x187)[0x4c7f37]
 /home/y/bin/traffic_server[0x5cf7ea]
 /home/y/bin/traffic_server[0x5d0a6a]
 /home/y/bin/traffic_server(_ZN7FetchSM15InvokePluginExtEi+0x33a)[0x4c7a5a]
 /home/y/bin/traffic_server(_ZN7FetchSM18process_fetch_readEi+0x187)[0x4c7f37]
 /home/y/bin/traffic_server(_ZN7FetchSM13fetch_handlerEiPv+0x8b)[0x4c80eb]
 /home/y/bin/traffic_server(_ZN8PluginVC17process_read_sideEb+0x375)[0x4dd0d5]
 /home/y/bin/traffic_server(_ZN8PluginVC18process_write_sideEb+0x57a)[0x4dda2a]
 /home/y/bin/traffic_server(_ZN7FetchSM13fetch_handlerEiPv+0x8b)[0x4c80eb]
 /home/y/bin/traffic_server(_ZN8PluginVC12main_handlerEiPv+0x372)[0x4de272]
 /home/y/bin/traffic_server(_ZN8PluginVC17process_read_sideEb+0x375)[0x4dd0d5]
 /home/y/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x8f)[0x712f7f]
 /home/y/bin/traffic_server(_ZN8PluginVC18process_write_sideEb+0x57a)[0x4dda2a]
 /home/y/bin/traffic_server(_ZN7EThread7executeEv+0x61b)[0x713aab]
 /home/y/bin/traffic_server(_ZN8PluginVC12main_handlerEiPv+0x372)[0x4de272]
 /home/y/bin/traffic_server[0x71232a]
 /home/y/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x8f)[0x712f7f]
 /lib64/libpthread.so.0(+0x3297207851)[0x2b7372601851]
 /home/y/bin/traffic_server(_ZN7EThread7executeEv+0x61b)[0x713aab]
 /lib64/libc.so.6(clone+0x6d)[0x3296ee890d]
 /home/y/bin/traffic_server[0x71232a]
 /lib64/libpthread.so.0(+0x3297207851)[0x2b7372601851]



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (TS-2743) Crash with SPDY on production environment in SpdyNV::SpdyNV

2014-04-26 Thread Yunkai Zhang (JIRA) 

[ 
https://issues.apache.org/jira/browse/TS-2743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13982179#comment-13982179
 ] 

Yunkai Zhang commented on TS-2743:
--

All legal HTTP headers should be in string format, that is why we should use 
'TSMimeHdrFieldValueStringGet' to extract value.

The root cause of this issue is that SpdyNV does not check and ignore invalid 
HTTP headers carefully. 

What we should do is to check the return value of 
'TSMimeHdrFieldValueStringGet' and ignore bad HTTP headers, instead of 
replacing a new method.

I'll give a patch later, please test it.



 Crash with SPDY on production environment in SpdyNV::SpdyNV
 ---

 Key: TS-2743
 URL: https://issues.apache.org/jira/browse/TS-2743
 Project: Traffic Server
  Issue Type: Bug
  Components: Core, SPDY
Reporter: Sudheer Vinukonda
Assignee: Yunkai Zhang
  Labels: SPDY_ATS

 After patching a fix (by removing the apparent incorrect assert failure on 
 non-ipv4 address in TsFetchCreate - jira ts-2742), noticed a new crash, this 
 time in SpdyNV::SpdyNV with the below stack trace. Will update more upon 
 investigation.
 FATAL: SpdyCommon.cc:133: failed assert `value  value_len`
 FATAL: SpdyCommon.cc:133: failed assert `value  value_len`
 /home/y/bin/traffic_server - STACK TRACE:
 /home/y/bin/traffic_server - STACK TRACE:
 /home/y/lib64/libtsutil.so.5(+0x1f5c8)[0x2b7371f6a5c8]
 /home/y/lib64/libtsutil.so.5(+0x1f5c8)[0x2b7371f6a5c8]
 /home/y/lib64/libtsutil.so.5(+0x1da8f)[0x2b7371f68a8f]
 /home/y/lib64/libtsutil.so.5(+0x1da8f)[0x2b7371f68a8f]
 /home/y/bin/traffic_server[0x4af739]
 /home/y/bin/traffic_server[0x5cf7ea]
 /home/y/bin/traffic_server[0x5d0a6a]
 /home/y/bin/traffic_server(_ZN7FetchSM15InvokePluginExtEi+0x33a)[0x4c7a5a]
 /home/y/bin/traffic_server[0x4af739]
 /home/y/bin/traffic_server(_ZN7FetchSM18process_fetch_readEi+0x187)[0x4c7f37]
 /home/y/bin/traffic_server[0x5cf7ea]
 /home/y/bin/traffic_server[0x5d0a6a]
 /home/y/bin/traffic_server(_ZN7FetchSM15InvokePluginExtEi+0x33a)[0x4c7a5a]
 /home/y/bin/traffic_server(_ZN7FetchSM18process_fetch_readEi+0x187)[0x4c7f37]
 /home/y/bin/traffic_server(_ZN7FetchSM13fetch_handlerEiPv+0x8b)[0x4c80eb]
 /home/y/bin/traffic_server(_ZN8PluginVC17process_read_sideEb+0x375)[0x4dd0d5]
 /home/y/bin/traffic_server(_ZN8PluginVC18process_write_sideEb+0x57a)[0x4dda2a]
 /home/y/bin/traffic_server(_ZN7FetchSM13fetch_handlerEiPv+0x8b)[0x4c80eb]
 /home/y/bin/traffic_server(_ZN8PluginVC12main_handlerEiPv+0x372)[0x4de272]
 /home/y/bin/traffic_server(_ZN8PluginVC17process_read_sideEb+0x375)[0x4dd0d5]
 /home/y/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x8f)[0x712f7f]
 /home/y/bin/traffic_server(_ZN8PluginVC18process_write_sideEb+0x57a)[0x4dda2a]
 /home/y/bin/traffic_server(_ZN7EThread7executeEv+0x61b)[0x713aab]
 /home/y/bin/traffic_server(_ZN8PluginVC12main_handlerEiPv+0x372)[0x4de272]
 /home/y/bin/traffic_server[0x71232a]
 /home/y/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x8f)[0x712f7f]
 /lib64/libpthread.so.0(+0x3297207851)[0x2b7372601851]
 /home/y/bin/traffic_server(_ZN7EThread7executeEv+0x61b)[0x713aab]
 /lib64/libc.so.6(clone+0x6d)[0x3296ee890d]
 /home/y/bin/traffic_server[0x71232a]
 /lib64/libpthread.so.0(+0x3297207851)[0x2b7372601851]



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (TS-2743) Crash with SPDY on production environment in SpdyNV::SpdyNV

2014-04-26 Thread Sudheer Vinukonda (JIRA) 

[ 
https://issues.apache.org/jira/browse/TS-2743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13982189#comment-13982189
 ] 

Sudheer Vinukonda commented on TS-2743:
---

As noted in my comments above, I'm already using a temp fix to ignore null 
values from TSMimeHdrFieldValueStringGet() (for headers such as Content-Type) 
and use time(0) for NULL date values (inline with TSMimeHdrFieldValueDateGet() 
) in my prod tests. The reason I didnt submit patch for this is  that I was 
expecting a solution that replaces the TS API with core API, based on jpeach's 
comments.

In any case, at the moment, I'm blocked by TS-2750. Will appreciate any help on 
that core.

 Crash with SPDY on production environment in SpdyNV::SpdyNV
 ---

 Key: TS-2743
 URL: https://issues.apache.org/jira/browse/TS-2743
 Project: Traffic Server
  Issue Type: Bug
  Components: Core, SPDY
Reporter: Sudheer Vinukonda
Assignee: Yunkai Zhang
  Labels: SPDY_ATS
 Attachments: 
 0001-TS-2743-Ignore-invalid-HTTP-headers-in-SpdyNV-carefu.patch


 After patching a fix (by removing the apparent incorrect assert failure on 
 non-ipv4 address in TsFetchCreate - jira ts-2742), noticed a new crash, this 
 time in SpdyNV::SpdyNV with the below stack trace. Will update more upon 
 investigation.
 FATAL: SpdyCommon.cc:133: failed assert `value  value_len`
 FATAL: SpdyCommon.cc:133: failed assert `value  value_len`
 /home/y/bin/traffic_server - STACK TRACE:
 /home/y/bin/traffic_server - STACK TRACE:
 /home/y/lib64/libtsutil.so.5(+0x1f5c8)[0x2b7371f6a5c8]
 /home/y/lib64/libtsutil.so.5(+0x1f5c8)[0x2b7371f6a5c8]
 /home/y/lib64/libtsutil.so.5(+0x1da8f)[0x2b7371f68a8f]
 /home/y/lib64/libtsutil.so.5(+0x1da8f)[0x2b7371f68a8f]
 /home/y/bin/traffic_server[0x4af739]
 /home/y/bin/traffic_server[0x5cf7ea]
 /home/y/bin/traffic_server[0x5d0a6a]
 /home/y/bin/traffic_server(_ZN7FetchSM15InvokePluginExtEi+0x33a)[0x4c7a5a]
 /home/y/bin/traffic_server[0x4af739]
 /home/y/bin/traffic_server(_ZN7FetchSM18process_fetch_readEi+0x187)[0x4c7f37]
 /home/y/bin/traffic_server[0x5cf7ea]
 /home/y/bin/traffic_server[0x5d0a6a]
 /home/y/bin/traffic_server(_ZN7FetchSM15InvokePluginExtEi+0x33a)[0x4c7a5a]
 /home/y/bin/traffic_server(_ZN7FetchSM18process_fetch_readEi+0x187)[0x4c7f37]
 /home/y/bin/traffic_server(_ZN7FetchSM13fetch_handlerEiPv+0x8b)[0x4c80eb]
 /home/y/bin/traffic_server(_ZN8PluginVC17process_read_sideEb+0x375)[0x4dd0d5]
 /home/y/bin/traffic_server(_ZN8PluginVC18process_write_sideEb+0x57a)[0x4dda2a]
 /home/y/bin/traffic_server(_ZN7FetchSM13fetch_handlerEiPv+0x8b)[0x4c80eb]
 /home/y/bin/traffic_server(_ZN8PluginVC12main_handlerEiPv+0x372)[0x4de272]
 /home/y/bin/traffic_server(_ZN8PluginVC17process_read_sideEb+0x375)[0x4dd0d5]
 /home/y/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x8f)[0x712f7f]
 /home/y/bin/traffic_server(_ZN8PluginVC18process_write_sideEb+0x57a)[0x4dda2a]
 /home/y/bin/traffic_server(_ZN7EThread7executeEv+0x61b)[0x713aab]
 /home/y/bin/traffic_server(_ZN8PluginVC12main_handlerEiPv+0x372)[0x4de272]
 /home/y/bin/traffic_server[0x71232a]
 /home/y/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x8f)[0x712f7f]
 /lib64/libpthread.so.0(+0x3297207851)[0x2b7372601851]
 /home/y/bin/traffic_server(_ZN7EThread7executeEv+0x61b)[0x713aab]
 /lib64/libc.so.6(clone+0x6d)[0x3296ee890d]
 /home/y/bin/traffic_server[0x71232a]
 /lib64/libpthread.so.0(+0x3297207851)[0x2b7372601851]



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (TS-2743) Crash with SPDY on production environment in SpdyNV::SpdyNV

2014-04-26 Thread Yunkai Zhang (JIRA) 

[ 
https://issues.apache.org/jira/browse/TS-2743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13982191#comment-13982191
 ] 

Yunkai Zhang commented on TS-2743:
--

I don't agree to use 'TSMimeHdrFieldValueDateGet'.

I don't think replacing with internal API is urgent, we can use TS API to fix 
these problem firstly.

After fixing all of these series issues, we can open a new jira ticket to 
replace TS API with internals.



 Crash with SPDY on production environment in SpdyNV::SpdyNV
 ---

 Key: TS-2743
 URL: https://issues.apache.org/jira/browse/TS-2743
 Project: Traffic Server
  Issue Type: Bug
  Components: Core, SPDY
Reporter: Sudheer Vinukonda
Assignee: Yunkai Zhang
  Labels: SPDY_ATS
 Attachments: 
 0001-TS-2743-Ignore-invalid-HTTP-headers-in-SpdyNV-carefu.patch


 After patching a fix (by removing the apparent incorrect assert failure on 
 non-ipv4 address in TsFetchCreate - jira ts-2742), noticed a new crash, this 
 time in SpdyNV::SpdyNV with the below stack trace. Will update more upon 
 investigation.
 FATAL: SpdyCommon.cc:133: failed assert `value  value_len`
 FATAL: SpdyCommon.cc:133: failed assert `value  value_len`
 /home/y/bin/traffic_server - STACK TRACE:
 /home/y/bin/traffic_server - STACK TRACE:
 /home/y/lib64/libtsutil.so.5(+0x1f5c8)[0x2b7371f6a5c8]
 /home/y/lib64/libtsutil.so.5(+0x1f5c8)[0x2b7371f6a5c8]
 /home/y/lib64/libtsutil.so.5(+0x1da8f)[0x2b7371f68a8f]
 /home/y/lib64/libtsutil.so.5(+0x1da8f)[0x2b7371f68a8f]
 /home/y/bin/traffic_server[0x4af739]
 /home/y/bin/traffic_server[0x5cf7ea]
 /home/y/bin/traffic_server[0x5d0a6a]
 /home/y/bin/traffic_server(_ZN7FetchSM15InvokePluginExtEi+0x33a)[0x4c7a5a]
 /home/y/bin/traffic_server[0x4af739]
 /home/y/bin/traffic_server(_ZN7FetchSM18process_fetch_readEi+0x187)[0x4c7f37]
 /home/y/bin/traffic_server[0x5cf7ea]
 /home/y/bin/traffic_server[0x5d0a6a]
 /home/y/bin/traffic_server(_ZN7FetchSM15InvokePluginExtEi+0x33a)[0x4c7a5a]
 /home/y/bin/traffic_server(_ZN7FetchSM18process_fetch_readEi+0x187)[0x4c7f37]
 /home/y/bin/traffic_server(_ZN7FetchSM13fetch_handlerEiPv+0x8b)[0x4c80eb]
 /home/y/bin/traffic_server(_ZN8PluginVC17process_read_sideEb+0x375)[0x4dd0d5]
 /home/y/bin/traffic_server(_ZN8PluginVC18process_write_sideEb+0x57a)[0x4dda2a]
 /home/y/bin/traffic_server(_ZN7FetchSM13fetch_handlerEiPv+0x8b)[0x4c80eb]
 /home/y/bin/traffic_server(_ZN8PluginVC12main_handlerEiPv+0x372)[0x4de272]
 /home/y/bin/traffic_server(_ZN8PluginVC17process_read_sideEb+0x375)[0x4dd0d5]
 /home/y/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x8f)[0x712f7f]
 /home/y/bin/traffic_server(_ZN8PluginVC18process_write_sideEb+0x57a)[0x4dda2a]
 /home/y/bin/traffic_server(_ZN7EThread7executeEv+0x61b)[0x713aab]
 /home/y/bin/traffic_server(_ZN8PluginVC12main_handlerEiPv+0x372)[0x4de272]
 /home/y/bin/traffic_server[0x71232a]
 /home/y/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x8f)[0x712f7f]
 /lib64/libpthread.so.0(+0x3297207851)[0x2b7372601851]
 /home/y/bin/traffic_server(_ZN7EThread7executeEv+0x61b)[0x713aab]
 /lib64/libc.so.6(clone+0x6d)[0x3296ee890d]
 /home/y/bin/traffic_server[0x71232a]
 /lib64/libpthread.so.0(+0x3297207851)[0x2b7372601851]



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (TS-2743) Crash with SPDY on production environment in SpdyNV::SpdyNV

2014-04-26 Thread Yunkai Zhang (JIRA) 

[ 
https://issues.apache.org/jira/browse/TS-2743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13982195#comment-13982195
 ] 

Yunkai Zhang commented on TS-2743:
--

And it seems that TS-2750 may caused by the side-effect of TS-2743，so let's fix 
them one by one. 

 Crash with SPDY on production environment in SpdyNV::SpdyNV
 ---

 Key: TS-2743
 URL: https://issues.apache.org/jira/browse/TS-2743
 Project: Traffic Server
  Issue Type: Bug
  Components: Core, SPDY
Reporter: Sudheer Vinukonda
Assignee: Yunkai Zhang
  Labels: SPDY_ATS
 Attachments: 
 0001-TS-2743-Ignore-invalid-HTTP-headers-in-SpdyNV-carefu.patch


 After patching a fix (by removing the apparent incorrect assert failure on 
 non-ipv4 address in TsFetchCreate - jira ts-2742), noticed a new crash, this 
 time in SpdyNV::SpdyNV with the below stack trace. Will update more upon 
 investigation.
 FATAL: SpdyCommon.cc:133: failed assert `value  value_len`
 FATAL: SpdyCommon.cc:133: failed assert `value  value_len`
 /home/y/bin/traffic_server - STACK TRACE:
 /home/y/bin/traffic_server - STACK TRACE:
 /home/y/lib64/libtsutil.so.5(+0x1f5c8)[0x2b7371f6a5c8]
 /home/y/lib64/libtsutil.so.5(+0x1f5c8)[0x2b7371f6a5c8]
 /home/y/lib64/libtsutil.so.5(+0x1da8f)[0x2b7371f68a8f]
 /home/y/lib64/libtsutil.so.5(+0x1da8f)[0x2b7371f68a8f]
 /home/y/bin/traffic_server[0x4af739]
 /home/y/bin/traffic_server[0x5cf7ea]
 /home/y/bin/traffic_server[0x5d0a6a]
 /home/y/bin/traffic_server(_ZN7FetchSM15InvokePluginExtEi+0x33a)[0x4c7a5a]
 /home/y/bin/traffic_server[0x4af739]
 /home/y/bin/traffic_server(_ZN7FetchSM18process_fetch_readEi+0x187)[0x4c7f37]
 /home/y/bin/traffic_server[0x5cf7ea]
 /home/y/bin/traffic_server[0x5d0a6a]
 /home/y/bin/traffic_server(_ZN7FetchSM15InvokePluginExtEi+0x33a)[0x4c7a5a]
 /home/y/bin/traffic_server(_ZN7FetchSM18process_fetch_readEi+0x187)[0x4c7f37]
 /home/y/bin/traffic_server(_ZN7FetchSM13fetch_handlerEiPv+0x8b)[0x4c80eb]
 /home/y/bin/traffic_server(_ZN8PluginVC17process_read_sideEb+0x375)[0x4dd0d5]
 /home/y/bin/traffic_server(_ZN8PluginVC18process_write_sideEb+0x57a)[0x4dda2a]
 /home/y/bin/traffic_server(_ZN7FetchSM13fetch_handlerEiPv+0x8b)[0x4c80eb]
 /home/y/bin/traffic_server(_ZN8PluginVC12main_handlerEiPv+0x372)[0x4de272]
 /home/y/bin/traffic_server(_ZN8PluginVC17process_read_sideEb+0x375)[0x4dd0d5]
 /home/y/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x8f)[0x712f7f]
 /home/y/bin/traffic_server(_ZN8PluginVC18process_write_sideEb+0x57a)[0x4dda2a]
 /home/y/bin/traffic_server(_ZN7EThread7executeEv+0x61b)[0x713aab]
 /home/y/bin/traffic_server(_ZN8PluginVC12main_handlerEiPv+0x372)[0x4de272]
 /home/y/bin/traffic_server[0x71232a]
 /home/y/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x8f)[0x712f7f]
 /lib64/libpthread.so.0(+0x3297207851)[0x2b7372601851]
 /home/y/bin/traffic_server(_ZN7EThread7executeEv+0x61b)[0x713aab]
 /lib64/libc.so.6(clone+0x6d)[0x3296ee890d]
 /home/y/bin/traffic_server[0x71232a]
 /lib64/libpthread.so.0(+0x3297207851)[0x2b7372601851]



--
This message was sent by Atlassian JIRA
(v6.2#6252)