Gary, Timeout case could happened by any abruptly interrupted (e.g. network disconnection, cable plug/unplug...).
Some exception behavior may happened while Timeout triggered. The rootcause as I descripted below: If TCP CompletionToken is not signaled, it should not be closed directly by calling CloseEvent after timeout happened (Still in the TCP RcvTokenList). If not, any exception behavior may be triggered. We should cancel it by calling Tcp->Cancel(), then close it. but unfortunately, Tcp->Cancel() is unsupported currently. However, we can flush the tokens in TCP token list(what we want to do) by closing the existing TCP connection(Actually we should!). It's the safe operation. Please help to verify the corresponding patch in edk2 mailing list. Title as below: [edk2] [Patch] NetworkPkg: Need update Http token status while timeout happened Thanks. Jiaxin > -----Original Message----- > From: Gary Lin [mailto:g...@suse.com] > Sent: Thursday, May 19, 2016 12:11 PM > To: Wu, Jiaxin <jiaxin...@intel.com> > Cc: El-Haj-Mahmoud, Samer <samer.el-haj-mahm...@hpe.com>; Laszlo > Ersek <ler...@redhat.com>; Hegde, Nagaraj P <nagaraj- > p.he...@hpe.com>; edk2-devel@lists.01.org > Subject: Re: [edk2] HTTP Boot crashed when downloading remote image > > On Thu, May 19, 2016 at 02:11:34AM +0000, Wu, Jiaxin wrote: > > Thanks Gary. We also found that if TCP CompletionToken is not signaled, it > should not be closed directly by calling CloseEvent after timeout > happened(Still in the TCP RcvTokenList). If not, any exception behavior may > be triggered while more packets continue received but only timeout > happened (do you root cause? I'm glad to know.). We should cancel it by > calling Tcp4->Cancel(), then close it. but unfortunately, Tcp4->Cancel() is > unsupported currently. > > > Hi Jiaxin, > > The case I encountered is that HttpTcpReceiveNotifyDpc freed Wrap > mistakenly and caused the instability of the system (just sent the patch). > Actually the timeout shouldn't show in my case since I used a VM to connect > to the host to download a small EFI file. > > Anyway, the problem you mentioned looks valid to me and maybe I should > try to make a timeout test. > > Thanks, > > Gary Lin > > > Thanks. > > Jiaxin > > > > > -----Original Message----- > > > From: Gary Lin [mailto:g...@suse.com] > > > Sent: Thursday, May 19, 2016 10:06 AM > > > To: El-Haj-Mahmoud, Samer <samer.el-haj-mahm...@hpe.com>; Laszlo > > > Ersek <ler...@redhat.com> > > > Cc: edk2-devel@lists.01.org; Wu, Jiaxin <jiaxin...@intel.com> > > > Subject: Re: [edk2] HTTP Boot crashed when downloading remote image > > > > > > On Wed, May 18, 2016 at 01:54:24PM +0000, El-Haj-Mahmoud, Samer > wrote: > > > > Gary, > > > > > > > > The EDK2 list blocked the wireshark attachment. Can you put it on > > > > a share > > > and send a link please? We are trying to look at this internally as well. > > > Hi Samer and Laszlo, > > > > > > I've found the root cause. I'll send a patch later and Cc you guys. > > > > > > Thanks, > > > > > > Gary Lin > > > > > > > > > > > Thanks, > > > > --Samer > > > > > > > > -----Original Message----- > > > > From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On > > > > Behalf Of Gary Lin > > > > Sent: Tuesday, May 17, 2016 11:29 PM > > > > To: edk2-devel@lists.01.org > > > > Cc: Jiaxin Wu <jiaxin...@intel.com> > > > > Subject: Re: [edk2] HTTP Boot crashed when downloading remote > > > > image > > > > > > > > On Thu, May 12, 2016 at 06:26:36PM +0800, Gary Lin wrote: > > > > > Hi, > > > > > > > > > > I was testing HTTP Boot with the latest OMVF and found that it > > > > > crashed when downloading the remote image from the http server. > > > > > Here is my bisect result: > > > > > > > > > > commit b347a22aecbfac9aac47831fee9a30aa810d6d0b > > > > > NetworkPkg: Avoid the indefinite wait case in HttpDxe > > > > > > > > > > Actually, it sometimes worked but was broken for the most of time. > > > > > Reverting this patch makes HTTP Boot always work. > > > > > > > > > > It seems the IP4 driver tried to remove a mnp event, but the > > > > > event list was empty so the assert was triggered. > > > > > > > > > Some findings: > > > > > > > > 1. OVMF could crash without b347a22ae in a slight different way. > > > > Several "TcpInput: Discard a packet" showed without the assert. > > > > > > > > 2. Rx4Token->CompletionToken in HttpTcpReceiveBody() corrupted > after > > > > the timeout. I saved Rx4Token->CompletionToken.Event before the > > > > loop of Tcp4->Poll() and set a assert like this: > > > > > > > > ASSERT(Rx4Token->CompletionToken.Event == event); > > > > > > > > right after "if (!Wrap->TcpWrap.IsRxDone)". The assert was raised > > > > after the timeout. > > > > > > > > 3. I attached wireshark to the tap interface. The first few packets were > > > > alright, and then the window size from ACK of OVMF decreased > rapidly > > > > and "TCP ZeroWindow" showed right before the crash. I added debug > > > > message to HttpTcpReceiveNotify and found that the function wasn't > > > > triggered for some reason when the packets arrived and so > > > > Wrap->TcpWrap.IsRxDone was never set to TRUE. In the end, the > socket > > > > RCVD buffer was used up, and then the timeout event triggered. > > > > > > > > I'll keep digging for more information. > > > > > > > > Thanks, > > > > > > > > Gary Lin > > > > > > > > > The debug message: > > > > > > > > > > TcpToSendAck: scheduled a delayed ACK for TCB 3E2D2418 > > > > > TcpToSendAck: scheduled a delayed ACK for TCB 3E2D2418 > > > > > TcpInput: sequence acceptance test failed for segment of TCB > > > > > 3E2D2418 > > > > > TcpInput: Discard a packet > > > > > TcpInput: sequence acceptance test failed for segment of TCB > > > > > 3E2D2418 > > > > > TcpInput: Discard a packet > > > > > TcpInput: sequence acceptance test failed for segment of TCB > > > > > 3E2D2418 > > > > > TcpInput: Discard a packet > > > > > TcpInput: sequence acceptance test failed for segment of TCB > > > > > 3E2D2418 > > > > > TcpInput: Discard a packet > > > > > ASSERT > /home/gary/git/edk2/MdePkg/Library/BaseLib/LinkedList.c(62): > > > > > List->BackLink != ((void *) 0) > > > > > > > > > > The backtrace (I cut the trace before EfiBootManagerBoot to make > > > > > it > > > > > shorter): > > > > > > > > > > (gdb) bt > > > > > #0 0x000000003ff7f702 in CpuDeadLoop () at > > > > > /home/gary/git/edk2/MdePkg/Library/BaseLib/CpuDeadLoop.c:37 > > > > > #1 0x000000003ff80a34 in DebugAssert ( > > > > > FileName=0x3ff93b50 > > > "/home/gary/git/edk2/MdePkg/Library/BaseLib/LinkedList.c", > > > LineNumber=62, > > > > > Description=0x3ff93bb0 "List->BackLink != ((void *) 0)") > > > > > at > > > > > > > > > /home/gary/git/edk2/OvmfPkg/Library/PlatformDebugLibIoPort/DebugLib.c: > > > > > 153 > > > > > #2 0x000000003ff7f7e3 in InternalBaseLibIsNodeInList > > > > > (List=0x3e2c40a0, > > > Node=0x3e2c40a0, > > > > > VerifyNodeInList=0 '\000') at > > > > > /home/gary/git/edk2/MdePkg/Library/BaseLib/LinkedList.c:62 > > > > > #3 0x000000003ff7fc49 in IsListEmpty (ListHead=0x3e2c40a0) > > > > > at > > > > > /home/gary/git/edk2/MdePkg/Library/BaseLib/LinkedList.c:361 > > > > > #4 0x000000003ff7fec5 in RemoveEntryList (Entry=0x3e2c40a0) > > > > > at > > > > > /home/gary/git/edk2/MdePkg/Library/BaseLib/LinkedList.c:545 > > > > > #5 0x000000003ff7a5e5 in CoreCloseEvent (UserEvent=0x3e2c4018) > > > > > at > > > > > /home/gary/git/edk2/MdeModulePkg/Core/Dxe/Event/Event.c:752 > > > > > #6 0x000000003e99a910 in Ip4FreeFrameRxToken > (Token=0x3e2c4118) > > > > > at > > > > > > > > > /home/gary/git/edk2/MdeModulePkg/Universal/Network/Ip4Dxe/Ip4If.c:38 > > > > > 0 > > > > > #7 0x000000003e99bdfe in Ip4RecycleFrame (Context=0x3e2c4118) > > > > > at > > > > > > > > > /home/gary/git/edk2/MdeModulePkg/Universal/Network/Ip4Dxe/Ip4If.c:11 > > > > > 24 > > > > > #8 0x000000003e9b26db in NetbufFreeVector (Vector=0x3e2c4798) > > > > > at > > > > > > > > > /home/gary/git/edk2/MdeModulePkg/Library/DxeNetLib/NetBuffer.c:176 > > > > > #9 0x000000003e9b285e in NetbufFree (Nbuf=0x3e2c4698) > > > > > at > > > > > > > > > /home/gary/git/edk2/MdeModulePkg/Library/DxeNetLib/NetBuffer.c:220 > > > > > #10 0x000000003e9a39b6 in Ip4OnRecyclePacket (Event=0x3e2c4198, > > > Context=0x3e2c4398) > > > > > at > > > > > > > > > /home/gary/git/edk2/MdeModulePkg/Universal/Network/Ip4Dxe/Ip4Input. > > > c > > > > > :1 > > > > > 146 > > > > > #11 0x000000003ff79bd5 in CoreDispatchEventNotifies (Priority=16) > > > > > at > > > > > /home/gary/git/edk2/MdeModulePkg/Core/Dxe/Event/Event.c:200 > > > > > #12 0x000000003ff79179 in CoreRestoreTpl (NewTpl=8) at > > > > > /home/gary/git/edk2/MdeModulePkg/Core/Dxe/Event/Tpl.c:131 > > > > > #13 0x000000003ff660e0 in CoreReleaseLock (Lock=0x3ff98990) > > > > > at > > > > > > /home/gary/git/edk2/MdeModulePkg/Core/Dxe/Library/Library.c:102 > > > > > #14 0x000000003ff7996b in CoreReleaseEventLock () at > > > > > /home/gary/git/edk2/MdeModulePkg/Core/Dxe/Event/Event.c:119 > > > > > #15 0x000000003ff7a32c in CoreSignalEvent (UserEvent=0x3e2c4198) > > > > > at > > > > > /home/gary/git/edk2/MdeModulePkg/Core/Dxe/Event/Event.c:572 > > > > > #16 0x000000003e929c28 in IpIoExtFree (Event=0x3e2c4198) > > > > > at > > > > > > > > > /home/gary/git/edk2/MdeModulePkg/Library/DxeIpIoLib/DxeIpIoLib.c:596 > > > > > #17 0x000000003e9307fb in NetbufFreeVector (Vector=0x3e2c4598) > > > > > at > > > > > > > > > /home/gary/git/edk2/MdeModulePkg/Library/DxeNetLib/NetBuffer.c:176 > > > > > #18 0x000000003e93097e in NetbufFree (Nbuf=0x3e2c4498) > > > > > at > > > > > > > > > /home/gary/git/edk2/MdeModulePkg/Library/DxeNetLib/NetBuffer.c:220 > > > > > #19 0x000000003e932551 in NetbufQueTrim (NbufQue=0x3e2d2698, > > > Len=59695) > > > > > at > > > > > > > > > /home/gary/git/edk2/MdeModulePkg/Library/DxeNetLib/NetBuffer.c:1605 > > > > > #20 0x000000003e923fcd in SockProcessRcvToken (Sock=0x3e2d1018, > > > RcvToken=0x3e2aa098) > > > > > at /home/gary/git/edk2/NetworkPkg/TcpDxe/SockImpl.c:346 > > > > > #21 0x000000003e919f5f in SockRcv (Sock=0x3e2d1018, > > > Token=0x3e2aa098) > > > > > at /home/gary/git/edk2/NetworkPkg/TcpDxe/SockInterface.c:727 > > > > > #22 0x000000003e91ef11 in Tcp4Receive (This=0x3e2d1138, > > > Token=0x3e2aa098) > > > > > at /home/gary/git/edk2/NetworkPkg/TcpDxe/TcpMain.c:440 > > > > > #23 0x000000003e8636e2 in HttpTcpReceiveBody (Wrap=0x3e2aa018, > > > HttpMsg=0x3ef7e0d0, Timeout=0x3e2c6218) > > > > > at /home/gary/git/edk2/NetworkPkg/HttpDxe/HttpProto.c:1797 > > > > > #24 0x000000003e85f86c in HttpResponseWorker (Wrap=0x3e2aa018) > > > > > at /home/gary/git/edk2/NetworkPkg/HttpDxe/HttpImpl.c:1157 > > > > > #25 0x000000003e85fd2e in EfiHttpResponse (This=0x3e2d12a0, > > > Token=0x3ef7e0b8) > > > > > at /home/gary/git/edk2/NetworkPkg/HttpDxe/HttpImpl.c:1323 > > > > > #26 0x000000003e845578 in HttpIoRecvResponse (HttpIo=0x3ef7e050, > > > RecvMsgHeader=0 '\000', ResponseData=0x3ff593b0) > > > > > at > > > > > > /home/gary/git/edk2/NetworkPkg/HttpBootDxe/HttpBootSupport.c:921 > > > > > #27 0x000000003e847bb3 in HttpBootGetBootFile > > > > > (Private=0x3ef7e018, > > > HeaderOnly=0 '\000', BufferSize=0x3ff59550, > > > > > Buffer=0x3e16a018 "MZ\220", ImageType=0x3ff594b8) > > > > > at > > > > > > /home/gary/git/edk2/NetworkPkg/HttpBootDxe/HttpBootClient.c:1006 > > > > > #28 0x000000003e83ff69 in HttpBootLoadFile (Private=0x3ef7e018, > > > BufferSize=0x3ff59550, Buffer=0x3e16a018, > > > > > ImageType=0x3ff594b8) at > > > > > /home/gary/git/edk2/NetworkPkg/HttpBootDxe/HttpBootImpl.c:304 > > > > > #29 0x000000003e8404b2 in HttpBootDxeLoadFile (This=0x3ef7d030, > > > FilePath=0x3ee1706e, BootPolicy=1 '\001', > > > > > BufferSize=0x3ff59550, Buffer=0x3e16a018) at > > > > > /home/gary/git/edk2/NetworkPkg/HttpBootDxe/HttpBootImpl.c:489 > > > > > #30 0x000000003e2682b9 in BmGetFileBufferFromLoadFile > > > (LoadFileHandle=0x3ef7d118, FilePath=0x3ee1706e, > > > > > FullPath=0x3ff596c0, FileSize=0x3ff596a8) > > > > > at > > > > > > > > > /home/gary/git/edk2/MdeModulePkg/Library/UefiBootManagerLib/BmBoo > > > t.c > > > > > :1 > > > > > 320 > > > > > #31 0x000000003e26859d in BmGetFileBufferFromLoadFiles > > > (FilePath=0x3ee1706e, FullPath=0x3ff596c0, > > > > > FileSize=0x3ff596a8) at > > > > > > > > > /home/gary/git/edk2/MdeModulePkg/Library/UefiBootManagerLib/BmBoo > > > t.c > > > > > :1 > > > > > 412 > > > > > #32 0x000000003e26899a in EfiBootManagerGetLoadOptionBuffer > > > (FilePath=0x3ee17018, FullPath=0x3ff596c0, > > > > > FileSize=0x3ff596a8) at > > > > > > > > > /home/gary/git/edk2/MdeModulePkg/Library/UefiBootManagerLib/BmBoo > > > t.c > > > > > :1 > > > > > 529 > > > > > #33 0x000000003e268ce2 in EfiBootManagerBoot > > > (BootOption=0x3e2d6c50) > > > > > at > > > > > > > > > /home/gary/git/edk2/MdeModulePkg/Library/UefiBootManagerLib/BmBoo > > > t.c > > > > > :1 > > > > > 652 > > > > > > > > > > Cheers, > > > > > > > > > > Gary Lin > > > > > _______________________________________________ > > > > > edk2-devel mailing list > > > > > edk2-devel@lists.01.org > > > > > https://lists.01.org/mailman/listinfo/edk2-devel > > > > > > > > > _______________________________________________ > > > > edk2-devel mailing list > > > > edk2-devel@lists.01.org > > > > https://lists.01.org/mailman/listinfo/edk2-devel > > > > _______________________________________________ > > > > edk2-devel mailing list > > > > edk2-devel@lists.01.org > > > > https://lists.01.org/mailman/listinfo/edk2-devel > > > > > > _______________________________________________ edk2-devel mailing list edk2-devel@lists.01.org https://lists.01.org/mailman/listinfo/edk2-devel
