Re: [edk2] Strange behavior of the DS-5 debugger on AARCH64 with step-by-step debugging in uefi

[Andrew Fish]

> On Nov 23, 2015, at 11:31 PM, Ard Biesheuvel  
> wrote:
> 
> On 24 November 2015 at 00:38, Vladimir Olovyannikov
>  wrote:
>> 
>> 
>>> -Original Message-
>>> From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
>>> Sent: Tuesday, November 10, 2015 10:04 AM
>>> To: Vladimir Olovyannikov
>>> Cc: Cohen, Eugene; edk2-devel@lists.01.org
>>> Subject: Re: [edk2] Strange behavior of the DS-5 debugger on AARCH64 with
>>> step-by-step debugging in uefi
>>> 
>>> On 10 November 2015 at 18:41, Vladimir Olovyannikov
>>>  wrote:
 Ard,
 Many thanks for your help. It works.
 
>>> 
>>> Great! Thanks for reporting back.
>>> 
>>> 
 -Original Message-
 From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
 Sent: Monday, November 09, 2015 10:31 PM
 To: Vladimir Olovyannikov
 Cc: Cohen, Eugene; edk2-devel@lists.01.org
 Subject: Re: [edk2] Strange behavior of the DS-5 debugger on AARCH64
>>> with step-by-step debugging in uefi
 
 On 9 November 2015 at 19:01, Vladimir Olovyannikov
  wrote:
> -Original Message-
> From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
> Sent: Sunday, November 08, 2015 10:52 PM
> To: Vladimir Olovyannikov
> Cc: Cohen, Eugene; edk2-devel@lists.01.org
> Subject: Re: [edk2] Strange behavior of the DS-5 debugger on AARCH64
>>> with step-by-step debugging in uefi
> 
> On 6 November 2015 at 21:32, Vladimir Olovyannikov
> > wrote:
>>> Hello Ard, Eugene,
>>> Thank you for explanation.
>>> 
>>> Ard, I tried the patch, but it cannot be applied to the latest (pulled a
>>> minute ago, git-svn-id:
>>> https://svn.code.sf.net/p/edk2/code/trunk/edk2@18732 6f19259b-4bc3-
>>> 4df7-8a09-765794883524)
>>> tree: all 3 hunks failed. Which commit should I be based on to apply the
>>> patch?
>>> 
>>> Anyway I found the lines manually and changed them. However, when
>>> I try to
>>> 
>>> source /uefi/ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py -f
>>> (0x8500,0x0028) -m (0x8000,0x4000) -a
>>> I am getting
>>> 
>>> ERROR(?): ValueError: need more than 1 value to unpack
>>>  File " /uefi/ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py", line
>>> 94, in >
>>>armplatform_debugger.load_all_symbols()
>>> ERROR(CMD656):
>>> # in /uefi/BroadcomPlatformPkg/NS2Pkg/Scripts/armpkg_syms.ds:2
>>> while executing: source
>>> /uefi/ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py -f
>>> (0x8500,0x0028) -m (0x8000,0x4000) -a
>>> ! The script /uefi/ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py
>>> failed to complete due to an error during execution of the script
>>> 
>> [...]
>> Ard, I got a pretty much the same issue when I tried to do some debugging in 
>> the ShellPkg.
>> Except Shell I can perfectly debug everything.
>> 
>> 1. source / uefi/ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py -f 
>> (0x8500,0x0028) -m (0x8000,0x4000) -a
>> loads symbols fine, but does not recognize any module matching the 
>> current PC if stopped in the shell.
>> 2. Loading symbols with "add-symbol-file 
>> /uefi/Build/NS2Pkg/DEBUG_GCC49/AARCH64/ShellPkg/Application/Shell/Shell/DEBUG/Shell.dll
>>  0xB6926000"
>>"recognizes" modules (wrong ones though) but the source code does not 
>> match disassembly.
>> 
>> So with Shell debug using DS-5 the code does not match the source.
>> Is there a special linker setting I am missing or a technique?
>> I am using the latest UEFI code from
>> https://github.com/tianocore/edk2.git
>> 
> 
> I am sorry, but since I don't have access to DS-5, I am not sure how
> to debug this.
> 
> Is there any way for you to figure how much the offset is between the
> current and the correct location? I.e., by looking at the ELF asm dump
> and the instructions around the PC? Something in the order of 0x260
> perhaps?
> 

The map file in the build output directory is useful for tracking down these 
kind of issues. 

Issues like this usually relate to the fact that PE/COFF images load the 
PE/COFF header into memory when the code is loaded, while ELF (and Mach-O) do 
not do this. This means you have to shift the link address from 0 to the size 
of the PE/COFF header (0x260 is one possible size of the PE/COFF header). 
Basically it is part of the ELF to PE/COFF conversion magic. Maybe the ELF to 
PE/COFF conversion tool and the debugger script our out of sync? 

The other thing to look out for is TE (Tiano Executable) images. They have a 
stripped down version of a PE/COFF header to save space, and they are mostly 
used for PEI modules that run from FLASH. You have to add a negative adjustment 
to get the symbols loaded correctly. 

Thanks,

Andrew Fish

> -- 
> Ard.
> ___
> edk2-devel mailing list
> edk2-devel@lists.01.org
> 

Re: [edk2] Strange behavior of the DS-5 debugger on AARCH64 with step-by-step debugging in uefi

[Vladimir Olovyannikov]

Thank you Andrew, Ard.

The problem was my wrong interpretation of the offset in DS-5 for the ShellPkg,
and therefore wrong entry point arithmetic.

Vladimir

> -Original Message-
> From: af...@apple.com [mailto:af...@apple.com]
> Sent: Monday, November 30, 2015 10:02 AM
> To: Vladimir Olovyannikov
> Cc: edk2-devel@lists.01.org
> Subject: Re: [edk2] Strange behavior of the DS-5 debugger on AARCH64 with
> step-by-step debugging in uefi
> 
> 
> > On Nov 23, 2015, at 11:31 PM, Ard Biesheuvel 
> wrote:
> >
> > On 24 November 2015 at 00:38, Vladimir Olovyannikov
> >  wrote:
> >>
> >>
> >>> -Original Message-
> >>> From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
> >>> Sent: Tuesday, November 10, 2015 10:04 AM
> >>> To: Vladimir Olovyannikov
> >>> Cc: Cohen, Eugene; edk2-devel@lists.01.org
> >>> Subject: Re: [edk2] Strange behavior of the DS-5 debugger on AARCH64
> with
> >>> step-by-step debugging in uefi
> >>>
> >>> On 10 November 2015 at 18:41, Vladimir Olovyannikov
> >>>  wrote:
>  Ard,
>  Many thanks for your help. It works.
> 
> >>>
> >>> Great! Thanks for reporting back.
> >>>
> >>>
>  -Original Message-
>  From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
>  Sent: Monday, November 09, 2015 10:31 PM
>  To: Vladimir Olovyannikov
>  Cc: Cohen, Eugene; edk2-devel@lists.01.org
>  Subject: Re: [edk2] Strange behavior of the DS-5 debugger on AARCH64
> >>> with step-by-step debugging in uefi
> 
>  On 9 November 2015 at 19:01, Vladimir Olovyannikov
>   wrote:
> > -Original Message-
> > From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
> > Sent: Sunday, November 08, 2015 10:52 PM
> > To: Vladimir Olovyannikov
> > Cc: Cohen, Eugene; edk2-devel@lists.01.org
> > Subject: Re: [edk2] Strange behavior of the DS-5 debugger on
> AARCH64
> >>> with step-by-step debugging in uefi
> >
> > On 6 November 2015 at 21:32, Vladimir Olovyannikov
> > > wrote:
> >>> Hello Ard, Eugene,
> >>> Thank you for explanation.
> >>>
> >>> Ard, I tried the patch, but it cannot be applied to the latest 
> >>> (pulled a
> >>> minute ago, git-svn-id:
> >>> https://svn.code.sf.net/p/edk2/code/trunk/edk2@18732 6f19259b-
> 4bc3-
> >>> 4df7-8a09-765794883524)
> >>> tree: all 3 hunks failed. Which commit should I be based on to apply
> the
> >>> patch?
> >>>
> >>> Anyway I found the lines manually and changed them. However,
> when
> >>> I try to
> >>>
> >>> source /uefi/ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py -f
> >>> (0x8500,0x0028) -m (0x8000,0x4000) -a
> >>> I am getting
> >>>
> >>> ERROR(?): ValueError: need more than 1 value to unpack
> >>>  File " /uefi/ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py",
> line
> >>> 94, in >
> >>>armplatform_debugger.load_all_symbols()
> >>> ERROR(CMD656):
> >>> # in
> /uefi/BroadcomPlatformPkg/NS2Pkg/Scripts/armpkg_syms.ds:2
> >>> while executing: source
> >>> /uefi/ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py -f
> >>> (0x8500,0x0028) -m (0x8000,0x4000) -a
> >>> ! The script
> /uefi/ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py
> >>> failed to complete due to an error during execution of the script
> >>>
> >> [...]
> >> Ard, I got a pretty much the same issue when I tried to do some
> debugging in the ShellPkg.
> >> Except Shell I can perfectly debug everything.
> >>
> >> 1. source / uefi/ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py -f
> (0x8500,0x0028) -m (0x8000,0x4000) -a
> >> loads symbols fine, but does not recognize any module matching the
> current PC if stopped in the shell.
> >> 2. Loading symbols with "add-symbol-file
> /uefi/Build/NS2Pkg/DEBUG_GCC49/AARCH64/ShellPkg/Application/Shell/Sh
> ell/DEBUG/Shell.dll 0xB6926000"
> >>"recognizes" modules (wrong ones though) but the source code does
> not match disassembly.
> >>
> >> So with Shell debug using DS-5 the code does not match the source.
> >> Is there a special linker setting I am missing or a technique?
> >> I am using the latest UEFI code from
> >> https://github.com/tianocore/edk2.git
> >>
> >
> > I am sorry, but since I don't have access to DS-5, I am not sure how
> > to debug this.
> >
> > Is there any way for you to figure how much the offset is between the
> > current and the correct location? I.e., by looking at the ELF asm dump
> > and the instructions around the PC? Something in the order of 0x260
> > perhaps?
> >
> 
> The map file in the build output directory is useful for tracking down these
> kind of issues.
> 
> Issues like this usually relate to the fact that PE/COFF images load the
> PE/COFF header into memory when the code is loaded, while ELF (and Mach-
> O) do not do this. This means you have to shift the link address from 0 to the
> 

Re: [edk2] Strange behavior of the DS-5 debugger on AARCH64 with step-by-step debugging in uefi

[Ard Biesheuvel]

On 1 December 2015 at 01:24, Vladimir Olovyannikov
 wrote:
> Thank you Andrew, Ard.
>
> The problem was my wrong interpretation of the offset in DS-5 for the 
> ShellPkg,
> and therefore wrong entry point arithmetic.
>

OK, thanks for clearing that up.

Regards,
Ard.


>> -Original Message-
>> From: af...@apple.com [mailto:af...@apple.com]
>> Sent: Monday, November 30, 2015 10:02 AM
>> To: Vladimir Olovyannikov
>> Cc: edk2-devel@lists.01.org
>> Subject: Re: [edk2] Strange behavior of the DS-5 debugger on AARCH64 with
>> step-by-step debugging in uefi
>>
>>
>> > On Nov 23, 2015, at 11:31 PM, Ard Biesheuvel 
>> wrote:
>> >
>> > On 24 November 2015 at 00:38, Vladimir Olovyannikov
>> >  wrote:
>> >>
>> >>
>> >>> -Original Message-
>> >>> From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
>> >>> Sent: Tuesday, November 10, 2015 10:04 AM
>> >>> To: Vladimir Olovyannikov
>> >>> Cc: Cohen, Eugene; edk2-devel@lists.01.org
>> >>> Subject: Re: [edk2] Strange behavior of the DS-5 debugger on AARCH64
>> with
>> >>> step-by-step debugging in uefi
>> >>>
>> >>> On 10 November 2015 at 18:41, Vladimir Olovyannikov
>> >>>  wrote:
>>  Ard,
>>  Many thanks for your help. It works.
>> 
>> >>>
>> >>> Great! Thanks for reporting back.
>> >>>
>> >>>
>>  -Original Message-
>>  From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
>>  Sent: Monday, November 09, 2015 10:31 PM
>>  To: Vladimir Olovyannikov
>>  Cc: Cohen, Eugene; edk2-devel@lists.01.org
>>  Subject: Re: [edk2] Strange behavior of the DS-5 debugger on AARCH64
>> >>> with step-by-step debugging in uefi
>> 
>>  On 9 November 2015 at 19:01, Vladimir Olovyannikov
>>   wrote:
>> > -Original Message-
>> > From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
>> > Sent: Sunday, November 08, 2015 10:52 PM
>> > To: Vladimir Olovyannikov
>> > Cc: Cohen, Eugene; edk2-devel@lists.01.org
>> > Subject: Re: [edk2] Strange behavior of the DS-5 debugger on
>> AARCH64
>> >>> with step-by-step debugging in uefi
>> >
>> > On 6 November 2015 at 21:32, Vladimir Olovyannikov
>> > > wrote:
>> >>> Hello Ard, Eugene,
>> >>> Thank you for explanation.
>> >>>
>> >>> Ard, I tried the patch, but it cannot be applied to the latest 
>> >>> (pulled a
>> >>> minute ago, git-svn-id:
>> >>> https://svn.code.sf.net/p/edk2/code/trunk/edk2@18732 6f19259b-
>> 4bc3-
>> >>> 4df7-8a09-765794883524)
>> >>> tree: all 3 hunks failed. Which commit should I be based on to apply
>> the
>> >>> patch?
>> >>>
>> >>> Anyway I found the lines manually and changed them. However,
>> when
>> >>> I try to
>> >>>
>> >>> source /uefi/ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py -f
>> >>> (0x8500,0x0028) -m (0x8000,0x4000) -a
>> >>> I am getting
>> >>>
>> >>> ERROR(?): ValueError: need more than 1 value to unpack
>> >>>  File " /uefi/ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py",
>> line
>> >>> 94, in >
>> >>>armplatform_debugger.load_all_symbols()
>> >>> ERROR(CMD656):
>> >>> # in
>> /uefi/BroadcomPlatformPkg/NS2Pkg/Scripts/armpkg_syms.ds:2
>> >>> while executing: source
>> >>> /uefi/ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py -f
>> >>> (0x8500,0x0028) -m (0x8000,0x4000) -a
>> >>> ! The script
>> /uefi/ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py
>> >>> failed to complete due to an error during execution of the script
>> >>>
>> >> [...]
>> >> Ard, I got a pretty much the same issue when I tried to do some
>> debugging in the ShellPkg.
>> >> Except Shell I can perfectly debug everything.
>> >>
>> >> 1. source / uefi/ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py -f
>> (0x8500,0x0028) -m (0x8000,0x4000) -a
>> >> loads symbols fine, but does not recognize any module matching the
>> current PC if stopped in the shell.
>> >> 2. Loading symbols with "add-symbol-file
>> /uefi/Build/NS2Pkg/DEBUG_GCC49/AARCH64/ShellPkg/Application/Shell/Sh
>> ell/DEBUG/Shell.dll 0xB6926000"
>> >>"recognizes" modules (wrong ones though) but the source code does
>> not match disassembly.
>> >>
>> >> So with Shell debug using DS-5 the code does not match the source.
>> >> Is there a special linker setting I am missing or a technique?
>> >> I am using the latest UEFI code from
>> >> https://github.com/tianocore/edk2.git
>> >>
>> >
>> > I am sorry, but since I don't have access to DS-5, I am not sure how
>> > to debug this.
>> >
>> > Is there any way for you to figure how much the offset is between the
>> > current and the correct location? I.e., by looking at the ELF asm dump
>> > and the instructions around the PC? Something in the order of 0x260
>> > perhaps?
>> >
>>
>> The map file in the build output directory is useful for tracking down these
>> 

Re: [edk2] Strange behavior of the DS-5 debugger on AARCH64 with step-by-step debugging in uefi

[Vladimir Olovyannikov]

> -Original Message-
> From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
> Sent: Tuesday, November 10, 2015 10:04 AM
> To: Vladimir Olovyannikov
> Cc: Cohen, Eugene; edk2-devel@lists.01.org
> Subject: Re: [edk2] Strange behavior of the DS-5 debugger on AARCH64 with
> step-by-step debugging in uefi
> 
> On 10 November 2015 at 18:41, Vladimir Olovyannikov
>  wrote:
> > Ard,
> > Many thanks for your help. It works.
> >
> 
> Great! Thanks for reporting back.
> 
> 
> > -Original Message-
> > From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
> > Sent: Monday, November 09, 2015 10:31 PM
> > To: Vladimir Olovyannikov
> > Cc: Cohen, Eugene; edk2-devel@lists.01.org
> > Subject: Re: [edk2] Strange behavior of the DS-5 debugger on AARCH64
> with step-by-step debugging in uefi
> >
> > On 9 November 2015 at 19:01, Vladimir Olovyannikov
> >  wrote:
> >> -Original Message-
> >> From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
> >> Sent: Sunday, November 08, 2015 10:52 PM
> >> To: Vladimir Olovyannikov
> >> Cc: Cohen, Eugene; edk2-devel@lists.01.org
> >> Subject: Re: [edk2] Strange behavior of the DS-5 debugger on AARCH64
> with step-by-step debugging in uefi
> >>
> >> On 6 November 2015 at 21:32, Vladimir Olovyannikov
> >> > wrote:
>  Hello Ard, Eugene,
>  Thank you for explanation.
> 
>  Ard, I tried the patch, but it cannot be applied to the latest (pulled a
> minute ago, git-svn-id:
> https://svn.code.sf.net/p/edk2/code/trunk/edk2@18732 6f19259b-4bc3-
> 4df7-8a09-765794883524)
>  tree: all 3 hunks failed. Which commit should I be based on to apply the
> patch?
> 
>  Anyway I found the lines manually and changed them. However, when
> I try to
> 
>  source /uefi/ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py -f
> (0x8500,0x0028) -m (0x8000,0x4000) -a
>  I am getting
> 
>  ERROR(?): ValueError: need more than 1 value to unpack
>    File " /uefi/ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py", line
> 94, in >
>  armplatform_debugger.load_all_symbols()
>  ERROR(CMD656):
>  # in /uefi/BroadcomPlatformPkg/NS2Pkg/Scripts/armpkg_syms.ds:2
> while executing: source
> /uefi/ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py -f
> (0x8500,0x0028) -m (0x8000,0x4000) -a
>  ! The script /uefi/ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py
> failed to complete due to an error during execution of the script
> 
[...]
Ard, I got a pretty much the same issue when I tried to do some debugging in 
the ShellPkg.
Except Shell I can perfectly debug everything. 

1. source / uefi/ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py -f 
(0x8500,0x0028) -m (0x8000,0x4000) -a
 loads symbols fine, but does not recognize any module matching the current 
PC if stopped in the shell.
2. Loading symbols with "add-symbol-file 
/uefi/Build/NS2Pkg/DEBUG_GCC49/AARCH64/ShellPkg/Application/Shell/Shell/DEBUG/Shell.dll
 0xB6926000"
"recognizes" modules (wrong ones though) but the source code does not match 
disassembly.

So with Shell debug using DS-5 the code does not match the source. 
Is there a special linker setting I am missing or a technique? 
I am using the latest UEFI code from
https://github.com/tianocore/edk2.git

Thank you,
Vladimir
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] Strange behavior of the DS-5 debugger on AARCH64 with step-by-step debugging in uefi

[Vladimir Olovyannikov]

Ard,
Many thanks for your help. It works.

Vladimir

-Original Message-
From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org] 
Sent: Monday, November 09, 2015 10:31 PM
To: Vladimir Olovyannikov
Cc: Cohen, Eugene; edk2-devel@lists.01.org
Subject: Re: [edk2] Strange behavior of the DS-5 debugger on AARCH64 with 
step-by-step debugging in uefi

On 9 November 2015 at 19:01, Vladimir Olovyannikov
 wrote:
> -Original Message-
> From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
> Sent: Sunday, November 08, 2015 10:52 PM
> To: Vladimir Olovyannikov
> Cc: Cohen, Eugene; edk2-devel@lists.01.org
> Subject: Re: [edk2] Strange behavior of the DS-5 debugger on AARCH64 with 
> step-by-step debugging in uefi
>
> On 6 November 2015 at 21:32, Vladimir Olovyannikov
> > wrote:
>>> Hello Ard, Eugene,
>>> Thank you for explanation.
>>>
>>> Ard, I tried the patch, but it cannot be applied to the latest (pulled a 
>>> minute ago, git-svn-id: 
>>> https://svn.code.sf.net/p/edk2/code/trunk/edk2@18732 
>>> 6f19259b-4bc3-4df7-8a09-765794883524)
>>> tree: all 3 hunks failed. Which commit should I be based on to apply the 
>>> patch?
>>>
>>> Anyway I found the lines manually and changed them. However, when I try to
>>>
>>> source /uefi/ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py -f 
>>> (0x8500,0x0028) -m (0x8000,0x4000) -a
>>> I am getting
>>>
>>> ERROR(?): ValueError: need more than 1 value to unpack
>>>   File " /uefi/ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py", line 94, in 
>>> >
>>> armplatform_debugger.load_all_symbols()
>>> ERROR(CMD656):
>>> # in /uefi/BroadcomPlatformPkg/NS2Pkg/Scripts/armpkg_syms.ds:2 while 
>>> executing: source /uefi/ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py -f 
>>> (0x8500,0x0028) -m (0x8000,0x4000) -a
>>> ! The script /uefi/ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py failed to 
>>> complete due to an error during execution of the script
>>>
>>> Replacing the script with the older version makes it work as before, but 
>>> again the debugger is useless.
>>>
>>> Any idea what is wrong?
>>>
>
>>No, not a clue. But could you please share the diff between the
>>version of the script that works for you and the version that doesn't?
>>I don't think the patch I sent could trigger the error you are seeing,
>>so it must be something else that changed.
> Here is the diff.
>
> --- firmware_volume_old.py  2015-06-24 15:15:37.0 -0700
> +++ firmware_volume_new.py  2015-11-06 11:49:24.321730539 -0800
> @@ -91,12 +91,7 @@
>  return filename[0:string.find(filename,'\0')]
>
>  def get_debug_elfbase(self):
> -stripped_size = struct.unpack(" self.ec.getMemoryService().read(self.base_te + 0x6, 2, 32))[0]
> -stripped_size -= EfiSectionTE.SIZEOF_EFI_TE_IMAGE_HEADER
> -
> -base_of_code = self.ec.getMemoryService().readMemory32(self.base_te 
> + 0xC)
> -
> -return self.base_te + base_of_code - stripped_size
> +return self.base_pe32
>

This hunk is incorrect. Could you please try the patch instead?
(attached, this time, so it should apply more easily)

Thanks,
Ard.

>  class EfiSectionPE32:
>  def __init__(self, ec, base_pe32):
> @@ -131,17 +126,7 @@
>  return filename[0:string.find(filename,'\0')]
>
>  def get_debug_elfbase(self):
> -# Offset from dos hdr to PE file hdr
> -pe_file_header = self.base_pe32 + 
> self.ec.getMemoryService().readMemory32(self.base_pe32 + 0x3C)
> -
> -base_of_code = self.base_pe32 + 
> self.ec.getMemoryService().readMemory32(pe_file_header + 0x28)
> -base_of_data = self.base_pe32 + 
> self.ec.getMemoryService().readMemory32(pe_file_header + 0x2C)
> -
> -if (base_of_code < base_of_data) and (base_of_code != 0):
> -return base_of_code
> -else:
> -return base_of_data
> -
> +return self.base_pe32
>  class EfiSectionPE64:
>  def __init__(self, ec, base_pe64):
>  self.ec = ec
> @@ -176,16 +161,7 @@
>  return filename[0:string.find(filename,'\0')]
>
>  def get_debug_elfbase(self):
> -# Offset from dos hdr to PE file hdr
> -pe_file_header = self.base_pe64 + 
> self.ec.getMemoryService().readMemory32(self.base_pe64 + 0x3C)
> -
> -base_of_code = self.base_pe64 + 
> self.ec.getMemoryService().readMemory32(pe_file_header + 0x28)
> -base_of_data = self.base_pe64 + 
> self.ec.getMemoryService().readMemory32(pe_file_header + 0x2C)
> -
> -if (base_of_code < base_of_data) and (base_of_code != 0):
> -return base_of_code
> -else:
> -return base_of_data
> +return self.base_pe64
>
>  class FirmwareFile:
>  EFI_FV_FILETYPE_RAW   = 0x01
>
>>> Eugene, so you do not use the so convenient cmd_load_symbols.py script to 
>>> load all symbols?
>>> I dumped efi and dll as you suggested. The .text and .data sections in 
>>> 

Re: [edk2] Strange behavior of the DS-5 debugger on AARCH64 with step-by-step debugging in uefi

[Ard Biesheuvel]

On 10 November 2015 at 18:41, Vladimir Olovyannikov
 wrote:
> Ard,
> Many thanks for your help. It works.
>

Great! Thanks for reporting back.

-- 
Ard.

> -Original Message-
> From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
> Sent: Monday, November 09, 2015 10:31 PM
> To: Vladimir Olovyannikov
> Cc: Cohen, Eugene; edk2-devel@lists.01.org
> Subject: Re: [edk2] Strange behavior of the DS-5 debugger on AARCH64 with 
> step-by-step debugging in uefi
>
> On 9 November 2015 at 19:01, Vladimir Olovyannikov
>  wrote:
>> -Original Message-
>> From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
>> Sent: Sunday, November 08, 2015 10:52 PM
>> To: Vladimir Olovyannikov
>> Cc: Cohen, Eugene; edk2-devel@lists.01.org
>> Subject: Re: [edk2] Strange behavior of the DS-5 debugger on AARCH64 with 
>> step-by-step debugging in uefi
>>
>> On 6 November 2015 at 21:32, Vladimir Olovyannikov
>> > wrote:
 Hello Ard, Eugene,
 Thank you for explanation.

 Ard, I tried the patch, but it cannot be applied to the latest (pulled a 
 minute ago, git-svn-id: 
 https://svn.code.sf.net/p/edk2/code/trunk/edk2@18732 
 6f19259b-4bc3-4df7-8a09-765794883524)
 tree: all 3 hunks failed. Which commit should I be based on to apply the 
 patch?

 Anyway I found the lines manually and changed them. However, when I try to

 source /uefi/ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py -f 
 (0x8500,0x0028) -m (0x8000,0x4000) -a
 I am getting

 ERROR(?): ValueError: need more than 1 value to unpack
   File " /uefi/ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py", line 94, 
 in >
 armplatform_debugger.load_all_symbols()
 ERROR(CMD656):
 # in /uefi/BroadcomPlatformPkg/NS2Pkg/Scripts/armpkg_syms.ds:2 while 
 executing: source /uefi/ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py -f 
 (0x8500,0x0028) -m (0x8000,0x4000) -a
 ! The script /uefi/ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py failed 
 to complete due to an error during execution of the script

 Replacing the script with the older version makes it work as before, but 
 again the debugger is useless.

 Any idea what is wrong?

>>
>>>No, not a clue. But could you please share the diff between the
>>>version of the script that works for you and the version that doesn't?
>>>I don't think the patch I sent could trigger the error you are seeing,
>>>so it must be something else that changed.
>> Here is the diff.
>>
>> --- firmware_volume_old.py  2015-06-24 15:15:37.0 -0700
>> +++ firmware_volume_new.py  2015-11-06 11:49:24.321730539 -0800
>> @@ -91,12 +91,7 @@
>>  return filename[0:string.find(filename,'\0')]
>>
>>  def get_debug_elfbase(self):
>> -stripped_size = struct.unpack("> self.ec.getMemoryService().read(self.base_te + 0x6, 2, 32))[0]
>> -stripped_size -= EfiSectionTE.SIZEOF_EFI_TE_IMAGE_HEADER
>> -
>> -base_of_code = self.ec.getMemoryService().readMemory32(self.base_te 
>> + 0xC)
>> -
>> -return self.base_te + base_of_code - stripped_size
>> +return self.base_pe32
>>
>
> This hunk is incorrect. Could you please try the patch instead?
> (attached, this time, so it should apply more easily)
>
> Thanks,
> Ard.
>
>>  class EfiSectionPE32:
>>  def __init__(self, ec, base_pe32):
>> @@ -131,17 +126,7 @@
>>  return filename[0:string.find(filename,'\0')]
>>
>>  def get_debug_elfbase(self):
>> -# Offset from dos hdr to PE file hdr
>> -pe_file_header = self.base_pe32 + 
>> self.ec.getMemoryService().readMemory32(self.base_pe32 + 0x3C)
>> -
>> -base_of_code = self.base_pe32 + 
>> self.ec.getMemoryService().readMemory32(pe_file_header + 0x28)
>> -base_of_data = self.base_pe32 + 
>> self.ec.getMemoryService().readMemory32(pe_file_header + 0x2C)
>> -
>> -if (base_of_code < base_of_data) and (base_of_code != 0):
>> -return base_of_code
>> -else:
>> -return base_of_data
>> -
>> +return self.base_pe32
>>  class EfiSectionPE64:
>>  def __init__(self, ec, base_pe64):
>>  self.ec = ec
>> @@ -176,16 +161,7 @@
>>  return filename[0:string.find(filename,'\0')]
>>
>>  def get_debug_elfbase(self):
>> -# Offset from dos hdr to PE file hdr
>> -pe_file_header = self.base_pe64 + 
>> self.ec.getMemoryService().readMemory32(self.base_pe64 + 0x3C)
>> -
>> -base_of_code = self.base_pe64 + 
>> self.ec.getMemoryService().readMemory32(pe_file_header + 0x28)
>> -base_of_data = self.base_pe64 + 
>> self.ec.getMemoryService().readMemory32(pe_file_header + 0x2C)
>> -
>> -if (base_of_code < base_of_data) and (base_of_code != 0):
>> -return base_of_code
>> -else:
>> -return base_of_data
>> +return self.base_pe64
>>

Re: [edk2] Strange behavior of the DS-5 debugger on AARCH64 with step-by-step debugging in uefi

[Vladimir Olovyannikov]

-Original Message-
From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org] 
Sent: Sunday, November 08, 2015 10:52 PM
To: Vladimir Olovyannikov
Cc: Cohen, Eugene; edk2-devel@lists.01.org
Subject: Re: [edk2] Strange behavior of the DS-5 debugger on AARCH64 with 
step-by-step debugging in uefi

On 6 November 2015 at 21:32, Vladimir Olovyannikov
> wrote:
>> Hello Ard, Eugene,
>> Thank you for explanation.
>>
>> Ard, I tried the patch, but it cannot be applied to the latest (pulled a 
>> minute ago, git-svn-id: https://svn.code.sf.net/p/edk2/code/trunk/edk2@18732 
>> 6f19259b-4bc3-4df7-8a09-765794883524)
>> tree: all 3 hunks failed. Which commit should I be based on to apply the 
>> patch?
>>
>> Anyway I found the lines manually and changed them. However, when I try to
>>
>> source /uefi/ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py -f 
>> (0x8500,0x0028) -m (0x8000,0x4000) -a
>> I am getting
>>
>> ERROR(?): ValueError: need more than 1 value to unpack
>>   File " /uefi/ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py", line 94, in 
>> >
>> armplatform_debugger.load_all_symbols()
>> ERROR(CMD656):
>> # in /uefi/BroadcomPlatformPkg/NS2Pkg/Scripts/armpkg_syms.ds:2 while 
>> executing: source /uefi/ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py -f 
>> (0x8500,0x0028) -m (0x8000,0x4000) -a
>> ! The script /uefi/ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py failed to 
>> complete due to an error during execution of the script
>>
>> Replacing the script with the older version makes it work as before, but 
>> again the debugger is useless.
>>
>> Any idea what is wrong?
>>

>No, not a clue. But could you please share the diff between the
>version of the script that works for you and the version that doesn't?
>I don't think the patch I sent could trigger the error you are seeing,
>so it must be something else that changed.
Here is the diff.

--- firmware_volume_old.py  2015-06-24 15:15:37.0 -0700
+++ firmware_volume_new.py  2015-11-06 11:49:24.321730539 -0800
@@ -91,12 +91,7 @@
 return filename[0:string.find(filename,'\0')]
 
 def get_debug_elfbase(self):
-stripped_size = struct.unpack("> Eugene, so you do not use the so convenient cmd_load_symbols.py script to 
>> load all symbols?
>> I dumped efi and dll as you suggested. The .text and .data sections in 
>> PE-COFF and ELF match 1:1 for me. Still the debugger is useless as it points 
>> to non-relevant code.
>>

>Indeed, this is expected. The PE/COFF and ELF versions of the binary
>should look entirely the same, it's only the DS-5 scripts that needs
>to have the hack removed to deal with the difference.
OK, I see.

Thank you,

Vladimir
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] Strange behavior of the DS-5 debugger on AARCH64 with step-by-step debugging in uefi

[Ard Biesheuvel]

On 9 November 2015 at 19:01, Vladimir Olovyannikov
 wrote:
> -Original Message-
> From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
> Sent: Sunday, November 08, 2015 10:52 PM
> To: Vladimir Olovyannikov
> Cc: Cohen, Eugene; edk2-devel@lists.01.org
> Subject: Re: [edk2] Strange behavior of the DS-5 debugger on AARCH64 with 
> step-by-step debugging in uefi
>
> On 6 November 2015 at 21:32, Vladimir Olovyannikov
> > wrote:
>>> Hello Ard, Eugene,
>>> Thank you for explanation.
>>>
>>> Ard, I tried the patch, but it cannot be applied to the latest (pulled a 
>>> minute ago, git-svn-id: 
>>> https://svn.code.sf.net/p/edk2/code/trunk/edk2@18732 
>>> 6f19259b-4bc3-4df7-8a09-765794883524)
>>> tree: all 3 hunks failed. Which commit should I be based on to apply the 
>>> patch?
>>>
>>> Anyway I found the lines manually and changed them. However, when I try to
>>>
>>> source /uefi/ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py -f 
>>> (0x8500,0x0028) -m (0x8000,0x4000) -a
>>> I am getting
>>>
>>> ERROR(?): ValueError: need more than 1 value to unpack
>>>   File " /uefi/ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py", line 94, in 
>>> >
>>> armplatform_debugger.load_all_symbols()
>>> ERROR(CMD656):
>>> # in /uefi/BroadcomPlatformPkg/NS2Pkg/Scripts/armpkg_syms.ds:2 while 
>>> executing: source /uefi/ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py -f 
>>> (0x8500,0x0028) -m (0x8000,0x4000) -a
>>> ! The script /uefi/ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py failed to 
>>> complete due to an error during execution of the script
>>>
>>> Replacing the script with the older version makes it work as before, but 
>>> again the debugger is useless.
>>>
>>> Any idea what is wrong?
>>>
>
>>No, not a clue. But could you please share the diff between the
>>version of the script that works for you and the version that doesn't?
>>I don't think the patch I sent could trigger the error you are seeing,
>>so it must be something else that changed.
> Here is the diff.
>
> --- firmware_volume_old.py  2015-06-24 15:15:37.0 -0700
> +++ firmware_volume_new.py  2015-11-06 11:49:24.321730539 -0800
> @@ -91,12 +91,7 @@
>  return filename[0:string.find(filename,'\0')]
>
>  def get_debug_elfbase(self):
> -stripped_size = struct.unpack(" self.ec.getMemoryService().read(self.base_te + 0x6, 2, 32))[0]
> -stripped_size -= EfiSectionTE.SIZEOF_EFI_TE_IMAGE_HEADER
> -
> -base_of_code = self.ec.getMemoryService().readMemory32(self.base_te 
> + 0xC)
> -
> -return self.base_te + base_of_code - stripped_size
> +return self.base_pe32
>

This hunk is incorrect. Could you please try the patch instead?
(attached, this time, so it should apply more easily)

Thanks,
Ard.

>  class EfiSectionPE32:
>  def __init__(self, ec, base_pe32):
> @@ -131,17 +126,7 @@
>  return filename[0:string.find(filename,'\0')]
>
>  def get_debug_elfbase(self):
> -# Offset from dos hdr to PE file hdr
> -pe_file_header = self.base_pe32 + 
> self.ec.getMemoryService().readMemory32(self.base_pe32 + 0x3C)
> -
> -base_of_code = self.base_pe32 + 
> self.ec.getMemoryService().readMemory32(pe_file_header + 0x28)
> -base_of_data = self.base_pe32 + 
> self.ec.getMemoryService().readMemory32(pe_file_header + 0x2C)
> -
> -if (base_of_code < base_of_data) and (base_of_code != 0):
> -return base_of_code
> -else:
> -return base_of_data
> -
> +return self.base_pe32
>  class EfiSectionPE64:
>  def __init__(self, ec, base_pe64):
>  self.ec = ec
> @@ -176,16 +161,7 @@
>  return filename[0:string.find(filename,'\0')]
>
>  def get_debug_elfbase(self):
> -# Offset from dos hdr to PE file hdr
> -pe_file_header = self.base_pe64 + 
> self.ec.getMemoryService().readMemory32(self.base_pe64 + 0x3C)
> -
> -base_of_code = self.base_pe64 + 
> self.ec.getMemoryService().readMemory32(pe_file_header + 0x28)
> -base_of_data = self.base_pe64 + 
> self.ec.getMemoryService().readMemory32(pe_file_header + 0x2C)
> -
> -if (base_of_code < base_of_data) and (base_of_code != 0):
> -return base_of_code
> -else:
> -return base_of_data
> +return self.base_pe64
>
>  class FirmwareFile:
>  EFI_FV_FILETYPE_RAW   = 0x01
>
>>> Eugene, so you do not use the so convenient cmd_load_symbols.py script to 
>>> load all symbols?
>>> I dumped efi and dll as you suggested. The .text and .data sections in 
>>> PE-COFF and ELF match 1:1 for me. Still the debugger is useless as it 
>>> points to non-relevant code.
>>>
>
>>Indeed, this is expected. The PE/COFF and ELF versions of the binary
>>should look entirely the same, it's only the DS-5 scripts that needs
>>to have the hack removed to deal with the difference.
> OK, I see.
>
> Thank you,
>
> Vladimir
From 

Re: [edk2] Strange behavior of the DS-5 debugger on AARCH64 with step-by-step debugging in uefi

[Ard Biesheuvel]

On 6 November 2015 at 21:32, Vladimir Olovyannikov
 wrote:
> Hello Ard, Eugene,
> Thank you for explanation.
>
> Ard, I tried the patch, but it cannot be applied to the latest (pulled a 
> minute ago, git-svn-id: https://svn.code.sf.net/p/edk2/code/trunk/edk2@18732 
> 6f19259b-4bc3-4df7-8a09-765794883524)
> tree: all 3 hunks failed. Which commit should I be based on to apply the 
> patch?
>
> Anyway I found the lines manually and changed them. However, when I try to
>
> source /uefi/ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py -f 
> (0x8500,0x0028) -m (0x8000,0x4000) -a
> I am getting
>
> ERROR(?): ValueError: need more than 1 value to unpack
>   File " /uefi/ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py", line 94, in 
> 
> armplatform_debugger.load_all_symbols()
> ERROR(CMD656):
> # in /uefi/BroadcomPlatformPkg/NS2Pkg/Scripts/armpkg_syms.ds:2 while 
> executing: source /uefi/ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py -f 
> (0x8500,0x0028) -m (0x8000,0x4000) -a
> ! The script /uefi/ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py failed to 
> complete due to an error during execution of the script
>
> Replacing the script with the older version makes it work as before, but 
> again the debugger is useless.
>
> Any idea what is wrong?
>

No, not a clue. But could you please share the diff between the
version of the script that works for you and the version that doesn't?
I don't think the patch I sent could trigger the error you are seeing,
so it must be something else that changed.

> Eugene, so you do not use the so convenient cmd_load_symbols.py script to 
> load all symbols?
> I dumped efi and dll as you suggested. The .text and .data sections in 
> PE-COFF and ELF match 1:1 for me. Still the debugger is useless as it points 
> to non-relevant code.
>

Indeed, this is expected. The PE/COFF and ELF versions of the binary
should look entirely the same, it's only the DS-5 scripts that needs
to have the hack removed to deal with the difference.

-- 
Ard.
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] Strange behavior of the DS-5 debugger on AARCH64 with step-by-step debugging in uefi

[Cohen, Eugene]

Vladimir,

Since the UEFI images are PE-COFF but DS-5 builds ELF there is a conversion 
process in the edk2 build.  This can result in a couple of issues with debug if 
not handled correctly:

1. The start of the image for debugger load purposes must be adjusted to 
reflect the PE/TE header in the image.  If this isn't done correctly the code 
and data will be shifted and the debugger is useless.

2. The relative position of .text and .data must be consistent across ELF and 
PE-COFF.  If this isn't done correctly you may be able to get correlated source 
code but all references to data (like dumping global variables) are messed up.

The way I go about debugging this is to dump the PE-COFF image (.efi) with 
dumpbin (from Visual Studio) and dump the ELF image (.dll) with fromelf (from 
RVCT) and compare the positions of the .text and .data sections.

Based on Ard's response I think you are hitting issue #1.

Eugene

-Original Message-
From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of Vladimir 
Olovyannikov
Sent: Thursday, November 05, 2015 3:52 PM
To: edk2-devel@lists.01.org
Subject: [edk2] Strange behavior of the DS-5 debugger on AARCH64 with 
step-by-step debugging in uefi

Hello All,

I faced a very strange behavior of the DS-5 debugger when I debug AARCH64  UEFI 
with the latest (well, the one which contains
DEFINE GCC_DLINK2_FLAGS_COMMON = 
--script=$(EDK_TOOLS_PATH)/Scripts/GccBase.lds
)
tools_def.template.
Whenever I step in the debugger it never matches the source where the execution 
point currently is.
It is impossible to step-by-step debug with this...
If I switch to the older tools_def.template, I don't have those issues, and can 
debug with no problem.
However, the ShellPkg cannot be built with older tools_def.template if I want 
to have ShellPkg (New Shell) instead of ShellBinPkg.

Please advise if I am doing anything wrong?

This is how I debug if step-by-step debug is needed:

I place a while(1) in a place of interest. Build UEFI, and then reboot the 
board and run the uefi; then when it reaches the point of while1, I connect 
DS-5 to the device and execute ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py 
-f (0x8500,0x0028) -m (0x8000,0x4000) -a

And expect to be at the (while1 line). With the latest tools_def.tempalte the 
source point is weird; with the previous ones - OK.
Any advice would be appreciated.

Thank you,
Vladimir

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] Strange behavior of the DS-5 debugger on AARCH64 with step-by-step debugging in uefi

[Vladimir Olovyannikov]

Hello Ard, Eugene,
Thank you for explanation.

Ard, I tried the patch, but it cannot be applied to the latest (pulled a minute 
ago, git-svn-id: https://svn.code.sf.net/p/edk2/code/trunk/edk2@18732 
6f19259b-4bc3-4df7-8a09-765794883524)
tree: all 3 hunks failed. Which commit should I be based on to apply the patch?

Anyway I found the lines manually and changed them. However, when I try to 

source /uefi/ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py -f 
(0x8500,0x0028) -m (0x8000,0x4000) -a
I am getting

ERROR(?): ValueError: need more than 1 value to unpack
  File " /uefi/ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py", line 94, in 

armplatform_debugger.load_all_symbols()
ERROR(CMD656): 
# in /uefi/BroadcomPlatformPkg/NS2Pkg/Scripts/armpkg_syms.ds:2 while executing: 
source /uefi/ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py -f 
(0x8500,0x0028) -m (0x8000,0x4000) -a
! The script /uefi/ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py failed to 
complete due to an error during execution of the script

Replacing the script with the older version makes it work as before, but again 
the debugger is useless.

Any idea what is wrong?

Eugene, so you do not use the so convenient cmd_load_symbols.py script to load 
all symbols? 
I dumped efi and dll as you suggested. The .text and .data sections in PE-COFF 
and ELF match 1:1 for me. Still the debugger is useless as it points to 
non-relevant code.

Please help me to get back on track with debugger...

Thank you,
Vladimir

-Original Message-
From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org] 
Sent: Friday, November 06, 2015 5:36 AM
To: Cohen, Eugene
Cc: Vladimir Olovyannikov; edk2-devel@lists.01.org
Subject: Re: [edk2] Strange behavior of the DS-5 debugger on AARCH64 with 
step-by-step debugging in uefi

On 6 November 2015 at 14:27, Cohen, Eugene  wrote:
> Vladimir,
>
> Since the UEFI images are PE-COFF but DS-5 builds ELF there is a conversion 
> process in the edk2 build.  This can result in a couple of issues with debug 
> if not handled correctly:
>
> 1. The start of the image for debugger load purposes must be adjusted to 
> reflect the PE/TE header in the image.  If this isn't done correctly the code 
> and data will be shifted and the debugger is useless.
>
> 2. The relative position of .text and .data must be consistent across ELF and 
> PE-COFF.  If this isn't done correctly you may be able to get correlated 
> source code but all references to data (like dumping global variables) are 
> messed up.
>
> The way I go about debugging this is to dump the PE-COFF image (.efi) with 
> dumpbin (from Visual Studio) and dump the ELF image (.dll) with fromelf (from 
> RVCT) and compare the positions of the .text and .data sections.
>
> Based on Ard's response I think you are hitting issue #1.
>

Indeed. I made some changes to the GCC linker scripts recently to
ensure that the PE/COFF and ELF layouts are identical in memory. The
primary reason for doing so was the recent introduction of a security
feature in UEFI that requires the PE/COFF sections to be page aligned,
making it more likely that you would hit #2 in Eugene's list above.

-- 
Ard.

> -Original Message-
> From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of 
> Vladimir Olovyannikov
> Sent: Thursday, November 05, 2015 3:52 PM
> To: edk2-devel@lists.01.org
> Subject: [edk2] Strange behavior of the DS-5 debugger on AARCH64 with 
> step-by-step debugging in uefi
>
> Hello All,
>
> I faced a very strange behavior of the DS-5 debugger when I debug AARCH64  
> UEFI with the latest (well, the one which contains
> DEFINE GCC_DLINK2_FLAGS_COMMON = 
> --script=$(EDK_TOOLS_PATH)/Scripts/GccBase.lds
> )
> tools_def.template.
> Whenever I step in the debugger it never matches the source where the 
> execution point currently is.
> It is impossible to step-by-step debug with this...
> If I switch to the older tools_def.template, I don't have those issues, and 
> can debug with no problem.
> However, the ShellPkg cannot be built with older tools_def.template if I want 
> to have ShellPkg (New Shell) instead of ShellBinPkg.
>
> Please advise if I am doing anything wrong?
>
> This is how I debug if step-by-step debug is needed:
>
> I place a while(1) in a place of interest. Build UEFI, and then reboot the 
> board and run the uefi; then when it reaches the point of while1, I connect 
> DS-5 to the device and execute ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py 
> -f (0x8500,0x0028) -m (0x8000,0x4000) -a
>
> And expect to be at the (while1 line). With the latest tools_def.tempalte the 
> source point is weird; with the previous ones - OK.
> Any advice would be appreciated.
>
> Thank you,
> Vladimir
>
> ___
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel
> ___
> edk2-devel 

Re: [edk2] Strange behavior of the DS-5 debugger on AARCH64 with step-by-step debugging in uefi

[Ard Biesheuvel]

On 6 November 2015 at 14:27, Cohen, Eugene  wrote:
> Vladimir,
>
> Since the UEFI images are PE-COFF but DS-5 builds ELF there is a conversion 
> process in the edk2 build.  This can result in a couple of issues with debug 
> if not handled correctly:
>
> 1. The start of the image for debugger load purposes must be adjusted to 
> reflect the PE/TE header in the image.  If this isn't done correctly the code 
> and data will be shifted and the debugger is useless.
>
> 2. The relative position of .text and .data must be consistent across ELF and 
> PE-COFF.  If this isn't done correctly you may be able to get correlated 
> source code but all references to data (like dumping global variables) are 
> messed up.
>
> The way I go about debugging this is to dump the PE-COFF image (.efi) with 
> dumpbin (from Visual Studio) and dump the ELF image (.dll) with fromelf (from 
> RVCT) and compare the positions of the .text and .data sections.
>
> Based on Ard's response I think you are hitting issue #1.
>

Indeed. I made some changes to the GCC linker scripts recently to
ensure that the PE/COFF and ELF layouts are identical in memory. The
primary reason for doing so was the recent introduction of a security
feature in UEFI that requires the PE/COFF sections to be page aligned,
making it more likely that you would hit #2 in Eugene's list above.

-- 
Ard.

> -Original Message-
> From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of 
> Vladimir Olovyannikov
> Sent: Thursday, November 05, 2015 3:52 PM
> To: edk2-devel@lists.01.org
> Subject: [edk2] Strange behavior of the DS-5 debugger on AARCH64 with 
> step-by-step debugging in uefi
>
> Hello All,
>
> I faced a very strange behavior of the DS-5 debugger when I debug AARCH64  
> UEFI with the latest (well, the one which contains
> DEFINE GCC_DLINK2_FLAGS_COMMON = 
> --script=$(EDK_TOOLS_PATH)/Scripts/GccBase.lds
> )
> tools_def.template.
> Whenever I step in the debugger it never matches the source where the 
> execution point currently is.
> It is impossible to step-by-step debug with this...
> If I switch to the older tools_def.template, I don't have those issues, and 
> can debug with no problem.
> However, the ShellPkg cannot be built with older tools_def.template if I want 
> to have ShellPkg (New Shell) instead of ShellBinPkg.
>
> Please advise if I am doing anything wrong?
>
> This is how I debug if step-by-step debug is needed:
>
> I place a while(1) in a place of interest. Build UEFI, and then reboot the 
> board and run the uefi; then when it reaches the point of while1, I connect 
> DS-5 to the device and execute ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py 
> -f (0x8500,0x0028) -m (0x8000,0x4000) -a
>
> And expect to be at the (while1 line). With the latest tools_def.tempalte the 
> source point is weird; with the previous ones - OK.
> Any advice would be appreciated.
>
> Thank you,
> Vladimir
>
> ___
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel
> ___
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

[edk2] Strange behavior of the DS-5 debugger on AARCH64 with step-by-step debugging in uefi

[Vladimir Olovyannikov]

Hello All,

I faced a very strange behavior of the DS-5 debugger when I debug AARCH64  UEFI 
with the latest
(well, the one which contains
DEFINE GCC_DLINK2_FLAGS_COMMON = 
--script=$(EDK_TOOLS_PATH)/Scripts/GccBase.lds
)
tools_def.template.
Whenever I step in the debugger it never matches the source where the execution 
point currently is.
It is impossible to step-by-step debug with this...
If I switch to the older tools_def.template, I don't have those issues, and can 
debug with no problem.
However, the ShellPkg cannot be built with older tools_def.template if I want 
to have ShellPkg (New Shell) instead of ShellBinPkg.

Please advise if I am doing anything wrong?

This is how I debug if step-by-step debug is needed:

I place a while(1) in a place of interest. Build UEFI, and then reboot the 
board and run the uefi; then when it reaches the point of while1,
I connect DS-5 to the device and execute
ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py -f (0x8500,0x0028) -m 
(0x8000,0x4000) -a

And expect to be at the (while1 line). With the latest tools_def.tempalte the 
source point is weird; with the previous ones - OK.
Any advice would be appreciated.

Thank you,
Vladimir

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] Strange behavior of the DS-5 debugger on AARCH64 with step-by-step debugging in uefi

[Ard Biesheuvel]

On 5 November 2015 at 23:52, Vladimir Olovyannikov
 wrote:
> Hello All,
>
> I faced a very strange behavior of the DS-5 debugger when I debug AARCH64  
> UEFI with the latest
> (well, the one which contains
> DEFINE GCC_DLINK2_FLAGS_COMMON = 
> --script=$(EDK_TOOLS_PATH)/Scripts/GccBase.lds
> )
> tools_def.template.
> Whenever I step in the debugger it never matches the source where the 
> execution point currently is.
> It is impossible to step-by-step debug with this...
> If I switch to the older tools_def.template, I don't have those issues, and 
> can debug with no problem.
> However, the ShellPkg cannot be built with older tools_def.template if I want 
> to have ShellPkg (New Shell) instead of ShellBinPkg.
>
> Please advise if I am doing anything wrong?
>
> This is how I debug if step-by-step debug is needed:
>
> I place a while(1) in a place of interest. Build UEFI, and then reboot the 
> board and run the uefi; then when it reaches the point of while1,
> I connect DS-5 to the device and execute
> ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py -f (0x8500,0x0028) -m 
> (0x8000,0x4000) -a
>
> And expect to be at the (while1 line). With the latest tools_def.tempalte the 
> source point is weird; with the previous ones - OK.
> Any advice would be appreciated.
>

Hello Vladimir,

This is likely caused by the fact that the ARM ELF images now have a
hole before the first section to line up with the PE/COFF version of
the binary.
Can you try the patch I am sending out separately?

Thanks,
Ard.
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel