Hi Jesse

On 09/08/17 06:45, Jesse Millwood wrote:
>
> Hello,
>
> I realize that the only officially supported SMP platform is the i.mx6
> but I did see some code on 6.0 to master for SMP and the zynq7000 so I
> am trying to test out some SMP functionality on my zc702 board.
>
> I am wondering if anyone has this working because my system is failing
> when compiling from master and the 6.0 and compatible tags.
>
>  
>

6.0.x release doesn't support SMP on Zynq, it's supposed to be in the
next release. However, you can still use master branches to get SMP/Zynq
work. Another option is to just use the master branch in the elfloader
since it fixes SMP related bugs and adds a reset functionality to reset
zynq's secondary core.

>From the config you provided, it seems like you enabled printing and
debug mode. Zynq/SMP has only been tested in release mode on
sel4test/zc706. This means it might not work if you enabled printing
and/or debug mode. Furthermore, we haven't tested it on camkes-based
projects yet.

>From your thorough output (thanks!), everything seems to be fine; core1
should be going to restore_user_context and proceed with the idleThread
after releasing the lock, and core0 should get it, and go to the root
thread.

I'd suggest you try disabling printing/debug mode and run on
sel4test/master (bamboo_zynq7000_smp_release_xml_defconfig), just to
make sure this works on your board.


Please let us know if you still have the same issue.

Cheers,
Hesham

> My second core seems to be coming up but the system ultimately fails
> and prints out:
>
> Bo
>
>   ot
>
> nKgE RaNlEl L fDinAiTsAh eAdBO, RdT!r
>
> ppFaeud lttio nugs eirn stspraucctei o
>
> n: 0xe001d1b0                       o
>
> FAR: 0xfffffff8 DFSR: 0x807
>
> halting...
>
> Kernel entry via Syscall, number: 1, Call
>
> Cap type: 1, Invocation tag: 37
>
> Which seems to be “Booting all finished, dropped to user space” from
> core0 and “KERNEL DATA ABORT!” from core1.
>
> The “0xe001d1b0” seems to be the label of the “idle_thread” function.
>
>  
>
> While stepping through via JTAG, I have verified that core1 gets
> through “init_kernel” and then enters “restore_user_context” at some
> point in “restore_user_context” the fault registers as shown in the
> printed output are set. I think it is either in the c_exit_hook in
> restore_user_context or after the program branches to “0xFFF0010”
> which is “ldr pc,0xFFFF0030”. This branches to the
> “arm_data_abort_exception” label, which goes to “kernel_data_fault”
> label and then to “kernel data abort”.
>
> I’m having trouble exactly pin pointing where the fault occurs but it
> seems to be close to there.
>
> Has anyone had similar issues with SMP?It seems to get fairly far
> without setting the fault registers.
>
>  
>
> I have tried to step through the execution over JTAG and here are some
> of my (verbose) notes
>
>  
>
>    | CORE0 Address | Core0 Function              | Core0
> Instruction      | CORE1 Address | Core1 Function
>                     | Core1 Instruction    |       DFSR |       DFAR |
> Note |
>
>   
> |---------------+-----------------------------+------------------------+---------------+------------------------------------+----------------------+------------+------------+------|
>
>    |    0x10000000 | label: start                | =cpsid
> aif=            |    0xFFFFFF34 |                                    |
> =mvn r0,#0x0f=       | 0x00000000 | 0x00005000 |      |
>
>    |    0x10003A2C | call: platform_init         | =bl
> -x10003DD8=        |    0xFFFFFF30
> |                                    | =wfe=                |
> 0x00000000 | 0x00005000 |      |
>
>    |    0x10003ACC | call: smp_boot              | =bl
> 0x100039FC=        |    0xFFFFFF34
> |                                    | =mvn r0,#0x0f=       |
> 0x00000000 | 0x00005000 |      |
>
>    |    0x10003ADO | ret: smp_boot               | =bl
> 0x10005C54=        |    0x10000020 | in:
> non_boot_core                  | =orr r0,r0,#0x40=    | 0x00000000 |
> 0x00005000 |    2 |
>
>    |    0x10003ADC | =if(is_hyp_mode())=         | =beq
> 0x10003AF0=       |    0x10002200 | label: arm_disable_dcaches        
> | =push {r14}=         | 0x00000000 | 0x00005000 |      |
>
>    |    0x10003AFC | call: arm_enable_mmu        | =bl
> 0x10002174=        |    0xE0006190 | in:
> try_init_kernel_secondary_core | =beq 0xE0001680=     | 0x00000000 |
> 0x00005000 |    1 |
>
>    |    0xE0001D70 | label: init_kernel          | =push
> {r11,r14}=       |    0xE0001680 | in: try_init_kernel_secondary_core
> | =beq 0xE0001680=     | 0x00000000 | 0x00005000 |    1 |
>
>    |    0xE0001814 | label: try_init_kernel      | =push
> {r11,r14}=       |    0xE0001690 | in: try_init_kernel_secondary_core
> | =beq 0xE0001680=     | 0x00000000 | 0x00005000 |    1 |
>
>    |    0xE0001B80 | call: create_initial_thread | =str r0,
> [r11,#-0x14]= |    0xE0001680 | in: try_init_kernel_secondary_core |
> =beq 0xE0001680=     | 0x00000000 | 0x00005000 |    1 |
>
>    |    0xE0001C48 | call: SMP_COND_STATEMENT    | =bl
> 0xE0003C20=        |    0xE0001680 | in:
> try_init_kernel_secondary_core | =beq 0xE0001680=     | 0x00000000 |
> 0x00005000 | 1, 3 |
>
>    |    0xE0001C4C | call: SMP_COND_STATEMENT    | =bl
> 0xE00017D8=        |    0xE0001680 | in:
> try_init_kernel_secondary_core | =beq 0xE0001680=     | 0x00000000 |
> 0x00005000 | 1, 4 |
>
>    |    0xE0001C50 | NODE_LOCK_SYS               | =bl
> 0xE0019280=        |    0xE0019288 | in:
> getCurrentCPUIndex             | =sub r13,r13,#0x8=   | 0x00000000 |
> 0x00005000 |    5 |
>
>    |    0xE0001D1C | call: arch_pause            | =bl
> 0xE0019DB0=        |    0xE0019290 | in:
> getCurrentCPUIndex             | =str r0,[r11,#-0x8]= | 0x00000000 |
> 0x00005000 |    6 |
>
>    |    0xE0001D40 | in: clh_lock_acquire        | =uxtb
> r3,r3=           |    0xE00037F4 | in: init_core_state               
> | =pop {r4,r11,pc}=    | 0x00000000 | 0x00005000 |      |
>
>    |    0xE0001D20 | in: clh_lock_acquire        | =mov
> r2,#0xE800=       |    0xE0003754 | in: init_core_state               
> | =movw r2,#0xE8E0=    | 0x00000000 | 0x00005000 |    7 |
>
>    |    0xE0001D40 | in: clh_lock_acquire        | =uxtb
> r3,r3=           |    0xE00017C4 | in: try_init_kernel_secondary     
> | =mov r3,#0x1=        | 0x00000000 | 0x00005000 |    8 |
>
>    |    0xE0001D40 | in: clh_lock_acquire        | =uxtb
> r3,r3=           |    0xE002A06C | in: schedule                      
> | =push {r11,r14}=     | 0x00000000 | 0x00005000 |    9 |
>
>    |    0xE0001D40 | in: clh_lock_acquire        | =uxtb
> r3,r3=           |    0xE002979C | in: activateThread                
> | =push {r11, r14}=    | 0x00000000 | 0x00005000 |   10 |
>
>    |    0xE0001D40 | in: clh_lock_acquire        | =uxtb
> r3,r3=           |    0xE001D24C | label: Arch_activateIdleThread    
> | =push {r11}=         | 0x00000000 | 0x00005000 |   11 |
>
>    |    0xE0001D38 | in: clh_lock_acquire        | =ldr
> r3,[r3,#0x4]=     |    0xE0000054 | in: start                         
> | =b 0xE001CEC8=       | 0x00000000 | 0x00005000 |   12 |
>
>   
>
>  
>
> Notes  
>
>     1. Core1 is in a =while (!node_boot_lock)= loop
>
>     2. In =smp_boot=, CORE1 changes after =init_cpus= (branch
> location: ZSR:10003A08)
>
>        - In =smp_boot=, =boot_cpus= is called
>
>        - This sets the =CPU_JUMP_PTR= =*((volatile
> uint32_t*)CPU_JUMP_PTR) = (uint32_t)entry;=
>
>        - calls =dsb= (data synchronization barrier)
>
>                - After this call, CPU1 goes to =FFFFFF2C: dsb sy=
>
>        - And then =sev=
>
>                - After this call, CPU1 goes to the =non_boot_core= label
>
>                - SEV
>
>                   - SEV causes an event to be signaled to all cores
> within a multiprocessor system. If SEV is implemented, WFE must also
> be implemented.
>
>                - WFE
>
>                   - If the Event Register is not set, WFE suspends
> execution until one of the following events occurs:
>
>                     - an IRQ interrupt, unless masked by the CPSR I-bit
>
>                     - an FIQ interrupt, unless masked by the CPSR F-bit
>
>                     - an Imprecise Data abort, unless masked by the
> CPSR A-bit
>
>                   - a Debug Entry request, if Debug is enabled
>
>                   - an Event signaled by another processor using the
> SEV instruction.
>
>        - If the Event Register is set, WFE clears it and returns
> immediately.
>
>        - If WFE is implemented, SEV must also be implemented.
>
>        - After CPU0 executes =arm_enable_mmu()= from the =main= function
>
>        - by the end of =smp_boot= core1 is just starting =non_boot_main=
>
>     3. The =SMP_COND_STATEMENT= is calling =clh_lock_init=
>
>     4. The =SMP_COND_STATEMENT= is calling =release_secondary_cpus=
>
>     5. right after Core0 returned from releasing secondary cpus
>
>        - First time Core1 has exited the loop
>
>        - Core1's stack is
>
>                - =getCurrentCPUINdex=
>
>                - =init_core_state=
>
>                - =try_init_kernel_secondary_core=
>
>                - =init_kernel=
>
>     6. Core0 is in a
> =while(big_kernel_lock.node_owners[cpu].next->value ! = CLHState_Granted)=
>
>        - Core0 is in a static inline function =clh_lock_acquire= in
> =try_init_kernel=
>
>        - Core1 is in =getCurrentCPUIndex= but being called from
> =tcbDebugAppend=
>
>                - =tcbDebugAppend= is being called from a =for= loop in
> =init_core_state=
>
>     7. Core0 is still in the previously mentioned while loop
>
>        - Core1 is in "init_core_state" and has exited the for loop
> that called =tcbDebugAppend(NODE_STATE_ON_CORE(ksIdleThread, i))=
>
>     8. Core0 is still in the previously mentioned while loop
>
>        - Core1 has returned to =try_init_kernel_secondary_core= from
> =init_core_state= and is at the end of the function
>
>     9. Core0 is still in the previously mentioned while loop
>
>        - Core1 has entered the =init_kernel= call and then the
> =schedule= function.
>
>     10. Core0 is still in the previously mentioned while loop
>
>                - Core1 has entered the =activateThread= call after
> =schedule= in =init_kernel=
>
>     11. Core0 is still in the previously mentioned while loop
>
>                - Core1 seems to have dropped into the =case
> ThreadState_IdleThreadState:= case when switching on =switch
> (thread_state_get_tsType(NODE_STATE(ksCurThread)->tcbState))=
>
>                  - This was in =activateThread=
>
>     12. Core0 is still in the previously mentioned while loop
>
>                - Core1 has exited =init_kernel= and is now branching
> to =restore_user_context=
>
>  
>
>  
>
> To test everything out I am using the  “camkes-sols-master” manifest
> and building the “CAmkES Hello World application with events and
> dataports”.
>
> The changes I made are
>
> ·        I edited the top level CAmkES file to set the affinity for
> two separate cores.
>
> ·        Upped the Max Number of CPU nodes to 2
>
> The rest of the config is pretty standard. I have it attached to this
> message.
>
> The FSBL and ps7_init script I use are the standard ones created for
> the zc702 from the 2017.2 version of the Xilinx XSDK.
>
> I am booting from jtag and first run the ps7_init script and then
> flash the fsbl and then the
> “capdl-loader-experimental-image-arm-zynq7000” that was built.
>
>  
>
> I am wondering if anyone is using a modified fsbl or ps7_init that
> does something else, if there is config value that I missed, or if it
> is still in development? If it is still in development I’d like to
> work with whoever is
>
>  
>
> Thanks,
>
> Jesse Millwood
>
>
>
> _______________________________________________
> Devel mailing list
> Devel@sel4.systems
> https://sel4.systems/lists/listinfo/devel

_______________________________________________
Devel mailing list
Devel@sel4.systems
https://sel4.systems/lists/listinfo/devel

Reply via email to