Re: Duplicate task_spawn()

2020-05-30 Thread Yang Chung Fan
>
> Any static should be conditioned on CONFIG_LIB_SYSCALL for the
> task_spawn() version in sched/task/task_spawn.c, however, that is not
> really necessary either because that version is not linked into the same
> binary as is the version in libs/libc/spawn.
>
> I suppose a user could enable CONFIG_LIB_SYSCALL in a FLAT build.  Then
> both would be linked into the same blob, but that is kind of a useless
> configuration.

Yes, this is what my current config has.

I am trying to port my cRTOS design to newer nuttx.

While running in FLAT build, I load Linux binaries in SystemV ABI and
hook the system calls redirect them to Nuttx APIs.

Of course, I can do this in a kernel build, but I want to keep things simple.

I agree that using system calls in a FLAT build is quite meaning less.

However, I think we can somehow keep this flexibility for exploiting
for binary compability?

-- 
Yang

2020年5月31日(日) 1:07 Gregory Nutt :
>
>
> > Any static should be conditioned on CONFIG_LIB_SYSCALL for the
> > task_spawn() version in sched/task/task_spawn.c, however, that is not
> > really necessary either because that version is not linked into the
> > same binary as is the version in libs/libc/spawn.
> >
> > I suppose a user could enable CONFIG_LIB_SYSCALL in a FLAT build.
> > Then both would be linked into the same blob, but that is kind of a
> > useless configuration.
> >
> So, I think the preferred fix would simply to make CONFIG_LIB_SYSCALL
> dependent on !CONFIG_BUILD_FLAT in the Kconfig file.
>
>


-- 
Yang Chung Fan (楊宗凡) (ヤン ゾン ファン)
Member of Softlab, Tsukuba University , Japan
Email: sonic.tw...@gmail.com


Re: Duplicate task_spawn()

2020-05-30 Thread Gregory Nutt



Any static should be conditioned on CONFIG_LIB_SYSCALL for the 
task_spawn() version in sched/task/task_spawn.c, however, that is not 
really necessary either because that version is not linked into the 
same binary as is the version in libs/libc/spawn.


I suppose a user could enable CONFIG_LIB_SYSCALL in a FLAT build.  
Then both would be linked into the same blob, but that is kind of a 
useless configuration.


So, I think the preferred fix would simply to make CONFIG_LIB_SYSCALL 
dependent on !CONFIG_BUILD_FLAT in the Kconfig file.





Re: Duplicate task_spawn()

2020-05-30 Thread Gregory Nutt

On 5/30/2020 9:22 AM, Gregory Nutt wrote:

On 5/30/2020 3:44 AM, Xiang Xiao wrote:
It seems that the version inside sched/task/task_spawn.c need become 
a static function.


I think that the version of task_spawn in lib/libc/spawn need to exist 
only if CONFIG_LIB_SYSCALL is selected.  In that case, the one in 
sched/task/task_spawn.c should be static (really doesn't matter in 
that case because they are build into separate blobs).


The version of task_spawn.c in libs/libc/spawn simply marshals the 
parameters into a structure and calls nx_task_spawn().  If 
CONFIG_LIB_SYSCALL is defined then nx_task_spawn() will un-marshal the 
data can call the really task spawn.  This nonsense is only necessary 
because task_spawn has 8 parameters and the maximum number of 
parameters in a system call is only 6.


Without syscalls:  Application should call directly in task_spawn() in 
sched/task/task_spawn.c


With syscalls:  Application should call the marshalling task_spawn() 
in libs/libc/spawn/lib_task_spawn.c -> That will call the 
autogenerated nx_task_spawn() proxy -> And generate a system call -> 
The system call will the unmarshalling nx_task_spawn() in 
sched/task/task_spawn.c -> Which will, finally, call the real 
task_spawn().


I will add that condition logic via a PR.

Greg


Update:

No changes are required in libs/libc/spawn because the conditional logic 
is already in place.  in Make.defs:


   ifneq ($(CONFIG_BUILD_KERNEL),y)
   CSRCS += lib_psa_getstacksize.c lib_psa_setstacksize.c
   ifeq ($(CONFIG_LIB_SYSCALL),y)
   CSRCS += lib_task_spawn.c
   endif
   endif

In libs/libc/spawn/task_spawn.c:

   #if defined(CONFIG_LIB_SYSCALL) && !defined(CONFIG_BUILD_KERNEL)

Similarly in sched/task/task_spawn.c:

#ifdef CONFIG_LIB_SYSCALL
int nx_task_spawn(FAR const struct spawn_syscall_parms_s *parms)
{
  ...
}
#endif

Any static should be conditioned on CONFIG_LIB_SYSCALL for the 
task_spawn() version in sched/task/task_spawn.c, however, that is not 
really necessary either because that version is not linked into the same 
binary as is the version in libs/libc/spawn.


I suppose a user could enable CONFIG_LIB_SYSCALL in a FLAT build.  Then 
both would be linked into the same blob, but that is kind of a useless 
configuration.


The side-effect of making task_spawn() static is that it then cannot be 
used within the OS.  But as far as I can tell, nothing in the OS itself 
currently uses task_spawn() so I think it is safe to make it 
conditionally static.  But that only protects from duplicate symbols in 
the useless case mentioned above.


Greg






Re: Duplicate task_spawn()

2020-05-30 Thread Gregory Nutt

On 5/30/2020 3:44 AM, Xiang Xiao wrote:

It seems that the version inside sched/task/task_spawn.c need become a static 
function.


I think that the version of task_spawn in lib/libc/spawn need to exist 
only if CONFIG_LIB_SYSCALL is selected.  In that case, the one in 
sched/task/task_spawn.c should be static (really doesn't matter in that 
case because they are build into separate blobs).


The version of task_spawn.c in libs/libc/spawn simply marshals the 
parameters into a structure and calls nx_task_spawn().  If 
CONFIG_LIB_SYSCALL is defined then nx_task_spawn() will un-marshal the 
data can call the really task spawn.  This nonsense is only necessary 
because task_spawn has 8 parameters and the maximum number of parameters 
in a system call is only 6.


Without syscalls:  Application should call directly in task_spawn() in 
sched/task/task_spawn.c


With syscalls:  Application should call the marshalling task_spawn() in 
libs/libc/spawn/lib_task_spawn.c -> That will call the autogenerated 
nx_task_spawn() proxy -> And generate a system call -> The system call 
will the unmarshalling nx_task_spawn() in sched/task/task_spawn.c -> 
Which will, finally, call the real task_spawn().


I will add that condition logic via a PR.

Greg




Re: MSYS2 build slow

2020-05-30 Thread Gregory Nutt



Without specifying -g CONFIG_HOST_LINUX is assumed.
This worked well in the past with MSYS2.
But I will try -g later today.


Not if CONFIG_HOST_WINDOWS=y in the defconfig file.  That is what is set 
in stm32f4discovery:nsh:


$ grep -r CONFIG_HOST 
boards/arm/stm32/stm32f4discovery/configs/nsh/defconfig

CONFIG_HOST_WINDOWS=y

I also double checked Xiang's recommendation using this patch:

   $ diff -u boards/arm/stm32/stm32f4discovery/scripts/Make.defs .
   --- boards/arm/stm32/stm32f4discovery/scripts/Make.defs 2020-05-26
   07:39:23.201472900 -0600
   +++ ./Make.defs 2020-05-30 08:34:55.802400700 -0600
   @@ -39,10 +39,11 @@

 LDSCRIPT = ld.script

   -ARCHINCLUDES += ${shell $(INCDIR) -s "$(CC)" $(TOPDIR)$(DELIM)include}
   +CINCPATH := ${shell $(INCDIR) -s "$(CC)" $(TOPDIR)$(DELIM)include}
   +CXXINCPATH := ${shell $(INCDIR) -s "$(CC)"
   $(TOPDIR)$(DELIM)include$(DELIM)cxx}

   -ARCHXXINCLUDES += ${shell $(INCDIR) -s "$(CC)"
   $(TOPDIR)$(DELIM)include}
   -ARCHXXINCLUDES += ${shell $(INCDIR) -s "$(CC)"
   $(TOPDIR)$(DELIM)include$(DELIM)cxx}
   +ARCHINCLUDES += $(CINCPATH)
   +ARCHXXINCLUDES += $(CINCPATH) $(CXXINCPATH)

 ifeq ($(CONFIG_CYGWIN_WINTOOL),y)
   ARCHSCRIPT = -T "${shell cygpath -w
   $(BOARD_DIR)$(DELIM)scripts$(DELIM)$(LDSCRIPT)}"

That also gives a smaller, but consistent improvement in build time:

BEFORE:

   time make
   real    3m35.242s
   user    0m26.620s
   sys 1m9.932s

   time make -j
   real    0m47.548s
   user    0m30.717s
   sys 1m45.736s

AFTER:

   time make
   real    3m14.522s
   user    0m21.474s
   sys 0m56.380s

   time make -j
   real    0m46.532s
   user    0m24.673s
   sys 1m24.048s

Again, that is using Cygwin on 32Gb Rizen 5 3600

Greg





RE: MSYS2 build slow

2020-05-30 Thread Schock, Johannes - NIVUS GmbH
Without specifying -g CONFIG_HOST_LINUX is assumed.
This worked well in the past with MSYS2.
But I will try -g later today.

Johannes

> -Original Message-
> From: Gregory Nutt [mailto:spudan...@gmail.com]
> > For MSYS2 on my i7-3770 it's (every tests with jobs enabled):
> > # make distclean
> > # tools/configure.sh stm32f4discovery/nsh
> You really should specific -g on the configure.sh command line.
> Otherwise, the system will assume you are building for Cygwin. There are
> only a few differences between Cygwin and MSYS2, but though you should
> know.


Re: MSYS2 build slow

2020-05-30 Thread Gregory Nutt




For MSYS2 on my i7-3770 it's (every tests with jobs enabled):
# make distclean
# tools/configure.sh stm32f4discovery/nsh
You really should specific -g on the configure.sh command line. 
Otherwise, the system will assume you are building for Cygwin. There are 
only a few differences between Cygwin and MSYS2, but though you should know.


RE: Duplicate task_spawn()

2020-05-30 Thread Xiang Xiao
It seems that the version inside sched/task/task_spawn.c need become a static 
function.

> -Original Message-
> From: Yang Chung Fan 
> Sent: Saturday, May 30, 2020 3:27 PM
> To: dev@nuttx.apache.org
> Subject: Duplicate task_spawn()
> 
> Hi,
> 
> Did anyone also noticed that when building with CONFIG_LIB_SYSCALL=y, the 
> linker is unhappy about the duplicated task_spawn()
> symbols?
> 
> One of them is in sched/ and other one is in libs/libc.
> 
> --
> Yang



Duplicate task_spawn()

2020-05-30 Thread Yang Chung Fan
Hi,

Did anyone also noticed that when building with CONFIG_LIB_SYSCALL=y,
the linker is unhappy about the duplicated task_spawn() symbols?

One of them is in sched/ and other one is in libs/libc.

-- 
Yang


RE: MSYS2 build slow

2020-05-30 Thread Schock, Johannes - NIVUS GmbH
For MSYS2 on my i7-3770 it's (every tests with jobs enabled):
# make distclean
# tools/configure.sh stm32f4discovery/nsh
# time make -j
real6m24,970s
user1m5,615s
sys 3m18,369s

# make clean -j
# time make -j
real4m26,780s
user0m58,376s
sys 2m54,523s

For comparison I checked out one commit before the "-I to INCDIR" change:
#git checkout 23668a4b9b3c15c7367ff21befb4a6f48d01ffb7
# make distclean
# tools/configure.sh stm32f4discovery/nsh
# time make -j
real3m52,370s
user0m40,846s
sys 2m1,092s

# make clean -j
# time make -j
real3m0,449s
user0m34,298s
sys 1m39,438s

What is interesting:
With the change to incdir.c portions of the build process are still slower than 
before the "-I to INCDIR" change, but other portions are faster 
(libdrivers,libboard).
It would be interesting if someone could verify my results (or am I the only 
MSYS2 user?).
Overall it's still slower than before, but it's back in the ballpark.

Thanks.

Johannes

> -Original Message-
> From: Gregory Nutt [mailto:spudan...@gmail.com]
> Sent: Saturday, May 30, 2020 4:25 AM
> To: dev@nuttx.apache.org
> Subject: Re: MSYS2 build slow
> 
> Johannes,
> 
> PR 1149 has been merged.  That means that the incdir.c program has now
> been incorporated into the build system.  It all checks out:  I have
> pretty thoroughly verified it on Cygwin and Abdelatif and Alan Carvalho
> de Assis have checked it out well on Linux.  The PR also passes all of
> the PR build checks which verifies that it works correctly with Linux
> and macOS.
> 
> The only untested platform is MSYS and there could very well be issues
> there.  I have MSYS2 installed on two systems.  I tried testing on my
> laptop tonight but there is something screwed up in the MSYS2
> installation.  I have both MSYS2 and Cygwin installed and I am thinking
> that when I build with Cygwin , it messes up the MSYS2 configuration
> (and vice versa).  They share the same repository files.  I am not sure
> what is going on but I was not able to verify on MSYS2.  I will try
> again on my desktop in the morning.
> 
> If you find problems, you can be pretty much assured that the problem is
> in tools/incdir.c and I will work with you to resolve any issues that
> you run across.  If you do have problems, try using 'make V=1'.  That
> will show us the arguments to and the output from incdir.exe.
> 
> Greg