Re: [PATCH v4 02/79] scripts/headers_compile_test.sh: compile test script for exported headers

2015-11-10 Thread Gabriel Laskar 
Hi Mikko,

On Thu, 15 Oct 2015 09:17:18 +0200
Alexander Stein  wrote:

> On Thursday 15 October 2015 07:55:40, Mikko Rapeli wrote:
> > Users of kernel header files would be happier if they did not
> > contain kernel specific parts and would contain #include statements
> > for all other header files that they depend on, and in general
> > would compile.
> > 
> > For each header file exported to userspace, this script creates
> > a simple .c file which just includes the header file. Then it
> > tries to compile it together with minimal header files from GCC
> > and libc, and reports results.  
> 
> Just an idea: Why not try to create a pre-compiled header (pch)
> instead. So you can get rid of creating temporary .c files for each
> header.

I am working on exactly the same thing, here is my suggestions for this:

* you can use -fsyntax-only in order to avoid file creation, it will be
  faster, and don't clutter the directories.
* For some headers, for example tegra_drm.h, it is expected to have
  include drm.h before, so you have to maintain a list of the headers
  that should be included before each headers.
* finally I don't think that including stdint.h/stddef.h is the right
  solution when a header is using types like size_t or uint32_t, fixing
  the headers to user __kernel_size_t and __u32 should be cleaner.

-- 
Gabriel Laskar
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH v4 02/79] scripts/headers_compile_test.sh: compile test script for exported headers

2015-11-10 Thread Gabriel Laskar 
Hi Mikko,

On Thu, 15 Oct 2015 09:17:18 +0200
Alexander Stein  wrote:

> On Thursday 15 October 2015 07:55:40, Mikko Rapeli wrote:
> > Users of kernel header files would be happier if they did not
> > contain kernel specific parts and would contain #include statements
> > for all other header files that they depend on, and in general
> > would compile.
> > 
> > For each header file exported to userspace, this script creates
> > a simple .c file which just includes the header file. Then it
> > tries to compile it together with minimal header files from GCC
> > and libc, and reports results.  
> 
> Just an idea: Why not try to create a pre-compiled header (pch)
> instead. So you can get rid of creating temporary .c files for each
> header.

I am working on exactly the same thing, here is my suggestions for this:

* you can use -fsyntax-only in order to avoid file creation, it will be
  faster, and don't clutter the directories.
* For some headers, for example tegra_drm.h, it is expected to have
  include drm.h before, so you have to maintain a list of the headers
  that should be included before each headers.
* finally I don't think that including stdint.h/stddef.h is the right
  solution when a header is using types like size_t or uint32_t, fixing
  the headers to user __kernel_size_t and __u32 should be cleaner.

-- 
Gabriel Laskar
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH v4 02/79] scripts/headers_compile_test.sh: compile test script for exported headers

2015-10-15 Thread Mikko Rapeli 
On Thu, Oct 15, 2015 at 09:17:18AM +0200, Alexander Stein wrote:
> Hi Mikko,
> 
> On Thursday 15 October 2015 07:55:40, Mikko Rapeli wrote:
> > Users of kernel header files would be happier if they did not contain
> > kernel specific parts and would contain #include statements for all
> > other header files that they depend on, and in general would compile.
> > 
> > For each header file exported to userspace, this script creates
> > a simple .c file which just includes the header file. Then it
> > tries to compile it together with minimal header files from GCC
> > and libc, and reports results.
> 
> Just an idea: Why not try to create a pre-compiled header (pch) instead. So 
> you can get rid of creating temporary .c files for each header.

Good idea. I'll explore this too. Currently the scripts abuse the
headers_install target directory and leave cruft like these .c files and
the hacked up libc headers around. I was planning to clean these up when
actual headers are clean of compiler errors and when integrating
this compile check to 'make headers_check'.

-Mikko
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH v4 02/79] scripts/headers_compile_test.sh: compile test script for exported headers

2015-10-15 Thread Alexander Stein 
Hi Mikko,

On Thursday 15 October 2015 07:55:40, Mikko Rapeli wrote:
> Users of kernel header files would be happier if they did not contain
> kernel specific parts and would contain #include statements for all
> other header files that they depend on, and in general would compile.
> 
> For each header file exported to userspace, this script creates
> a simple .c file which just includes the header file. Then it
> tries to compile it together with minimal header files from GCC
> and libc, and reports results.

Just an idea: Why not try to create a pre-compiled header (pch) instead. So you 
can get rid of creating temporary .c files for each header.

Best regards,
Alexander

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH v4 02/79] scripts/headers_compile_test.sh: compile test script for exported headers

2015-10-15 Thread Alexander Stein 
Hi Mikko,

On Thursday 15 October 2015 07:55:40, Mikko Rapeli wrote:
> Users of kernel header files would be happier if they did not contain
> kernel specific parts and would contain #include statements for all
> other header files that they depend on, and in general would compile.
> 
> For each header file exported to userspace, this script creates
> a simple .c file which just includes the header file. Then it
> tries to compile it together with minimal header files from GCC
> and libc, and reports results.

Just an idea: Why not try to create a pre-compiled header (pch) instead. So you 
can get rid of creating temporary .c files for each header.

Best regards,
Alexander

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH v4 02/79] scripts/headers_compile_test.sh: compile test script for exported headers

2015-10-15 Thread Mikko Rapeli 
On Thu, Oct 15, 2015 at 09:17:18AM +0200, Alexander Stein wrote:
> Hi Mikko,
> 
> On Thursday 15 October 2015 07:55:40, Mikko Rapeli wrote:
> > Users of kernel header files would be happier if they did not contain
> > kernel specific parts and would contain #include statements for all
> > other header files that they depend on, and in general would compile.
> > 
> > For each header file exported to userspace, this script creates
> > a simple .c file which just includes the header file. Then it
> > tries to compile it together with minimal header files from GCC
> > and libc, and reports results.
> 
> Just an idea: Why not try to create a pre-compiled header (pch) instead. So 
> you can get rid of creating temporary .c files for each header.

Good idea. I'll explore this too. Currently the scripts abuse the
headers_install target directory and leave cruft like these .c files and
the hacked up libc headers around. I was planning to clean these up when
actual headers are clean of compiler errors and when integrating
this compile check to 'make headers_check'.

-Mikko
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[PATCH v4 02/79] scripts/headers_compile_test.sh: compile test script for exported headers

2015-10-14 Thread Mikko Rapeli 
Users of kernel header files would be happier if they did not contain
kernel specific parts and would contain #include statements for all
other header files that they depend on, and in general would compile.

For each header file exported to userspace, this script creates
a simple .c file which just includes the header file. Then it
tries to compile it together with minimal header files from GCC
and libc, and reports results.

Default libc and GCC header file locations are parsed from compiler
configuration.

Kernel headers depend on GCC headers so their path is included in
the test compiler command line.

Some gcc and kernel headers depend on libc headers which are made available
by copying from the compiler default location to a temporary location and
removing possibly existing kernel headers from this directory. This is a bit
of a hack but seems to work in multiple environments.

Tested natively on:

Debian unstable, i586-linux-gnu and gcc 4.9.2
Raspbian Wheezy, arm-linux-gnueabihf and gcc 4.6.3
Ubuntu 12.04 LTS, x86_64-linux-gnu and gcc 4.6.3

Tested cross compilation using standard CROSS_COMPILE=/path/to/gcc with:

arm-linux-gnueabihf-gcc (crosstool-NG linaro-1.13.1-4.9-2014.09 - Linaro GCC 
4.9-2014.09) 4.9.2 20140904 (prerelease)

Execute the script in the directory where kernel header files are installed.
For example:

$ make headers_install
$ cd usr/include
$ ../../scripts/headers_compile_test.sh

Example statistics from 3.18.4 kernel:

130 files failed the compile test.
609 files passed the compile test.

Example error types from 3.18.4 kernel:

$ ../../scripts/headers_compile_test.sh 2>&1 | \
grep error: | sed -e 's/.*error://g' | sort | uniq -c | sort -rn

352  unknown type name ‘uint32_t’
134  unknown type name ‘uint64_t’
103  unknown type name ‘size_t’
 43  unknown type name ‘__kernel_ulong_t’
 38  unknown type name ‘uint8_t’
 24  unknown type name ‘int32_t’
 22  field ‘addr’ has incomplete type
 18  field ‘tstamp’ has incomplete type
 16  unknown type name ‘__kernel_time_t’
 16  field ‘in’ has incomplete type
 16  field ‘in6’ has incomplete type
 14  unknown type name ‘__be16’
 13  ‘IFNAMSIZ’ undeclared here (not in a function)
 12  unknown type name ‘uint16_t’
  9  field ‘ifru_netmask’ has incomplete type
  9  field ‘ifru_hwaddr’ has incomplete type
  9  field ‘ifru_dstaddr’ has incomplete type
  9  field ‘ifru_broadaddr’ has incomplete type
  9  field ‘ifru_addr’ has incomplete type
  8  unknown type name ‘__kernel_pid_t’
  7  unknown type name ‘u_short’
  7  unknown type name ‘pid_t’
  6  invalid application of ‘sizeof’ to incomplete type ‘struct timespec’
  6  field ‘src’ has incomplete type
  6  field ‘audio_tstamp’ has incomplete type
  6  array type has incomplete element type
  5  unknown type name ‘__kernel_long_t’
  5  requested alignment is not an integer constant
  5  field ‘smsk’ has incomplete type
  4  unknown type name ‘__kernel_uid32_t’
  4  unknown type name ‘__kernel_gid32_t’
  4  ‘ETH_ALEN’ undeclared here (not in a function)
  3  unknown type name ‘int64_t’
  3  unknown type name ‘caddr_t’
  3  ‘IPSET_ERR_TYPE_SPECIFIC’ undeclared here (not in a function)
  3  field ‘trigger_tstamp’ has incomplete type
  3  field ‘src_addr’ has incomplete type
  3  field ‘sin_addr’ has incomplete type
  3  field ‘laddr’ has incomplete type
  3  field ‘id’ has incomplete type
  3  field ‘dmsk’ has incomplete type
  3  field ‘bssid’ has incomplete type
  3  expected specifier-qualifier-list before ‘uint64_t’
  2  unknown type name ‘u_long’
  2  unknown type name ‘stack_t’
  2  unknown type name ‘sigset_t’
  2  unknown type name ‘sa_family_t’
  2  unknown type name ‘__kernel_mode_t’
  2  unknown type name ‘__kernel_key_t’
  2  unknown type name ‘elf_gregset_t’
  2  unknown type name ‘bool’
  2  ‘uint64_t’ undeclared here (not in a function)
  2  ‘true’ undeclared (first use in this function)
  2  ‘NAME_MAX’ undeclared here (not in a function)
  2  ‘__kernel_mode_t’ undeclared here (not in a function)
  2  invalid application of ‘sizeof’ to incomplete type ‘struct sockaddr’
  2  field ‘uc_mcontext’ has incomplete type
  2  field ‘tmsk’ has incomplete type
  2  field ‘tgt’ has incomplete type
  2  field ‘shm_perm’ has incomplete type
  2  field ‘sem_perm’ has incomplete type
  2  field ‘raddr’ has incomplete type
  2  field ‘msg_perm’ has incomplete type
  2  field ‘grp’ has incomplete type
  2  field ‘dst’ has incomplete type
  2  field ‘dst_addr’ has incomplete type
  2  field ‘arp_pa’ has incomplete type
  2  field ‘arp_netmask’ has incomplete type
  2  field ‘arp_ha’ has incomplete type
  2  ‘false’ undeclared (first use in this function)
  1  xen/interface/xen.h: No such file or directory
  1

[PATCH v4 02/79] scripts/headers_compile_test.sh: compile test script for exported headers

2015-10-14 Thread Mikko Rapeli 
Users of kernel header files would be happier if they did not contain
kernel specific parts and would contain #include statements for all
other header files that they depend on, and in general would compile.

For each header file exported to userspace, this script creates
a simple .c file which just includes the header file. Then it
tries to compile it together with minimal header files from GCC
and libc, and reports results.

Default libc and GCC header file locations are parsed from compiler
configuration.

Kernel headers depend on GCC headers so their path is included in
the test compiler command line.

Some gcc and kernel headers depend on libc headers which are made available
by copying from the compiler default location to a temporary location and
removing possibly existing kernel headers from this directory. This is a bit
of a hack but seems to work in multiple environments.

Tested natively on:

Debian unstable, i586-linux-gnu and gcc 4.9.2
Raspbian Wheezy, arm-linux-gnueabihf and gcc 4.6.3
Ubuntu 12.04 LTS, x86_64-linux-gnu and gcc 4.6.3

Tested cross compilation using standard CROSS_COMPILE=/path/to/gcc with:

arm-linux-gnueabihf-gcc (crosstool-NG linaro-1.13.1-4.9-2014.09 - Linaro GCC 
4.9-2014.09) 4.9.2 20140904 (prerelease)

Execute the script in the directory where kernel header files are installed.
For example:

$ make headers_install
$ cd usr/include
$ ../../scripts/headers_compile_test.sh

Example statistics from 3.18.4 kernel:

130 files failed the compile test.
609 files passed the compile test.

Example error types from 3.18.4 kernel:

$ ../../scripts/headers_compile_test.sh 2>&1 | \
grep error: | sed -e 's/.*error://g' | sort | uniq -c | sort -rn

352  unknown type name ‘uint32_t’
134  unknown type name ‘uint64_t’
103  unknown type name ‘size_t’
 43  unknown type name ‘__kernel_ulong_t’
 38  unknown type name ‘uint8_t’
 24  unknown type name ‘int32_t’
 22  field ‘addr’ has incomplete type
 18  field ‘tstamp’ has incomplete type
 16  unknown type name ‘__kernel_time_t’
 16  field ‘in’ has incomplete type
 16  field ‘in6’ has incomplete type
 14  unknown type name ‘__be16’
 13  ‘IFNAMSIZ’ undeclared here (not in a function)
 12  unknown type name ‘uint16_t’
  9  field ‘ifru_netmask’ has incomplete type
  9  field ‘ifru_hwaddr’ has incomplete type
  9  field ‘ifru_dstaddr’ has incomplete type
  9  field ‘ifru_broadaddr’ has incomplete type
  9  field ‘ifru_addr’ has incomplete type
  8  unknown type name ‘__kernel_pid_t’
  7  unknown type name ‘u_short’
  7  unknown type name ‘pid_t’
  6  invalid application of ‘sizeof’ to incomplete type ‘struct timespec’
  6  field ‘src’ has incomplete type
  6  field ‘audio_tstamp’ has incomplete type
  6  array type has incomplete element type
  5  unknown type name ‘__kernel_long_t’
  5  requested alignment is not an integer constant
  5  field ‘smsk’ has incomplete type
  4  unknown type name ‘__kernel_uid32_t’
  4  unknown type name ‘__kernel_gid32_t’
  4  ‘ETH_ALEN’ undeclared here (not in a function)
  3  unknown type name ‘int64_t’
  3  unknown type name ‘caddr_t’
  3  ‘IPSET_ERR_TYPE_SPECIFIC’ undeclared here (not in a function)
  3  field ‘trigger_tstamp’ has incomplete type
  3  field ‘src_addr’ has incomplete type
  3  field ‘sin_addr’ has incomplete type
  3  field ‘laddr’ has incomplete type
  3  field ‘id’ has incomplete type
  3  field ‘dmsk’ has incomplete type
  3  field ‘bssid’ has incomplete type
  3  expected specifier-qualifier-list before ‘uint64_t’
  2  unknown type name ‘u_long’
  2  unknown type name ‘stack_t’
  2  unknown type name ‘sigset_t’
  2  unknown type name ‘sa_family_t’
  2  unknown type name ‘__kernel_mode_t’
  2  unknown type name ‘__kernel_key_t’
  2  unknown type name ‘elf_gregset_t’
  2  unknown type name ‘bool’
  2  ‘uint64_t’ undeclared here (not in a function)
  2  ‘true’ undeclared (first use in this function)
  2  ‘NAME_MAX’ undeclared here (not in a function)
  2  ‘__kernel_mode_t’ undeclared here (not in a function)
  2  invalid application of ‘sizeof’ to incomplete type ‘struct sockaddr’
  2  field ‘uc_mcontext’ has incomplete type
  2  field ‘tmsk’ has incomplete type
  2  field ‘tgt’ has incomplete type
  2  field ‘shm_perm’ has incomplete type
  2  field ‘sem_perm’ has incomplete type
  2  field ‘raddr’ has incomplete type
  2  field ‘msg_perm’ has incomplete type
  2  field ‘grp’ has incomplete type
  2  field ‘dst’ has incomplete type
  2  field ‘dst_addr’ has incomplete type
  2  field ‘arp_pa’ has incomplete type
  2  field ‘arp_netmask’ has incomplete type
  2  field ‘arp_ha’ has incomplete type
  2  ‘false’ undeclared (first use in this function)
  1  xen/interface/xen.h: No such file or directory
  1