Re: netbsd32_{,u}int64 in sys/types.h for compat/sys/siginfo.h

[Kamil Rytarowski]

On 18.11.2019 05:38, Rin Okuyama wrote:
> Thank you for your comment!
> 
> On 2019/11/17 22:42, Kamil Rytarowski wrote:
>> Please check it also with picotrace/truss:
>>
>> http://pkgsrc.se/devel/picotrace
> 
> netbsd32_signal.c needed to catch up with kern_sig.c so that syscall
> information is provided with SIGTRAP TRAP_SCE/TRAP_SCX. I committed
> the fix and picotrace/truss works fine now on COMPAT_NETBSD32.
> 

Thanks! I have submitted a mail how to further improve it.

> On 2019/11/17 22:42, Kamil Rytarowski wrote:
>> On 17.11.2019 04:34, Rin Okuyama wrote:
>>> Hi,
>>>
>>> In order to distangle circular dependency between
>>>  v.s. ,
>>> I propose
>>>
>>> (1) Move NETBSD32_INT64_ALIGN from  to
>>> 
>>>
>>> (2) Move netbsd32_{,u}int64 from  to
>>> 
>>>
>>> See attached patch for example on amd64.
>>>
>>
>> What do you think about duplicating the defines of netbsd32_uint64
>> inside sys/compat/sys/siginfo.h + adding a comment about keeping it in
>> sync with netbsd32.h?
>>
>> I think that avoiding spaghetti dependencies is a benefit.
>>
>> We already duplicated there _ptrace_state, removing circular
>> dependencies between sys/ptrace.h and sys/siginfo.h.
> 
> I don't think this is a good idea in this case. If we want to have
> duplicate define of netbsd32_uint64, and to avoid an "#ifdef __x86_64__"
> mess in , we need to move NETBSD32_INT64_ALIGN to
>  other than . If so,
> why not ?
> 

No need to move anything out of machine/netbsd32_machdep.h. It's
sufficient to define netbsd32_int64 with a custom non-conflicting name
or protect it with #ifdef before typedefing/defining twice.

> Also, in my proposal, spaghetti dependencies are avoided in the end;
> everyone depends on , and not on each other.
> 

However I have no strong opinions here. I would personally avoid
compat32 definitions in sys/types.h.

Compat code tends to need hacks, so it is sensible imho to restrict them
to compat headers (I am aware that it's not always followed).

> Thanks,
> rin




signature.asc
Description: OpenPGP digital signature

Re: adding linux syscall fallocate

[HRISHIKESH GOYAL]

Hello,

As per the stackoverflow answer (https://stackoverflow.com/a/11779492),
`truncate` doesn't write anything to file but it pretends(block allocation
doesn't happen) to write nulls (/dev/zero) in case if file is extended. As
oppose to this, `fallocate` actually allocates the space for file.

To understand the depth of the fallocate() ffs project, I tried following
on Linux:

*Initial disk:*
$ df -h
Filesystem  Size  Used Avail Use% Mounted on
/dev/sda1   9.8G 1004M  8.3G  11% /

*fallocate:*
$ touch bar
$ fallocate -l 5G bar

$ ls -lh
-rw-r--r-- 1 hrishikesh.c Tech 5.0G Nov 17 22:45 bar

$ df -h
Filesystem  Size  Used Avail Use% Mounted on
/dev/sda1   9.8G  6.0G  3.3G  65% /

I see that 5G of space is allocated to file bar(as Avail space is reduced
from 8.3G to 3.3G).

*truncate:*
$ touch baz
$ truncate -s 3G baz

$ ls -lh
total 5.1G
-rw-r--r-- 1 hrishikesh.c Tech 5.0G Nov 17 22:45 bar
-rw-r--r-- 1 hrishikesh.c Tech 3.0G Nov 17 22:49 baz

$ df -h
Filesystem  Size  Used Avail Use% Mounted on
/dev/sda1   9.8G  6.0G  3.3G  65% /

But in case of `truncate` disk available space is not reduced.

*Questions:*
1. As what I follow from the above stackoverflow answer and truncate man
page, even though `truncate` doesn't allocate space for file baz but
filesystem should still update the free space by reducing it to
0.3G(otherwise filesystem metadata are not consistent with file metadata).
Could anyone please correct me?

2. Does it mean that `truncate` only updates file vnode (i.e. size)
attribute and doesn't update super block (free_space) attribute?

3. I checked first 100 bytes in both above files using c lang fread()
function, all are filled with NULL character ( '\0' ), how file bar
(previously fallocate'ed file) got initialised with NULLs(as per my
understanding since they are uninitialised, they should be some random
bytes.. and not all nulls right?).

Could anyone please help me by answering above questions.
Thanks in advance.

Regards,
Hrishikesh










On Sun, Nov 17, 2019 at 2:00 AM Jaromír Doleček 
wrote:

> Le sam. 16 nov. 2019 à 17:14, HRISHIKESH GOYAL  a
> écrit :
>
>> Does *posix_fallocate()* implemented (has support) for any of the
>> underlying filesystems in NetBSD/FreeBSD?
>>
>
> As far as I know no filesystem actually implements the support in NetBSD.
>
>
>> Also, as per my understanding, all calls to *posix_fallocate()*
>> internally go to *fallocate()* systemcall which are redirected to 
>> *vop_fallocate_desc().
>> *So in the project
>> https://wiki.netbsd.org/projects/project/ffs-fallocate/ we only need to
>> implement *ffs_fallocate() *right?
>>
>
> Yes, pretty much.
>
> Jaromir
>

Proposal for a new project: Improving Process Scheduling in BSD using Machine Learning Techniques

[Manikishan Ghantasala]

Hello tech-kern,

I am Manikishan a GSoC 2019 student in NetBSD, and I have a proposal for a
project for NetBSD.


Recently, I and a peer of mine were trying to learn about process
schedulers and we came across a research paper “Applying Machine Learning
Techniques to Improve Process Scheduling"[LINK
]  which grabbed our
attention and we went through the paper and surfed a few more research
papers related to it.

Coming to the project idea, the idea is to improve process scheduling using
machine learning techniques that will help the scheduler to categorize the
processes more wisely and allocate the time_slices or priority accordingly
(which will depend on the scheduler),

The following will be the tentative workflow of the project :

   1. Study the scheduler and the way it handles different processes.
   2. Categorizing processes into different types that should be handled
   differently
   3. Finding out suitable program attributes that are most significant for
   the prediction and characterization of the processes
   4. Finding the best model  and tuning if necessary that helps to the
   characterization of the processes
   5. Creating the dataset and test benches to train and test the model
   during implementation.
   6. Implementing the model into the kernel.

The integrated model will replace the role of the scheduler as the
decision-maker and the new scheduler will be allocating resources according
to the data provided by the model.


So, at last, the idea of the project is to improve process scheduling in an
operating system. All the papers we have gone through dealt with LINUX
kernels having priority scheduler and new CFS scheduler.


During the process, we might come across several bugs in the existing
scheduler, which can be rectified improving the present scheduler in the
initial phase of the project itself.

It would take a great effort in coming up with our own approach, as we are
at our beginner levels of understanding of these advanced concepts, we want
to take help from several other developers too. I hope NetBSD will like
this project and this project will be a major contribution to the BSD.

We will be overwhelmed to get support and guidance for the project. The
idea is open to suggestions.

Regards,
Manikishan

Re: netbsd32_{,u}int64 in sys/types.h for compat/sys/siginfo.h

[Rin Okuyama]

Thank you for your comment!

On 2019/11/17 22:42, Kamil Rytarowski wrote:

Please check it also with picotrace/truss:

http://pkgsrc.se/devel/picotrace


netbsd32_signal.c needed to catch up with kern_sig.c so that syscall
information is provided with SIGTRAP TRAP_SCE/TRAP_SCX. I committed
the fix and picotrace/truss works fine now on COMPAT_NETBSD32.

On 2019/11/17 22:42, Kamil Rytarowski wrote:

On 17.11.2019 04:34, Rin Okuyama wrote:

Hi,

In order to distangle circular dependency between
 v.s. ,
I propose

(1) Move NETBSD32_INT64_ALIGN from  to


(2) Move netbsd32_{,u}int64 from  to


See attached patch for example on amd64.



What do you think about duplicating the defines of netbsd32_uint64
inside sys/compat/sys/siginfo.h + adding a comment about keeping it in
sync with netbsd32.h?

I think that avoiding spaghetti dependencies is a benefit.

We already duplicated there _ptrace_state, removing circular
dependencies between sys/ptrace.h and sys/siginfo.h.


I don't think this is a good idea in this case. If we want to have
duplicate define of netbsd32_uint64, and to avoid an "#ifdef __x86_64__"
mess in , we need to move NETBSD32_INT64_ALIGN to
 other than . If so,
why not ?

Also, in my proposal, spaghetti dependencies are avoided in the end;
everyone depends on , and not on each other.

Thanks,
rin

Re: __{read,write}_once

[Mindaugas Rasiukevicius]

David Holland  wrote:
>  > I see the potential source of confusion, but just think about: what
>  > could "atomic" possibly mean for loads or stores?  A load is just a
>  > load, there is no RMW cycle, conceptually; inter-locking the operation
>  > does not make sense.  For anybody who attempts to reason about this,
>  > it should not be too confusing, plus there are man pages.
> 
> ...that it's not torn.
> 
> As far as names... there are increasingly many slightly different
> types of atomic and semiatomic operations.
> 
> I think it would be helpful if someone came up with a comprehensive
> naming scheme for all of them (including ones we don't currently have
> that we're moderately likely to end up with later...)

Yes, the meaning of "atomic" has different flavours and describes different
set of properties in different fields (operating systems vs databases vs
distributed systems vs ...) and, as we can see, even within the fields.

Perhaps not ideal, but "atomic loads/stores" and "relaxed" are already the
dominant terms.  I think it is pointless to attempt to reinvent the wheel
here.  It is terminology used by C11 (and C++11) and accepted by various
technical literature and, at this point, by academia (if you look at the
recent papers on memory models -- it's pretty much settled).  These terms
are not too bad; it could be worse; and this bit is certainly not the worst
part of C11.  So, I would just move on.

-- 
Mindaugas

Re: netbsd32_{,u}int64 in sys/types.h for compat/sys/siginfo.h

[Kamil Rytarowski]

On 17.11.2019 04:34, Rin Okuyama wrote:
> Hi,
> 
> In order to distangle circular dependency between
>  v.s. ,
> I propose
> 
> (1) Move NETBSD32_INT64_ALIGN from  to
> 
> 
> (2) Move netbsd32_{,u}int64 from  to
> 
> 
> See attached patch for example on amd64.
> 

What do you think about duplicating the defines of netbsd32_uint64
inside sys/compat/sys/siginfo.h + adding a comment about keeping it in
sync with netbsd32.h?

I think that avoiding spaghetti dependencies is a benefit.

We already duplicated there _ptrace_state, removing circular
dependencies between sys/ptrace.h and sys/siginfo.h.

> Background is:
> 
> Now, gdb for i386 does not work again on amd64 (both on -current and
> netbsd-9) with:
> 
> ptrace: Invalid arguments.
> 
> This is because sizeof(struct netbsd32_ptrace_siginfo) is 128+4+4=136
> on amd64 whereas sizeof(struct ptrace_siginfo) is 128+4=132 on i386;
> netbsd32_ptrace_siginfo has uint64_t members via siginfo32_t via
> __ksiginfo32 since compat/sys/siginfo.h rev 1.5:
> 
> http://cvsweb.netbsd.org/bsdweb.cgi/src/sys/compat/sys/siginfo.h#rev1.5
> 
> As a result, netbsd32_ptrace_siginfo requires 4-byte tail padding on
> amd64. However, tail padding does not appear on i386, since 64-bit
> objects only need 4-byte alignment on i386.
> 
> Actually, gdb/i386 becomes sane with this hack:
> 

Please check it also with picotrace/truss:

http://pkgsrc.se/devel/picotrace


> 
> Index: sys/compat/sys/siginfo.h
> ===
> RCS file: /home/netbsd/src/sys/compat/sys/siginfo.h,v
> retrieving revision 1.8
> diff -p -u -r1.8 siginfo.h
> --- sys/compat/sys/siginfo.h    30 Sep 2019 21:13:33 -    1.8
> +++ sys/compat/sys/siginfo.h    12 Nov 2019 11:04:58 -
> @@ -34,6 +34,12 @@
>  
>  #ifdef _KERNEL
>  
> +typedef uint64_t _args_t
> +#ifdef __x86_64__
> +    __attribute__((__aligned__(4)))
> +#endif
> +    ;
> +
>  typedef union sigval32 {
>  int sival_int;
>  uint32_t sival_ptr;
> @@ -73,7 +79,7 @@ struct __ksiginfo32 {
>  int    _sysnum;
>  int    _retval[2];
>  int    _error;
> -    uint64_t _args[8]; /* SYS_MAXSYSARGS */
> +    _args_t    _args[8]; /* SYS_MAXSYSARGS */
>  } _syscall;
>  
>  struct {
> 
> 
> We provide netbsd32_uint64 for this purpose:
> 
> typedef uint64_t netbsd32_uint64 NETBSD32_INT64_ALIGN;
> 
> and NETBSD32_INT64_ALIGN is __attribute__((__aligned__(4))) on amd64.
> However, unfortunately, NETBSD32_INT64_ALIGN is defined in
> , and  requires
> .
> 
> Thoughts? Any comments or objections?
> 
> Thanks,
> rin




signature.asc
Description: OpenPGP digital signature