Re: Using binary attributes for configuration sysfs entries

[valdis . kletnieks]

On Wed, 13 Mar 2019 12:02:02 +0530, Jay Aurabind said:

> For the corresponding sysfs interface, since there are a lot of
> parameters, would it be justified to use the same binary format though
> sysfs_create_binary_file() ? The rationale is that it would be easier
> to simply pack all the config options in the struct and send it in
> once rather than individually write 40 files. This is what the
> attached patch follows. Interface is added only for reception
> parameters as of now.

Sysfs has a rule of "one value per file" - and saying "the one value is the
config struct we read/write to the file" is stretching that quite a bit.   By
the time you're doing all the marshaling of values in and out of this struct,
there's no real benefit to do it via sysfs rather than the existing ioctl()
call.

Make sure you double-check that "the same binary format" means what you
think it does when considering 32/64 bit architectures. While you're there, 
double
check that the ioctl() works correctly for a 32-bit userspace program running
on a 64-bit kernel - we have historically had an incredible number of API 
botches
for that case.

Of course, it helps your case considerably if you can point at something else
in sysfs and say "The Foobar 9934 driver already does this".. :)

On the other hand, having 40 files is just a massive race condition waiting to
happen, especially if the device has hardware weirdness that imply that certain
sets of (say) 3 or 5 parameters have to be changed in unison.


___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: [PATCH 1/1] staging: erofs: Add function comment for erofs/super.c

[valdis . kletnieks]

On Wed, 13 Mar 2019 08:48:26 +0800, Gao Xiang said:
(Reversing order of  comments)
> "Release slab Memory" is not a very useful description for this function.

Agreed.

> i_callback is used to free inode structure and other memory related to this 
> inode

However, that's not a bad description at all. :)

(Though I'd consider renaming the function to 'reap_inode_mem()' or something
like that so no description is needed. :)




___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: [PATCH 1/1] staging: erofs: Add function comment for erofs/super.c

[valdis . kletnieks]

On Tue, 12 Mar 2019 23:33:47 +0530, arshad hussain said:
> Re-sending. Previous patch I put in very old kernel-newbies address.
>
> Subject: [PATCH 1/1] staging: erofs: Add function comment for
>  erofs/super.c
>
> This patch adds functions comment for file erofs/super.c in
> sphinx format.
>
> Signed-off-by: Arshad Hussain 
> ---
>  drivers/staging/erofs/super.c | 133 
+-
>  1 file changed, 130 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/staging/erofs/super.c b/drivers/staging/erofs/super.c
> index 15c784fba879..6d2b93d94839 100644
> --- a/drivers/staging/erofs/super.c
> +++ b/drivers/staging/erofs/super.c
> @@ -30,6 +30,11 @@ static void init_once(void *ptr)
> inode_init_once(>vfs_inode);
>  }
>
> +/*

Actually, if you're trying to do sphinx/kerneldoc format, the first line of the
block comment should be '/**' with two asterisks.

Also, all kernel patches should be cc'd to linux-ker...@vger.kernel.org as well 
as
the maintainers and subsystem lists.


___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Finding all modules which consume kernel lib?

[valdis . kletnieks]

On Sat, 09 Mar 2019 13:04:44 +0300, Lev Olshvang said:

> There is also modulus.dep file which depmod builds. You can just grep to find
> which modulus depends.

Note that the modules.dep file only tells modprobe "If you're loading module A,
you need to load B first to get some symbols registered". If B is built-in to
the kernel, its EXPORT_SYMBOLS are already available, so it doesn't need to be
listed in modules.dep.

Also, it doesn't record build-time dependencies - it totally assumes that A and 
B
were built against the same source tree and that Kbuild took care of making sure
that any source code changes to B that affect A caused a rebuild of A to happen.

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: exit-code of 251

[valdis . kletnieks]

On Wed, 06 Mar 2019 17:20:07 +0530, prashantkumar dhotre said:

> Could you please let me know if any system call can exit with 251 ?
> I am trying to find out who exited with 251 ?

strace is your friend.

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: undefined reference to `ioctl_tty'

[valdis . kletnieks]

On Wed, 06 Mar 2019 03:50:10 -0500, Jeffrey Walton said:

> Yeah, the race seems to be the downside to ioctl and TIOEXCL.
>
> It is too bad Linux does not honor O_EXCL for the device, or provide a
> similar open flag like SOCK_NONBLOCK was added to avoid the race in
> sockets. It would avoid a lot of problems.

-ENOPATCH :)

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: undefined reference to `ioctl_tty'

[valdis . kletnieks]

On Wed, 06 Mar 2019 01:58:41 -0500, Jeffrey Walton said:

>TIOCEXCL  void
>   Put the terminal into exclusive mode.  No further open(2)
>   operations on the terminal are permitted.  (They fail with
>   EBUSY, except for a process with the CAP_SYS_ADMIN
>   capability.)

Note that this is still slightly racy - if two processes both try to do an open
and an ioctl, this can happen:

Process 1   open()
Process 2 open()
Process 1 ioctl(TIOEXCL)
Process 2 ioctl()

But at this point, the second process has already succeeded in opening the
device, so "no further opens" doesn't save you.

Usually, this sort of race would be really hard to hit, especially if the
ioctl() happens right after the open() - except that many systems will use cron
or similar to launch stuff "once per hour" or similar.  And they end up
batching together two processes that each try to do this. Suddenly, it's a lot
easier to hit that hole when cron launches two processes at 12:54:17 PM.

If there's one thing I've learned from almost 4 decades of sysadmin'ing on all
sorts of systems, saying "Oh that timing hole is so tiny it won't ever get hit
in production" guarantees that the Clock Gods will smite you for your hubris.

:)


___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Unimplemented system call

[valdis . kletnieks]

On Tue, 05 Mar 2019 19:52:13 -0500, Umair Khan said:

> I was just wondering why are these system calls not implemented?

Ah. So now we get to the question you *should* have asked. :)

Depends on the system call and the hardware and the kernel config.
Often, on embedded systems, you know from the start that your system
won't be needed certain optional features, so you can build the kernel
without those features to save space.

However, if some program tries to issue a syscall that's related to those
optional features, you have to return *some* sort of error. So 'not implemented'
it is

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Unimplemented system call

[valdis . kletnieks]

On Tue, 05 Mar 2019 19:08:20 -0500, Umair Khan said:

> I was just roaming around the linux source code when I stumbled upon
> this line - 
> https://github.com/torvalds/linux/blob/master/include/linux/syscalls.h#L1200
>
> May I know what is the use of this system call?

/*
 * Not a real system call, but a placeholder for syscalls which are
 * not implemented -- see kernel/sys_ni.c
 */
asmlinkage long sys_ni_syscall(void);

Well, did you go look at kernel/sys_ni.c like it told you to?  Because if you
bothered reading the first 2 dozen lines of that file, you would find:

/*
 * Non-implemented system calls get redirected here.
 */
asmlinkage long sys_ni_syscall(void)
{
return -ENOSYS;
}

Are there any remining questions?

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: How to make /dev/ttyACM0 (and friends) exclusive?

[valdis . kletnieks]

On Mon, 04 Mar 2019 16:38:44 +0100, Yann Droneaud said:

> O_EXCL is intended to be used to prevent opening an existing file. Said
> differently, it's used to ensure a new file is created, useful to
> prevent race condition, where multiple processes compete to create a
> file. For example think of temporary file created with random name.

Which means that if you're trying to regulate exclusive use of a character
device or similar, you use O_EXCL|O_CREAT to create a lock file, and make sure
that all the programs competing for the character device use the same pathname
to create the lock file - which has been a tried-and-true way of handling
exclusive locking for serial ports for 4 decades now.

The fact that 'strings /usr/sbin/ModemManager | grep /' proves pretty clearly
that ModemManager is trusting DBus to keep track of things rather than using
lock files on the modems is almost certainly part of the reason it's a steaming
pile of dingo's kidneys rather than a helpful piece of software.

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: How to make /dev/ttyACM0 (and friends) exclusive?

[valdis . kletnieks]

On Sat, 02 Mar 2019 14:36:12 -0500, Jeffrey Walton said:

> I feel like I am missing something... Does Linux consider the modem a
> shared resource instead of an exclusive resource? What use cases
> support two different programs sending commands to the modem at the
> same time?

The Linux kernel has exactly zero clue what a "modem" is.  It's talking to a
serial port, and doesn't care where the other end of the serial cable is. If
you have a onboard modem, that cable may be all of 2 mm long and consist of a
bunch of traces between two chips on a PCB, or even internal connections
between two sides of a chip, but it's still there.

So the correct question is "what use cases have two programs talking to the
same serial port"?

And the answer is:  A lot.  For a long time, there were these things called
"terminals", that the younger folk may not have encountered. And a very common
use case was to login via a terminal.  At that point, you usually had a login
shell like bash or similar running and often doing I/O to the terminal - and if
you ran any sub-processes, they also would do I/O to the terminal. So consider
the following bash one-liner:

% for i in `seq 1 10`; do echo "Loop number $i"; date; sleep 1; done

How and why does this work? (Hint 1:  'echo' is a bash builtin, Hint 2:
think about how a shell handles stdin/out for child processes)


___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: How to make /dev/ttyACM0 (and friends) exclusive?

[valdis . kletnieks]

On Fri, 01 Mar 2019 23:29:53 -0500, Jeffrey Walton said:

> I cannot seem to open the device in exclusive mode. The current open
> is (I also tried with O_EXCL):
>
> int modem = open(device_path, O_RDWR | O_NOCTTY | O_SYNC);

So what happens?  Does the open fail? If so, what does perror() say about it?
Or does the open work but the flag is ignored?

Have you tried making the *other* program open it with O_EXCL so your
caller-id program can't mess with its state?

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: SELinux, LSM and ima_policy rules

[valdis . kletnieks]

On Mon, 04 Feb 2019 11:38:19 +0300, Lev Olshvang said:
> I learned recently that IMA kernel security  subsystem can be integrated with 
> LSMs, such as SELinux, Smack, ...
> https://sourceforge.net/p/linux-ima/wiki/Home/
>
> https://www.kernel.org/doc/Documentation/ABI/testing/ima_policy
>
> It was present in kernel since v3.8 but not google does not know much about 
> the usability.

Note that although it's been in the tree since v3.8, the ability to stack LSMs
is much more recent.  That means that if you had IMA running, you couldn't have
SELinux or AppArmor active. Thus the lack of usability documentation.

You'll need a working and enabled TPM chipset in your system to use this. If
your BIOS has a 'secure boot' option, you have a TPM (though secure boot isn't
needed for IMA, but if you're deploying IMA, you may as well go the whole way
and do secure boot as well).

I'm not sure anybody has reliable overhead numbers, as it will be fairly system
specific.  Also, the sort of people who would run IMA are more concerned about
security than throughput.


___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Userspace configurable GPIO pinout

[valdis . kletnieks]

On Mon, 28 Jan 2019 13:26:55 +, Laurence Rochfort said:
> I'm writing a driver for an ePaper display, that will attach via GPIO.
>
> I've read the docs in Documentation/GPIO, and
> Documentation/devicetree/bindings/gpio.
>
> The ePaper itself just has a 34pin flat-flex cable, so people will have to
> use an adapter board to convert to the GPIO headers for a given system.

At that point, you're really stuck with "userspace gets to connect to 34 GPIO
pins and start bit-bashing".  I don't think there's any sane way for a single 
driver
to claim 34 GPIO pins, especially when *which* 34 pins get claimed will depend
on what their system is and how the adapter board is wired.

Having said that, the problem then becomes telling userspace how the 34
pins are wired.  About the only way to sort that out is if you're working for
the company that is building this, and you provide adapters for systems, and
a userspace installer program that says "Oh, this is a beaglebone/RPi/whatever,
so write a /etc/Epaper.conf file to match".

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: drivers: iio: Adding channels to the simple_dummy module

[valdis . kletnieks]

On Sat, 26 Jan 2019 12:12:07 +0530, Bharath Vedartham said:
> On Fri, Jan 25, 2019 at 11:24:33AM -0500, valdis.kletni...@vt.edu wrote:
> > On Fri, 25 Jan 2019 17:11:25 +0530, Bharath Vedartham said:
> > > VM has low RAM. Need to adjust :)
> > 
> > Upgrade the hypervisor system so the VM can be properly configured :)
> > 
> I think I ll rather build a native version. The reason I chose a VM was
> cause I was told that if I break the kernel it would be much safer. But
> I think GRUB will save my life if I screw things up. Thanks for your
> time!

Well, your /boot should be big enough at install time, and remember to keep
multiple versions around (mine is big enough to keep 3-4 known-good kernels and
up to 12-15 more during a git bisect).  And remember to take backups of your
system. (Seriously - the first time you do an 'rm *' in the wrong directory
will make up for the US$80 or so an external USB drive will cost you)

> > (As a side note, few people send their diffs from inside their test VM.
> > They usually build outside the VM and copy the needed pieces into
> > a shared area that the build environment and the VM can both see.)
> That is an interesting usecase. I wonder why they need to do that?

As I said, I know of very few (approximately zero) who send the diffs from
inside their test VM. The vast majority do the devel work outside and copy it.
About the only use case of sending from inside the VM I can think of is the
hypervisor being Windows HyperV. Though why somebody would be doing Linux
kernel devel in a Windows shop is beyond me. :)


___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: drivers: iio: Adding channels to the simple_dummy module

[valdis . kletnieks]

On Fri, 25 Jan 2019 17:11:25 +0530, Bharath Vedartham said:
> On Thu, Jan 24, 2019 at 04:21:32PM +0530, Himanshu Jha wrote:
> > Well, you can try other email clients such as Evolution, Claws ...
> > 
> > There is no rule saying to use Mutt exclusively.

> VM has low RAM. Need to adjust :)

Upgrade the hypervisor system so the VM can be properly configured :)

(As a side note, few people send their diffs from inside their test VM.
They usually build outside the VM and copy the needed pieces into
a shared area that the build environment and the VM can both see.)

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: drivers: iio: dummy: Unable to add channels to simple_dummy_channel

[valdis . kletnieks]

On Wed, 23 Jan 2019 23:25:59 +0530, Bharath Vedartham said:
> I am trying to add the 3-axis compass data channels to the
> simple_dummy_channel. I have mounted configfs and am able to load the
> modules correctly. Is this the right approach? printk is not printing
> anything to syslogs.

Do you have some printk's in your probing/open routines that show that you
in fact properly found and acquired the compass hardware? Remember that
the mere fact that a module will modprobe does not mean it's correct. Bonus
points if you have the stuff in place for udev or whatever to autoload your
driver, that's an additional indication you're on the right track.

If I wanted to, I could built the driver for a 622mbit/sec ATM connection, even
though I haven't seen one of those in a *long* time. They were only ever in
wide use in long-haul networking, as it was at the time the sort of thing that
major providers used for their core backbones, sold as an OC12, and evaporated
when gigabit interfaces became cheap enough for large corporations). I've seen
ATM cards for desktop PCs, but never for a laptop.

But if I built it, it would modprobe.

For the curious: (Prices were for the days when these things were still common)

T-1 - 1.544 megabits per second (24 DS0 lines) Ave. cost $250.-$500./mo.
T-3 - 43.232 megabits per second (28 T-1s) Ave. cost $4,000.-$16,000./mo.
OC-3 - 155 megabits per second (100 T-1s) Ave. cost $20,000.-$45,000./mo.
OC-12 - 622 megabits per second (4 OC3s) no estimated price available
OC-48 - 2.5 gigabits per seconds (4 OC12s) no estimated price available
OC-192 - 9.6 gigabits per second (4 OC48s)

(A DS0 was 64 kilobits per second, and the digitized form of 1 regular phone 
line. It's
no accident that in modem days, 56K was the highest download speed)

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Security-What can be done in kernel to disable forever executable memory modificaton

[valdis . kletnieks]

On Sat, 12 Jan 2019 16:19:00 +0300, Lev Olshvang said:

> The fact that the text segment could be modified is bad news from the
> security standpoint.

We've known that for at least a decade now. Maybe longer. And we
already had this discussion once, about a week ago.

> I am not sure whether it is actually happening, perhaps instead new pages are
> allocated, sort of COW (copy on write).

In which case, you should probably stop and verify if it's happening.

> And here I am getting to the point :

> Is there any way to disable the change of permission bits of PTE? Is it
> possible in the hardware (ARM) or should kernel be patched?

Are you sure you want to disable *all* changes of a PTE?
Hint: Figure out how shared libraries are loaded before you go any further.

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: mount /proc at boot as read-only

[valdis . kletnieks]

On Sun, 06 Jan 2019 21:13:26 +0300, Lev Olshvang said:

> I am trying to harden the embedded system.
> I have embedded system with systemd .

OK, you've already got a problem right there.

It's an embedded system.  Therefor, you know everything that should be running,
and what order it should start in.  If you don't already know that, you have 
bigger
design issues.

So you probably want to reduce system complexity and save both RAM and flash
memory space by heaving systemd over the side and using something simpler
(sysvinit, or upstart, or even use '/bin/make' if you want to guarantee that
certain tasks don't start till others have actually launched successfully, or
use a custom-written system launcher).

That's going to do more to reduce the attack surface than any amount of 
monkeying
around with the permissions in /proc will do.

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: How to forbid user space and kernel executable pages from becoming writable?

[valdis . kletnieks]

On Sat, 05 Jan 2019 18:30:01 +0300, Lev Olshvang said:

> Some articles, ex 
> https://shanetully.com/2013/12/writing-a-self-mutating-x86_64-c-program/
> state that mprotect() can change protection of executable section.

Note that appears to be a 5 year old article, and one that tries to be 
operating system
agnostic.

> As I understanf pte entry has page protection bits set to RO so  mprotect
> should change pte which is loaded to MMU/TLB. Why kernel can not refuse do
> perform this mprotect call(). Whu we do norhave kernel config options to 
> forbid
> user-space mutable code as security feature?

Hint: Read up on how shared libraries get loaded when a process starts up, and
also read up on dlopen() 

Think about how you get those to work if you can't have user-space mutable code
at all.

Are you ready to accept the performance hit of statically linked binaries?  Or 
the
maintenance nightmare of replacing every relevant binary whenever a bug is
found in a library (use glibc as an example)?

> From the other side,  when  run-time linker or elf_loader loads the
> executable it uses MAP_DENYWRITE which protect executable file from being
> overwritten. 

Now, That's a reasonably cheap protection that closes a fairly large attack 
surface.
It's a lot harder to exploit a buffer overrun or similar if it's difficult to 
find or create
a page that's both writable and executable.  Of course, there's always ways to 
change
a page mapping to allow it - the other half of the protection is that the vast 
majority
of exploits are severely limited in how many bytes can be injected.

> To add to the confusion, the following quote from the LWN articlle 
> https://lwn.net/Articles/422487/ about CONFIG_DEBUG_SET_MODULE_RONX 
> "Marking the kernel module pages as RO and/or NX is important not only because

> But I am not sure that some variant of pte_clear(), pte_mkexec() can not 
> disable it.

So how would modprobe and (possibly more importantly) rmmod work if it couldn't
be disabled?

> So let me cut to final qiestion:
>
> Suppose I want to cut off dynamic code instrumentation, like ftrace and 
> friends.
> Is it achievable at least at ARM architecture to enforce RO+X at hardware or 
> kernel? 

And here is where we get into a discussion of computer security, and this thing
called a "treat model".  Basically, it boils down to several questions:

1) What sensitive data are you trying to protect?
2) Who/what are you trying to protect it from?
3) Why do you think "I should prevent XYZ" is a reasonable protection?

If you're trying to harden a given feature, it's important to ask whether it's 
worth the
effort (and include "amount of inconvenience caused" in there).  

And an often overlooked part is "how easy is it to bypass?".  Say you manage to 
fix
dlopen() and the dynamic loader to totally prevent code outside dlopen() from 
finding
a writable executable page.  What stops an attacker from simply handing an 
existing
program a booby-trapped shared library via LD_LIBRARY_PATH?

Similarly, disabling ftrace in the kernel doesn't buy you much if an attacker 
can
modprobe code that re-enables it, or write a new kernel down into /boot and
waiting for (or causing) a reboot.

So now your security measure needs to include modifying the shared library 
loader
to ignore environment variables when loading into a setuid binary, and add 
support
for secure boot and crypto signing of kernel modules and other similar 
things

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: why are some stat.h "S_*" perm macros not exported via uapi?

[valdis . kletnieks]

On Mon, 31 Dec 2018 14:53:30 -0500, "Robert P. J. Day" said:

>   #define S_IRWXUGO   (S_IRWXU|S_IRWXG|S_IRWXO)
>   #define S_IALLUGO   (S_ISUID|S_ISGID|S_ISVTX|S_IRWXUGO)
>   #define S_IRUGO (S_IRUSR|S_IRGRP|S_IROTH)
>   #define S_IWUGO (S_IWUSR|S_IWGRP|S_IWOTH)
>   #define S_IXUGO (S_IXUSR|S_IXGRP|S_IXOTH)

Can you give an actual non-contrived example of code where one of these would
be preferable to inline coding the mask? And do you plan to include all the
various combos? And why should that be done in the uapi headers, rather than
glibc/musl/other_c_lib providing it?

I'll go out on a limb and say that most programmers would have an easier time
mentally parsing the octal constant 0660 than

((S_IRUGO | S_IWUGO) & ~(S_IXUSR | S_IXGRP | S_IRWXO ))

(Damn, that's ugly.  The sort of thing where I've been hacking C code since
1984, and I'm *still* looking at it and going "Did I get that right, or did I
just screw the pooch?"

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: In a process context kernel_read returns -EINTR, how to proceed?

[valdis . kletnieks]

On Sun, 23 Dec 2018 11:41:30 +0300, Lev Olshvang said:
> I use  security_mmap_file hook.

And what are you trying to accomplish in there?

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: In a process context kernel_read returns -EINTR, how to proceed?

[valdis . kletnieks]

On Fri, 21 Dec 2018 16:51:29 +0300, Lev Olshvang said:

> I need to read file  inside LSM hook and I can not do it in user space

Why? And which LSM hook are you trying to do this?

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: In a process context kernel_read returns -EINTR, how to proceed?

[valdis . kletnieks]

On Thu, 20 Dec 2018 13:33:09 +0300, Lev Olshvang said:
> I use kernel_read to read  file in chunks of 4K size in a process context
> On several files, like libc, libm,  I got -EINTR error.

What are you trying to accomplish?  This is not a recommended way to do things.

> I do not understand who may interrupt such a small read, I suppose these
> libraries are already mapped in other processes so no real I/O should be done.

That should be a hint

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Why existing filesystem encryption tools do not limit run-time file access while user/session might be used?

[valdis . kletnieks]

On Fri, 07 Dec 2018 23:13:45 +0300, Lev Olshvang said:
> Existing file encryption tools, like dm-crypt, fscrypt and eCryptfs provide
> only encryption of files only until file system is mounted.  (data at rest)
> The moment it became mounted, every user of computer can try to access the
> data.

There's this thing called "threat model" - what are you trying to protect, and
who are you protecting from.

Most filesystem encryption is designed to defend against a stolen device such
as a laptop, so that the thief cannot read the data.

> I do not understand why linux kernel key belonging  to only one user can not 
> be
> used at every read/write to decrypt data only for him?  

> Evidently I do not understand the reason why mount of user home directory
> during login (Ubunty's eCrypfs) do not  use  user's  session kernel key to
> allow only holder of this key to encrypt/decrypt files?

Note - a *session* kernel key doesn't work, because those change session to
session, and thus there's no guarantee that they'll function as a key to
decrypt files that last across multiple sessions.  For a thought experiment,
figure out how to have 2 sessions with different keys to both be able to
decrypt the same file.  A lot harder than it looks.  So you end up using a
reasonably permanent key for the files.

Now you're talking a different threat model - two users who distrust each other
on the same powered-on system.  However, this proposed crypto behavior doesn't
provide any *actual* additional security over and above the already existing
permission bits and ACLs.

Consider two users A and B, and a file /home/a/secretfile that's owned by A and
mode 600. B is an attacker, who can't read secretfile because of the
permissions.

Now let's say that B gets root.  Now they can bypass the permissions and read
secretfile. However, adding a permanent kernel key and crypto doesn't add
security, because unless you have a whole mess of other things in place (for
instance SELinux) preventing root from doing so, B-as-root can just 'sudo A
/bin/bash' and get access.

So the short answer of why we don't do it - because you have to do a lot of
*other* hardening to make it impossible to bypass it, and that adds to the
total cost (in sysadmin time and other factors).  So unless you have some
*really* valuable data to protect it's not worth the effort.

And usually, it's a heck of a lot cheaper to just buy a whole second server so
A and B are on different hardware which neutralizes things like Spectre attacks
as well



pgp7Y1KdmBCUi.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: When kernel deletes unused kernel module from RAM?

[valdis . kletnieks]

On Fri, 30 Nov 2018 20:35:54 +0300, Lev Olshvang said:

> I saw many times that kernel keeps kernel module with reference count of 0 in 
> a
> running system until explicit rmmod command is entered/

> Is there any way to require that unused module will be removed from kernel by
> the kernel itself ?

No, because that's a really good way to shoot yourself in the foot.

Hint 1:  What's the refcount the instant after modprobe finishes?  Even if it's 
being
auto-probed by modprobe because it's a pre-req module for the one that modprobe
has been asked to load, there's a race condition.

lsmod | grep dell
dell_laptop32768  0
dell_smbios20480  1 dell_laptop

So if smbios got loaded due to a 'modprobe dell_laptop', the refcount will
still be 0 for an instant, and it gets auto-rmmod'ed before dell_laptop can try
(and then fail) to load)

Hint 2:

What will break if that dell_laptop module gets auto-rmmod'ed?

The *correct* solution here is to have a userspace daemon that cleans up if you 
*really*
care about a module being unloaded.  Except for the beached-whale module from 
NVidia
that's 14M in size, most modules are only 8K to 200K or so in size - if you're 
really that
stressed for RAM, you're probably an embedded system and have *bigger* things 
to worry
about.

Now, there's *other* good reasons to want to remove a module - for instance,  
only have
certain security-related modules loaded when a security USB widget is plugged 
in, for
example, or time-of-day restrictions on when the feature(s) provided by a 
module are
available.  But that sort of thing is best done from userspace, not "when the 
refcount
reaches zero".

So the question becomes:  What actual problem are you trying to solve by doing 
an
auto-rmmod of a module? There's almost certainly a better solution


pgp4CTr3M2AV8.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Kernel default security configuration - how it affects LSM policy?

[valdis . kletnieks]

On Wed, 21 Nov 2018 17:20:50 +0300, Lev Olshvang said:

> So the questioned config option seems obsolete ?
> Wheher LSM always consulted last ?

If an LSM is configured/loaded, it is always consulted *after* applying
standard DAC file permission bits checks. (Discretionary Access Control- the
owner of the file/object is allowed to make their own decisions)

LSMs are always restrictive MAC (Mandatory access control - they are applied by
the system regardless of what the user/owner wants) calls. Restrictive means
they can only prohibit a call that has already passed the DAC check, they
cannot allow a call that would otherwise be failed by DAC.

LSMs are called after DAC checks for a number of reasons. One big one
is that when the LSM hooks were designed, the file permission checks were
(and still are) incredibly cheap - 3-4 opcodes or so.  So it makes sense to
do the cheap check first, as things like SELinux or AppArmor take a lot
more cycles to do the check. (There's also a few oddball corner cases where
doing the MAC check first results in non-intuitive results)





pgpjOHypjXIXL.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: [ARM64] Printing IRQ stack usage information

[valdis . kletnieks]

On Tue, 20 Nov 2018 18:21:33 +0530, Pintu Agarwal said:

> + sp = current_stack_pointer;
> + if (on_irq_stack(sp, cpu)) {
> + stack_start = (unsigned long)per_cpu(irq_stack, cpu);
> + last_usage = per_cpu(irq_stack_usage, cpu);
> + curr_usage = sp - stack_start;
> + pr_info("cpu:%d : sp: %lu, stack_start: %lu, usage: %lu\n", cpu, 
> sp, stack_start, (sp - stack_start));
> + if (curr_usage > last_usage)
> + per_cpu(irq_stack_usage, cpu) = curr_usage;
> + }

This code only works if called from the function that uses the deepest
amount of stack, because curr_usage depends on the current stack
pointer.  If some other routine went 2,934 bytes further into the stack
and then returned, you'll never know.

What you want instead is a way to see what the deepest point reached during
the current call, and compare that to the previous low.

How to detect the used stack? Go look at how the code currently
detects how deep the kernel stacks reached. (Hint 1: look at check_stack_usage()
in kernel/exit.c and stack_not_used().  Hint 2: what other support code
is implied by the implementation of stack_not used()?

> Which is the best place to invoke this ?
> I have the following option:
> 1. kernel/softirq.c => __do_softirq()
> 2. arch/arm64/kernel/smp.c => handle_IPI()
> 3. kernel/softirq.c => irq_exit()
> 4. ???

Pondering the above, what other implementations might work? Hint: if
you have a similar irq_stack_not_used() function, do you need any
additional code in *any* of those 4 places?

(And yes, I could just splat out the needed code - but you'd learn nowhere
near as much that way.. :)





pgpF7C47lQyPl.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: deferred pages

[valdis . kletnieks]

On Tue, 20 Nov 2018 10:16:41 -0500, Cindy-Sue Causey said:

> First reference is near the top under "AIX 4.3.2+ Deferred Paging
> #2) AIX 5L Differences Guide Version 5.3 Edition

I feel sorry for anybody still running either of these AIX versions (4.3.2
is late 1998, 5.3 is from 2005).

Word of warning - AIX is *highly* different from Linux - it's even diverged an
amazing amount internally from its SYSV Unix roots.  I wouldn't take any AIX
internals documentation as helpful for Linux, unless you also found the Bach
book on the SYSV kernel and Kirk McKusic's book on the BSD4.3 kernel useful in
understanding the Linux kernel

I've done AIX for a living from (roughly) the RT days, through AIX/370 (now
*that* was a beta test that generated a wealth of bar stories :), and AIX 3
through AIX 5 on RS6K/Power based systems.. A lot of the stuff they do doesn't
make any sense at all unless you know how IBM's mainframe systems do things
(And yes, I spent a number of years doing OS/VS1/HASP and VM (SP, HPO, and XA
variants)


pgpMMG1VZqjyG.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Why eeprom driver is read-only ?

[valdis . kletnieks]

On Tue, 20 Nov 2018 00:04:03 +0200, Ranran said:

> That's interesting...
> I think the name is confusing, because this chips are also writable.
>
> Not only this, but in arm the eeprom (at24) is writable!
> But in the x86 I am using, it is readonly in kernel code:
> https://android.googlesource.com/kernel/omap/+/glass-omap-xrr02/drivers/misc/eeprom/eeprom.c
> static struct bin_attribute eeprom_attr = {
> .attr = {
> .name = "eeprom",
> .mode = S_IRUGO,
> },
> .size = EEPROM_SIZE,
> .read = eeprom_read,
> };

Well, at least in the mainline kernel, we have this at the top of the eeprom.c 
file:

/* Addresses to scan */
static const unsigned short normal_i2c[] = { 0x50, 0x51, 0x52, 0x53, 0x54,
0x55, 0x56, 0x57, I2C_CLIENT_END };


/* Size of EEPROM in bytes */
#define EEPROM_SIZE 256

which is pretty obviously intended for "very small eeprom hanging off an I2C 
adapter",
and thus probably *NOT* the boot ROM that the BIOS lives in.


pgpuBwbSJesU6.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: [ARM64] Printing IRQ stack usage information

[valdis . kletnieks]

On Fri, 16 Nov 2018 23:13:48 +0530, Pintu Agarwal said:
> On Fri, Nov 16, 2018 at 10:16 PM  wrote:

> > Congrats. You just re-invented DEBUG_STACK_USAGE, which just keeps a 
> > high-water mark
> > for stack usage.
>
> So, you mean to say, my implementation is good enough to get the
> irq_stack usage, from the interrupt handler ?

No - your code doesn't keep a high-water mark (which should probably be
hooked into the IRQ exit code.

> But my concern is that if I dump it from irq handler, I will get
> information only for the current cpu.
> How do I store and get the information for all the cpu from the boot time ?

Make the high-water mark a per-cpu variable.

> From where do I call my dump_irq_stack_info() [some where during the
> entry/exit part of the irq handler], so that I could dump information
> for all the handler at boot time itself ?

No, you don't do a dump-stack during entry/exit.  You just maintain a high-water
value in the exit, and then you create a /proc/something or similar that when
read does a 'foreach CPU do print_high_water_irq'.

> Like I would to capture these information:
> - What was the name of the handler ?
> - Which cpu was executing it ?
> - How much irq stack (max value, same like high water mark) were used
> at that time ?

First, do the easy part and find out if you even *care* once you see actual
numbers.  If your IRQ stack is 8K but you never use more than 2500 bytes,
do you *really* care about the name of the handler anymore?

Also, see the code for /proc/interrupts to see how it keeps track of the
interrupts per CPU - maybe all you need to do is change each entry from
a 'count' to 'count, highwater'.


pgp_W8ryM0fu5.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: [ARM64] Printing IRQ stack usage information

[valdis . kletnieks]

On Fri, 16 Nov 2018 20:10:28 +0530, Pintu Agarwal said:

> > Look at the code controlled by '#ifdef CONFIG_DEBUG_STACK_USAGE'

> For example: is it possible to keep storing the irq_stack_usage (for
> each cpu in a variable) information from boot time, and then use this
> variable to dump the irq_stack information, after the system booted,
> may be from proc entry ?

Congrats. You just re-invented DEBUG_STACK_USAGE, which just keeps a high-water 
mark
for stack usage.


pgpWrHR9Ets3M.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: [ARM64] Printing IRQ stack usage information

[valdis . kletnieks]

On Fri, 16 Nov 2018 11:44:36 +0530, Pintu Agarwal said:

> > If your question is "Did one
> > of the CPUs blow out its IRQ stack (or come close to doing so)?" there's 
> > better
> > approaches.
> >
> Yes, exactly, this is what the main intention.
> If you have any better idea about this approach, please refer me.
> It will be of great help.

Look at the code controlled by '#ifdef CONFIG_DEBUG_STACK_USAGE'
which does the same thing for process stacks, or CONFIG_SCHED_STACK_END_CHECK
or the use of guard pages for detecting stack overrun

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: a question on mapping from PTE to swap entry

[valdis . kletnieks]

On Thu, 15 Nov 2018 23:53:56 +0800, "Larry" said:

> I'm curious when multiple process has the same PTE which points to the 
> samepage,
> how can kernel differenciate which page from swap space should be swappedin?

The PTE tells where in /dev/swap to find the page.  If it's a shared page, the
references all point at the same place, so "process 1's PAGE_A info" is the
same as "process 2's PAGE_A" info.  They may have different addresses in the
two processes, but they're both referencing the same swap page.

> Process 2 get scheduled, PAGE_A was from Process 1, so it should be swapped 
> out

Why? In general, there's no need to do so - and if it's shared, it's probably a 
hot page
so it won't be chosen for swapout.

(Remember - the kernel doesn't do swapin/swapout decisions in schedule(), it
only takes action once Process 2 starts running and references a page that 
requires
something be done.


pgpB0PhyBmYRp.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: [ARM64] Printing IRQ stack usage information

[valdis . kletnieks]

On Thu, 15 Nov 2018 18:52:39 +0530, Pintu Agarwal said:

> Currently, when I tested this (as a proc interface), I got the below output:
> CPUUNUSED-STACKACTUAL-STACK
>  0 16368 16384

> 3) How should I test it to get the different usage values for unused stack ?
> Can I get these values by implementing a sample interrupt handler,
> and printing information from there?

Hint 1:  If you're in a state where seq_printf() is legal, how many IRQ's are
on this processor's IRQ stack?

Hint 2:  What are the chances that some other CPU is currently in an IRQ?
(run 'top' and look for what percent of time that's happening)

Hint 3: what are the chances that the value of irq_stack_ptr is already stale
by the time seq_printf() finishes running?

Hint 4: what happens to the validity of your output if you get rescheduled
in the middle of that for_each loop?

(In other words, this code is terribly racy and is probably not going to answer
whatever debugging question you were working on..  If your question is "Did one
of the CPUs blow out its IRQ stack (or come close to doing so)?" there's better
approaches.




pgpgnND87yFdu.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: function stack frames in the kernel

[valdis . kletnieks]

On Mon, 12 Nov 2018 02:00:02 +0800, Carter Cheng said:
> Thanks for the reply but the link doesn't quite answer the question. I am
> wondering how the pointer is handled so that there is one per thread by the
> compiler. I perhaps was under the perhaps mistaken impression that the
> stack pointer frame pointer management inside the compiler makes certain
> assumptions in user space- but i am unsure how this applies to kernel space.

For regular function calls, a kernel stack works exactly the same as a function
stack in userspace (remember, it's the same compiler, and other tools like the
linker and gdb need to be able to understand function calls).

Where things are different are what happens if an interrupt or a call to
schedule() or enter/exit the kernel (or a few other similar situations I can't
remember at 2AM) causes a different thread to start running - for those cases,
there's architecture-dependent shim code (usually involving at least a bit of
assembler) to do the register swapping needed.



pgpn3gz1qohHY.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Creating compressed backing_store as swapfile

[valdis . kletnieks]

On Mon, 05 Nov 2018 11:28:49 -0500, "Austin S. Hemmelgarn" said:

> Also, it's probably worth noting that BTRFS doesn't need to decompress
> the entire file to read or write blocks in the middle, it splits the
> file into 128k blocks and compresses each of those independent of the
> others, so it can just decompress the 128k block that holds the actual
> block that's needed.

Presumably it does something sane with block allocation for the now-compressed
128K that's presumably much smaller.  Also, that limits the damage from writing 
to
the middle of a compression unit

That *does* however increase the memory requirement - you can OOM or
deadlock if your read/write from the swap needs an additional 128K for the
compression buffer at an inconvenient time...



pgpLJYMAk5PeA.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Creating compressed backing_store as swapfile

[valdis . kletnieks]

On Mon, 05 Nov 2018 11:07:12 -0500, "Austin S. Hemmelgarn" said:

> Performance isn't _too_ bad for the BTRFS case though (I've actually
> tested this before), just make sure you disable direct I/O mode on the
> loop device, otherwise you run the risk of data corruption.

Did you test that for random-access. or just sequential read/write?
(Also, see the note in my other mail regarding doing a random-access
write to the middle of the file...)



pgp9db3HLgH6J.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Creating compressed backing_store as swapfile

[valdis . kletnieks]

On Mon, 05 Nov 2018 20:31:46 +0530, Pintu Agarwal said:
> I wanted to have a swapfile (64MB to 256MB) on my system.
> But I wanted the data to be compressed and stored on the disk in my swapfile.
> [Similar to zram, but compressed data should be moved to disk, instead of 
> RAM].

What platform are you on that you're both storage constrained enough to need
swap, and also so short on disk space that compressing it makes sense?
Understanding the hardware constraints here would help in advising you.

> Note: I wanted to optimize RAM space, so performance is not important
> right now for our requirement.
>
> So, what are the options available, to perform this in 4.x kernel version.
> My Kernel: 4.9.x

Given that this is a greenfield development, why are you picking a kernel
that's 2 years out of date?  You *do* realize that 4.9.135 does *not* contain
all the bugfixes since then, only that relatively small subset that qualify for
'stable' (see Documentation/process/stable-kernel-rules.rst for the gory
details).

One possible total hack would be to simply use a file-based swap area,
but put the file on a filesystem that supports automatic inline compression.

Note that this will probably *totally* suck on performance, because there's
no good way to find where 4K block 11,493 starts inside the compressed
file, so it would have to read/decompress from the file beginning.  Also,
if you write data to a previously unused location (or even a previously used
spot that compressed the 4K page to a different length), you have a bad time
inserting it.  (Note that zram can avoid most of this because it can (a) keep
a table of pointers to where each page starts and (b) it isn't constrained to
writing to 4K blocks on disk, so if the current compression takes a 4K page down
to 1,283 bytes, it doesn't have to care *too* much if it stores that someplace
that crosses a page boundary.

Another thing that you will need to worry about is what happens in low-memory
situations - the time you *most* need to do a swap operation, you may not have
enough memory to do the I/O.  zram basically makes sure it *has* the memory
needed beforehand, and swap directly to pre-allocated disk doesn't need much
additional memory.


pgp9ikzGEsGfC.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Backporting stacked security patch

[valdis . kletnieks]

On Tue, 30 Oct 2018 21:19:08 +0300, Lev Olshvang said:
> As I stated, I have no option to come to customer and ask to upgrade his
> kernel, but I will try to put his attention to mentioned by you cease of
> security updates.

If you can't get him to upgrade the installed kernel, it's not worth your effort
to backport patches.  And just updating the patches to the 4.2 level doesn't
do your customer any favors, as there's been a lot of security patches
between 4.2 and 4.19.

And many of the important ones do *NOT* apply to the security/ branch of
the tree - it's stuff like the Spectre fixes that show up elsewhere in the tree,
networking fixes that resolve "packet of death" issues, and so on.



pgpxEMjEzrXa2.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: MMU related code

[valdis . kletnieks]

On Mon, 29 Oct 2018 00:08:12 +0800, Carter Cheng said:

> Where do I find the code in the kernel related to the MMU and resolving
> memory addresses? I am trying to understand what the implications are if
> code like this has bugs and the impact on the various functions that return
> chunks of memory for use via pointers (either as pages or kmalloc chunks)
> etc.

The results are easy enough to predict even without looking at the code.  If 
your
memory allocations are buggy, you get random memory overlays and corruption,
attempts to access non-mapped physical or virtual memory addresses, and so on.

Basically, all the same sorts of issues beginning C programmers encounter before
they understand pointers.


pgpi3KDBhJEKb.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Boot all yes config kernel with QEMU

[valdis . kletnieks]

On Tue, 23 Oct 2018 13:56:42 +1100, "Tobin C. Harding" said:

> I'd like to build and boot an allyesconfig kernel with QEMU.  Building
> is no problem but when I try to boot it I get problems because the host
> system does not support features requested by the VM.
>
> How does one go about testing an allyesconfig kernel?

As you have noticed, there's no guarantee that a allyesconfig kernel will boot 
in
a VM because the VM doesn't support something.  Note that you're going to have
the exact same issues booting on real hardware - you'll discover that there's 
configs
that won't boot on real hardware either - and even if it boots, you'll probably 
not have
the I/O devices to test even a third of the drivers unless you have a really 
well stocked
test lab.

But then, all(yes|mod|no)config aren't intended for actual booting and testing 
- they're
pretty much build testing all the options in one build.

Basically, one of two things happen after building an all-something-config:

1) The build completes and you get on with your life

2) The build fails, and you get to send an email telling the maintainer that
their driver for the Frobnozz 1300 Widget won't build in =y in kernel 4.21-rc2,
or whatever your build died on.


pgpGHG445qbUa.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Modify a file content from block layer

[valdis . kletnieks]

On Mon, 22 Oct 2018 22:32:14 +0530, jitendra kumar khasdev said:

> Working on a custom driver that tracks I/Os on block level. For some use
> case, I need to write metadata of tracked I/Os from driver in shutdown
> sequence (more likely in reboot handler). In reboot handler, performing I/O
> on disk is tricky (no userspace, no VFS. no sysfs).

Now take a step back.

What problem are you trying to solve by scribbling on blocks?  What's in the
metadata that (a) needs to be preserved and (b) can't be done by feeding it
to a userspace process?

Hint:  A number of dm targets need to track I/Os and manage to do so without
writing to a file system.

> Although, I am able to write one sector, by creating custom bio and giving
> manually sector to disk. Still, looking some correct API like
> *write_IO_on_disk(buffer, dev, offset, len)*

That's just *looking* for an fsck. :)


pgpSpIKL1iP08.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: changing parameters in build

[valdis . kletnieks]

On Wed, 17 Oct 2018 20:02:46 +0800, Carter Cheng said:

> The problem is I have to do something special with the clang options. I
> have to add an interprocedural link time optimization pass spitting out
> bitcode files and tying them together using llvm-link.

As I said back on Friday, this is work that's already been done:

> There's no LTO support in the stock 4.19 tree, but Andi Kleen did a patchset
> for 4.15, and there's another patchset to enable LTO when using Clang rather
> than gcc. (I haven't tried either one, don't use on a production machine, as
> the resulting kernel may crash, eat filesystems, and/or turn your dog 
> green...)

http://lmgtfy.com/?q=andi+kleen+linux+4.15+lto

What you're probably going to run into is that adding the options
isn't the hard part of the project.  The hard part will be fixing all
the places where LTO exposes issues in the code, such as this
(already-fixed) problem:

https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg1620485.html




pgpJ4Y1_TyfcJ.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Testing the performance impact of kernel modifications

[valdis . kletnieks]

On Tue, 16 Oct 2018 01:23:45 +0800, Carter Cheng said:
> I am actually looking at some changes that litter the kernel with short
> code snippets and thus according to papers i have read can result in CPU
> hits of around 48% when applied is userspace.

You're going to need to be more specific.  Note that 48% increase in a 
micro-benchmark
doesn't necessarily translate to a measurable performance change - for example, 
I have a
kernel build running right now with a cold file cache, and it's only using 6-8% 
of the CPU in
kernel mode (the rest being gcc in userspace and waiting for the spinning-oxide 
disk). If the
entire kernel slowed down by 50% that would only be 3-4% change visible at the 
macro level.

>  but I haven't seen any kernel space papers measuring degradations in overall
> system performance when adding safety checks(perhaps redundant sometimes) into
> the kernel

Well.. here's the thing.  Papers are usually written by academics and trade
journal pundits, not people who write code for a living.  As a result, they end
up comparing released code versions.  As a worked example, see how the whole
Spectre thing turned out - the *initial* fears were that we'd see a huge
performance drop.  But the patches that finally shipped for the Linux kernel
were after a bunch of clever people had thought about it and come up with less
intrusive ways to close the security issue.

(Having said that, the guys at Phoronix do a reasonable job of doing
macro-level benchmarks of each kernel release and pointing out if there's a big
hit in a subsystem).

And as I said earlier - sometimes it doesn't matter, because correctness trumps 
performance.


pgpJm5ih01G0F.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Testing the performance impact of kernel modifications

[valdis . kletnieks]

On Mon, 15 Oct 2018 23:42:03 +0800, Carter Cheng said:

> I was wondering what are some good ways to assess the performance impact of
> kernel modifications. Are there some papers in the literature where this is
> done? Does one need to differentiate between CPU bound and different types
> of I/O bound processes etc?

That is *so* totally dependent on exactly what the modification is, that
there's no right answer here.

The things you will want to measure for a new TCP flow control module (to
measure the difference between, say, cubic and new_reno and fq_codel and
your new module) will be *totally* different from changes to an LSM, which again
will be different from an overhaul of a disk I/O scheduler.

And then, the environment matters as well.  The performance metrics that I care
about on my laptop (which is used as a desktop replacement) are "can I do a
kernel build and my desktop environment still work well" type things.  But the
numbers I care about on the machines I maintain across the hall in the data
center are different - those are disk storage, backup, and archive - so I'm
willing to burn a lot of CPU in both kernel and userspace if it gets me more
IOPs and throughput - important when you have 500+ million files in a single
petabyte-plus file system.  Meanwhile, the guys a few cubicles down are doing
HPC, which means they want as little kernel CPU usage as possible because that
gets in the way of user computations.

And sometimes, it doesn't matter in the slightest what the performance impact 
is,
because the change is required for correctness - running incorrect code faster 
is
still running incorrect code.  See the recent Spectre patches for an example.




pgpHZrKZUdV7y.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: link time analysis for the kernel.

[valdis . kletnieks]

On Thu, 11 Oct 2018 21:45:16 +0800, Carter Cheng said:

> There are some detaills about the current procedures for linking the kernel
> that I am unfamiliar with. My understanding is that GCC and Clang both have
> the ability to do link time analysis and transforms on code but is it
> possible to write link time passes that will run on the kernel since the
> linking phase is a bit different (i.e. doesnt produce an ELF file)?

The fact that the kernel gets linked is an existence proof that it is possible
to do link time processing on the kernel.

There's no LTO support in the stock 4.19 tree, but Andi Kleen did a patchset
for 4.15, and there's another patchset to enable LTO when using Clang rather
than gcc. (I haven't tried either one, don't use on a production machine, as
the resulting kernel may crash, eat filesystems, and/or turn your dog green...)

Note that 'vmlinux' is a statically linked ELF binary. That  plus a bootstrap
code gets merged to create a bzImage or similar thing that can be loaded by
Grub2 or whatever boot loader.



pgpDhe8KBlmOu.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Learning Linux Kernel Development

[valdis . kletnieks]

On Thu, 11 Oct 2018 14:42:56 +0400, o...@goosey.org said:
> 10.10.2018, 19:36, "Carter Cheng" :
> >2. Is there some good way to figure out how to update knowledge gained 
> > from
> >this book to what is in the 4.x series of kernels?

> I think all C code-based drivers will work on all Linux versions after
> downloading kernel please checkout the staging/ folder.

Nowhere close.  Working on "all Linux versions" would imply a stable API.

The internal kernel APIs change all the time - a non-toy driver written for a
2.6.10 kernel has little to no chance of working on 4.19 without updating to
use the APIs presented by the 4.19 kernel.

See Documentation/process/stable-api-nonsense.rst for the details.


pgp57mAUy8HMr.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Fwd: Learning Linux Kernel Development

[valdis . kletnieks]

On Wed, 10 Oct 2018 23:35:20 +0800, Carter Cheng said:
> 1. After finishing the book and perhaps Understanding the Linux Kernel and
> Linux Device Drivers. What is the best way to dig deeper.

There's multiple answers to that question, as it depends on the questioner's 
preferred
learning style and motivation for digging deeper.

(I'll just add a link so I don't have to copy-paste here)
https://lists.kernelnewbies.org/pipermail/kernelnewbies/2017-April/017765.html

> 2. Is there some good way to figure out how to update knowledge gained from
> this book to what is in the 4.x series of kernels?

If you've studied enough different kernels so that you can generalize from an
example, the fact that 4.19 is 12 million lines different from 2.6.whatever
won't be much of an issue. If however you're still at the cargo-cult
programming level of writing code the way the LDD says via cut-n-paste without
really understanding it, you're going to have a bad time.

Somebody was keeping sample code from the LDD, updated to recent kernels. Not
sure if it's much help - if you understand the concepts involved, you probably
wouldn't need sample code, and if you didn't understand the concepts, you just
end up cut-n-pasting from a newer version

(Two really good kernel books are Bach's book on the Unix SYSV kernel and
McKusick's book on the BSD kernel - both spend a good amount of time doing "and
if we don't take a lock here, this race condition can happen, and if it
happens, *this* is what your file system looks like afterwards" type
discussion...)



pgpyxgT_noaeo.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: What is "#define alloc_ieee80211 alloc_ieee80211_rsl"?

[valdis . kletnieks]

On Tue, 09 Oct 2018 20:07:08 +0100, John Whitmore said:

> #define alloc_ieee80211   alloc_ieee80211_rsl
>
> So what am I missing or why are a number of functions being redefined
> as another name, which doesn't exist?

There's areas in the kernel which use the preprocessor ## to glue stuff
together. see include/asm-generic/atomic-long.h  and ATOMIC_LONG_FETCH_OP for
an example. It's also use in some code to generate function names to initialize
a structure of function pointers.  This can make grepping for a function name
troublesome if you don't realize there's ## abuse going on to create a function
name on the fly.

However, in this particular case, it looks like they're redirecting an
allocator so that for rtl8192u, when common/copypasted code calls
alloc_ieee80211 it gets redirected to their own version...

[/usr/src/linux-next] git grep alloc_ieee80211
drivers/staging/rtl8192u/ieee80211/ieee80211.h:#define alloc_ieee80211  
alloc_ieee80211_rsl
drivers/staging/rtl8192u/ieee80211/ieee80211.h:  * allocated beyond this 
structure by alloc_ieee80211
drivers/staging/rtl8192u/ieee80211/ieee80211.h:struct net_device 
*alloc_ieee80211(int sizeof_priv);
drivers/staging/rtl8192u/ieee80211/ieee80211_module.c:struct net_device 
*alloc_ieee80211(int sizeof_priv)
drivers/staging/rtl8192u/r8192U_core.c: dev = alloc_ieee80211(sizeof(struct 
r8192_priv));




pgpqxfuEnqDl8.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: How to make linux stop all processes/activities except the one I am benchmarking?

[valdis . kletnieks]

On Mon, 08 Oct 2018 23:05:56 +0300, Lev Olshvang said:

> Of cause the simplest way which comes to my mind  is to send SIGPAUSE to all
> processes,  except mine.

Actually, the *simplest* way is to just boot the machine into single-user mode 
and
run your test there.


pgp1ddCl5hh2I.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Diploma project with the Linux kernel

[valdis . kletnieks]

On Fri, 05 Oct 2018 02:58:23 +0100, Mike Krinkin said:

> This might be of interest to you: https://www.criu.org

That's got two problems - first, it's userspace. And second, it's fairly
mature software, which means it's not suitable for a student project
by itself, and all the low-hanging fruit for improvements has probably
already been done (meaning that further extensions will be technically
challenging...)

A better source for project ideas is to do a literature search and find
proof-of-concept projects that *didn't* turn into mature software, and
need work to turn them into actual running code


pgpYFDYSz5xRi.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Diploma project with the Linux kernel

[valdis . kletnieks]

On Thu, 04 Oct 2018 21:44:14 +0300, Boian Karatotev said:

> I am a Computer Science student and for my last year I need to make and
> present a 'diploma project' at the end of June. So far I want to make a
> kernel module, whose description is in the following paragraph. I feel
> comfortable with C and my OS knowledge is maybe slightly better than my OS
> course. My question is: Would it possible to pull this off? I have no
> experience with the kernel and I want to get into kernel development, so
> this would be a perfect opportunity for that. My only issue is that this
> may be too complex for my experience.

> My idea: Something along the lines of checkpoint-restart as a kernel
> module. I want to ultimately be able to migrate a running process to a
> different machine (assuming same at least some basic similarity). I know of
> BLCR 
> and I am planning on using it as a guide, although I am unsure about
> working on it directly. As far as I know, the grading process does not
> require this to be 100% complete, so I am aiming at transferring at least
> all the memory, restoring file descriptors and maybe child
> processes/threads.

You mean you want to re-invent the current checkpoint-restart code that's been
in the kernel since v3.10 back in June 2013? (see kernel/kcmp.c for the gory
details).

Note that migrating a running process to a different machine is a *lot*
trickier, especially if it has things like open files or network connections.
"Assume at least some basic similarity" isn't anywhere *near* good enough - if
the process has /home/fred/wombats/my_terabyte_database open, you're going to
need to have it at the same place in the filesystem and data synced across to
the new target (particularly fun if the process scribbles some more on the file
while you're busy migrating it, or if it hasn't done an fsync). Similarly, if
it has a TCP connection open to someplace else, you're going to have to figure
out what to do with the IP 4-tuple and sequence numbers to avoid breaking the
connection. And if it's HPC software using MPI configured to do RDMA over
Infiniband, that's even uglier

In fact, migrating an entire virtual machine is easier than migrating one
process, because you don't have to worry about recovering the process state,
that's all in kernel memory that you migrate with the VM.  Move the VM, take
down the IP on the old hypervisor, set up the IP on the new one, toss out a
gratuitous ARP packet so other machines on the subnet notice, and you're ready
to go...

There's a *reason* why VMWare gets away with charging lots of money for their
enterprise-class software that supports migrating a live VM across hypervisors.
It's a lot harder to do than you think.



pgpWK8lb45fL3.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: How to enable max98357a in sound/soc/codecs

[valdis . kletnieks]

On Thu, 04 Oct 2018 12:44:00 +0100, Paul Nader said:

> I'm trying to enable the max98357a codec in sound/soc/codecs.

> In the Kconfig file it is listed with an empty tristate so it doesn't show
> up when I do a make menu config.

That's usually a result of the variable being enabled by a 'select' elsewhere.

> How can I enable it without adding a description to the tristate?

Enable something that does a 'select'.  Look in a 'git grep MAX98357A'
or "find . -name 'Kconfig*' | xargs grep MAX98357A' .  It looks like that
part is mostly used for some arm64 (it's in the defconfig) and some
Intel SoC boards.  Look at sound/soc/intel/boards/Kconfig.


pgp7Xu_ZWHAWp.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Checkpatch.pl warning

[valdis . kletnieks]

On Sun, 30 Sep 2018 16:04:13 +0300, Shubham Singh said:

> Regarding the warning of checkpatch.pl, 
> " 'str' object has no attribute 'decode' "
> This warning is related to spdxcheck.py which is because of version of python
> In python3 str has no attribute decode(), while it works fine in python2, 
> without any warning.
> I checked the patch related to this
> https://lore.kernel.org/lkml/20180717190635.29467-1-jcl...@redhat.com/T/#u
> We can resolve the warning by just putting an if statement above 
> line.decode() to resolve the warning. 

I'm assuming you're using a 4.19-rc1 or later kernel, as Cline's patch first 
appeared
in a mainline kernel at that tag, and next-20180820 for the -next tree.

Are you saying this commit:

commit bed95c43c15eb6b1ccc5b09e5ae08cac726c456d
Author: Jeremy Cline 
Date:   Fri Aug 17 15:44:01 2018 -0700

scripts: add Python 3 compatibility to spdxcheck.py

is incomplete and needs patches?

> Am I right or I missed something?

If you are right, there *is* something missing - a patch. :)

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Problem with netconsole and eth0 timing

[valdis . kletnieks]

On Wed, 26 Sep 2018 21:38:27 -0700, don fisher said:

> Thanks. I tried building with the driver embedded in the kernel, but the
> compile failed with a halt. No crash is apparent, just a halt. It turned
> out that this was repeated until I removed the netconsole command during
> boot. System appears stable now. I will try tomorrow to embed the
> driver, then add netconsole option in the command line.

Wait, what?  The *compile* "failed with a halt"?  What the heck does that mean?


pgp5U1nXlk_eM.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Problem with netconsole and eth0 timing

[valdis . kletnieks]

On Wed, 26 Sep 2018 13:25:35 -0700, don fisher said:
> Would you tell me how to tell the driver that it is to be eth0, ip
> address etc. Maybe on linux command line, but I do not know the format.

To quote some guy named Don Fisher:

> my kernel and including the proper command (as shown below) in the linux boot 
> string:
>   netconsole=64001@192.168.7.60/eth0,64001@192.168.7.55/34:e6:d7:01:2a:dd

That's how. The netconsole command gets the info it needs from there, and tells
the network layer how to configure the ethernet device and the network layer -
although mostly the network layer.  And the devices will auto-name themselves
during boot, so all you need to do is know *which* name the kernel gives to the
port you want to use, and then use that name.

So grovel around in dmesg, and look for lines like (2 examples I have handy 
here)

 grep eth /var/log/dmesg
[7.278395] igb :07:00.0: added PHC on eth0
[7.278398] igb :07:00.0: eth0: (PCIe:5.0Gb/s:Width x2) 24:6e:96:10:db:64
[7.278916] igb :07:00.0: eth0: PBA No: G61346-000
[7.368911] igb :07:00.1: added PHC on eth1
[7.368913] igb :07:00.1: eth1: (PCIe:5.0Gb/s:Width x2) 24:6e:96:10:db:65
[7.369372] igb :07:00.1: eth1: PBA No: G61346-000

dmesg | grep eth
[2.642006] e1000e :00:19.0 eth0: (PCI Express:2.5GT/s:Width x1) 
f0:1f:af:0c:8a:da
[2.642076] e1000e :00:19.0 eth0: Intel(R) PRO/1000 Network Connection
[2.642118] e1000e :00:19.0 eth0: MAC: 10, PHY: 11, PBA No: 7011FF-0FF
[5.071095] e1000e :00:19.0 eno1: renamed from eth0
[   44.516004] e1000e :00:19.0 eth0: renamed from eno1

And find the one that has the MAC address of the port you want to use. Note
that you want the kernel-assigned device name, not the one that udev/systemd
finally assign to the device. So for the second example (my laptop), you'd want
either the eth0 or eno1 name (depending which one was in effect when the
netconsole module initializes.  The initial eth0 is from pci enumeration, the
rename to eno1 is courtesy of the kernel, and then the rename back to eth0 by
systemd.  Some experimentation may be needed (I've got a few servers that have
4 1G ports on the motherboard and multiple 10G/40G dual-port cards, so
sometimes the port wired to our management network ends up at eth6 or eth7...)

So if the port you want to use gets named eth4 by the kernel, you use
netconsole=64001@192.168.7.60/eth4,64001@192.168.7.55/34:e6:d7:01:2a:dd

(Gory ethernet details follow :)

Remember that strictly speaking, the ethernet device itself doesn't *need* to
know what its IP address is - it only needs to know its own MAC address so it
knows which packets on the wire to accept to hand to the network stack, and
*maybe* a list of other MAC addresses it should accept.

And the hardware already knows its own MAC address.. :)

You can get ethernet devices working with a *very* small set of functions:

0) Tell the kernel your hardware state (link/no link, MAC address, a few other 
things)
1) Receive packets for your own MAC address
2) Receive broadcast packets
3) Receive packets for another specified MAC address (semi-optional)
4) Receive packets in promiscuous mode (semi-optional)
5) Transmit packet to the MAC address provided

Pretty much everything else can be done in kernelspace (though modern
cards often provide offload of some IP and even TCP handling, interrupt
coalescing, and all sorts of other stuff)

(I learned far too much about minimalist Ethernet when the Clarkson Packet
Drivers were getting created in the cubicle next to mine. :)




pgp7phgOz_EgO.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: staging:rtl8192u: Possible memory leak?

[valdis . kletnieks]

On Wed, 26 Sep 2018 16:52:03 +0100, John Whitmore said:

> This might all be a mute point as I seem to remember someone saying
> that memory allocation never fails in Linux and this can only happen
> if the first two allocations work and the third fails.

If memory allocation never fails, it would be a moot point, not a mute point.

However, memory allocation can indeed fail, and often does.

And you are correct, there is a memory leak.  While looking at that, I found
another minor issue. Patch submitted.


___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

[PATCH] fix error handling in drivers/staging/rtl8192u/ieee80211/ieee80211_module.c

[valdis . kletnieks]

John notes that if the kzalloc of ieee->pHTInfo fails, we fail to call
ieee80211_networks_free().  In addition, that function has an un-needed check
before kfree().

Reported-by: John Whitmore 
Signed-off-by: Valdis Kletnieks 
---
diff --git a/drivers/staging/rtl8192u/ieee80211/ieee80211_module.c 
b/drivers/staging/rtl8192u/ieee80211/ieee80211_module.c
index 90a097f2cd4e..97ff0371b5bb 100644
--- a/drivers/staging/rtl8192u/ieee80211/ieee80211_module.c
+++ b/drivers/staging/rtl8192u/ieee80211/ieee80211_module.c
@@ -78,8 +78,6 @@ static inline int ieee80211_networks_allocate(struct 
ieee80211_device *ieee)
 
 static inline void ieee80211_networks_free(struct ieee80211_device *ieee)
 {
-   if (!ieee->networks)
-   return;
kfree(ieee->networks);
ieee->networks = NULL;
 }
@@ -180,6 +178,7 @@ struct net_device *alloc_ieee80211(int sizeof_priv)
return dev;
 
  failed:
+   ieee80211_networks_free(ieee);
if (dev)
free_netdev(dev);
 


___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Problem with netconsole and eth0 timing

[valdis . kletnieks]

On Tue, 25 Sep 2018 18:26:06 -0700, don fisher said:

> The wicked message eth0: up comes at Sep 24 22:02:01.173616. The
> difference is maybe 36 seconds? There is an eth0: avail message at Sep
> 24 22:01:34.112744, don't know if that would suffice for netconsole Both
> are after netconsole has bailed out. Any obvious solutions I am missing?
> The documentation is pretty clear on how to set this up, so there must
> be some way to get it to work. I could find nothing on Google.

Here's the big clue:

Sep 24 22:01:25 dfpc60 systemd-modules-load[185]: Module 'netconsole' is builtin

which means that it's going to get initialized during *very* early boot, before
the initramfs gets called. netconsole is able to do the equivalent of
'ifconfig' (or at least enough of it to set the IPs/ports/ARP entries before
the rest of networking comes up), but it can't also get the physical device up 
and
running if the hardware driver isn't present.

Since it's builtin, this is probably a custom-built kernel.  So make sure that
the driver for eth0 is also builtin.



pgpxHC9SDy10Q.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: 4.19-rc3 incompatible with nvidia driver

[valdis . kletnieks]

On Sun, 16 Sep 2018 20:47:41 +0200, Daan Wendelen said:

> The latest nvidia driver is incompatible with 4.19-rc3 because
> "drm_mode_connector_update_edid_property" got renamed to
> "drm_connector_update_edid_property" and
> "drm_mode_connector_attach_encoder" to "drm_connector_attach_encoder" in
> the linux headers.
>
> Is this considered the fault of linux or is it considered nvidia's
> responsibility to fix this?

The short form:

Out of tree modules are the problem of their maintainers. Kernel support
is basically limited to not intentionally breaking the basic mechanism for
compiling an out-of-tree module.

The long form:

Documentation/process/stable-api-nonsense.rst - go read it.


pgpMBdqUywzUY.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: dev process - reduce mistakes

[valdis . kletnieks]

On Wed, 19 Sep 2018 09:35:36 +1000, "Tobin C. Harding" said:
> Hi,
>
> I'm after some advice from those more experienced with [kernel]
> development please.
>
> What systems do you have in place to help catch mistakes?  In other
> words; what processes do you use when coding and submitting patches to
> help eliminate simple mistakes?

Same way you verify it for any other large programming project.

'make clean && make && make install && shutdown -r now'

1) Verify no compile errors.

2) Verify no unexplained compile warnings. gcc *does* screw up and whine about
things incorrectly on occasion because it doesn't know everything - "variable
may be used uninitialized" is a famous one, usually emitted because there's
program logic it can't see. For instance, code like:

int oddball;
for (i=0;i 5) oddball = 7;
 if (frobozz) quux = oddball;
}

and there's a reason known to programmer but not compiler
that guarantees that wombat > 5 will always happen at least once
before frobozz becomes true.

But in general, unless you can *prove* the compiler is false-positive on
a warning, fix the warning. ;)

2a) (optional) Install 'sparse' and do a 'make C=2' and see if that complains 
about
anything that gcc didn't...

3) Test that it actually boots and does whatever your patch is supposed to do.

See? And here you thought it was difficult :)


pgpMR2GNU9yYY.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: regarding const variables/structures

[valdis . kletnieks]

On Sat, 15 Sep 2018 01:19:24 +0530, inventsekar said:

> I didnt get you... I thanked Valdis for his detailed mail..

A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?


___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Linux FS design and implementation book

[valdis . kletnieks]

On Sat, 15 Sep 2018 07:33:51 +0530, inventsekar said:
> Around last year I was searching for Linux Kernel FS design and
> implementation, and I found out a book by someone,.. a full length book,
> particularly, at the end of the book he/she included source code as well...

That sounds like the Lions book.

John Lions - A Commentary on the Sixth Edition UNIX Operating System

https://en.wikipedia.org/wiki/Lions%27_Commentary_on_UNIX_6th_Edition,_with_Source_Code

Googling for 'lions unix book pdf' will find many sources.  There's
dead-tree versions available as well at high cost...


pgpKxt65Oub1J.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: regarding const variables/structures

[valdis . kletnieks]

On Wed, 12 Sep 2018 22:40:50 +0530, inventsekar said:

> So, please suggest some subsystems or some small puedes of code, where i
> can "dwell" sometimes and submit my first patch.

0) subsystems?  Anything under drivers/staging is fair game and certain to
provide hours of amusement...

1) Install sparse
2) Get yourself a linux-next tree (so you don't submit fixes for already fixed 
stuff)

$ git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
$ cd linux
$ git remote add linux-next 
git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git
$ git fetch linux-next
$ git fetch --tags linux-next

If you want to update it later, do *not* use 'git pull'. Due to the way
linux-next is rebased every day, doing that will result in a Lovecraftian 
horror

$ git remote update
$ git checkout next-20180911   (or whatever)

3) (optional)  Add the following patch to the top level Makefile:

diff --git a/Makefile b/Makefile
index 9e71826f67d7..f28b2ab9c369 100644
--- a/Makefile
+++ b/Makefile
@@ -434,6 +434,7 @@ KBUILD_CFLAGS   := -Wall -Wundef -Wstrict-prototypes 
-Wno-trigraphs \
   -Werror-implicit-function-declaration \
   -Wno-format-security \
   -std=gnu89
+KBUILD_CFLAGS   +=  -Wimplicit-fallthrough=2 -Wvla
 KBUILD_CPPFLAGS := -D__KERNEL__
 KBUILD_AFLAGS_KERNEL :=
 KBUILD_CFLAGS_KERNEL :=

4) Figure out why I said (3)  (There's an actual reason for that)

5) Build your kernel, running sparse on the code before compiling with extra
warnings:

$ make C=2 W=1 |& tee build.output

6) Start looking at the output.  Note however that sparse and gcc
sometimes throw warnings for perfectly good code, so do *not*
submit random "shut up warning" patches.  Verify the warning is
in fact correct, *then* submit the patch.

If you get ambitious, find out what Coverty and Trinity are, and other similar
tools.  If you use Trinity, remember to always mount a scratch monkey. :)



pgpJXM7KLOAR4.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: interruptible sleep tasks

[valdis . kletnieks]

On Tue, 11 Sep 2018 16:26:40 +0530, Muni Sekhar said:
> During kernel drivers testing I see lot of user mode applications
> blocks forever in interruptible sleep mode.  Is there any way to get
> the corresponding kernel functions stack trace ?

cat /proc/NNN/stack   should do the trick.

> Similarly is it possible to get a warning or kernel functions stack
> trace when a task in “S - interruptible sleep mode” and did not get
> scheduled for certain duration?

Take a look at the number of processes in that state *normally* on a desktop
system:

[~] ps ax | grep S | wc -l
218

And that's running e16 as a window manager, not a full Gnome environment.

That's the *normal* state for a process or kernel thread waiting for work.


___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Implementing tight time constraints for read/write with USB serial (ftdi_sio)

[valdis . kletnieks]

On Mon, 03 Sep 2018 21:12:35 +0200, Thomas Bracht Laumann Jespersen said:

> The challenge is that my application needs to act within 100 tbits, ie in 
> around
> ~.066ms. That is the window in which I need to begin transmitting a reply. Is
> there a way to achieve this with the ftdi_sio driver or do I need dedicated
> hardware for this?

(Hopefully you've already done all of the below - it's mostly for the
others out there thinking about doing real-time work...)

Step 0 is quantifying what a failure means.  Do you just miss a data point for
a graph, or does something on the assembly line explode or equally bad, or
something in between?

Step 1 is firming up exactly what that 0.66ms is.  Does it start at the
beginning of receiving the packet you care about, or when the last byte has
arrived?  And are you counting until the first byte of the response leaves, or
the last byte? Or are you counting until it's actually received at the target?
(Note that some statistical modeling may be needed, depending how PROFIBUS
behaves under load - what happens if another station is transmitting a very
large packet at the time you need to send your packet, etc etc..)

Step 2 is doing some userspace benchmarking of your response code - can the
target hardware do whatever processing is needed in the time allowed (remember
to subtract off any hardware-imposed latency like your PROFIBUS or your
RS485-USB converter).  This can be easy to meet if you have a beefy CPU and
just need to check the values of 3 bytes, add 2 other bytes together, and send
the result.  On the other hand, if you're on a low-end ARM and need to do a
512x512 FFT - the Realtime Gods may not smile favorably on your endeavor.

If your project is still alive at this point, *now* you start asking stuff like
realtime scheduling for your kernel thread, how to reduce interrupt
latency, and so on



pgpHpvDZhtfnk.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Queries on bottom halves

[valdis . kletnieks]

On Sun, 02 Sep 2018 20:58:14 +0530, Abhinav Misra said:

> But if in new kernel this implementation is changed then why we need so
> many options to defer the work as all of them are basically getting
> executed in almost the same way.* In that case code running softirq,
> tasklet, workqueue and kernel thread are all same ?*
> If this is true then why we just remove all these these option and keep one
> or two alternative.

-ENOPATCH

Often the gain from simplification isn't enough to motivate somebody to
actually do all the work of coding and exhaustively testing the patch series.



pgp0Q_VO4bNNu.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Question about DMA mapping.

[valdis . kletnieks]

On Fri, 31 Aug 2018 08:53:29 +0800, "phind@gmail.com" said:

> Thank for your time, I'm new to linux kernel. I am reading LDD3 chapter
> 15, Dirrect Memory Access section. I see that when I call function
> /dma_map_single/ and /dma_unmap_single/, I need to pass a direction as a
> parameter.

Please note that the LDD is current as of 2.6.10, which is from 2005, and
before git was used to maintain the kernel.

[/usr/src/linux-next] git diff --shortstat v2.6.12 v4.19-rc1
 70553 files changed, 23122660 insertions(+), 4318380 deletions(-)

The *concepts* in LDD3 are still valid.  However, the *exact* APIs
are almost certainly changed by now.

[/usr/src/linux-next] ls -l Documentation/DMA*
-rw-r--r--. 1 source source 34951 Jul 18  2017 Documentation/DMA-API-HOWTO.txt
-rw-rw-r--. 1 source source 28310 Sep  7  2017 Documentation/DMA-API.txt
-rw-r--r--. 1 source source  5209 Jul 18  2017 Documentation/DMA-ISA-LPC.txt
-rw-r--r--. 1 source source  7062 Jul 18  2017 Documentation/DMA-attributes.txt

Start there.


pgpcgeWPGP9dv.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: suggestions for good in-kernel examples of simple ethernet/PHY drivers?

[valdis . kletnieks]

On Thu, 30 Aug 2018 13:27:13 -0400, "Robert P. J. Day" said:

>   i'm already looking at that, but what i wanted was an example of an
> existing, physical driver that shows how simple the design can be (if
> such a thing exists).

For some reason, engineers building network devices almost never manage to
create a device that has a clean design with few warts.  It's almost as if they
were following in the steps of Persian rug makers, where hand-woven rugs always
have at least one (intentional) error, because only Allah is perfection



pgpwVbSX6t4jN.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: System call vs POSIX call

[valdis . kletnieks]

On Thu, 16 Aug 2018 16:29:29 -, Nicholas Mc Guire said:

> Note that you can do system calls directly with system() but that is
> generaly not how you do it - you to through the glibc calls
> which do some checks before invoking the actual system call.

system() does a fork/exec of a process.

syscall() issues an arbitrary system call, but all marshalling of
parameters and return codes becomes your problem


pgpb84q0Wz515.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: A question on the function clean_bdev_bh_alias

[valdis . kletnieks]

On Fri, 10 Aug 2018 22:48:43 +0800, "Larry" said:

> Does it intend to remove existent pages in block device's page cache
> if some new page cache of a vfs file was created whose block number
> range has overlaps with its block device page cache?

That's a situation you *really* hope doesn't arise.

Consider if you have (for example) an ext4 filesystem on /dev/sda4 mounted on
/foobar.

For there to be page cache for the ext4 filesystem and also page cache for
the block device, that means that something is writing to a file on /foobar
while something else is writing directly to /dev/sda4.

That's a really good way to corrupt the filesystem (and is why it's a Really
Bad Idea to fsck a mounted filesystem).  And no, merely trimming cache
pages that overlap doesn't even come *close* to fixing the problem...



pgp8RJnm3pU8F.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Problem of compiling linux2.6.11.12 kernel

[valdis . kletnieks]

On Mon, 06 Aug 2018 01:10:40 +0800, "He Huanyu" said:

> linux 4.16.9-1-ARCH
> gcc (GCC) 8.1.1 20180531
> GNU ld (GNU Binutils) 2.30

> And I get a problem like this, don't know if it is because of the version of 
> gcc or ld:

The problem is the version of Linux.  2.6.11 probably won't build with gcc 8.

Why are you trying to build 2.6.11 if you have 4.16 available? That dates to 
June 2005,
and is only of interest to software archaeologists.


pgpgAnEc40V6z.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: How to unwind stack to seek for specific caller?

[valdis . kletnieks]

On Fri, 03 Aug 2018 15:14:45 +0300, Lev Olshvang said:

> But I need to find out whether  my function y() was called from linux  kernel
> function x().\

What problem are you trying to solve?  If it's your function, don't you already
*know* which functions call it?

And if being called from code A requires different behavior than if it was
called from B, the *right*  answer is to create a flag and pass it.  That
requires less code, is simpler than trying to unwind the stack, and makes it
easier if 6 months from now you call a call from routine C.

For an example, see the memory-allocation flags GFP_ATOMIC and similar - if
you're adding another call site that needs it, you just add GFP_ATOMIC to the
flags.

And I'm willing to bet that most of those GFP_ flags were *all* originally "we
have to do something special if we try to do memory allocation from That One
Problem Spot. :)



pgpG_JgpbR3tq.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: First patch

[valdis . kletnieks]

On Thu, 02 Aug 2018 23:49:03 +0200, Arkadiusz Lis said:

> From 231fa77d595536cdaacf364b02dd64fd45a6adc4 Mon Sep 17 00:00:00 2001
> From: Arkadiusz Lis 
> Date: Thu, 2 Aug 2018 23:38:17 +0200
> Subject: [PATCH] Staging: One Laptop Per Child: fix coding style and license
>  issues

One thing per patch.  That's two things, therefor


pgpmVyPo4CL6q.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Sharing memory between two kernel module - Better approach

[valdis . kletnieks]

On Thu, 02 Aug 2018 23:12:23 +0530, Abhinav Misra said:

> I want to know is there any alternative for sharing the memory between two
> modules.

Why are two modules trying to share memory?  In general, each module should
be managing its own data objects.  If two modules need to share, maybe it really
should be one merged module

(And note that there's a number of kernel APIs for memory allocation and
management, optimized for various use cases - so "what are you trying to share"
and "why are you trying to share it" actually matter to get the code correct


pgp5LOFCzi8gA.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: virtual address space allocation

[valdis . kletnieks]

On Tue, 31 Jul 2018 15:41:28 +0200, Babis Chalios said:

> I am working on the runtime system of a distributed memory programming
> language. For some reason we need to create a big hole on the address space of
> the application. We are trying to do so by using an mmap during the
> initialization of the runtime and then the runtime handles itself those
> addresses.

How big a big hole, exactly?  And when you say "For some reason", does that
mean "For reasons I don't understand", or "For reasons I don't want to explain"?
(Note that in American colloquial English, it usually means the first...)

> However, we do have problems if memory overcommitment is disabled.

Well, if your "big hole" plus everything already running is big enough to not
fit in (RAM+SWAP), and you disable overcommit, you're going to have a bad
day

> So here goes the question: Is there any other mechanism to allow us to
> do what we want to?

It's unclear what you're trying to *do*.  Trying to re-invent MPI?  If your
goal is RDMA, there's already drivers available for Infiniband and other
fabrics.



pgpH94vf4e22a.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Need Suggestions

[valdis . kletnieks]

On Thu, 26 Jul 2018 03:07:27 +0530, Himanshu Jha said:

> Now, this part of section is really important and somehow if you think
> you really learnt C well in the course offered at University. Then
> please take a look at implicit conversions:
> https://en.cppreference.com/w/c/language/conversion
>
>   assert(sizeof(int) > -1);
>
> That should be enough to shatter your confidence.

If you want your confidence shaken:

https://www.lysator.liu.se/c/duffs-device.html

Fortunately, most maintainers will frown on stuff like that unless there's
a *damned* good reason - and Tom Duff had a very good reason indeed.

(Intro-to-c pop quiz - why is strcpy() not usable here?)

gcc 8.1.1 will accept Tom's code (with 3 warnings due to it
being pre-ANSI C).

For those who took a class on compiler design, and *think* they
know how LALR parsers and that stuff works - Examine the lexical
nesting of that, and figure out how the devil the compiler doesn't
get indigestion.. :)

And the K book on C does successive trimming of an strcpy(),
and finishes with

void strcpy(char *a, *b) { while (*a++ = *b++); }

The thing that's actually used a lot in the Linux kernel that gives
a lot of C newcomers heartburn is the widespread use of
structures of function pointers.. For example, this from include/linux/fs.h:

struct lock_manager_operations {
int (*lm_compare_owner)(struct file_lock *, struct file_lock *);
unsigned long (*lm_owner_key)(struct file_lock *);
fl_owner_t (*lm_get_owner)(fl_owner_t);
void (*lm_put_owner)(fl_owner_t);
void (*lm_notify)(struct file_lock *);  /* unblock callback */
int (*lm_grant)(struct file_lock *, int);
bool (*lm_break)(struct file_lock *);
int (*lm_change)(struct file_lock *, int, struct list_head *);
void (*lm_setup)(struct file_lock *, void **);
};

That's the structure definition.  Novice challenge: Find one or more places
where this structure is initialized, and understand how and why that works.


pgpR7TnySojEa.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: link_path_walk/dentry_path_raw TOCTTOU race question

[valdis . kletnieks]

On Tue, 24 Jul 2018 16:59:27 -0400, riya khanna said:

> I'm trying to understand what prevents TOCTTOU race conditions in
> dentry_path_raw
> and link_path_walk? What happens when somebody points a symlink path
> component from a dir that has the attacker is allowed to read to a dir that
> they are not allowed to read while link_path_walk() is doing its job?

The important part is that each step of the walk is atomic - it will go where
the link points at that point in the walk (or possibly fail if one of the steps
points to a now non-existent or non-accessible location).  If there's 5 parts
in the symlink path, say a/b/c/d/e, it will go to where 'a' points then, then
attempt to lookup 'b' in whatever directory following 'a' left it, then if it
finds 'b', it tries to lookup 'c', lather rinse repeat.

So in your case, the reader will attempt to land in one directory or the other,
at which point the permissions get applied.  So if the directory is unreadable,
the effect is the same as if you were following 
'ln -s /foo/bar/unreadable/baz /tmp/quux'

Another good way to confuse many programs is to have them cd to
a directory that's a bunch of levels deep (5-6 is usually sufficient), and
once they get there, have something chmod one of the intermediate
directories to mode 000.  Things that chase '..' to find / then have a bad 
day...

Most of the fun with TOCTTOU games with symlinks involves landing the
victim in a directory they *can* do damage in, but not the intended one.
For instance, getting root to do something in //lib/modules rather than in
/tmp  (note that this can be done in auto-ambush mode with things like
tar or cpio, where you deliver first a symlink off to someplace else, and
then extract a file relative to the symlink)


pgpnxOuqBM6MK.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Synaptics Driver Problem

[valdis . kletnieks]

On Mon, 23 Jul 2018 19:29:17 +0400, o...@goosey.org said:

> So synaptics is not working, I want to write a driver for synaptics.
> Can I modify a generic synaptics driver?

Step 0 should be to figure out *why* the generic driver isn't working,
because you'll need that information to fix the existing driver or
writing a new one.

When you say "not working", how exactly does it fail? Does it not
even get seen by the operating system?  Does it get seen, but not
as a HID input device?  Does it get registered as a HID, but touches
and taps don't get detected?


pgpRcWi8NrSwu.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: undefined reference to `__divdi3'

[valdis . kletnieks]

On Sun, 22 Jul 2018 17:48:21 +0530, Himanshu Jha said:

> I am currently working on my GSoC project and while testing through
> 0-day test service, I hit the following error:

>424  static u32 bme680_compensate_gas(struct bme680_data *data, u16 
> gas_res_adc,
>425   u8 gas_range)
>426  {
>427  struct bme680_calib *calib = >bme680;
>428  s64 var1;
>429  u64 var2;
>430  s64 var3;
>431  u32 calc_gas_res;
>432
>433  /* Look up table 1 for the possible gas range values */
>434  u32 lookupTable1[16] = {2147483647u, 2147483647u, 2147483647u,
>435  2147483647u, 2147483647u, 2126008810u,
>436  2147483647u, 2130303777u, 2147483647u,
>437  2147483647u, 2143188679u, 2136746228u,
>438  2147483647u, 2126008810u, 2147483647u,
>439  2147483647u};

As an aside, making this a 'static u32' will get rid of the mess of mov and 
movabs
instructions each time.  'static const u32' would be even better, as then the
compiler knows it can optimize assuming it never changes any of the values...

(In addition, with all that initialization out of the picture, figuring out
which  assembler code does what should be easier...



pgpmYmuBmCwYy.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Need Suggestions

[valdis . kletnieks]

On Wed, 18 Jul 2018 22:16:07 -0300, "Daniel." said:
> Sorry. Do I did something bad? I wasn't intended to.. I didn't think that
> trimming was a bad idea, I just want to reinforce the book suggestion ...
> Sorry

It's OK.. just remember to trim out the non-important stuff next time. :)


pgprhQKnEoGUb.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Need Suggestions

[valdis . kletnieks]

On Wed, 18 Jul 2018 14:29:41 -0700, Dave Stevens said:
> On Wed, 18 Jul 2018 18:19:05 -0300 "Daniel."  wrote:
>
> > > Hi Athul...
> > > This is my collegemate's writing, ... This will solve all your
> > > confusions
> > >
>
> is it the policy on this list to trim?

Only the irrelevant parts, leave the parts that are the context for the reply.
So that *should* have been trimmed down to:

---
> 4) Without hesitation buy, Robert Love's Linux Kernel Programming book. It is
> one of the best beginner material and start reading it parallel to the OS 
> book.
> This is easier to read than the previous one and more practical. But the
> previous one adds more value and is more theoretical. Handle (3) and (4) in
> parallel without blocking on any of the other activities.

I can confirm, this book is GOOD! Also check LDD3,  it's open and online
available, you may find some of it's authors in this mailist! :-). http://
www.makelinux.net/ldd3/ it's a outdated but it was a very valuable source to
me. You can find the examples updated on the internet. 
---

Also, MUAs the encourage top-posting and allowing quoting without a '> ' or
other indicator of quoting depth are a pox on email.


pgpa79V4rBBP8.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Need Suggestions

[valdis . kletnieks]

On Tue, 17 Jul 2018 21:10:05 -0700, Dave Stevens said:

> $ make help
> make: *** No rule to make target 'help'.  Stop.

Were you cd'ed into the directory that had the base of the kernel source tree?

Hint: the directory should look something like this:

[/usr/src/linux-next] ls
COPYINGModule.symvers  certs kernel patches   
signing_key.x509  vmlinux.o
CREDITSNextcryptolibreverts   
sound x509.genkey
Documentation  README  drivers   localversion-next  samples   
tags
Kbuild System.map  firmware  mm sched.h.patch 
tools
Kconfigarchfsmodules.builtinscripts   
usr
LICENSES   block   include   modules.order  security  
virt
MAINTAINERSbuild.default   init  netsigning_key.pem   
vmlinux
Makefile   built-in.a  ipc   null.dwo   signing_key.priv  
vmlinux-gdb.py

(Some files won't be there until you have a successful build - but the COPYING,
CREDITS, and MAINTAINERS files should be there.  If they aren't there, you're
in the wrong directory


pgpgGEkxND4Sj.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Netpoll APIs

[valdis . kletnieks]

On Tue, 17 Jul 2018 18:35:15 +0530, jitendra kumar khasdev said:

> Can netpoll apis send data over the internet. It worked for me in local
> network but when I try to send data over server which is in cloud not able
> to send it.
>
> Can anybody help me out ?

If it works over the local net, time to do the standard network debugging:

Does ping work to/from your cloud target?
Does traceroute look sane?
Does ssh/scp work?
Are your routing tables correct at both ends?
Is a firewall blocking it somewhere?
Does 'netstat' show a program listening for your packets at the far end?


pgpKXCb7jf4hx.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: newly add MAP_SYNC flag to mmap

[valdis . kletnieks]

On Tue, 17 Jul 2018 02:15:17 -, David Frank said:

> inking. I'm checking out if the flag does what is is said to do-- I don't 
> have 
> to call msync function, which would boost performance.

Note that this can actually *kill* performance, because this means that the
kernel has to flush to backing store every single time it notes a change,
whereas if you use msync only at those points your software needs a sync point,
it can do it at only those points

Thought experiment:  Imagine a workflow that needs to checkpoint every 1000
changes to the shared segment (for instance, if you've mapped an array with
1000 rows, do a for() loop across it incrementing one item, and checkpoint when
they're all incremented).  msync after the loop completes is one sync, while a 
worst-case
using MAP_SYNC could result in a flush after every single increment (if the 
system
is rescheduling the process over and over - for instance, if there's also 
another
syscall inside the loop).



pgpQaF6G7wWPa.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: newly add MAP_SYNC flag to mmap

[valdis . kletnieks]

On Tue, 17 Jul 2018 00:57:31 -, David Frank said:

> I installed LTS 18.04 with headers.  I'm trying to use the new flag MAP_SYNC
> with mmap call, but it complained MAP_SYNC undefined.What #define do I need to
> enable this?

You need more than a #define. You need a 4.15 kernel and matching
kernel-headers. But if you installed 18.04 correctly, those should be in place
already, as that apparently shipped with 4.15...

Or if you're not really a kernel person, but a misplaced userspace person,
'man 2 mmap' tells us:

NAME
   mmap, munmap - map or unmap files or devices into memory

SYNOPSIS
   #include 


pgpytrRfJFruC.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Need Suggestions

[valdis . kletnieks]

On Sat, 14 Jul 2018 22:07:12 +0400, oz...@goosey.org said:

> LFS is a good places to prepare a new linux distribution and have fun as you
> talk about :)

Note that *building* a LFS system is a bit of work.

Properly *maintaining* an LFS system is an ongoing *ton* of work.  Basically,
you end up maintaining your own distro - which means you need to keep up
to date on security issues on a per-program basis, and backport patches
when you discover problems with shared library mismatches..

For example:  Look at how many programs on your system use OpenSSL.
There's been more than a few security issues with it - and if the fix is "update
to the next release", you get to figure out what needs to be patched to use
the next version if there's an API change, or backport the OpenSSL fix for
the packages that haven't upgraded upstream to use the new OpenSSL API...


pgpuChf4nZ35W.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Question about memcpy

[valdis . kletnieks]

On Fri, 13 Jul 2018 11:02:13 +0800, bing zhu said:

> I‘m trying to write a simple fs in user space,if memcpy is slower than
> kernel , i think it's unfair,as for only cpu for my task,
> it's a bit of arbitrary ，i just want my task not interrupted during a
> specific time is that possible ?

Not getting interrupted is an *entirely* different issue than making memcpy 
fast.

Note that in general, systems code should be able to deal with interruptions
during most parts of the code, and locking used and disabling pre-emption for
sections of code that can't deal with being interrupted.  Remember that if your
filesystem code turns off interrupts for long enough, you can start losing
things like I/O completions.  Fortunately for those who write systems code,
the vast majority of interrupts are totally transparent to the vast majority
of the kernel code.

And if you're doing a file system in userspace, you're going to fail to notice
hundreds or even thousands of interrupts happening. If you don't believe me,
'cat /proc/interrupts', and realize that userspace didn't notice *any* of them
happening.



pgpKxTF7O0vLQ.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: mmap huge page and MAP_SYNC flag support

[valdis . kletnieks]

On Thu, 12 Jul 2018 20:33:32 -, David Frank said:
>  I got it from hereHow To Install Kernel 4.15 RC7 on Ubuntu, Linux Mint, 
> Elementary OS And Other Ubuntu Derivatives | LinuxG.net

Looks like you got the linux-image-... .deb but didn't do the linux-headers-... 
.deb(s).

(And I have no idea why they list 2 different linux-headers .deb's. Anybody 
understand
Ubuntu's mindset enough to explain that? I get paid to deal with RedHat-based 
stuff...)


pgphNd7RamH3v.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: mmap huge page and MAP_SYNC flag support

[valdis . kletnieks]

On Thu, 12 Jul 2018 20:10:36 -, David Frank said:

>  Thanks Valdis.Looks like the failure is that I can not write to a 2GB buffer
> (char bf[2GB size]).Where do I get the distro include file?

>From the same exact place you got the distro kernel from. (I'm guessing
it's a distro kernel from the -generic in the uname -r).  If you built it
yourself from a Linus source git/tarball, 'make headers_install' will do it, but
note that if the rest of the distro install still thinks you have a distro 
kernel-headers
installed, the package management system may whine


pgpFBzTfjbaTQ.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: mmap huge page and MAP_SYNC flag support

[valdis . kletnieks]

On Thu, 12 Jul 2018 18:44:43 -, David Frank said:
> Hi,How do I enable the MAP_SYNC flag in mmap for linux 4.15, my code does not
> compile with that flag on linux 4.15.0-041500rc7-generic #201801072330

Get your distro to ship an update to /usr/include/linux  (kernel-headers) that
contains the flag.  Your kernel supports it, but your toolchain doesn't until
you get an include file that includes the update for MAP_SYNC

> Also, how do I make mmap support map size of 2GB or larger, currently the
> call to mmap fails with mapsize of 2GB.

What error code does it fail with?


pgpMZLwLw0PuU.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Question about memcpy

[valdis . kletnieks]

On Thu, 12 Jul 2018 22:27:37 +0800, bing zhu said:

> as for memcpy ,kernel is faster than user ,might because schedule ,can i
> try to make user as fast as kernel ?

Do you have an actual issue where the difference in speed of these two
things makes a difference?  Or is this primarily a mental curiosity thing?


pgpA1hBztvIuQ.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Question about memcpy

[valdis . kletnieks]

On Tue, 10 Jul 2018 22:51:34 +0800, bing zhu said:

> Thank you ,I use this func for both kernel and user ,result are same.
> void *memcpy(void *dest, const void *src, size_t n)
> {

Might want to use 'void *my_memcpy(..)' instead, just in case the build
environment plays #define games with you and causes a different memcpy()
to get invoked instead.

[/usr/src/linux-next] egrep -r '#define\s*memcpy\(' include/ arch/*/include
arch/arm64/include/asm/string.h:#define memcpy(dst, src, len) __memcpy(dst, 
src, len)
arch/m68k/include/asm/string.h:#define memcpy(d, s, n) __builtin_memcpy(d, s, n)
arch/sparc/include/asm/string.h:#define memcpy(t, f, n) __builtin_memcpy(t, f, 
n)
arch/x86/include/asm/string_64.h:#define memcpy(dst, src, len)  
\
arch/x86/include/asm/string_64.h:#define memcpy(dst, src, len) __memcpy(dst, 
src, len)
arch/x86/include/asm/string_32.h:#define memcpy(t, f, n)
\
arch/x86/include/asm/string_32.h:#define memcpy(t, f, n) __builtin_memcpy(t, f, 
n)
arch/x86/include/asm/string_32.h:#define memcpy(t, f, n)
\
arch/xtensa/include/asm/string.h:#define memcpy(dst, src, len) __memcpy(dst, 
src, len)



pgp63qxFnoXbQ.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: [linux-3.12.19] - kernel panic observed

[valdis . kletnieks]

On Tue, 10 Jul 2018 12:08:25 +0530, Sriram said:
> I m working on linux-3.12.19 kernel.

You have our condolences.  Note that the fix for your problem has
probably already been found, somewhere in the millions of lines of
changes since then.

[/usr/src/linux-next] git diff --shortstat v3.12 next-20180710
 64362 files changed, 12400150 insertions(+), 4758460 deletions(-)

If finding the fix in all that is too difficult for you, complain to the vendor 
who
you're paying to support that kernel.



pgpRxIfk7ww_O.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: [PATCH] vsprintf: fix build warning

[valdis . kletnieks]

On Tue, 10 Jul 2018 09:42:03 +1000, "Tobin C. Harding" said:

> I was under the impression that each maintainer constantly rebased their
> next branches and that was why one has to checkout the tagged linux-next
> each day instead of just pulling.

Close, but no cigar.  The maintainers don't rebase their -next branches, but
due to the way linux-next merges 200+ trees on top of current Linus tree,
*that* ends up rebasing every day.


pgpMgaIICxzey.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Question about watchdog

[valdis . kletnieks]

On Mon, 09 Jul 2018 09:30:51 -0400, Ruben Safir said:
> On 07/08/2018 04:44 PM, valdis.kletni...@vt.edu wrote:
> > Error while parsing statement.,  What is a "filesystem bus" and when does it
> > issue a HW interrupt?

> You have a hard drive on the system bus and it sends interupts...

That's probably on the PCI or USB buses - Linux doesn't have a "filesystem bus".


pgpg0v7cVqune.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Question about memcpy

[valdis . kletnieks]

On Mon, 09 Jul 2018 19:34:44 +0530, Himanshu Jha said:

> I think for these benchmarking stuff, to evaluate the cycles and time
> correctly you should use the __rdtscp(more info at "AMD64 Architecture
> Programmer’s Manual Volume 3: General-Purpose and System Instructions"
> Pg 401)

Just beware that many Intel (and maybe some AMD) chipsets have a non-constant
TSC frequency.  Check /proc/cpuinfo for 'constant_tsc' before relying on the 
value.



pgpQ1gxUPeraD.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Question about watchdog

[valdis . kletnieks]

On Sun, 08 Jul 2018 15:46:32 -0400, Ruben Safir said:

> What are you saying?  That is the filesystem bus sends a HW interupt on

Error while parsing statement.,  What is a "filesystem bus" and when does it
issue a HW interrupt?


pgpAXjJvARBym.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Regarding Signing Linux kernel with Microsoft secure boot keys for UEFI

[valdis . kletnieks]

On Sun, 08 Jul 2018 11:21:08 +0530, inventsekar said:

> I read this page few times but I am unable to understand what's Linus's
> idea..Why he disagree ...
> whether the Linux kernel should include code that makes it easier to boot
> Linux on Windows PCs.

The issue is "trusted boot", and it doesn't actually make it easier to boot 
Linux.

The problem is that the obvious way to implement it for a distro requires an
intermediate key signed by Microsoft.

In other words, you can't do it easily without Microsoft's permission. Although
pretty much all UEFI boxes that support secure boot allow installing trusted
private keys, it's not something you can do in the middle of an Ubuntu install -
it requires dropping down into the BIOS screens and setting a bunch of stuff.

So the only way to do it in a distro-friendly manner without involving
Microsoft is to have the Linux Foundation or similar non-distro entity create a
public/private key pair, and somebody gets *all* the vendors to include that
key as well as Mirosoft's key.  Dell, Lenovo, Toshiba, And all the others.
Because any vendor that doesn't include it will get reports on the web "Trusted
boot of Linux on Zen-Cheap doesn't work."

Which, of course, most hardware manufacturers don't give a rat's tail about,
because if they did, they'd fix their buggy BIOS that create pages on the web
"suspend doesn't work on Zen-Cheap".

(In actual practice, what happened was that somebody got Microsoft to sign
an intermediate UEFI blob that allows bootstrapping a Linux kernel, and distros
have included that blob.  However, just like linux-firmware is packaged 
separately
from the kernel due to the differing license on most firmware (which isn't GPL),
that blob has to be distributed separate from the kernel as well.


pgpLF76XsxIaz.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies