hrtimer + Kernel Networking - freeze

2016-04-09 Thread Thomas Schmidt 
Hello,

i try to do some experiment generating high throughput network traffic and try 
to analyze the impact of system utilization on the network performance 
especially inter packet delay variation. One part of that is that i implement 
basically same functionality in user space and kernel space - where UDP packets 
are sent (not received) with high cadence and i compare the behavior. the UDP 
packets have always fixed content.

For the kernel implementation i have done several experiments with netpoll API 
and alternatively with socket API, where the netpoll_send_udp() respective 
sock_sendmsg() is triggered by hrtimer. Basically this works like expected, 
however especially if timer value is extremely low (e.g. < 1  nsec - well 
knowing that this is VERY challenging for the kernel) the system freezes 
totally after few seconds. 

I tried as well to separate the hrtimer + network send into an own kernel 
thread - but no success - still the system freezes after a while.
This behavior is only observed in case network functions _and_ hrtimer are in 
kernel space - i made one experiment where only hrtimer is in kernel space and 
triggers over a signal a user space process to send packets - which is stable 
(even with extremely optimistic low nano second interval settings of hrtimer) 
and does not freeze the system.
Unfortunately kernel remote debugging gives not too much idea. Last entry is 
"hrtimer: interrupt took 11398 ns“...

I assume that the system is just overloaded and not able to process that many 
timer interrupts. When the network send process runs in user space there is (i 
guess) a protection but (naturally ?!) when it runs all in kernel space there 
is no protection against overload.

Questions:

- is there any way to detect that hrtimer interrupts (+ the callback process) 
taking too much processing time - so that i could prevent a freeze ?

- may i have not used an elegant way to implement my experiment. (I tried 2 
alternative approaches A.: start module + basic network setup —>  start hrtimer 
—> callback sending udp packet and re-setup timer; approach B.: start module —> 
start thread —> thread function does basic network setup and starts timer —> 
callback sending udp packet and re-setup time ) is there anything else which 
would may works better ?

- with the netpoll API and kernel socket API I outsource IP+UDP packet assembly 
to kernel. How is it impacting if i do own thread ? because the IP/UDP assembly 
for sure is handled in other thread and there is as well also IO operation from 
network device module needed - which runs eventually in some other kernel 
thread  - it might not influence anything if i have my module doing very low 
amount of logic (assembling IP  is surely more effort) in own thread.


Would be very nice if somebody gives some ideas.

Many Thanks
Thomas S.


___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: system call

2016-04-09 Thread Nitin Varyani 
I am using Ubuntu

On Sat, Apr 9, 2016 at 8:04 PM, Pranay Srivastava  wrote:

> On Sat, Apr 9, 2016 at 7:51 PM, Nitin Varyani 
> wrote:
> > I have a 64 bit machine
> >
>
> Before changing the source try to build, install and boot your kernel.
> I'm sure there are some extra steps you might need to perform to boot your
> compiled kernel. Which distro are you using? I used OpenSuse for this work.
>
>
> > I am changing linux 3.5.4 source tree.
> >
> > I modified syscall_64.tbl
> > I had put your code in linux_3.5.4/arch/x86/pks_first/pks_first_call.c
> >
> > Then, I created pks_first/Makefile
> >
> > Modified the arch/x86/Kbuild
> >
> > Modified include/linux/syscalls.h
> >
> >
> > I ran "make menuconfig" and then simply exit.
> >
> > Then I ran
> >
> > "make"
> >
> > I saw that pks_first_call.o was created
> >
> > I then ran
> >
> > "make modules_install"
> > "make install"
> >
> > After then I restarted my system and booted linux 3.5.4. But it was stuck
> > half way.
> >
> > On Sat, Apr 9, 2016 at 7:39 PM, Pranay Srivastava 
> wrote:
> >>
> >> Hi Nitin
> >>
> >> On Sat, Apr 9, 2016 at 5:03 PM, Nitin Varyani  >
> >> wrote:
> >> > Neither of the solution is working.
> >> > @ Pranay: kernel is not booting after making the changes you have
> >> > mentioned.
> >> > somethings like
> >> > "dropping to shell
> >> > initramfs:"
> >> > is displayed on booting.
> >>
> >> I don't think this is related to the changes you made. I would advise
> >> you just build the sources for your
> >> distro and try to get to boot the kernel you compiled. Perhaps some
> >> steps you might have missed specific to your
> >> distro?
> >>
> >> >
> >> >
> >> > On Thu, Apr 7, 2016 at 1:08 PM, Pranay Srivastava 
> >> > wrote:
> >> >>
> >> >> Nitin
> >> >>
> >> >>
> >> >> On Thu, Apr 7, 2016 at 11:53 AM, Nitin Varyani
> >> >> 
> >> >> wrote:
> >> >> >
> >> >> > Hi,
> >> >> >   I want to implement a system call as explained in Linux
> kernel
> >> >> > development by Robert Love.
> >> >> >
> >> >> > He does three things
> >> >> >  adding entry to entry.S
> >> >> > adding entry to asm/unistd.h
> >> >> > and adding the system call code to sched.c
> >> >> >
> >> >> >
> >> >> > and then make + make install
> >> >> >
> >> >> > I do not want to implement for all architectures but only for my PC
> >> >> > which is 64 bit. I am not able to locate files entry. S and
> unistd.h
> >> >> > which
> >> >> > he is telling in his tutorial.
> >> >> > Please help me out to figure out the exact steps. Please also
> mention
> >> >> > the linux kernel version I should use.
> >> >> >
> >> >>
> >> >> Please refer this. I wrote this quite a while back but should be good
> >> >> to
> >> >> go.
> >> >>
> >> >>
> >> >>
> >> >>
> http://codewithkernel.blogspot.my/2014/06/adding-new-system-call-in-linux-x86-and.html
> >> >>
> >> >> > Nitin
> >> >> >
> >> >> > ___
> >> >> > Kernelnewbies mailing list
> >> >> > Kernelnewbies@kernelnewbies.org
> >> >> > http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
> >> >> >
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> ---P.K.S
> >> >
> >> >
> >>
> >>
> >>
> >> --
> >> ---P.K.S
> >
> >
>
>
>
> --
> ---P.K.S
>
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: system call

2016-04-09 Thread Pranay Srivastava 
On Sat, Apr 9, 2016 at 8:11 PM, Nitin Varyani  wrote:
> I am using Ubuntu
>
> On Sat, Apr 9, 2016 at 8:04 PM, Pranay Srivastava  wrote:
>>
>> On Sat, Apr 9, 2016 at 7:51 PM, Nitin Varyani 
>> wrote:
>> > I have a 64 bit machine
>> >
>>
>> Before changing the source try to build, install and boot your kernel.
>> I'm sure there are some extra steps you might need to perform to boot your
>> compiled kernel. Which distro are you using? I used OpenSuse for this
>> work.
>>
>>
>> > I am changing linux 3.5.4 source tree.
>> >
>> > I modified syscall_64.tbl
>> > I had put your code in linux_3.5.4/arch/x86/pks_first/pks_first_call.c
>> >
>> > Then, I created pks_first/Makefile
>> >
>> > Modified the arch/x86/Kbuild
>> >
>> > Modified include/linux/syscalls.h
>> >
>> >
>> > I ran "make menuconfig" and then simply exit.
>> >
>> > Then I ran
>> >
>> > "make"
>> >
>> > I saw that pks_first_call.o was created
>> >
>> > I then ran
>> >
>> > "make modules_install"
>> > "make install"
>> >
>> > After then I restarted my system and booted linux 3.5.4. But it was
>> > stuck
>> > half way.
>> >
>> > On Sat, Apr 9, 2016 at 7:39 PM, Pranay Srivastava 
>> > wrote:
>> >>
>> >> Hi Nitin
>> >>
>> >> On Sat, Apr 9, 2016 at 5:03 PM, Nitin Varyani
>> >> 
>> >> wrote:
>> >> > Neither of the solution is working.
>> >> > @ Pranay: kernel is not booting after making the changes you have
>> >> > mentioned.
>> >> > somethings like
>> >> > "dropping to shell
>> >> > initramfs:"
>> >> > is displayed on booting.
>> >>
>> >> I don't think this is related to the changes you made. I would advise
>> >> you just build the sources for your
>> >> distro and try to get to boot the kernel you compiled. Perhaps some
>> >> steps you might have missed specific to your
>> >> distro?
>> >>
>> >> >
>> >> >
>> >> > On Thu, Apr 7, 2016 at 1:08 PM, Pranay Srivastava 
>> >> > wrote:
>> >> >>
>> >> >> Nitin
>> >> >>
>> >> >>
>> >> >> On Thu, Apr 7, 2016 at 11:53 AM, Nitin Varyani
>> >> >> 
>> >> >> wrote:
>> >> >> >
>> >> >> > Hi,
>> >> >> >   I want to implement a system call as explained in Linux
>> >> >> > kernel
>> >> >> > development by Robert Love.
>> >> >> >
>> >> >> > He does three things
>> >> >> >  adding entry to entry.S
>> >> >> > adding entry to asm/unistd.h
>> >> >> > and adding the system call code to sched.c
>> >> >> >
>> >> >> >
>> >> >> > and then make + make install
>> >> >> >
>> >> >> > I do not want to implement for all architectures but only for my
>> >> >> > PC
>> >> >> > which is 64 bit. I am not able to locate files entry. S and
>> >> >> > unistd.h
>> >> >> > which
>> >> >> > he is telling in his tutorial.
>> >> >> > Please help me out to figure out the exact steps. Please also
>> >> >> > mention
>> >> >> > the linux kernel version I should use.
>> >> >> >
>> >> >>
>> >> >> Please refer this. I wrote this quite a while back but should be
>> >> >> good
>> >> >> to
>> >> >> go.
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >> http://codewithkernel.blogspot.my/2014/06/adding-new-system-call-in-linux-x86-and.html
>> >> >>
>> >> >> > Nitin
>> >> >> >
>> >> >> > ___
>> >> >> > Kernelnewbies mailing list
>> >> >> > Kernelnewbies@kernelnewbies.org
>> >> >> > http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>> >> >> >
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> ---P.K.S
>> >> >
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> ---P.K.S
>> >
>> >
>>
>>
>>
>> --
>> ---P.K.S
>
>

Check this

http://unix.stackexchange.com/questions/123632/newly-compiled-kernel-not-booting-up-vanilla-kernel-3-12-14



-- 
---P.K.S

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: system call

2016-04-09 Thread Pranay Srivastava 
On Sat, Apr 9, 2016 at 7:51 PM, Nitin Varyani  wrote:
> I have a 64 bit machine
>

Before changing the source try to build, install and boot your kernel.
I'm sure there are some extra steps you might need to perform to boot your
compiled kernel. Which distro are you using? I used OpenSuse for this work.


> I am changing linux 3.5.4 source tree.
>
> I modified syscall_64.tbl
> I had put your code in linux_3.5.4/arch/x86/pks_first/pks_first_call.c
>
> Then, I created pks_first/Makefile
>
> Modified the arch/x86/Kbuild
>
> Modified include/linux/syscalls.h
>
>
> I ran "make menuconfig" and then simply exit.
>
> Then I ran
>
> "make"
>
> I saw that pks_first_call.o was created
>
> I then ran
>
> "make modules_install"
> "make install"
>
> After then I restarted my system and booted linux 3.5.4. But it was stuck
> half way.
>
> On Sat, Apr 9, 2016 at 7:39 PM, Pranay Srivastava  wrote:
>>
>> Hi Nitin
>>
>> On Sat, Apr 9, 2016 at 5:03 PM, Nitin Varyani 
>> wrote:
>> > Neither of the solution is working.
>> > @ Pranay: kernel is not booting after making the changes you have
>> > mentioned.
>> > somethings like
>> > "dropping to shell
>> > initramfs:"
>> > is displayed on booting.
>>
>> I don't think this is related to the changes you made. I would advise
>> you just build the sources for your
>> distro and try to get to boot the kernel you compiled. Perhaps some
>> steps you might have missed specific to your
>> distro?
>>
>> >
>> >
>> > On Thu, Apr 7, 2016 at 1:08 PM, Pranay Srivastava 
>> > wrote:
>> >>
>> >> Nitin
>> >>
>> >>
>> >> On Thu, Apr 7, 2016 at 11:53 AM, Nitin Varyani
>> >> 
>> >> wrote:
>> >> >
>> >> > Hi,
>> >> >   I want to implement a system call as explained in Linux kernel
>> >> > development by Robert Love.
>> >> >
>> >> > He does three things
>> >> >  adding entry to entry.S
>> >> > adding entry to asm/unistd.h
>> >> > and adding the system call code to sched.c
>> >> >
>> >> >
>> >> > and then make + make install
>> >> >
>> >> > I do not want to implement for all architectures but only for my PC
>> >> > which is 64 bit. I am not able to locate files entry. S and unistd.h
>> >> > which
>> >> > he is telling in his tutorial.
>> >> > Please help me out to figure out the exact steps. Please also mention
>> >> > the linux kernel version I should use.
>> >> >
>> >>
>> >> Please refer this. I wrote this quite a while back but should be good
>> >> to
>> >> go.
>> >>
>> >>
>> >>
>> >> http://codewithkernel.blogspot.my/2014/06/adding-new-system-call-in-linux-x86-and.html
>> >>
>> >> > Nitin
>> >> >
>> >> > ___
>> >> > Kernelnewbies mailing list
>> >> > Kernelnewbies@kernelnewbies.org
>> >> > http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> ---P.K.S
>> >
>> >
>>
>>
>>
>> --
>> ---P.K.S
>
>



-- 
---P.K.S

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: system call

2016-04-09 Thread Pranay Srivastava 
Hi Nitin

On Sat, Apr 9, 2016 at 5:03 PM, Nitin Varyani  wrote:
> Neither of the solution is working.
> @ Pranay: kernel is not booting after making the changes you have mentioned.
> somethings like
> "dropping to shell
> initramfs:"
> is displayed on booting.

I don't think this is related to the changes you made. I would advise
you just build the sources for your
distro and try to get to boot the kernel you compiled. Perhaps some
steps you might have missed specific to your
distro?

>
>
> On Thu, Apr 7, 2016 at 1:08 PM, Pranay Srivastava  wrote:
>>
>> Nitin
>>
>>
>> On Thu, Apr 7, 2016 at 11:53 AM, Nitin Varyani 
>> wrote:
>> >
>> > Hi,
>> >   I want to implement a system call as explained in Linux kernel
>> > development by Robert Love.
>> >
>> > He does three things
>> >  adding entry to entry.S
>> > adding entry to asm/unistd.h
>> > and adding the system call code to sched.c
>> >
>> >
>> > and then make + make install
>> >
>> > I do not want to implement for all architectures but only for my PC
>> > which is 64 bit. I am not able to locate files entry. S and unistd.h which
>> > he is telling in his tutorial.
>> > Please help me out to figure out the exact steps. Please also mention
>> > the linux kernel version I should use.
>> >
>>
>> Please refer this. I wrote this quite a while back but should be good to
>> go.
>>
>>
>> http://codewithkernel.blogspot.my/2014/06/adding-new-system-call-in-linux-x86-and.html
>>
>> > Nitin
>> >
>> > ___
>> > Kernelnewbies mailing list
>> > Kernelnewbies@kernelnewbies.org
>> > http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>> >
>>
>>
>>
>> --
>> ---P.K.S
>
>



-- 
---P.K.S

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: system call

2016-04-09 Thread Nitin Varyani 
Neither of the solution is working.
@ Pranay: kernel is not booting after making the changes you have mentioned.
somethings like
"dropping to shell
initramfs:"
is displayed on booting.


On Thu, Apr 7, 2016 at 1:08 PM, Pranay Srivastava  wrote:

> Nitin
>
>
> On Thu, Apr 7, 2016 at 11:53 AM, Nitin Varyani 
> wrote:
> >
> > Hi,
> >   I want to implement a system call as explained in Linux kernel
> development by Robert Love.
> >
> > He does three things
> >  adding entry to entry.S
> > adding entry to asm/unistd.h
> > and adding the system call code to sched.c
> >
> >
> > and then make + make install
> >
> > I do not want to implement for all architectures but only for my PC
> which is 64 bit. I am not able to locate files entry. S and unistd.h which
> he is telling in his tutorial.
> > Please help me out to figure out the exact steps. Please also mention
> the linux kernel version I should use.
> >
>
> Please refer this. I wrote this quite a while back but should be good to
> go.
>
>
> http://codewithkernel.blogspot.my/2014/06/adding-new-system-call-in-linux-x86-and.html
>
> > Nitin
> >
> > ___
> > Kernelnewbies mailing list
> > Kernelnewbies@kernelnewbies.org
> > http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
> >
>
>
>
> --
> ---P.K.S
>
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Is the stack layout for an arch standard?

2016-04-09 Thread Abhimanyu Saini 
I was studying the dump_stack function in linux kernel (form arm) and I came 
across


struct stackframe {
/*
 * FP member should hold R7 when CONFIG_THUMB2_KERNEL is enabled
 * and R11 otherwise.
 */
unsigned long fp;
unsigned long sp;
unsigned long lr;
unsigned long pc;
};


If this is the complete stack frame, then where are the local variables stored? 
Is this format generic, for example will U-Boot compiled for the same arch use 
this exact format for maintaining stack?


-Abhimanyu
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: what is the purpose of "LOCALVERSION" variable in scripts/setlocalversion?

2016-04-09 Thread Chinmay V S 
> i was unaware of the apparently independent variable "LOCALVERSION"
> which is used in that assignment.
>
>   there's precious little mention of that variable ... under what
> circumstances does one set it?

I have sometimes seen a "+" or a short-commit-sha appended when
building kernel in a local git repository with uncommitted/committed
changes.

Also i suppose one could run make LOCALVERSION= to
help identify a kernel image by the embedded  if
required at a later time.

As the image name is one of the first things to be printed in the
kernel/dmesg logs, it helps map error logs to their respective builds.

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

what is the purpose of "LOCALVERSION" variable in scripts/setlocalversion?

2016-04-09 Thread Robert P. J. Day 

  i was perusing "scripts/setlocalversion" to see the variations for
setting the kernel version, and i ran across this snippet:

  # CONFIG_LOCALVERSION and LOCALVERSION (if set)
  res="${res}${CONFIG_LOCALVERSION}${LOCALVERSION}"

i'm well aware of the kernel *config* option CONFIG_LOCALVERSION, but
i was unaware of the apparently independent variable "LOCALVERSION"
which is used in that assignment.

  there's precious little mention of that variable ... under what
circumstances does one set it? is it meant to be set in the user's
environment and simply inherited that way? is anyone here using it on
a regular basis?

rday

-- 


Robert P. J. Day Ottawa, Ontario, CANADA
http://crashcourse.ca

Twitter:   http://twitter.com/rpjday
LinkedIn:   http://ca.linkedin.com/in/rpjday



___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies