Re: lua in kernel!

2014-06-19 Thread Lourival Vieira Neto
Hi Mayuresh,

 while i was drawn to netbsd because of the upcoming lua
 support in the kernel and userland,

I'm happy to read this =).

 i am quite lost about
 the probable use cases for real-world scenarios.

We have proposed some use cases, such as packet filtering, device
drivers, network protocols and file systems. Please note that use
cases depend on the creation of proper bindings between the kernel and
Lua. Currently, we have just few bindings committed on -current.

I'm currently working on the packet filtering use case by extending
NPF using Lua. I'll talk about this use case on EuroBSDCon 2014 [1]
and hope to make the code publicly available soon. Here is the talk's
abstract:

NetBSD recently added an experimental support for kernel scripting
based on the programming language Lua, which allows privileged users to load
and run Lua scripts in kernel. This talk presents a special use case on
scripting the NetBSD Packet Filter (NPF). It presents NPFLua, a NPF extension
module that allows users to define advanced rules to filter the
network traffic using Lua scripts.

This talk also presents Luadata, a Lua extension library that allows developers
to expose safely system memory for Lua scripts. This library also allows users
to describe data layouts declaratively in Lua. Luadata is used in combination
with NPFLua to allow users to inspect and modify network packet payload using
Lua.

[1] http://2014.eurobsdcon.org/

Marc is working on a line-disciplines use case.

Moreover, I previously worked on a kernel-scripting environment for
Linux, named Lunatik. I developed a CPU frequency scaling use case,
extending CPUfreq. There are also research groups that worked on
packet filtering [2] and file systems [3] on Linux, using Lunatik.

[2] A. Graf. PacketScript—a Lua Scripting Engine for in-Kernel
Packet Processing. Master’s thesis, Computer Science Depart-
ment, University of Basel, July 2010.

[3] M. Grawinkel, T. Suss, G. Best, I. Popov, and A. Brinkmann.
Towards Dynamic Scripted pNFS Layouts. In High Perfor-
mance Computing, Networking, Storage and Analysis (SCC),
2012 SC Companion:, pages 13–17. IEEE, 2012.

 prima-face, it feels quite strange to have a scriptable
 kernel and have that capability extended through out the
 userland.

Yes, it is not usual. But I think it can be quite useful =).

 i have been googling (via lynx) and haven't found anything
 which would suggest possible use cases for the lua-in-kernel
 effort. might be because my google skills are poor.

 can someone with access to such a document please share the
 details?

As Justin pointed, there are a Marc's presentation and some discussion
on the mailing lists.

We, I and Marc et al., are also working on a paper about Scriptable OS
that we hope to make publicly available soon. This paper introduces
the concept of Scriptable OS, which supports that OS can adequately
provide extensibility through kernel scripting. It also presents some
use cases and experiments.

Feel free to ask more questions here or contact me privately.

 also, if the lua-in-kernel effort does succeed, would there
 be some mechanism to turn it off while doing a customized
 build?

Actually, Lua in kernel is optional. If you want to use it, you need
to explicitly enable it.

 can't figure how useful such a feature might be in
 a production environment like web-app hosting or even an
 embedded system.

Suppose that you have discovered a new vulnerability on a specific
implementation of SSH. You can use a Lua script on a NPF firewall to
filter the SSH software version and then block the traffic from the
vulnerable SSH implementation. Here is a Lua script example that
implements this kind of filtering:

function filter(hdr, pld)
  -- get a segment of the payload
  local seg = pld:segment(0, 255)

  -- convert segment data to string
  local str = tostring(seg)

  -- pattern to capture the software version
  local pattern =
'SSH%-[%w%p]+%-([%w%p]+) ?[%w%p]*\r\n'

  -- get the software version
  local software_version = str:match(pattern)

  if software_version == 'OpenSSH_6.4' then
-- reject the packet
return false
  end

  -- accept the packet
  return true
end

Regards,
-- 
Lourival Vieira Neto


lua in kernel!

2014-06-18 Thread Mayuresh Kathe
hello,

while i was drawn to netbsd because of the upcoming lua
support in the kernel and userland, i am quite lost about
the probable use cases for real-world scenarios.

prima-face, it feels quite strange to have a scriptable
kernel and have that capability extended through out the
userland.

i have been googling (via lynx) and haven't found anything
which would suggest possible use cases for the lua-in-kernel
effort. might be because my google skills are poor.

can someone with access to such a document please share the
details?

also, if the lua-in-kernel effort does succeed, would there
be some mechanism to turn it off while doing a customized
build? can't figure how useful such a feature might be in
a production environment like web-app hosting or even an
embedded system.

thanks,

~mayuresh



Re: lua in kernel!

2014-06-18 Thread Justin Cormack
On Wed, Jun 18, 2014 at 12:06 PM, Mayuresh Kathe mayur...@kathe.in wrote:
 hello,

 while i was drawn to netbsd because of the upcoming lua
 support in the kernel and userland, i am quite lost about
 the probable use cases for real-world scenarios.

 prima-face, it feels quite strange to have a scriptable
 kernel and have that capability extended through out the
 userland.

 i have been googling (via lynx) and haven't found anything
 which would suggest possible use cases for the lua-in-kernel
 effort. might be because my google skills are poor.

 can someone with access to such a document please share the
 details?

Currently the best sources I am aware of are Marc Balmer's talks, eg
https://archive.fosdem.org/2013/schedule/event/lua_in_the_netbsd_kernel/attachments/slides/278/export/events/attachments/lua_in_the_netbsd_kernel/slides/278/kernel_mode_lua.pdf
(I think there is a video somewhere too)

There were also some discussions on the NetBSD lists too. But I think
we need to put together a better document, examples and actual code.
The current working code is for defining line disciplines.

I also have a userspace project for programming NetBSD via Lua
https://github.com/justincormack/ljsyscall which is being used in
various ways, eg for testing the NetBSD Linux compatibility layers and
programming rump kernels.

 also, if the lua-in-kernel effort does succeed, would there
 be some mechanism to turn it off while doing a customized
 build? can't figure how useful such a feature might be in
 a production environment like web-app hosting or even an
 embedded system.

Yes all the Lua support is optional, and will remain that way.

Justin