lua in kernel!

[Mayuresh Kathe]

hello,

while i was drawn to netbsd because of the upcoming lua
support in the kernel and userland, i am quite lost about
the probable use cases for real-world scenarios.

prima-face, it feels quite strange to have a scriptable
kernel and have that capability extended through out the
userland.

i have been googling (via lynx) and haven't found anything
which would suggest possible use cases for the lua-in-kernel
effort. might be because my google skills are poor.

can someone with access to such a document please share the
details?

also, if the lua-in-kernel effort does succeed, would there
be some mechanism to turn it off while doing a customized
build? can't figure how useful such a feature might be in
a production environment like web-app hosting or even an
embedded system.

thanks,

~mayuresh

[Justin Cormack]

Post by Mayuresh Kathe
hello,
while i was drawn to netbsd because of the upcoming lua
support in the kernel and userland, i am quite lost about
the probable use cases for real-world scenarios.
prima-face, it feels quite strange to have a scriptable
kernel and have that capability extended through out the
userland.
i have been googling (via lynx) and haven't found anything
which would suggest possible use cases for the lua-in-kernel
effort. might be because my google skills are poor.
can someone with access to such a document please share the
details?

Currently the best sources I am aware of are Marc Balmer's talks, eg
https://archive.fosdem.org/2013/schedule/event/lua_in_the_netbsd_kernel/attachments/slides/278/export/events/attachments/lua_in_the_netbsd_kernel/slides/278/kernel_mode_lua.pdf
(I think there is a video somewhere too)

There were also some discussions on the NetBSD lists too. But I think
we need to put together a better document, examples and actual code.
The current working code is for defining line disciplines.

I also have a userspace project for programming NetBSD via Lua
https://github.com/justincormack/ljsyscall which is being used in
various ways, eg for testing the NetBSD Linux compatibility layers and
programming rump kernels.

Post by Mayuresh Kathe
also, if the lua-in-kernel effort does succeed, would there
be some mechanism to turn it off while doing a customized
build? can't figure how useful such a feature might be in
a production environment like web-app hosting or even an
embedded system.

Yes all the Lua support is optional, and will remain that way.

Justin

[Lourival Vieira Neto]

Hi Mayuresh,

Post by Mayuresh Kathe
while i was drawn to netbsd because of the upcoming lua
support in the kernel and userland,

I'm happy to read this =).

Post by Mayuresh Kathe
i am quite lost about
the probable use cases for real-world scenarios.

We have proposed some use cases, such as packet filtering, device
drivers, network protocols and file systems. Please note that use
cases depend on the creation of proper bindings between the kernel and
Lua. Currently, we have just few bindings committed on -current.

I'm currently working on the packet filtering use case by extending
NPF using Lua. I'll talk about this use case on EuroBSDCon 2014 [1]
and hope to make the code publicly available soon. Here is the talk's
abstract:

"NetBSD recently added an experimental support for kernel scripting
based on the programming language Lua, which allows privileged users to load
and run Lua scripts in kernel. This talk presents a special use case on
scripting the NetBSD Packet Filter (NPF). It presents NPFLua, a NPF extension
module that allows users to define advanced rules to filter the
network traffic using Lua scripts.

This talk also presents Luadata, a Lua extension library that allows developers
to expose safely system memory for Lua scripts. This library also allows users
to describe data layouts declaratively in Lua. Luadata is used in combination
with NPFLua to allow users to inspect and modify network packet payload using
Lua."

[1] http://2014.eurobsdcon.org/

Marc is working on a line-disciplines use case.

Moreover, I previously worked on a kernel-scripting environment for
Linux, named Lunatik. I developed a CPU frequency scaling use case,
extending CPUfreq. There are also research groups that worked on
packet filtering [2] and file systems [3] on Linux, using Lunatik.

[2] A. Graf. PacketScript—a Lua Scripting Engine for in-Kernel
Packet Processing. Master’s thesis, Computer Science Depart-
ment, University of Basel, July 2010.

[3] M. Grawinkel, T. Suss, G. Best, I. Popov, and A. Brinkmann.
Towards Dynamic Scripted pNFS Layouts. In High Perfor-
mance Computing, Networking, Storage and Analysis (SCC),
2012 SC Companion:, pages 13–17. IEEE, 2012.

Post by Mayuresh Kathe
prima-face, it feels quite strange to have a scriptable
kernel and have that capability extended through out the
userland.

Yes, it is not usual. But I think it can be quite useful =).

Post by Mayuresh Kathe
i have been googling (via lynx) and haven't found anything
which would suggest possible use cases for the lua-in-kernel
effort. might be because my google skills are poor.
can someone with access to such a document please share the
details?

As Justin pointed, there are a Marc's presentation and some discussion
on the mailing lists.

We, I and Marc et al., are also working on a paper about Scriptable OS
that we hope to make publicly available soon. This paper introduces
the concept of Scriptable OS, which supports that OS can adequately
provide extensibility through kernel scripting. It also presents some
use cases and experiments.

Feel free to ask more questions here or contact me privately.

Post by Mayuresh Kathe
also, if the lua-in-kernel effort does succeed, would there
be some mechanism to turn it off while doing a customized
build?

Actually, Lua in kernel is optional. If you want to use it, you need
to explicitly enable it.

Post by Mayuresh Kathe
can't figure how useful such a feature might be in
a production environment like web-app hosting or even an
embedded system.

Suppose that you have discovered a new vulnerability on a specific
implementation of SSH. You can use a Lua script on a NPF firewall to
filter the SSH software version and then block the traffic from the
vulnerable SSH implementation. Here is a Lua script example that
implements this kind of filtering:

function filter(hdr, pld)
-- get a segment of the payload
local seg = pld:segment(0, 255)

-- convert segment data to string
local str = tostring(seg)

-- pattern to capture the software version
local pattern =
'SSH%-[%w%p]+%-([%w%p]+) ?[%w%p]*\r\n'

-- get the software version
local software_version = str:match(pattern)

if software_version == 'OpenSSH_6.4' then
-- reject the packet
return false
end

-- accept the packet
return true
end

Regards,