Re: LuaD: creating a flexible data filter system
On Saturday, 17 October 2015 at 02:02:16 UTC, Jakob Ovrum wrote: On Friday, 16 October 2015 at 10:45:52 UTC, Chris wrote: Later you call the function with the Lua C API like "lua_pcall(L, 0, 1, 0);". It's a bit tricky to move things around on the Lua stack, but you'll get there! ;) Or you could use LuaD which doesn't require you to mess around with the relatively unproductive, bug-prone C API :) I've used both, LuaD and DerelictLua + my own smaller D library that wraps all these nasty Lua stack operations (e.g. creating, accessing and adding to tables). The better I got to know the Lua C API while writing my own wrappers, the more I came to appreciate LuaD :-) However, LuaD is still 5.1 and I didn't want to be stuck with 5.1. So I rolled my own. It's not as comprehensive as LuaD but did the trick. The purpose was to experiment with Lua server pages + vibe.d which worked fine. Rikki gave me the idea to compile each page as a Lua function into memory for faster execution (etLua style[1]). It works fine with both LuaD and my own wrappers. In my own version I use the Lua C API directly in some places, though I don't know, if that's really a big speedup. If I set up my own homepage, I'd probably give vibe.d + Lua a shot. No more PHP and sh*t like that. [1] https://github.com/leafo/etlua
Re: LuaD: creating a flexible data filter system
many thanks for the valuable insights. so far i made a simple prototype with LuaD and classes, works nicely for when my niput what so far is not 100% clear is if there is a way to have a parsed msgpack or json documents being exposed in my lua code in a way so it behaves like a lua object. Ideally in a RW fashion so that changed then again can be processed by D code.
Re: LuaD: creating a flexible data filter system
On Saturday, 17 October 2015 at 13:15:17 UTC, yawniek wrote: many thanks for the valuable insights. so far i made a simple prototype with LuaD and classes, works nicely for when my niput what so far is not 100% clear is if there is a way to have a parsed msgpack or json documents being exposed in my lua code in a way so it behaves like a lua object. Ideally in a RW fashion so that changed then again can be processed by D code. http://luajit.org/ext_ffi_tutorial.html C meta methods
LuaD: creating a flexible data filter system
hi, i'm reading in a stream of data that is deserialized into individual frames. a frame is either of: a) a specific D datastructure ( struct with a few ulong,string,string[string] etc members), known at compile time b) json (preferably stdx.data.json) i now want to implement something where i can dynamically add lua filters that then get data out of these frames, create a new lua object send it back to D code where its either sent to another lua filter or at last being serialized again to json und then being processed further. ideally i would not like to copy all the data into a lua object but directly access it from lua. is there an elegant approach to do this and support both a and b cases? so far i did some benchmarks, mainly with string comparisons and it turned out that luaD is about 10x faster than mruby and python D bridges.
Re: LuaD: creating a flexible data filter system
On Friday, 16 October 2015 at 09:01:57 UTC, yawniek wrote: hi, i'm reading in a stream of data that is deserialized into individual frames. a frame is either of: a) a specific D datastructure ( struct with a few ulong,string,string[string] etc members), known at compile time b) json (preferably stdx.data.json) i now want to implement something where i can dynamically add lua filters that then get data out of these frames, create a new lua object send it back to D code where its either sent to another lua filter or at last being serialized again to json und then being processed further. ideally i would not like to copy all the data into a lua object but directly access it from lua. is there an elegant approach to do this and support both a and b cases? so far i did some benchmarks, mainly with string comparisons and it turned out that luaD is about 10x faster than mruby and python D bridges. You could use the Lua C API directly. I have a small project where I use DerelictLua[1] and access the Lua C API directly for fast processing (cf. http://www.lua.org/manual/5.3/ or http://www.lua.org/manual/5.2/ "C API"). Another performance trick is to compile lua functions and store them in memory, rather than having lua read and execute everything. lua_State* L = luaL_newstate(); auto func = luaL_loadstring(L, "Lua code as C-string\0"); // Lua file compiled as a function Later you call the function with the Lua C API like "lua_pcall(L, 0, 1, 0);". It's a bit tricky to move things around on the Lua stack, but you'll get there! ;) [1] https://github.com/DerelictOrg/DerelictLua
Re: LuaD: creating a flexible data filter system
On Friday, 16 October 2015 at 09:01:57 UTC, yawniek wrote: hi, i'm reading in a stream of data that is deserialized into individual frames. a frame is either of: a) a specific D datastructure ( struct with a few ulong,string,string[string] etc members), known at compile time b) json (preferably stdx.data.json) i now want to implement something where i can dynamically add lua filters that then get data out of these frames, create a new lua object send it back to D code where its either sent to another lua filter or at last being serialized again to json und then being processed further. ideally i would not like to copy all the data into a lua object but directly access it from lua. is there an elegant approach to do this and support both a and b cases? so far i did some benchmarks, mainly with string comparisons and it turned out that luaD is about 10x faster than mruby and python D bridges. I haven't done more than play with it so far, but see LuaJIT FFI. Obv LuaJIT is fast too. From memory I think you can access C structs directly. In addition you can write meta methods that provide nice sugar for accessing, constructors, etc.
Re: LuaD: creating a flexible data filter system
On Friday, 16 October 2015 at 09:01:57 UTC, yawniek wrote: hi, i'm reading in a stream of data that is deserialized into individual frames. a frame is either of: a) a specific D datastructure ( struct with a few ulong,string,string[string] etc members), known at compile time b) json (preferably stdx.data.json) i now want to implement something where i can dynamically add lua filters that then get data out of these frames, create a new lua object send it back to D code where its either sent to another lua filter or at last being serialized again to json und then being processed further. ideally i would not like to copy all the data into a lua object but directly access it from lua. is there an elegant approach to do this and support both a and b cases? so far i did some benchmarks, mainly with string comparisons and it turned out that luaD is about 10x faster than mruby and python D bridges. The class support in LuaD supports the kind of indirection you want, but it's probably not finished enough for your use case. Once closer to completion, it will be possible to use it with structs as well.
Re: LuaD: creating a flexible data filter system
On Friday, 16 October 2015 at 10:45:52 UTC, Chris wrote: Later you call the function with the Lua C API like "lua_pcall(L, 0, 1, 0);". It's a bit tricky to move things around on the Lua stack, but you'll get there! ;) Or you could use LuaD which doesn't require you to mess around with the relatively unproductive, bug-prone C API :)
