A better option for the shortest-paths server might be to use ConcurrentLua:
http://concurrentlua.luaforge.net/index.html See http://concurrentlua.luaforge.net/introduction.html, under "Distributed message passing". The "ping" processes/nodes would be your clients (handling the http requests), the "pong" process/node is your shortest-paths server. If you are on a multicore CPU you can have multiple shortest-paths nodes and do load-balancing between them. There is also Alua: http://translate.google.com/translate?hl=en&sl=pt&tl=en&u=http://alua.inf.puc-rio.br/, which I believe also does m+n threading now (so you can work with a single shortest-paths server even if you have several cores), but the english documentation is pretty lacking. -- Fabio Mascarenhas On Thu, Mar 11, 2010 at 4:29 PM, Andre Carregal <carre...@fabricadigital.com.br> wrote: > Dewey > > On Wed, Mar 10, 2010 at 10:37 PM, Dewey Gaedcke <de...@minggl.com> wrote: >> I can tell you are really busy and so I really hate to keep bugging you…. > > You certainly aren't bugging me... :o) > >> A friend who's a more advanced programmer than me has indicated that it's >> difficult (outside of true multi-threading) for one process to get access >> (even read-only) to the memory of another process. > > That's probably true and I wasn't proposing that kind of data sharing. > What I had in mind was to split your system in two parts: > > 1) One part responsible for a web interface that shows some view of > interpretation of the data managed by (2). > > 2) One part responsible for the data structure (your big Lua table) > access. This is what I'm assuming you have ready and running. > > My points about the inconveniences of CGI were related to the fact > that it could be used for (1), but certainly not for (2). > > My points about Copas and Xavante were related to the fact that one > way to make (1) and (2) talk to each other would be to use a socket > channel between them, in particular using HTTP as the protocol. > > In that scenario, (2) would have to be extended in order to listen to > the HTTP conversation, while (1) would have to act as a gateway > between two HTTP conversations. One with the browser, acquiring > parameters for example and responding with an HTML page, and one with > (2), using HTTP as a sort of RPC to obtain part of the whole data > structure and then present it to the user. > > I'm not sure in which part you would like to calculate the shortest > paths, but the architecture would be the same either way. > >> So where I'm still lost is that I don't understand how Copas, Xavante, >> FastCGI or any other technology is able to read (in parallel) a Lua table >> that exists in my dedicated persistent Lua memory process. > > Here is where HTTP could help you. On (2) you could use any socket > dispatcher (for example Copas) to implement a listener (aka a server) > and a protocol handler. If that protocol was HTTP, you could then use > Xavante as the handler, since it naturally works with Copas so you > would have to just write the "web part" of the problem. > > Note that you would be talking about two "web parts", one between the > client/browser/user and your site (1) and another between (1) and the > back end (2). > > So assuming (1) was implemented as say an Orbit application over > Apache with FastCGI and (2) implemented embedding Xavante in your > current application, the communication would go something like: > > Client -> HTTP -> Apache -> WSAPI -> Orbit -> Web App (1) > > This would get the client request into your public web application in > (1), where you could extract parameters and then use LuaSocket's > http.request() to call the web server in (2) using those parameters: > > http.request -> HTTP -> Xavante -> WSAPI -> Web App (2) > > Now your parameters have been transported through HTTP to the internal > server (Xavante), embedded into your application (2). There you would > access your table, find the shortest path for the received parameters > and then return the results using wsapi.response, still on (2). This > would be the data received by the http.request call by (1) done in the > previous step. > > With that data now in (1), you could process the results somehow and > generate your HTML response using wsapi.response in (1). This would > follow through HTTP to the client. > > Confusing? :o) > >> Please forgive my attempt at clarification if you feel you have already >> addressed that point, but I just wanted to confirm that we're on the same >> page before I do a deep dive to explore the products you have pointed me to. > > No need for apologies, I'm also having a bad time explaining things... > >> If you can confirm that either Copas or Xavante would (with customization) >> be able to gain parallel, read-only access to this huge, in-process Lua >> table, then I'll dig into those products and figure out the details to make >> my shortest-path code run against just one memory instance of this data. > > Note that while I'm proposing using Xavante (or just Copas), this > would not involve direct access to the table. Xavante (since it uses > Copas) can handle multiple requests at the same time, but it > multitasks collaboratively. > >> I'm not a low level programmer so perhaps it's easy stuff and I just don't >> realize how easy it is? > > I never said it was easy! :o) > >> Thanks very much for pointing me in the right direction. > > You are welcome, let me know if this made any sense. > > André > > _______________________________________________ > Kepler-Project mailing list > Kepler-Project@lists.luaforge.net > http://lists.luaforge.net/cgi-bin/mailman/listinfo/kepler-project > http://www.keplerproject.org/ > _______________________________________________ Kepler-Project mailing list Kepler-Project@lists.luaforge.net http://lists.luaforge.net/cgi-bin/mailman/listinfo/kepler-project http://www.keplerproject.org/
