[Lift] Re: Background jobs
On Jul 9, 5:27 pm, Vlad Seryakov vserya...@gmail.com wrote: I am building Web application using Lift (my first one) which is the port of existing Tcl/Aolserver based application. In the existing one i have two parts: Web and backend. Backedn part consists of jobs running in the background from time to time using schedule or on demand. I am trying to figure out the architecture and practical way of doing this in servlet/Lift environment. I used to have telnet interface but this is not a requirement, i can test using special temporary Web interface. Is there any recommendation or experience regarding such applications? In my opinion... Yes, drop the telnet and do a web interface to administrate anything, its worth it. for example, i have special job for importing huge Xml files into the database. How huge? The problem of loading big Xml files in Scala is already a challenge but for now i assume i have unlimited memory. The problem i have how to test it. Test it? like automated testing or test it want to know it works. If its the second, how about making a simple html file with a file submit form, add a sitemap entry to a lift application and create a snippet that parses that file and print the time it takes to crunch it! =) If you have no idea how to do this I would recomend buying the Lift book ( http://www.apress.com/book/view/9781430224211 which is not quite easy or polished but its a great way to go head first into the liftworld) Using scala interpreter mode i could test the Xml loading and parsing, now i need to load it into database and i have Mapper setup for my schema. Calling this job from simple Web snippet is my only choice? Calling this job from a web snippet is one way, other is to use mvn scala:console, but really is fast to set up that snippet, if you need any help let me know! Thanks --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Lift group. To post to this group, send email to liftweb@googlegroups.com To unsubscribe from this group, send email to liftweb+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/liftweb?hl=en -~--~~~~--~~--~--~---
[Lift] Re: Background jobs
You dont mention the kind of backend process you want to talk to, but you may well be interested in this post I wrote about lift-amqp: http://is.gd/CkPX Included is a neat example of how you can get inter-process communication using middleware messaging and it explains in fair detail the AMQP implementation in Lift. Perhaps you could send messages to and from your backend from your lift app? Cheers, Tim On Jul 10, 1:27 am, Vlad Seryakov vserya...@gmail.com wrote: I am building Web application using Lift (my first one) which is the port of existing Tcl/Aolserver based application. In the existing one i have two parts: Web and backend. Backedn part consists of jobs running in the background from time to time using schedule or on demand. I am trying to figure out the architecture and practical way of doing this in servlet/Lift environment. I used to have telnet interface but this is not a requirement, i can test using special temporary Web interface. Is there any recommendation or experience regarding such applications? for example, i have special job for importing huge Xml files into the database. The problem of loading big Xml files in Scala is already a challenge but for now i assume i have unlimited memory. The problem i have how to test it. Using scala interpreter mode i could test the Xml loading and parsing, now i need to load it into database and i have Mapper setup for my schema. Calling this job from simple Web snippet is my only choice? Thanks --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Lift group. To post to this group, send email to liftweb@googlegroups.com To unsubscribe from this group, send email to liftweb+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/liftweb?hl=en -~--~~~~--~~--~--~---
[Lift] Re: Background jobs
On Thu, Jul 9, 2009 at 5:27 PM, Vlad Seryakov vserya...@gmail.com wrote: I am building Web application using Lift (my first one) which is the port of existing Tcl/Aolserver based application. In the existing one i have two parts: Web and backend. Backedn part consists of jobs running in the background from time to time using schedule or on demand. I am trying to figure out the architecture and practical way of doing this in servlet/Lift environment. I used to have telnet interface but this is not a requirement, i can test using special temporary Web interface. Is there any recommendation or experience regarding such applications? My rule of thumb is that if the app needs to push something to the web interface, having them co-resident is easier. If they are communicating by writing RDBMS records, separation has advantages (bring down one of the apps, doesn't impact the other app.) You could also treat the apps as separate entities, but run them in the same Java Virtual Machine. OSGi is particularly good at handling stuff like this. for example, i have special job for importing huge Xml files into the database. The problem of loading big Xml files in Scala is already a challenge but for now i assume i have unlimited memory. The problem i have how to test it. Using scala interpreter mode i could test the Xml loading and parsing, now i need to load it into database and i have Mapper setup for my schema. Calling this job from simple Web snippet is my only choice? You can run a Mapper-based app separately from the web interface. The XML parsing issue with your data set is non-trivial. Have you looked at the Java SAX parser and hooking into events in the SAX parser so you're only keeping 1 record in memory at once? Thanks, David Thanks -- Lift, the simply functional web framework http://liftweb.net Beginning Scala http://www.apress.com/book/view/1430219890 Follow me: http://twitter.com/dpp Git some: http://github.com/dpp --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Lift group. To post to this group, send email to liftweb@googlegroups.com To unsubscribe from this group, send email to liftweb+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/liftweb?hl=en -~--~~~~--~~--~--~---
[Lift] Re: Background jobs
I think paulp has a scala.xml that doesn't require loading everything into memory at once. He once demonstrated it on the scala list. I think it's for 2.8. I'm not sure how you load but I think you process it with the regular scala pattern matching and XPath. Either look in the 2.8 sources or get in touch with him. - David Pollakfeeder.of.the.be...@gmail.com wrote: On Thu, Jul 9, 2009 at 5:27 PM, Vlad Seryakov vserya...@gmail.com wrote: I am building Web application using Lift (my first one) which is the port of existing Tcl/Aolserver based application. In the existing one i have two parts: Web and backend. Backedn part consists of jobs running in the background from time to time using schedule or on demand. I am trying to figure out the architecture and practical way of doing this in servlet/Lift environment. I used to have telnet interface but this is not a requirement, i can test using special temporary Web interface. Is there any recommendation or experience regarding such applications? My rule of thumb is that if the app needs to push something to the web interface, having them co-resident is easier. If they are communicating by writing RDBMS records, separation has advantages (bring down one of the apps, doesn't impact the other app.) You could also treat the apps as separate entities, but run them in the same Java Virtual Machine. OSGi is particularly good at handling stuff like this. for example, i have special job for importing huge Xml files into the database. The problem of loading big Xml files in Scala is already a challenge but for now i assume i have unlimited memory. The problem i have how to test it. Using scala interpreter mode i could test the Xml loading and parsing, now i need to load it into database and i have Mapper setup for my schema. Calling this job from simple Web snippet is my only choice? You can run a Mapper-based app separately from the web interface. The XML parsing issue with your data set is non-trivial. Have you looked at the Java SAX parser and hooking into events in the SAX parser so you're only keeping 1 record in memory at once? Thanks, David Thanks -- Lift, the simply functional web framework http://liftweb.net Beginning Scala http://www.apress.com/book/view/1430219890 Follow me: http://twitter.com/dpp Git some: http://github.com/dpp --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Lift group. To post to this group, send email to liftweb@googlegroups.com To unsubscribe from this group, send email to liftweb+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/liftweb?hl=en -~--~~~~--~~--~--~---
