I would suggest scala 1. extracting tables from web pages and converting them to text files -- DONE
http://www.hars.de/2009/01/html-as-xml-in-scala.html http://john-d-amos.blogspot.com/2009/08/vrbo-web-scraper-using-scala-and.html 2. transforming / restructuring the data into dimensional models 3. analytical regressions on the dimensional models This is where scala really shines, read about maps, reduce and flatmap. Plus the scalability of the JVM (running on something like Terracota, you can have literally thousands of nodes processing your stuff - if u do need that many) On Fri, Apr 23, 2010 at 9:00 AM, Raja Iskandar Shah <[email protected] > wrote: > thanks boh. > > this looks promising. the specific things that i am looking are small > backend processing stuff. examples are : > 1. extracting tables from web pages and converting them to text files > 2. transforming / restructuring the data into dimensional models > 3. analytical regressions on the dimensional models > > things that the example bi tools do. > > the instance you mentioned is a good example. > > are there any equivalent open source projects ? python applications that > does analysis on numbers ? > > > > > > > On Fri, Apr 23, 2010 at 3:35 AM, Boh Yap <[email protected]> wrote: > >> hi Raja, >> >> as always I wld suggest Python.... but what exactly do you have in >> mind. From what I can see, its a BI type application, but do you >> intend to rebuild the entire thing when there are some good tools >> already out there? >> >> Or do you have some specialised solution that these cannot solve? >> >> If its the 2nd case, then I think Python is a great tool. If its not a >> well defined problem, it could call for a fair bit of experimentation, >> different approaches to the problem, compare the approaches etc... >> Python's dynamic nature and rapid-prototyping capability allows for a >> great deal of experimentation, and exploration to arrive at the right >> solution. And you are not afraid to 'throw away' code, because the >> cost/speed/efficiency of writing Python code is is pretty low. >> >> Then once you have the solution/algorithms in mind, you can deploy it >> in Python, which has many choices of frameworks etc... and if you have >> performance bottlenecks, parts of it can be rewritten in C and you can >> call that from Python w/o problems. >> >> The only problem is you may find a lack of Python programmers, a >> problem that can be said of good Java, Ruby and even PHP programmers >> (and i don't mean cut & paste programmers;-). But they can be quite >> easily trained up. I have personally taught an ASP programmer Python, >> and he was able to do fairly sophisticated task after 2 weeks, it was >> to pull log data off a mobile app from a Postgres DB, rehash the data >> into another DB, to generate sales reports, delivered online with >> graphs in JS, live, on the web. (Mind you this guy had a far bit of >> initiative to go thru the docs and tutorial...) >> >> I can certainly help with the training, and I'm sure I can get others >> from the Python community. >> >> PM me if you wish to discuss further.... >> >> On Thu, Apr 22, 2010 at 9:34 AM, Raja Iskandar Shah >> <[email protected]> wrote: >> > good day all >> > >> > any suggestion for language to replace java >> > >> > the type of applications that we are looking at are >> > http://rapid-i.com/ >> > http://kettle.pentaho.org/ >> > >> > >> > -- >> > Join Open Source Developers Club Malaysia http://www.osdc.my/ >> > >> > Facebook Fan page >> > >> > http://www.facebook.com/group.php?gid=98685301577 >> > >> > http://www.facebook.com/OSDC.my >> > >> > You received this message because you are subscribed to the Google >> > >> > Groups "OSDC.my Mailing List" group. >> > To post to this group, send email to [email protected] >> > To unsubscribe from this group, send email to >> > [email protected]<osdcmy-list%[email protected]> >> > For more options, visit this group at >> > http://groups.google.com/group/osdcmy-list?hl=en >> >> >> >> -- >> #------- >> regds, >> >> Boh Heong, Yap >> >> -- >> Join Open Source Developers Club Malaysia http://www.osdc.my/ >> >> Facebook Fan page >> >> http://www.facebook.com/group.php?gid=98685301577 >> >> http://www.facebook.com/OSDC.my >> >> You received this message because you are subscribed to the Google >> >> Groups "OSDC.my Mailing List" group. >> To post to this group, send email to [email protected] >> To unsubscribe from this group, send email to >> [email protected]<osdcmy-list%[email protected]> >> For more options, visit this group at >> http://groups.google.com/group/osdcmy-list?hl=en > > > -- > Join Open Source Developers Club Malaysia http://www.osdc.my/ > > Facebook Fan page > > http://www.facebook.com/group.php?gid=98685301577 > > http://www.facebook.com/OSDC.my > > You received this message because you are subscribed to the Google > > Groups "OSDC.my Mailing List" group. > To post to this group, send email to [email protected] > To unsubscribe from this group, send email to > [email protected]<osdcmy-list%[email protected]> > For more options, visit this group at > http://groups.google.com/group/osdcmy-list?hl=en > -- Join Open Source Developers Club Malaysia http://www.osdc.my/ Facebook Fan page http://www.facebook.com/group.php?gid=98685301577 http://www.facebook.com/OSDC.my You received this message because you are subscribed to the Google Groups "OSDC.my Mailing List" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/osdcmy-list?hl=en
