I think Raja is talking about Business Intelligence like Pentaho http://www.pentaho.com/ , the dimensional is about the data that you can manipulate as you wish. Not my cup of tea though.
Ahmad Amran Kapi Art In Software Sdn Bhd Suite 2.5 Inkubator K-Ekonomi 75450 Ayer Keroh Melaka. 06-2322464 / 013-6102545 http://www.aist.com.my On Fri, Apr 23, 2010 at 3:32 PM, Boh Yap <[email protected]> wrote: > hi all, > > Raja, Swee Meng knows what he is talking about when 'web > scrapping'.... its a no-brainer in Python. > > 2. When you mean 'dimensional' do you mean putting stuff into arrays? > Is it only 2-dimensional or are you talking about more? > Are you gonna manipulate those arrays in memory before putting them in > RDBMS tables? > Are they gonna be huge? > > 3. If most of yr answers to above is yes... > you may find answers to Q3 as well in SciPy, Scientific Python. > Python is very strong in the scientific community where they need > to handle huge > datasets. The data manipulation problems, there beats most cases in biz > applications, that's why Wall Street hires rocket-scientists, literally). > NumPy is a Python lib that handles huge arrays in memory, for example... > > You may find something here... > > http://www.scipy.org/Topical_Software and at their main site. > > On Fri, Apr 23, 2010 at 10:42 AM, CL Chow <[email protected]> wrote: >> I suppose it's mathematical modelling??? >> Regards, >> CL Chow >> "Please do not send me Microsoft Office/Apple iWork documents. Send >> OpenDocument instead! http://fsf.org/campaigns/opendocument/"; >> >> >> >> >> On Fri, Apr 23, 2010 at 10:35 AM, sweemeng ng <[email protected]> wrote: >>> >>> 1) sounds like web scraping, you can cook one up easily with python, >>> python have a built-in http library, and html parser, though a better idea >>> is to use tools like scrapy, >>> 2) need clarification >>> 3) python itself don't do that, there is a numpy, which is something like >>> matlab for python(i am assuming you are talking about mathematical stuff) >>> >>> On Fri, Apr 23, 2010 at 9:00 AM, Raja Iskandar Shah >>> <[email protected]> wrote: >>>> >>>> thanks boh. >>>> >>>> this looks promising. the specific things that i am looking are small >>>> backend processing stuff. examples are : >>>> 1. extracting tables from web pages and converting them to text files >>>> 2. transforming / restructuring the data into dimensional models >>>> 3. analytical regressions on the dimensional models >>>> >>>> things that the example bi tools do. >>>> >>>> the instance you mentioned is a good example. >>>> >>>> are there any equivalent open source projects ? python applications that >>>> does analysis on numbers ? >>>> >>>> >>>> >>>> >>>> >>>> On Fri, Apr 23, 2010 at 3:35 AM, Boh Yap <[email protected]> wrote: >>>>> >>>>> hi Raja, >>>>> >>>>> as always I wld suggest Python.... but what exactly do you have in >>>>> mind. From what I can see, its a BI type application, but do you >>>>> intend to rebuild the entire thing when there are some good tools >>>>> already out there? >>>>> >>>>> Or do you have some specialised solution that these cannot solve? >>>>> >>>>> If its the 2nd case, then I think Python is a great tool. If its not a >>>>> well defined problem, it could call for a fair bit of experimentation, >>>>> different approaches to the problem, compare the approaches etc... >>>>> Python's dynamic nature and rapid-prototyping capability allows for a >>>>> great deal of experimentation, and exploration to arrive at the right >>>>> solution. And you are not afraid to 'throw away' code, because the >>>>> cost/speed/efficiency of writing Python code is is pretty low. >>>>> >>>>> Then once you have the solution/algorithms in mind, you can deploy it >>>>> in Python, which has many choices of frameworks etc... and if you have >>>>> performance bottlenecks, parts of it can be rewritten in C and you can >>>>> call that from Python w/o problems. >>>>> >>>>> The only problem is you may find a lack of Python programmers, a >>>>> problem that can be said of good Java, Ruby and even PHP programmers >>>>> (and i don't mean cut & paste programmers;-). But they can be quite >>>>> easily trained up. I have personally taught an ASP programmer Python, >>>>> and he was able to do fairly sophisticated task after 2 weeks, it was >>>>> to pull log data off a mobile app from a Postgres DB, rehash the data >>>>> into another DB, to generate sales reports, delivered online with >>>>> graphs in JS, live, on the web. (Mind you this guy had a far bit of >>>>> initiative to go thru the docs and tutorial...) >>>>> >>>>> I can certainly help with the training, and I'm sure I can get others >>>>> from the Python community. >>>>> >>>>> PM me if you wish to discuss further.... >>>>> >>>>> On Thu, Apr 22, 2010 at 9:34 AM, Raja Iskandar Shah >>>>> <[email protected]> wrote: >>>>> > good day all >>>>> > >>>>> > any suggestion for language to replace java >>>>> > >>>>> > the type of applications that we are looking at are >>>>> > http://rapid-i.com/ >>>>> > http://kettle.pentaho.org/ >>>>> > >>>>> > >>>>> > -- >>>>> > Join Open Source Developers Club Malaysia http://www.osdc.my/ >>>>> > >>>>> > Facebook Fan page >>>>> > >>>>> > http://www.facebook.com/group.php?gid=98685301577 >>>>> > >>>>> > http://www.facebook.com/OSDC.my >>>>> > >>>>> > You received this message because you are subscribed to the Google >>>>> > >>>>> > Groups "OSDC.my Mailing List" group. >>>>> > To post to this group, send email to [email protected] >>>>> > To unsubscribe from this group, send email to >>>>> > [email protected] >>>>> > For more options, visit this group at >>>>> > http://groups.google.com/group/osdcmy-list?hl=en >>>>> >>>>> >>>>> >>>>> -- >>>>> #------- >>>>> regds, >>>>> >>>>> Boh Heong, Yap >>>>> >>>>> -- >>>>> Join Open Source Developers Club Malaysia http://www.osdc.my/ >>>>> >>>>> Facebook Fan page >>>>> >>>>> http://www.facebook.com/group.php?gid=98685301577 >>>>> >>>>> http://www.facebook.com/OSDC.my >>>>> >>>>> You received this message because you are subscribed to the Google >>>>> >>>>> Groups "OSDC.my Mailing List" group. >>>>> To post to this group, send email to [email protected] >>>>> To unsubscribe from this group, send email to >>>>> [email protected] >>>>> For more options, visit this group at >>>>> http://groups.google.com/group/osdcmy-list?hl=en >>>> >>>> -- >>>> Join Open Source Developers Club Malaysia http://www.osdc.my/ >>>> >>>> Facebook Fan page >>>> >>>> http://www.facebook.com/group.php?gid=98685301577 >>>> >>>> http://www.facebook.com/OSDC.my >>>> >>>> You received this message because you are subscribed to the Google >>>> >>>> Groups "OSDC.my Mailing List" group. >>>> To post to this group, send email to [email protected] >>>> To unsubscribe from this group, send email to >>>> [email protected] >>>> For more options, visit this group at >>>> http://groups.google.com/group/osdcmy-list?hl=en >>> >>> -- >>> Join Open Source Developers Club Malaysia http://www.osdc.my/ >>> >>> Facebook Fan page >>> >>> http://www.facebook.com/group.php?gid=98685301577 >>> >>> http://www.facebook.com/OSDC.my >>> >>> You received this message because you are subscribed to the Google >>> >>> Groups "OSDC.my Mailing List" group. >>> To post to this group, send email to [email protected] >>> To unsubscribe from this group, send email to >>> [email protected] >>> For more options, visit this group at >>> http://groups.google.com/group/osdcmy-list?hl=en >> >> -- >> Join Open Source Developers Club Malaysia http://www.osdc.my/ >> >> Facebook Fan page >> >> http://www.facebook.com/group.php?gid=98685301577 >> >> http://www.facebook.com/OSDC.my >> >> You received this message because you are subscribed to the Google >> >> Groups "OSDC.my Mailing List" group. >> To post to this group, send email to [email protected] >> To unsubscribe from this group, send email to >> [email protected] >> For more options, visit this group at >> http://groups.google.com/group/osdcmy-list?hl=en > > > > -- > #------- > regds, > > Boh Heong, Yap > > -- > Join Open Source Developers Club Malaysia http://www.osdc.my/ > > Facebook Fan page > > http://www.facebook.com/group.php?gid=98685301577 > > http://www.facebook.com/OSDC.my > > You received this message because you are subscribed to the Google > > Groups "OSDC.my Mailing List" group. > To post to this group, send email to [email protected] > To unsubscribe from this group, send email to > [email protected] > For more options, visit this group at > http://groups.google.com/group/osdcmy-list?hl=en -- Join Open Source Developers Club Malaysia http://www.osdc.my/ Facebook Fan page http://www.facebook.com/group.php?gid=98685301577 http://www.facebook.com/OSDC.my You received this message because you are subscribed to the Google Groups "OSDC.my Mailing List" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/osdcmy-list?hl=en
