Amit - if you can break off a part of what you want into a problem suitable for, say half an hour discussion, I could put it on the agenda for an NYCJUG meeting. We'll be meeting the 2nd Tuesday in May (5/13), in Hoboken this time - see http://www.jsoftware.com/jwiki/NYCJUG, and I usually start assembling the agenda about a week to three days in advance.
So, if you could define a suitably-sized sub-task by the Sunday before the thirteenth and provide me with any materials -like small examples of realistic inputs and transformations - by then, I'll put it on the agenda. Or you could do it for another meeting - please just give me some notice. Hope to see you again one of these days, Devon On 4/21/08, amit bolakani <[EMAIL PROTECTED]> wrote: > > Hi guys, > Thanks for all the responses. I really appreciate all you guys trying to > help me out with my problem. And I think I owe you guys a better > explanation > of the problem. This is an interesting set of requirements and I don't > know > if something like this has been done with J before. Here is what I am > trying > to achieve: > I want to build a tool for Windows which will have a GUI based interface > and > will be able to interface with: > 1. Databases (SQL, Oracle, Mysql etc) > 2. Excel and CSV files > 3. Text files > These interfaces will be used to pull data into the tool. While inputting > the data, I want the user to be able to specify some data transformations > on > on the data and show reports and produce output excel/csv/word files. > added using SQLite) and moreover what the performance of this would be > ... > NOTE: If there are any J experts in NYC, I would like to sit down with > them > (if they are kind enough and interested enough to spend time on this) and > work out more details > > On Mon, Apr 21, 2008 at 10:32 AM, Devon McCormick <[EMAIL PROTECTED]> > wrote: > > > > FWIW - since I have Excel 2007 installed on one of my machines, I > thought > > I'd mention that its row and column limits are 1048576 x 16384. > > > > I kept running out of memory trying to use Tara to write my 65536 by 256 > > matrix of floating point numbers to a spreadsheet. I also had problems > > using OLE but was able to do it in pieces (about 16K rows at a time) in > > about 1 minute (on a 2.66 GHz machine with 2 GB RAM). I'm sure it could > > be > > done in Tara much the same way (possibly more quickly) but I'm not > > familiar > > with that package. > > > > On 4/21/08, Sherlock, Ric <[EMAIL PROTECTED]> wrote: > > > > > > Amit, > > > >From the thread you started on comp.lang.apl it seems that your files > > are > > > only about 50,000 rows. As Richard suggested in that thread - that > isn't > > > very big. You shouldn't have to resort to memory-mapped files for > > something > > > that big. If you just want to import the data from the Excel > spreadsheet > > and > > > then work with it in J then I wouldn't imagine that will present too > > much of > > > a problem. > > > > > > I just used the Tara addon for J to create an Excel workbook with a > > sheet > > > containing 50,000 rows and 10 columns. I then used Tara to read that > > data > > > from the Excel worksheet back into J. > > > > > > Importing (reading) the file into J took a number of seconds using > Tara, > > > but once the array was in J, I was able to operate on the whole (or > > parts of > > > the) array quickly and easily. For example getting the sum of all 10 > > columns > > > over the 50,000 rows of the resulting array took 0.0057 seconds in J. > > > > > ... > > > > > > -- > > Devon McCormick, CFA > > ^me^ at acm. > > org is my > > preferred e-mail > > > ---------------------------------------------------------------------- > > For information about J forums see http://www.jsoftware.com/forums.htm > > > ---------------------------------------------------------------------- > For information about J forums see http://www.jsoftware.com/forums.htm > -- Devon McCormick, CFA ^me^ at acm. org is my preferred e-mail ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
