Hi Biren,
I haven't heard back from you about whether you can use the data in the log files that I sent along yesterday. I want to make sure that Grameen Koota and Amy do not spend any unnecessary time gathering more data for you if the data in the zip file can be distilled into what you need. Can you let me know your thoughts? Also, did you have any success in starting to research things by performing specific tasks (bulk entry, loan account creating) and watching how it affects the volume of data? Looking forward to an update! :-) Thanks, Aliya ________________________________ From: Aliya Walji Sent: Tuesday, January 29, 2008 5:37 PM To: Amy Bensinger (Contractor); 'Biren Patnik' Cc: 'Mifos dev'; '[EMAIL PROTECTED]'; Van Mittal-Henkle; 'mifos dev'; 'Developer' Subject: RE: Database Growth Problem Hi Biren, As Amy mentioned in her email below, GK was collecting the statistics you need for all the tables in the DB from December 13th to date. The log files for this data are in tab delimited format so I was thinking that rather than having GK modify their scripts to just collect the data from the 6 tables you need, perhaps you can pull the data from the log files into a DB and query the DB yourself to pull out the data yourself for just the tables in question. This will give you a wide date range of data to work with (more than just 5 days) for better trending information. I've attached a zip file which contains all of this data. Can you let me know whether it's possible to just filter out the extraneous data yourself (in excel or by importing into a DB)? This will save Amy and GK some time and effort and also ensure you have data to work with quickly to move forward. Thanks, Aliya P.S. I have copied the Mifos Developer list on this email since we should start involving the community in our discussions around this topic. ________________________________ From: Amy Bensinger (Contractor) Sent: Tuesday, January 29, 2008 2:08 AM To: Biren Patnik Cc: Aliya Walji; Mifos dev; [EMAIL PROTECTED]; Van Mittal-Henkle; mifos dev Subject: RE: Database Growth Problem Hi, Biren. I can write the query to include only those tables you need. I am available in Skype all day, but I am at the GK head office, and connectivity is intermittent. ________________________________ From: Biren Patnik [mailto:[EMAIL PROTECTED] Sent: Tuesday, January 29, 2008 3:22 PM To: Amy Bensinger (Contractor) Cc: Aliya Walji; Mifos dev; [EMAIL PROTECTED]; Van Mittal-Henkle; mifos dev Subject: RE: Database Growth Problem Hi Amy, Thanks for your mail. We need the figure only for those tables I had mentioned in the excel sheet. We don't need data for all tables. It would be good if we can connect to those systems remotely by VPN. Otherwise we need some contact point in GK who can do this or anybody(sys admin) who had already done this. Can we talk these issues today in skype when you are free? Please let me know about this. Thanks and Kind Regards Biren Patnaik Lead Software Developer IBM Technology Campus Damestown Industrial Park,Mulhuddart Dublin 15 Internal Tel: 511211 External Tel : (+353) - 1 - 815-1211 Email: [EMAIL PROTECTED] "Amy Bensinger (Contractor)" <[EMAIL PROTECTED]> 29/01/2008 07:20 To "Aliya Walji" <[EMAIL PROTECTED]>, Biren Patnik/Ireland/[EMAIL PROTECTED] cc "Mifos dev" <[EMAIL PROTECTED]>, "Van Mittal-Henkle" <[EMAIL PROTECTED]>, <[EMAIL PROTECTED]> Subject RE: Database Growth Problem Hi, Biren. I originally was sending you a full list of show table stats (the same information as you have provided UI instructions for). Would it be all right if I wrote a query for the specific tables you want and have it dumped into a delimited file? GK's generally short staffed, so any activities we can automate for them is helpful. Also, the full table stats for the past month *should* be available (log files taken three times per day, as discussed in thread in December). Do you want that data, too? Thanks! ________________________________ From: Aliya Walji Sent: Tuesday, January 29, 2008 12:21 AM To: Biren Patnik Cc: Mifos dev; Van Mittal-Henkle; [EMAIL PROTECTED]; Amy Bensinger (Contractor) Subject: RE: Database Growth Problem Thanks Biren. I'll figure out how to get this information from GK as soon as possible and pass along to you again. I'll see if I can also get this information from one of our other deployments so we can see how things compare for a small MFI versus a larger one like GK. One thing that you could do in the mean time would be to potentially perform some specific tasks on the site and see how this affects the size of the tables. The tasks that I can think that would be performed on a regular basis that would impact those tables are the following: * Bulk entry for the centers that have meetings on a particular day * Creation of new loan accounts I think it might make sense for you to start with the first task with the GK DB/application and pick one branch, and try doing bulk entry for each of the centers that have a meeting on a particular day, and then see how that affects those tables. Since this is one of the most frequently performed tasks in Mifos (if not the most frequent), my guess is that it might shed some light on things (or help narrow things down). I know that going about things in this way might not be the most efficient way, but I thought it could keep you moving while you wait for the data from Grameen Koota. Let me know your thoughts on whether that might be a reasonable approach to move forward (and certainly, if you have any questions about how to use the site to do bulk entry, I'd be happy to answer). In the mean time, I'll work on getting you the stats from the DB. Thanks, Aliya ________________________________ From: Biren Patnik [mailto:[EMAIL PROTECTED] Sent: Monday, January 28, 2008 4:26 AM To: Aliya Walji Cc: Mifos dev; Van Mittal-Henkle; [EMAIL PROTECTED] Subject: Database Growth Problem Hi Aliya, Thank you very much for sending the hard disk for analysing the data. We have found that 6 tables constitute 87% of total data length (17.4gb out of 19.9 gb of data length). We need further figure of these 6 tables for analysing more in detail. Currently we find difficulty understanding what causes the database growth. Unless we get information about all of these following tables, it will be very difficult for us to analyse properly. Tables that are affected are as below (This information is extracted from the hard disk production data). Table Name Total data length Total Rows 1)Account_trxn 2.2gb 21598509 2)Customer_schedule 1.6gb 16804882 3)financial_trxn 5.7gb 59328792 4)loan_activity_details 2.7gb 12905461 5)loan_schedule 3.5gb 15449897 6)loan_trxn_detail 1.7gb 20239053 I have attached the excel spreadsheet which needs to be filled out with Date, Total Rows and Total Data length. The data needs to be filled out at particular time on each day. The best timing is in the evening when the office is closed. I have also attached a word document about how to get information about those tables. Please follow the instructions. Thanks and Kind Regards Biren Patnaik Lead Software Developer IBM Technology Campus Damestown Industrial Park,Mulhuddart Dublin 15 Internal Tel: 511211 External Tel : (+353) - 1 - 815-1211 Email: [EMAIL PROTECTED]
------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
