[
http://mifosforge.jira.com/browse/MIFOSBI-119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
johnwoodlock updated MIFOSBI-119:
---------------------------------
Story Points: 3 (was: 0.5)
Description:
This is a placeholder/reminder to revisit once we are comfortable with the DW
and the reports running on it.
Current DW tables are 'InnoDB' (like mifos production schema). This should
give reasonable performance for mid size MFIs for quite some time e.g. Secdep
and smaller. As data volumes increase... we can still get boosts by tuning
various options including queries, indexes, by denormalising in the ETL and by
running reports out of hours. All valid but do involve work.
MyIsam was the other MySQL engine option but in my testing didn't perform
better than InnoDB as long as reasonable memory was allocated to InnoDB buffer.
Infobright (infobright.com) is a column database which is more geared to data
warehousing type use (expected to be factors quicker). Also it doesn't require
much admin mainenance (like indexes because there aren't any) I think we
should look into it for big MFI's like GK rather than doing a lot of 'summary
tables' (aggregates) which is the normal route if time becomes a major issue.
There are some things to ponder with ICE the community edition of InfoBright
e.g. you can't do insert/update/delete sql statements. This seems like a
show-stopper but not really as long as we can add to the main fact tables using
the ETL infobright bulk loading plugin.
Also there is an Infobright/Pentaho integration
http://www.infobright.com/Event/infobright_and_pentaho_deliver_virtual_machine_that_integrates_open_source_/
but I'm not sure if this is more useful to us than just installing the
MySQL/Infobright combo by itself (would need sys admin view).
was:Run Digamber data through the ETL process to see if anything needs to be
done prior to creating a data warehouse and scheduling the ETL.
Priority: Major (was: Minor)
Affects Version/s: (was: Mifos BI 1.0 BETA - Iteration 12)
(was: Mifos BI 1.0 BETA)
Fix Version/s: (was: Mifos BI 1.0 BETA - Iteration 12)
(was: Mifos BI 1.0 BETA)
Remaining Estimate: (was: 3 hours)
Original Estimate: (was: 3 hours)
> Spike to incorporate InfoBright column db into DW for Large MFI's
> -----------------------------------------------------------------
>
> Key: MIFOSBI-119
> URL: http://mifosforge.jira.com/browse/MIFOSBI-119
> Project: mifos business intelligence
> Issue Type: Story
> Components: Data Warehouse
> Reporter: johnwoodlock
> Assignee: johnwoodlock
> Priority: Major
>
> This is a placeholder/reminder to revisit once we are comfortable with the DW
> and the reports running on it.
> Current DW tables are 'InnoDB' (like mifos production schema). This should
> give reasonable performance for mid size MFIs for quite some time e.g. Secdep
> and smaller. As data volumes increase... we can still get boosts by tuning
> various options including queries, indexes, by denormalising in the ETL and
> by running reports out of hours. All valid but do involve work.
> MyIsam was the other MySQL engine option but in my testing didn't perform
> better than InnoDB as long as reasonable memory was allocated to InnoDB
> buffer.
> Infobright (infobright.com) is a column database which is more geared to data
> warehousing type use (expected to be factors quicker). Also it doesn't
> require much admin mainenance (like indexes because there aren't any) I
> think we should look into it for big MFI's like GK rather than doing a lot of
> 'summary tables' (aggregates) which is the normal route if time becomes a
> major issue.
> There are some things to ponder with ICE the community edition of InfoBright
> e.g. you can't do insert/update/delete sql statements. This seems like a
> show-stopper but not really as long as we can add to the main fact tables
> using the ETL infobright bulk loading plugin.
> Also there is an Infobright/Pentaho integration
> http://www.infobright.com/Event/infobright_and_pentaho_deliver_virtual_machine_that_integrates_open_source_/
> but I'm not sure if this is more useful to us than just installing the
> MySQL/Infobright combo by itself (would need sys admin view).
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://mifosforge.jira.com/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
------------------------------------------------------------------------------
Increase Visibility of Your 3D Game App & Earn a Chance To Win $500!
Tap into the largest installed PC base & get more eyes on your game by
optimizing for Intel(R) Graphics Technology. Get started today with the
Intel(R) Software Partner Program. Five $500 cash prizes are up for grabs.
http://p.sf.net/sfu/intelisp-dev2dev
_______________________________________________
Mifos-issues mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/mifos-issues
