> Hello Robin, > > On Dec 7, 2011, at 5:15 PM, Robin Baumgartner wrote: > > Hi > > We are currently facing some problems trying to generate > revenue statistics for a customer and we are stuck in some > kind of dilemma: > > Generating them using the Tryton model layer suffers from > performance issues (possibly due to the numerous dependencies > to other models that must be resolved) and delegating the > heavy work to the Postgres database is nearly impossible due > to the fact that a lot of the relevant fields are not stored > in the database. > > This is a common dilemna. We were hit by this sometime back too. > > So I would like to ask: > > Are there some plans on improving the conditions for data > analysis, such as storing the values of Function fields in > the database? > > Does anyone have some experience with such statistics in > Tryton and would share some tips? > > Tryton as such is designed with data integrity in recording of > transactions in mind rather > than speedy data analysis. Performing analytics at reasonable speeds > require denormalised > data (Star Schema ?) and the tryton schemas are not good candidates > for such purposes. > > Analytics/Statistics operations are almost always never performed on > production/realtime > databases and it is common practice to move data to a data warehouse > [1], via an ETL [2] process. > > How to do ETL ? > > > The schema for analysis really depends on waht you want to analyse > and is different from customer > to customer and project to project. > > > This required data can be "easily extracted" out of tryton using web > service > (XMLRPC/NetRPC(old)/JSON RPC) [3] or using proteus [4] which exposes > the models in a developer > friendly active record pattern. If you have a lot of data, then you > might want to directly use trytond as > a module and read records in the same program where you do your ETL. > [5] > > > Such statistical data is also usually done on historical data, and > the most recent date of such reports > is usually 'yesterday' (ex:product wise sales until yesterday) and > never last minute. So you could use > the write_date and create_date (coalesce) created by the tryton ORM > automatically to find the recent > records (Note: all records created by tryton models have this info). > > > [1] http://en.wikipedia.org/wiki/Data_warehouse > [2] http://en.wikipedia.org/wiki/Extract,_transform,_load > [3] http://code.google.com/p/tryton/wiki/RemoteCalls > [4] http://pypi.python.org/pypi/proteus/2.2.0 > [5] http://code.google.com/p/tryton/wiki/HowToUseTrytondAsAModule > > > Thanks >
Well, thank you for this very interessting read! We will certainly take these proposals into account. Best regards Robin Baumgartner Member of Delivery & Production division Open Work Place AG · Habsburgerstrasse 23 · CH-6003 Luzern T +41 41 226 50 93 [email protected] · www.openworkplace.ch -- [email protected] mailing list
