Hello Robin,
On Dec 7, 2011, at 5:15 PM, Robin Baumgartner wrote: > Hi > > We are currently facing some problems trying to generate > revenue statistics for a customer and we are stuck in some > kind of dilemma: > > Generating them using the Tryton model layer suffers from > performance issues (possibly due to the numerous dependencies > to other models that must be resolved) and delegating the > heavy work to the Postgres database is nearly impossible due > to the fact that a lot of the relevant fields are not stored > in the database. This is a common dilemna. We were hit by this sometime back too. > > So I would like to ask: > > Are there some plans on improving the conditions for data > analysis, such as storing the values of Function fields in > the database? > > Does anyone have some experience with such statistics in > Tryton and would share some tips? Tryton as such is designed with data integrity in recording of transactions in mind rather than speedy data analysis. Performing analytics at reasonable speeds require denormalised data (Star Schema ?) and the tryton schemas are not good candidates for such purposes. Analytics/Statistics operations are almost always never performed on production/realtime databases and it is common practice to move data to a data warehouse [1], via an ETL [2] process. How to do ETL ? The schema for analysis really depends on waht you want to analyse and is different from customer to customer and project to project. This required data can be "easily extracted" out of tryton using web service (XMLRPC/NetRPC(old)/JSON RPC) [3] or using proteus [4] which exposes the models in a developer friendly active record pattern. If you have a lot of data, then you might want to directly use trytond as a module and read records in the same program where you do your ETL. [5] Such statistical data is also usually done on historical data, and the most recent date of such reports is usually 'yesterday' (ex:product wise sales until yesterday) and never last minute. So you could use the write_date and create_date (coalesce) created by the tryton ORM automatically to find the recent records (Note: all records created by tryton models have this info). [1] http://en.wikipedia.org/wiki/Data_warehouse [2] http://en.wikipedia.org/wiki/Extract,_transform,_load [3] http://code.google.com/p/tryton/wiki/RemoteCalls [4] http://pypi.python.org/pypi/proteus/2.2.0 [5] http://code.google.com/p/tryton/wiki/HowToUseTrytondAsAModule Thanks Sharoon Thomas Director & CEO, Openlabs Technologies & Consulting Pvt Limited Regd Office: 2J-Skyline Daffodil, Trippunithura - Kochi - IN 682013 Mobile: +1 786 247 1317 http://openlabs.co.in
signature.asc
Description: Message signed with OpenPGP using GPGMail
