Normally I don't need big "messes".... but if you show us a sample
dataset (original data on the table) and a "required" resultset (data
in the format you want) at least we can analyze it and tell you the
"ways" of doing that.
Just looking at your request I can tell that if you have gazillions of
rows and you're about to build something like an analytics app (e.g.
take the resultset and show me subtotals for month, week, day or by
employee name) the best you can achieve is definitely from defining a
view right into the database using windowing functions. You can then
define_table('name_of_the_view', migrate=False) and use DAL to read
that table.
With an "ideal" db structure (references, indexed columns, etc) and
multiple DAL queries you can definitely achieve some kind of
performance (the kind of "let's not let the user take a coffee on each
page reload").
It's really only a matter of the things you want to achieve with the
things you've got, and finding the right balance from
ultraparametrized queries (better developed using multiple DAL
queries) or controlled-set-of-operations (meaning you don't have to
smash your head to the wall to achieve that behaviour in one single
view).
