[
https://issues.apache.org/jira/browse/TORQUE-364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17835748#comment-17835748
]
Georg Kallidis commented on TORQUE-364:
---------------------------------------
Thanks! I think there might be multiple improvements possible here:
* The workaround you mentioned might be already correct, if narrowing down to
only select queries (I think processRow is used in selects only, updates might
be different) to shortcut and use the simpler (cheap) mapping. Seems alreay
fine to be included in an updated BaseRecordMapper..
* Prebuilding the sqlExpressions in static variables might be done quite
easily in Torque templates and the replacing part of the conditions in the
else-branch in processRow
* It might be of good help, if you could provide an example implementation of
preprocess(criteria, offset). Integration could be done either e.g. by an
additional argument or a new interface and from there on further ..
* The loop in the else branch could improved (it was designed with the if-else
conditional to allow a little bit of improved evaluation IMO) also to skip the
already checked condition by using e.g. a lookup hashset (keys might be the
column name or the full sql expression for the column). This way indeed a good
part of the redundant condition evaulations is removed (it should be evaluated
only once not columns times condition)
If I do understand this issue correctly, I might already continue by changing
the code considering the last and the first one. For the other improvement
options I'll wait for other's opinions and more information ..
> RecordMapper very slow on many columns in table
> -----------------------------------------------
>
> Key: TORQUE-364
> URL: https://issues.apache.org/jira/browse/TORQUE-364
> Project: Torque
> Issue Type: Improvement
> Components: Runtime, Templates
> Affects Versions: 5.1
> Reporter: Max Philipp Wriedt
> Priority: Major
> Labels: criteria_api, om, performance, recordmapper, templates
>
> When "doSelect()" a large quantity of columns in a table the default
> RecordMappers generated by Om-Templates (processRow()) cause an
> !https://wikimedia.org/api/rest_v1/media/math/render/svg/4441d9689c0e6b2c47994e2f587ac5378faeefba!
> Problem. (technically O(rows * columns))
> Specifically, constantly generating the SQL expression of all possible
> columns for every row in the result causes excessive use of StringBuilders
> which slow the mapping process to a crawl.
> I currently have two ideas on how best to tackle this problem:
> # Either generate all SQL column expressions once when a (template
> generated) RecordMapper is first created (using final static String fields)
> thus reducing the cost for every row to generating all selected column SQL
> expressions once(instead of every selected column times every available
> column)
> # Or (in case the first approach generates unacceptably excessive number of
> fields for RecordMappers) adjust the RecordMapper API to allow a
> "prepare(Criteria, int offset)" method to be called once before processing
> any rows and implement it on generated RecordMappers to scan the Criteria and
> build two lists: One containing references to the setXXX methods of the
> mapper in the order they appear in the ResultSet (via the order in the
> Criteria) and a second list containing the corresponding column offsets. This
> would allow the processRow method to only iterated over both lists
> simultaneously and call the referenced methods with the result set and offset
> immediately. (Alternatively one list using lambdas could be used but I am
> currently not sure about the stance or impact of these lambdas in the Torque
> project.)
> credits to [~refarb]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: torque-dev-unsubscr...@db.apache.org
For additional commands, e-mail: torque-dev-h...@db.apache.org
