[
https://issues.apache.org/jira/browse/BEAM-8365?focusedWorklogId=328725&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-328725
]
ASF GitHub Bot logged work on BEAM-8365:
----------------------------------------
Author: ASF GitHub Bot
Created on: 15/Oct/19 18:26
Start Date: 15/Oct/19 18:26
Worklog Time Spent: 10m
Work Description: apilloud commented on pull request #9764: [BEAM-8365]
Project push-down for TestTableProvider
URL: https://github.com/apache/beam/pull/9764#discussion_r335092961
##########
File path:
sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamIOSourceRel.java
##########
@@ -109,10 +137,11 @@ public RelOptCost computeSelfCost(RelOptPlanner planner,
RelMetadataQuery mq) {
@Override
public BeamCostModel beamComputeSelfCost(RelOptPlanner planner,
RelMetadataQuery mq) {
NodeStats estimates = BeamSqlRelUtils.getNodeStats(this, mq);
- return BeamCostModel.FACTORY.makeCost(estimates.getRowCount(),
estimates.getRate());
+ return BeamCostModel.FACTORY.makeCost(
+ estimates.getRowCount() * getRowType().getFieldCount(),
estimates.getRate());
}
Review comment:
This is a good question. This needs to apply to the rate as well. I think
this should probably be passed into `makeCost` as `dIo` and factored into the
cost model. See here:
https://github.com/apache/beam/blob/031b3789c4191bc82d0e97f4cabd0ccbee6c9902/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/planner/BeamCostModel.java#L121
If just multiplying it into the sum in `getCostCombination` passes all the
tests that is probably good enough for now. I would expect we actually want it
to be a smaller factor but we can figure that out later.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 328725)
Time Spent: 4h 40m (was: 4.5h)
> Add project push-down capability to IO APIs
> -------------------------------------------
>
> Key: BEAM-8365
> URL: https://issues.apache.org/jira/browse/BEAM-8365
> Project: Beam
> Issue Type: New Feature
> Components: dsl-sql
> Reporter: Kirill Kozlov
> Assignee: Kirill Kozlov
> Priority: Major
> Time Spent: 4h 40m
> Remaining Estimate: 0h
>
> * InMemoryTable should implement a following method:
> {code:java}
> public PCollection<Row> buildIOReader(
> PBegin begin, BeamSqlTableFilter filters, List<String> fieldNames);{code}
> Which should return a `PCollection` with fields specified in `fieldNames`
> list.
> * Create a rule to push fields used by a Calc (in projects and in a
> condition) down into TestTable IO.
> * Updating that same Calc (from previous step) to have a proper input and
> output schemes, remove unused fields.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
