[
https://issues.apache.org/jira/browse/CALCITE-4551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17308344#comment-17308344
]
James Starr edited comment on CALCITE-4551 at 3/25/21, 4:27 AM:
----------------------------------------------------------------
I modified RelMetadataTest, removing calls to THREAD_LIST
{code:java}
@Test void testPerformance() {
final String sql = "select deptno, count(*) from emp where deptno > 10 "
+ "group by deptno having count(*) = 0";
final RelRoot root = tester
.withClusterFactory(cluster -> {
// Create a custom provider that includes ColType.
// Include the same provider twice just to be devious.
final ImmutableList<RelMetadataProvider> list =
ImmutableList.of(ColTypeImpl.SOURCE, ColTypeImpl.SOURCE,
cluster.getMetadataProvider());
cluster.setMetadataProvider(
ChainedRelMetadataProvider.of(list));
return cluster;
})
.convertSqlToRel(sql);
final RelNode rel = root.rel;
ColType.Handler handler =
JaninoMetadataHandlerCreator.newInstance(ColType.Handler.class,
ImmutableSet.of(new ColTypeImpl()));
//JaninoRelMetadataProvider.of(ColTypeImpl.SOURCE).(null, ColType.DEF);
for (int i = 0; i < 100; i++) {
long start = System.currentTimeMillis();
getColumnType(rel, handler);
System.out.println(System.currentTimeMillis() - start);
}
}
private final String deptRel = "DEPTNO-rel";
private final String exprRel = "EXPR$1-rel";
private final String deptAgg = "DEPTNO-agg";
private void getColumnType (RelNode relNode, ColType.Handler handler) {
relNode.getCluster().invalidateMetadataQuery();
RelMetadataQuery relMetadataQuery = relNode.getCluster().getMetadataQuery();
for (int i = 0; i < 1_000_000; i++) {
assert handler.getColType(relNode, relMetadataQuery, 0).equals(deptRel);
assert handler.getColType(relNode, relMetadataQuery, 1).equals(exprRel);
final RelNode input = relNode.getInput(0);
assert handler.getColType(input, relMetadataQuery,0).equals(deptAgg);
}
}{code}
|Key |String|FlattenList(Object, Integer)|Object|FlattenList(Object)|
FlattenList(Object, Integer)|
| Fly Wheel|Yes |No|Yes |Yes |Yes |
|Average|97.94|119.55|91.34|99.03|104.9|
|Standard Deviation|11.75062431|11.04844703|8.150379554|9.235914747|11.39865312|
[~julianhyde], even if the JVM is successfully create the object on the stack,
which I am fairly doubtful of, the object creation is expensive. I am still
getting a 25% improvement on metadata calls one way or the other.
was (Author: jamesstarr):
I modified RelMetadataTest, removing calls to THREAD_LIST
{code:java}
@Test void testPerformance() {
final String sql = "select deptno, count(*) from emp where deptno > 10 "
+ "group by deptno having count(*) = 0";
final RelRoot root = tester
.withClusterFactory(cluster -> {
// Create a custom provider that includes ColType.
// Include the same provider twice just to be devious.
final ImmutableList<RelMetadataProvider> list =
ImmutableList.of(ColTypeImpl.SOURCE, ColTypeImpl.SOURCE,
cluster.getMetadataProvider());
cluster.setMetadataProvider(
ChainedRelMetadataProvider.of(list));
return cluster;
})
.convertSqlToRel(sql);
final RelNode rel = root.rel;
ColType.Handler handler =
JaninoMetadataHandlerCreator.newInstance(ColType.Handler.class,
ImmutableSet.of(new ColTypeImpl()));
//JaninoRelMetadataProvider.of(ColTypeImpl.SOURCE).(null, ColType.DEF);
for (int i = 0; i < 100; i++) {
long start = System.currentTimeMillis();
getColumnType(rel, handler);
System.out.println(System.currentTimeMillis() - start);
}
}
private final String deptRel = "DEPTNO-rel";
private final String exprRel = "EXPR$1-rel";
private final String deptAgg = "DEPTNO-agg";
private void getColumnType (RelNode relNode, ColType.Handler handler) {
relNode.getCluster().invalidateMetadataQuery();
RelMetadataQuery relMetadataQuery = relNode.getCluster().getMetadataQuery();
for (int i = 0; i < 1_000_000; i++) {
handler.getColType(relNode, relMetadataQuery, 0);
handler.getColType(relNode, relMetadataQuery, 1);
final RelNode input = relNode.getInput(0);
handler.getColType(input, relMetadataQuery,0);
}
}{code}
|Key |String|FlattenList(Object, Integer)|Object|FlattenList(Object)|
FlattenList(Object, Integer)|
| Fly Wheel|Yes |No|Yes |Yes |Yes |
|Average|97.94|119.55|91.34|99.03|104.9|
|Standard Deviation|11.75062431|11.04844703|8.150379554|9.235914747|11.39865312|
[~julianhyde], even if the JVM is successfully create the object on the stack,
which I am fairly doubtful of, the object creation is expensive. I am still
getting a 25% improvement on metadata calls one way or the other.
> Fly Weight for MD Cache Keys
> ----------------------------
>
> Key: CALCITE-4551
> URL: https://issues.apache.org/jira/browse/CALCITE-4551
> Project: Calcite
> Issue Type: Improvement
> Reporter: James Starr
> Priority: Major
>
> Create cache keys for metadata calls generates a fair bit of object turn in
> trivial cases, more expensive than the actual metadata call. Many metadata
> calls could reuse their cache keys since the are functionally identical.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
