[jira] [Commented] (CALCITE-1288) Avoid doing the same join twice if count(distinct) exists
[
https://issues.apache.org/jira/browse/CALCITE-1288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15372104#comment-15372104
]
Gautam Kumar Parai commented on CALCITE-1288:
-
[~julianhyde] Yes, I will start working on it. I was caught up with other
issues so did not get a chance earlier.
> Avoid doing the same join twice if count(distinct) exists
> -
>
> Key: CALCITE-1288
> URL: https://issues.apache.org/jira/browse/CALCITE-1288
> Project: Calcite
> Issue Type: Improvement
>Reporter: Gautam Kumar Parai
>Assignee: Gautam Kumar Parai
>
> When the query has one distinct aggregate and one or more non-distinct
> aggregates, the join instance need not produce the join-based plan. We can
> generate multi-phase aggregates. Another approach would be to use grouping
> sets. However, this transformation will be useful when systems don't support
> grouping sets and instead rely on the join-based plans (see the plan below)
> {code}
> select emp.empno, count(*), avg(distinct dept.deptno)
> from sales.emp emp inner join sales.dept dept
> on emp.deptno = dept.deptno
> group by emp.empno
> LogicalProject(EMPNO=[$0], EXPR$1=[$1], EXPR$2=[$3])
> LogicalJoin(condition=[IS NOT DISTINCT FROM($0, $2)], joinType=[inner])
> LogicalAggregate(group=[{0}], EXPR$1=[COUNT()])
> LogicalProject(EMPNO=[$0], DEPTNO0=[$9])
> LogicalJoin(condition=[=($7, $9)], joinType=[inner])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> LogicalAggregate(group=[{0}], EXPR$2=[AVG($1)])
> LogicalAggregate(group=[{0, 1}])
> LogicalProject(EMPNO=[$0], DEPTNO0=[$9])
> LogicalJoin(condition=[=($7, $9)], joinType=[inner])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> {code}
> The more efficient form should look like
> {code}
> select emp.empno, count(*), avg(distinct dept.deptno)
> from sales.emp emp inner join sales.dept dept
> on emp.deptno = dept.deptno
> group by emp.empno
> LogicalAggregate(group=[{0}], EXPR$1=[SUM($2)], EXPR$2=[AVG($1)])
> LogicalAggregate(group=[{0, 1}], EXPR$1=[COUNT()])
> LogicalProject(EMPNO=[$0], DEPTNO0=[$9])
> LogicalJoin(condition=[=($7, $9)], joinType=[inner])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (CALCITE-1313) Validator should derive type of expression in ORDER BY
[
https://issues.apache.org/jira/browse/CALCITE-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Hyde resolved CALCITE-1313.
--
Resolution: Fixed
Fix Version/s: 1.9.0
Fixed in http://git-wip-us.apache.org/repos/asf/calcite/commit/b4f28d7e.
[~minjikim], thanks for the PR!
> Validator should derive type of expression in ORDER BY
> --
>
> Key: CALCITE-1313
> URL: https://issues.apache.org/jira/browse/CALCITE-1313
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: MinJi Kim
>Assignee: Julian Hyde
> Fix For: 1.9.0
>
>
> SqlValidator.expandOrderExpr() does not validate node type and properly set
> it. Queries like the following currently fail in SqlToRelConverter because
> the type in thought to be unknown. Simple fix is to ensure that the type is
> validated when we expand the expression.
> {code}
> java.lang.AssertionError: Parsing failed throwing error: class
> org.apache.calcite.sql.SqlBasicCall: ROW_NUMBER() OVER (PARTITION BY
> `employee`.`store_id` ORDER BY `employee`.`department_id` ROWS BETWEEN
> UNBOUNDED PRECEDING AND CURRENT ROW)
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at
> org.apache.calcite.rel.rel2sql.RelToSqlConverterTest.checkRel2Sql(RelToSqlConverterTest.java:58)
> at
> org.apache.calcite.rel.rel2sql.RelToSqlConverterTest.checkRel2Sql(RelToSqlConverterTest.java:63)
> at
> org.apache.calcite.rel.rel2sql.RelToSqlConverterTest.testOver(RelToSqlConverterTest.java:391)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69)
> at
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:234)
> at
> com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:74)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CALCITE-1313) Validator should derive type of expression in ORDER BY
[
https://issues.apache.org/jira/browse/CALCITE-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Hyde updated CALCITE-1313:
-
Summary: Validator should derive type of expression in ORDER BY (was:
expandOrderExpr() does not validate node type and properly set it)
> Validator should derive type of expression in ORDER BY
> --
>
> Key: CALCITE-1313
> URL: https://issues.apache.org/jira/browse/CALCITE-1313
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: MinJi Kim
>Assignee: Julian Hyde
>
> Queries like the following currently fail in SqlToRelConverter because the
> type in thought to be unknown. Simple fix is to ensure that the type is
> validated when we expand the expression.
> {code}
> java.lang.AssertionError: Parsing failed throwing error: class
> org.apache.calcite.sql.SqlBasicCall: ROW_NUMBER() OVER (PARTITION BY
> `employee`.`store_id` ORDER BY `employee`.`department_id` ROWS BETWEEN
> UNBOUNDED PRECEDING AND CURRENT ROW)
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at
> org.apache.calcite.rel.rel2sql.RelToSqlConverterTest.checkRel2Sql(RelToSqlConverterTest.java:58)
> at
> org.apache.calcite.rel.rel2sql.RelToSqlConverterTest.checkRel2Sql(RelToSqlConverterTest.java:63)
> at
> org.apache.calcite.rel.rel2sql.RelToSqlConverterTest.testOver(RelToSqlConverterTest.java:391)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69)
> at
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:234)
> at
> com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:74)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CALCITE-1313) expandOrderExpr() does not validate node type and properly set it
[
https://issues.apache.org/jira/browse/CALCITE-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371949#comment-15371949
]
MinJi Kim commented on CALCITE-1313:
Thanks for the review! I moved the test case to SqlToRelConverterTest.
> expandOrderExpr() does not validate node type and properly set it
> -
>
> Key: CALCITE-1313
> URL: https://issues.apache.org/jira/browse/CALCITE-1313
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: MinJi Kim
>Assignee: Julian Hyde
>
> Queries like the following currently fail in SqlToRelConverter because the
> type in thought to be unknown. Simple fix is to ensure that the type is
> validated when we expand the expression.
> {code}
> java.lang.AssertionError: Parsing failed throwing error: class
> org.apache.calcite.sql.SqlBasicCall: ROW_NUMBER() OVER (PARTITION BY
> `employee`.`store_id` ORDER BY `employee`.`department_id` ROWS BETWEEN
> UNBOUNDED PRECEDING AND CURRENT ROW)
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at
> org.apache.calcite.rel.rel2sql.RelToSqlConverterTest.checkRel2Sql(RelToSqlConverterTest.java:58)
> at
> org.apache.calcite.rel.rel2sql.RelToSqlConverterTest.checkRel2Sql(RelToSqlConverterTest.java:63)
> at
> org.apache.calcite.rel.rel2sql.RelToSqlConverterTest.testOver(RelToSqlConverterTest.java:391)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69)
> at
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:234)
> at
> com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:74)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CALCITE-1313) expandOrderExpr() does not validate node type and properly set it
[
https://issues.apache.org/jira/browse/CALCITE-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371824#comment-15371824
]
Julian Hyde commented on CALCITE-1313:
--
That test is using RelToSql not SqlToRel but yes, the validator should test
type when it expands. Add that query to SqlToRelConverterTest and you'll kill
two birds with one stone.
> expandOrderExpr() does not validate node type and properly set it
> -
>
> Key: CALCITE-1313
> URL: https://issues.apache.org/jira/browse/CALCITE-1313
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: MinJi Kim
>Assignee: Julian Hyde
>
> Queries like the following currently fail in SqlToRelConverter because the
> type in thought to be unknown. Simple fix is to ensure that the type is
> validated when we expand the expression.
> {code}
> java.lang.AssertionError: Parsing failed throwing error: class
> org.apache.calcite.sql.SqlBasicCall: ROW_NUMBER() OVER (PARTITION BY
> `employee`.`store_id` ORDER BY `employee`.`department_id` ROWS BETWEEN
> UNBOUNDED PRECEDING AND CURRENT ROW)
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at
> org.apache.calcite.rel.rel2sql.RelToSqlConverterTest.checkRel2Sql(RelToSqlConverterTest.java:58)
> at
> org.apache.calcite.rel.rel2sql.RelToSqlConverterTest.checkRel2Sql(RelToSqlConverterTest.java:63)
> at
> org.apache.calcite.rel.rel2sql.RelToSqlConverterTest.testOver(RelToSqlConverterTest.java:391)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69)
> at
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:234)
> at
> com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:74)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CALCITE-1313) expandOrderExpr() does not validate node type and properly set it
MinJi Kim created CALCITE-1313:
--
Summary: expandOrderExpr() does not validate node type and
properly set it
Key: CALCITE-1313
URL: https://issues.apache.org/jira/browse/CALCITE-1313
Project: Calcite
Issue Type: Bug
Components: core
Reporter: MinJi Kim
Assignee: Julian Hyde
Queries like the following currently fail in SqlToRelConverter because the type
in thought to be unknown. Simple fix is to ensure that the type is validated
when we expand the expression.
{code}
java.lang.AssertionError: Parsing failed throwing error: class
org.apache.calcite.sql.SqlBasicCall: ROW_NUMBER() OVER (PARTITION BY
`employee`.`store_id` ORDER BY `employee`.`department_id` ROWS BETWEEN
UNBOUNDED PRECEDING AND CURRENT ROW)
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at
org.apache.calcite.rel.rel2sql.RelToSqlConverterTest.checkRel2Sql(RelToSqlConverterTest.java:58)
at
org.apache.calcite.rel.rel2sql.RelToSqlConverterTest.checkRel2Sql(RelToSqlConverterTest.java:63)
at
org.apache.calcite.rel.rel2sql.RelToSqlConverterTest.testOver(RelToSqlConverterTest.java:391)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at
com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69)
at
com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:234)
at
com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:74)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CALCITE-1312) timestamp_add(UNIT, DATE) should return timestamp if the unit added is less than a date
[
https://issues.apache.org/jira/browse/CALCITE-1312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371677#comment-15371677
]
MinJi Kim commented on CALCITE-1312:
https://github.com/apache/calcite/pull/253
> timestamp_add(UNIT, DATE) should return timestamp if the unit added is less
> than a date
> ---
>
> Key: CALCITE-1312
> URL: https://issues.apache.org/jira/browse/CALCITE-1312
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: MinJi Kim
>Assignee: Julian Hyde
>
> timestamp_add("MINUTE", 1, date '2016-06-15') returns 2016-06-15 since it
> returns a date and therefore truncates the minute informtion. timestamp_add
> should return timestamp instead of date for units less than date.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CALCITE-1312) timestamp_add(UNIT, DATE) should return timestamp if the unit added is less than a date
MinJi Kim created CALCITE-1312:
--
Summary: timestamp_add(UNIT, DATE) should return timestamp if the
unit added is less than a date
Key: CALCITE-1312
URL: https://issues.apache.org/jira/browse/CALCITE-1312
Project: Calcite
Issue Type: Bug
Components: core
Reporter: MinJi Kim
Assignee: Julian Hyde
timestamp_add("MINUTE", 1, date '2016-06-15') returns 2016-06-15 since it
returns a date and therefore truncates the minute informtion. timestamp_add
should return timestamp instead of date for units less than date.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CALCITE-1311) Add high-level record of interactions with the avatica server
[ https://issues.apache.org/jira/browse/CALCITE-1311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371481#comment-15371481 ] Julian Hyde commented on CALCITE-1311: -- I'd like to see an argument made that this is the best/only place in the architecture where this can be done. > Add high-level record of interactions with the avatica server > - > > Key: CALCITE-1311 > URL: https://issues.apache.org/jira/browse/CALCITE-1311 > Project: Calcite > Issue Type: New Feature > Components: avatica >Reporter: Josh Elser > > It would be nice to have the ability to configure the Avatica server to > create what is equivalent to an "audit log". > This functionality would provide administrators the ability to inspect what > queries were run and the user (client address and optionally the Kerberos > identity). > It would be nice to implement this as a framework and provide an initial > "sink" binding which just writes to a file. This would let us push these logs > to other systems automatically which would be nice. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CALCITE-1311) Add high-level record of interactions with the avatica server
[ https://issues.apache.org/jira/browse/CALCITE-1311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371159#comment-15371159 ] Josh Elser commented on CALCITE-1311: - bq. It would very nice to have this, but is it scope creep? If the backend server already has auditing it wouldn't want Avatica doing it at another level. Good point. It might very well be duplicative of something that backend system is already doing. Definitely should be an optional thing (if built) bq. Are we talking about data auditing here (e.g. if there is a batch insert of 100 rows, write those 100 rows out somewhere)? bq. How does this capability compare with metrics? Events such as "connection created", "statement canceled" are similar to those that would be gathered by metrics. So, would auditing and metrics write to the same place, and would they use the same mechanism? For the read side, I think it's pretty straightforward to just log the SQL statement. For writes and metadata operations, I'm not sure what exactly these would look like (would we log the Avatica method call? The insert/update template assuming that it's a prepared statement?). How it fits in with metrics is also a concern to avoid two features potentially colliding. My initial thoughts would be that metrics are raw timing data for the Avatica calls whereas this would be a "user level" record of what happened. We would definitely need to make sure that the delineation between the two are clear. Definitely still hashing out what this would look like; more feedback/opinions are welcomed :) > Add high-level record of interactions with the avatica server > - > > Key: CALCITE-1311 > URL: https://issues.apache.org/jira/browse/CALCITE-1311 > Project: Calcite > Issue Type: New Feature > Components: avatica >Reporter: Josh Elser > > It would be nice to have the ability to configure the Avatica server to > create what is equivalent to an "audit log". > This functionality would provide administrators the ability to inspect what > queries were run and the user (client address and optionally the Kerberos > identity). > It would be nice to implement this as a framework and provide an initial > "sink" binding which just writes to a file. This would let us push these logs > to other systems automatically which would be nice. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CALCITE-1311) Add high-level record of interactions with the avatica server
[ https://issues.apache.org/jira/browse/CALCITE-1311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371122#comment-15371122 ] Julian Hyde edited comment on CALCITE-1311 at 7/11/16 4:45 PM: --- It would very nice to have this, but is it scope creep? If the backend server already has auditing it wouldn't want Avatica doing it at another level. Are we talking about data auditing here (e.g. if there is a batch insert of 100 rows, write those 100 rows out somewhere)? How does this capability compare with metrics? Events such as "connection created", "statement canceled" are similar to those that would be gathered by metrics. So, would auditing and metrics write to the same place, and would they use the same mechanism? was (Author: julianhyde): It would very nice to have this, but is it scope creep? If the backend server already has auditing it wouldn't want Avatica doing it at another level. > Add high-level record of interactions with the avatica server > - > > Key: CALCITE-1311 > URL: https://issues.apache.org/jira/browse/CALCITE-1311 > Project: Calcite > Issue Type: New Feature > Components: avatica >Reporter: Josh Elser > > It would be nice to have the ability to configure the Avatica server to > create what is equivalent to an "audit log". > This functionality would provide administrators the ability to inspect what > queries were run and the user (client address and optionally the Kerberos > identity). > It would be nice to implement this as a framework and provide an initial > "sink" binding which just writes to a file. This would let us push these logs > to other systems automatically which would be nice. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CALCITE-1311) Add high-level record of interactions with the avatica server
[ https://issues.apache.org/jira/browse/CALCITE-1311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371122#comment-15371122 ] Julian Hyde commented on CALCITE-1311: -- It would very nice to have this, but is it scope creep? If the backend server already has auditing it wouldn't want Avatica doing it at another level. > Add high-level record of interactions with the avatica server > - > > Key: CALCITE-1311 > URL: https://issues.apache.org/jira/browse/CALCITE-1311 > Project: Calcite > Issue Type: New Feature > Components: avatica >Reporter: Josh Elser > > It would be nice to have the ability to configure the Avatica server to > create what is equivalent to an "audit log". > This functionality would provide administrators the ability to inspect what > queries were run and the user (client address and optionally the Kerberos > identity). > It would be nice to implement this as a framework and provide an initial > "sink" binding which just writes to a file. This would let us push these logs > to other systems automatically which would be nice. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CALCITE-1311) Add high-level record of interactions with the avatica server
Josh Elser created CALCITE-1311: --- Summary: Add high-level record of interactions with the avatica server Key: CALCITE-1311 URL: https://issues.apache.org/jira/browse/CALCITE-1311 Project: Calcite Issue Type: New Feature Components: avatica Reporter: Josh Elser It would be nice to have the ability to configure the Avatica server to create what is equivalent to an "audit log". This functionality would provide administrators the ability to inspect what queries were run and the user (client address and optionally the Kerberos identity). It would be nice to implement this as a framework and provide an initial "sink" binding which just writes to a file. This would let us push these logs to other systems automatically which would be nice. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
