[jira] [Commented] (CALCITE-5137) EnumerableUncollect throws NPE if input has ((List) null)
[
https://issues.apache.org/jira/browse/CALCITE-5137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17533410#comment-17533410
]
Dmitry Sysolyatin commented on CALCITE-5137:
[~nobigo] CI fails for the same reason as in the main branch.
> EnumerableUncollect throws NPE if input has ((List) null)
> -
>
> Key: CALCITE-5137
> URL: https://issues.apache.org/jira/browse/CALCITE-5137
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Dmitry Sysolyatin
>Assignee: Dmitry Sysolyatin
>Priority: Major
> Labels: pull-request-available
>
> EnumerableUncollect throws NPE when an input has element = ((List) null)
> Example:
> {code}
> SELECT * FROM UNNEST(CAST(null AS INTEGER ARRAY))
> {code}
> In a real situation, it can appear if to use left join. For example:
> {code}
> SELECT ARRAY(SELECT * FROM UNNEST(t.x)) FROM (VALUES(1)) LEFT JOIN (SELECT
> ARRAY[1] as x, 2 as y) t ON t.y = 1
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (CALCITE-1242) Allow configuration of maximum allowed value for maxRowCount
[ https://issues.apache.org/jira/browse/CALCITE-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Chuang updated CALCITE-1242: Fix Version/s: avatica-1.22.0 (was: avatica-1.21.0) > Allow configuration of maximum allowed value for maxRowCount > > > Key: CALCITE-1242 > URL: https://issues.apache.org/jira/browse/CALCITE-1242 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Duo Xu >Assignee: Josh Elser >Priority: Major > Fix For: avatica-1.22.0 > > > There are several APIs having the maxRowCount parameter. Currently this > setting is hard coded in Avatica as 100. So if the user set maxRowCount > 100 > in PrepareAndExecuteRequest, for example, the server will still only return > 100 results. So the APIs are currently broken. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (CALCITE-1316) Better control over retried operations in Avatica client
[ https://issues.apache.org/jira/browse/CALCITE-1316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Chuang updated CALCITE-1316: Fix Version/s: avatica-1.22.0 (was: avatica-1.21.0) > Better control over retried operations in Avatica client > > > Key: CALCITE-1316 > URL: https://issues.apache.org/jira/browse/CALCITE-1316 > Project: Calcite > Issue Type: Improvement > Components: avatica >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: avatica-1.22.0 > > > We have at least two places in the Avatica client now where we will try to > re-issue "RPCs" in the attempt to work seamlessly with load-balanced servers. > Between these two places, we have finite retries, infinite retries and no > standardized back-off strategies. We should try to centralize this into one > place and make sure that users can override the logic, heaven forbid they > come up with some situation where it's necessary. > Need to investigate the retry-loops we have in the Avatica client now, > categorize the loops and come up with a minimal set of knobs to configure the > retries, expose those knobs in the JDBC URL string options, and update the > documentation. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (CALCITE-1811) Work around "latest" tag for Docker images
[
https://issues.apache.org/jira/browse/CALCITE-1811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francis Chuang updated CALCITE-1811:
Fix Version/s: avatica-1.22.0
(was: avatica-1.21.0)
> Work around "latest" tag for Docker images
> --
>
> Key: CALCITE-1811
> URL: https://issues.apache.org/jira/browse/CALCITE-1811
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Affects Versions: avatica-1.10.0
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Minor
> Fix For: avatica-1.22.0
>
>
> One wrinkle with the dockerhub integration with the ASF is that we only get a
> tag for the explicit version we released (e.g. 1.10.0).
> However, in the other Dockerfiles we have in the 1.10.0 release, I had
> (incorrectly) relied on that. Need to come up with a plan for how to better
> manage this in the future. I think that we need to just have {{FROM
> apache/calcite-avatica:$tag}} everywhere, relying on the parent eventually
> being published to dockerhub by the ASF infra.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (CALCITE-839) Remove Jackson annotations from POJO classes in Meta
[ https://issues.apache.org/jira/browse/CALCITE-839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Chuang updated CALCITE-839: --- Fix Version/s: avatica-1.22.0 (was: avatica-1.21.0) > Remove Jackson annotations from POJO classes in Meta > > > Key: CALCITE-839 > URL: https://issues.apache.org/jira/browse/CALCITE-839 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Julian Hyde >Assignee: Josh Elser >Priority: Major > Fix For: avatica-1.22.0 > > > The Meta interface contains several POJO classes that represent RPC requests > or responses. Currently a few of those classes have Jackson annotations such > as @JsonCreator, @JsonProperty to help Jackson serialize the POJO to JSON and > de-serialize from JSON to the object. > As [~ndimiduk] pointed out in > http://mail-archives.apache.org/mod_mbox/calcite-dev/201503.mbox/%3CCANZa=gvkgd+bkj4+ejmuo6ivhs+okgskg1vwdazcy-zijyy...@mail.gmail.com%3E > these annotations are a "code smell" and should be removed. It makes it look > as if Jackson is the only possible transport, which is not the case. We can > continue to use Jackson as a transport, just specify the mappings elsewhere, > not as annotations. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (CALCITE-1308) Implement remaining DatabaseMetaData operations in RemoteMeta
[ https://issues.apache.org/jira/browse/CALCITE-1308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Chuang updated CALCITE-1308: Fix Version/s: avatica-1.22.0 (was: avatica-1.21.0) > Implement remaining DatabaseMetaData operations in RemoteMeta > - > > Key: CALCITE-1308 > URL: https://issues.apache.org/jira/browse/CALCITE-1308 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Critical > Fix For: avatica-1.22.0 > > > Noticed in CALCITE-1291: sqlline normally highlights the column(s) which are > (part of the) primary key. Running a debugger over it quickly, showed that no > keys were returned over the DatabaseMetaData.getPrimaryKeys call. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (CALCITE-1928) Calling previous() on a ResultSet that came from an Array gives NullPointerException
[ https://issues.apache.org/jira/browse/CALCITE-1928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Chuang updated CALCITE-1928: Fix Version/s: avatica-1.22.0 (was: avatica-1.21.0) > Calling previous() on a ResultSet that came from an Array gives > NullPointerException > > > Key: CALCITE-1928 > URL: https://issues.apache.org/jira/browse/CALCITE-1928 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Julian Hyde >Assignee: Julian Hyde >Priority: Major > Fix For: avatica-1.22.0 > > > Calling previous() on a ResultSet that came from an Array gives > NullPointerException. I broke this when I committed CALCITE-1902 and changed > 'Helper.INSTANCE' to 'statement.connection.helper', forgetting that > 'statement' can be null. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (CALCITE-1286) Create self-contained test-harness for TCK
[ https://issues.apache.org/jira/browse/CALCITE-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Chuang updated CALCITE-1286: Fix Version/s: avatica-1.22.0 (was: avatica-1.21.0) > Create self-contained test-harness for TCK > -- > > Key: CALCITE-1286 > URL: https://issues.apache.org/jira/browse/CALCITE-1286 > Project: Calcite > Issue Type: Improvement > Components: avatica >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: avatica-1.22.0 > > > Should make a Vagrant VM or a Docker image which is capable of automatically > running the Avatica TCK. > Ideally, running the TCK should be as simple as a single command. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (CALCITE-1352) Clarify documentation for avatica's max_rows_total
[
https://issues.apache.org/jira/browse/CALCITE-1352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francis Chuang updated CALCITE-1352:
Fix Version/s: avatica-1.22.0
(was: avatica-1.21.0)
> Clarify documentation for avatica's max_rows_total
> --
>
> Key: CALCITE-1352
> URL: https://issues.apache.org/jira/browse/CALCITE-1352
> Project: Calcite
> Issue Type: Improvement
> Components: avatica
>Affects Versions: avatica-1.8.0
>Reporter: Francis Chuang
>Priority: Minor
> Fix For: avatica-1.22.0
>
>
> The {{max_rows_total}} parameter in the protocol buffer definitions should be
> updated to include more information on values that result in unlimited rows
> being returned.
> When testing against calcite 1.8.0, I observed the following:
> {code}
> -1: Unlimited number of rows
> 0: 0 rows
> 1: 1 row
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (CALCITE-1318) Build/Document Avatica Kerberos/SPNEGO authentication behind load balancer.
[ https://issues.apache.org/jira/browse/CALCITE-1318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Chuang updated CALCITE-1318: Fix Version/s: avatica-1.22.0 (was: avatica-1.21.0) > Build/Document Avatica Kerberos/SPNEGO authentication behind load balancer. > --- > > Key: CALCITE-1318 > URL: https://issues.apache.org/jira/browse/CALCITE-1318 > Project: Calcite > Issue Type: Improvement > Components: avatica >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: avatica-1.22.0 > > > I was talking with [~_alexm] this weekend about some work that he had > recently done in getting Apache Impala set up behind a load balancer. When > Kerberos is in the picture, he told me that the way that works is that the > impalad daemons actually have two Kerberos identities: one for the hostname > that the impalad service is actually running on and another for the load > balancer host. The load-balancer continues to just do a simple pass-through. > Right now, the Avatica server can only accept a single Kerberos > principal+keytab. This means that we can't use the Kerberos authentication > when the client can access the server via multiple hostnames -- invalidating > the use of 'dumb' load balancers (hypothetically, a smart loadbalancer could > make it work). We could configure the Avatica server to use a principal with > the load-balancer's hostname, but then users would be unable to connect > directly to the server. > I know that Impala uses (or at least exposes) Thrift which has its own SASL > implementation; maybe they do something tricky there? Maybe we can glean > something from their implementation (even though it's not HTTP based). I > don't think JAAS lets us have multiple active logins, so I'm not even sure > where to begin. > Ideally, this is something that would be great to understand and provide some > deployment guidance for users to have identical deployment scenario for > "secure" and "unsecure" scenarios. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (CALCITE-1385) Support password-based convenience logins for Kerberos-authenticated clients
[ https://issues.apache.org/jira/browse/CALCITE-1385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Chuang updated CALCITE-1385: Fix Version/s: avatica-1.22.0 (was: avatica-1.21.0) > Support password-based convenience logins for Kerberos-authenticated clients > > > Key: CALCITE-1385 > URL: https://issues.apache.org/jira/browse/CALCITE-1385 > Project: Calcite > Issue Type: Improvement > Components: avatica >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: avatica-1.22.0 > > > Had a request for the automatic Kerberos login process to also support > password-based authentication (instead of just keytab). > This would require some extra logic in KerberosConnection to do the Jaas > Configuration with the provided password, but not attempt to prompt the user > for it (as it might be a headless client, like some GUI application). > One problem is that GUI apps will likely try to use pass this value in via > the "password" key in some properties (in addition to the "user" field). Will > actually have to try to test this out to make sure it works as intended. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (CALCITE-1244) `offset` is ignored in FetchRequest
[
https://issues.apache.org/jira/browse/CALCITE-1244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francis Chuang updated CALCITE-1244:
Fix Version/s: avatica-1.22.0
(was: avatica-1.21.0)
> `offset` is ignored in FetchRequest
> ---
>
> Key: CALCITE-1244
> URL: https://issues.apache.org/jira/browse/CALCITE-1244
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Reporter: Josh Elser
>Priority: Major
> Fix For: avatica-1.22.0
>
>
> In looking into some issues reported by [~onpduo], we both noticed that
> {{FetchRequest.offset}} is never actually used by the server.
> We should make sure the semantics on this value are clear (is it an offset
> from the beginning of the ResultSet or the current position of the
> ResultSet?) and make sure it is used.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (CALCITE-1341) Improve mechanism for client/server to unwrap protobuf RPC message
[
https://issues.apache.org/jira/browse/CALCITE-1341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francis Chuang updated CALCITE-1341:
Fix Version/s: avatica-1.22.0
(was: avatica-1.21.0)
> Improve mechanism for client/server to unwrap protobuf RPC message
> --
>
> Key: CALCITE-1341
> URL: https://issues.apache.org/jira/browse/CALCITE-1341
> Project: Calcite
> Issue Type: Improvement
> Components: avatica
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Critical
> Fix For: avatica-1.22.0
>
>
> We just ran into a funny bug in PHOENIX-3136 as a fallout of some changes
> which relocated Avatica classes.
> Because the Avatica RPC classes were also relocated (from
> org.apache.calcite.avatica to org.apache.phoenix.org.apache.calcite.avatica),
> clients of an older version could no longer communicate with a server of the
> newer version (and vice versa). This stems from a decision made where the
> wire API included the class name of the Request and Response Java protobuf
> class in the message. The server would send back the Response class name with
> the relocated name, but the client would not know what that class is (because
> it doesn't have the same relocation). In other words, the current protobuf
> serialization approach requires that Avatica classes are not shaded and
> relocated.
> The JSON serialization doesn't have this problem because it uses the
> {{JsonSubType}} Jackson annotation to map the Request/Response class to a
> logical name (e.g. OpenConnectionResponse to openConnection).
> We could do a similar approach for protobuf that the JSON serialization does
> (maintain this mapping and guarantee that it is not changed incompatibly).
> Long-term, I believe I'd like to see specific Requests dispatched to certain
> URLs (instead of all HTTP requests send to {{/}}) and do away with this
> additional logic in the serialization layer, but I'm not sure if we have to
> re-architect the URL scheme just to fix this issue in the near-term.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (CALCITE-1164) Use setObject(int, Object, int) when binding parameters
[
https://issues.apache.org/jira/browse/CALCITE-1164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francis Chuang updated CALCITE-1164:
Fix Version/s: avatica-1.22.0
(was: avatica-1.21.0)
> Use setObject(int, Object, int) when binding parameters
> ---
>
> Key: CALCITE-1164
> URL: https://issues.apache.org/jira/browse/CALCITE-1164
> Project: Calcite
> Issue Type: Improvement
> Components: avatica
>Reporter: Josh Elser
>Priority: Minor
> Fix For: avatica-1.22.0
>
>
> Trying to capture some discussion from a recent pull request:
> https://github.com/apache/calcite/pull/209#issuecomment-195025402
> In a few places (such as
> https://github.com/apache/calcite/blob/master/avatica/server/src/main/java/org/apache/calcite/avatica/jdbc/JdbcMeta.java#L795-L800),
> we perform:
> {code}
> TypedValue o = parameterValues.get(i);
> preparedStatement.setObject(i + 1, o.toJdbc(calendar));
> {code}
> Vladimir stated that this is ambiguous (stored procedures differing by
> argument list and differentiating between the actual type when the value is
> null) and would be remedied by passing along the desired type when setting
> the object.
> We may also have to invoke setNull explicitly? This is unclear to me.
> h5. Reasons why "explicit sql type" is important
> h6. Calling the proper function
> Consider database has two functions that differ in type of argument only.
> For instance {{compute(varchar)}}, {{compute(numeric)}}, and
> {{compute(user_defined_struct)}}
> Which one should be executed if calling with just
> {{preparedStatement.setObject(i, null)}}?
> There is not enough information for the database to choose between varchar
> and numeric function.
> h6. Performance
> Execution plan depends on the types of bind parameters. For instance, in
> PostgreSQL, you must tell all the datatypes of the bind variables right in
> {{PREPARE}} message.
> That basically means, if you flip between datatypes, you have to use
> different prepared statement IDs.
> If just {{String val = ...; ps.setObject(1, val)}} is used, then for non-null
> it can result in {{String}} execution plan, while for null it can flip to
> unknown.
> Same for batched statement execution. PostgreSQL just cannot handle datatype
> flips right in the middle of the batch. It is handled in the pgjdbc driver,
> so it cuts batch in several sub batches, so it becomes less efficient.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (CALCITE-3162) Reading Arrays from Calcite through JdbcMeta generates AvaticaSqlException
[
https://issues.apache.org/jira/browse/CALCITE-3162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francis Chuang updated CALCITE-3162:
Fix Version/s: avatica-1.22.0
(was: avatica-1.21.0)
> Reading Arrays from Calcite through JdbcMeta generates AvaticaSqlException
> --
>
> Key: CALCITE-3162
> URL: https://issues.apache.org/jira/browse/CALCITE-3162
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Affects Versions: avatica-1.15.0
> Environment: Tested on OS X 10.14.5, OpenJDK Runtime Environment
> Zulu11.29+3-CA (build 11.0.2+7-LTS)
> Issue occurs with both Apache Calcite 1.19.0 & 1.20.0.
>Reporter: Ralph Gasser
>Priority: Major
> Labels: pull-request-available
> Fix For: avatica-1.22.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I'm trying to use _Apache Calcite_ as SQL Parser and Query Planner for a
> custom data store and in addition, I'm using _Apache Avatica_ to expose the
> entire functionality via JDBC for an arbitrary, potential remote client to
> use.
> We're working a lot with Array types, since we're using our backend to store
> high-dimensional vectors. However, it seems that currently, Apache Avatica
> has troubles handling such arrays.
> Take the following test-case that reproduces the problem pretty well.
> {code:java}
> @Test
> public void test() throws Exception {
> HttpServer server = null;
> try {
> Class.forName("org.apache.calcite.jdbc.Driver");
> server = new HttpServer.Builder<>()
> .withHandler(new LocalService(new JdbcMeta("jdbc:calcite:",
> newProperties())),Driver.Serialization.PROTOBUF)
> .withPort(1234)
> .build();
> server.start();
> final Connection connection =
> DriverManager.getConnection("jdbc:avatica:remote:serialization=protobuf;url=http://127.0.0.1:1234;);
> final Statement stmt = connection.createStatement();
> final ResultSet resultSet = stmt.executeQuery("select ARRAY[1.0, 1.0,
> 3.0, 2.0]");
> resultSet.next();
> resultSet.getArray(1);
> } catch (Exception e) {
> System.out.println(e.getMessage());
> } finally {
> if (server != null) {
>server.stop();
> }
> }
> }
> {code}
> Executing the statement will throw an AvaticaSqlException:
> {code:java}
> org.apache.calcite.avatica.AvaticaSqlException: Error -1 (0) : Error
> while executing SQL "select ARRAY[1.0, 1.0, 3.0, 2.0]": Remote driver error:
> RuntimeException: java.sql.SQLException: invalid column ordinal: 2 ->
> SQLException: invalid column ordinal: 2
> {code}
> The culprit seems to be the _org.apache.calcite.avatica.jdbc.JdbcResultSet_
> class. More specifically, the _JdbcResultSet#extractUsingResultSet()_ method.
> I am actually testing a potential fix but first I wanted to make sure, that
> there is nothing wrong with the way I'm using the two components.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (CALCITE-1350) Avoid closeConnection when openConnection doesn't finish/succeed
[ https://issues.apache.org/jira/browse/CALCITE-1350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Chuang updated CALCITE-1350: Fix Version/s: avatica-1.22.0 (was: avatica-1.21.0) > Avoid closeConnection when openConnection doesn't finish/succeed > > > Key: CALCITE-1350 > URL: https://issues.apache.org/jira/browse/CALCITE-1350 > Project: Calcite > Issue Type: Improvement > Components: avatica >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: avatica-1.22.0 > > > I've noticed during testing of Avatica, often times when SPNEGO > authentication is misconfigured, the client will get stuck in > openConnection(). > If we consider sqlline and the user control-C's it, sqlline will try to close > the driver as well which would do a closeConnection() (which would also > obviously fail). > I believe we should be able to short-circuit the the closeConnection() when > we know that the openConnection() didn't succeed properly. > Another scenario is when the Avatica server is down. openConnection will > fail, but we'll still attempt the closeConnection on exit. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (CALCITE-2983) Add Avatica compatibility page for TLS and IBM Java
[ https://issues.apache.org/jira/browse/CALCITE-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Chuang updated CALCITE-2983: Fix Version/s: avatica-1.22.0 (was: avatica-1.21.0) > Add Avatica compatibility page for TLS and IBM Java > --- > > Key: CALCITE-2983 > URL: https://issues.apache.org/jira/browse/CALCITE-2983 > Project: Calcite > Issue Type: Task > Components: avatica, site >Reporter: Kevin Risden >Priority: Major > Fix For: avatica-1.22.0 > > > With the Jetty upgrade in CALCITE-2972, there are some compatibility issues > between TLS support and IBM Java. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Resolved] (CALCITE-5097) Release Avatica 1.21.0
[ https://issues.apache.org/jira/browse/CALCITE-5097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Chuang resolved CALCITE-5097. - Resolution: Fixed > Release Avatica 1.21.0 > -- > > Key: CALCITE-5097 > URL: https://issues.apache.org/jira/browse/CALCITE-5097 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Julian Hyde >Assignee: Francis Chuang >Priority: Major > Fix For: avatica-1.21.0 > > > Release Avatica 1.21.0. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (CALCITE-5097) Release Avatica 1.21.0
[ https://issues.apache.org/jira/browse/CALCITE-5097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Chuang closed CALCITE-5097. --- > Release Avatica 1.21.0 > -- > > Key: CALCITE-5097 > URL: https://issues.apache.org/jira/browse/CALCITE-5097 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Julian Hyde >Assignee: Francis Chuang >Priority: Major > Fix For: avatica-1.21.0 > > > Release Avatica 1.21.0. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Assigned] (CALCITE-5116) Upgrade vlsi-release-plugins to 1.78
[
https://issues.apache.org/jira/browse/CALCITE-5116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francis Chuang reassigned CALCITE-5116:
---
Assignee: Francis Chuang
> Upgrade vlsi-release-plugins to 1.78
> -
>
> Key: CALCITE-5116
> URL: https://issues.apache.org/jira/browse/CALCITE-5116
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Reporter: Francis Chuang
>Assignee: Francis Chuang
>Priority: Major
> Fix For: avatica-1.21.0
>
>
> The release no longer builds after upgrading to Gradle 7.4.2.
> The build fails with this message:
> {noformat}
> > Configure project :
> Building Apache Calcite Avatica 1.21.0
> FAILURE: Build failed with an exception.
> * What went wrong:
> Could not determine the dependencies of task ':stageSvnDist'.
> > Could not resolve all dependencies for configuration ':releaseSignatures'.
>> Querying the mapped value of flatmap(provider(task 'distTar', class
> org.gradle.api.tasks.bundling.Tar)) before task ':release:distTar' has
> completed is not supported
> {noformat}
> This is because the version of vlsi-release-plugins is too old, updating to
> 1.78 fixes the build.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (CALCITE-5137) EnumerableUncollect throws NPE if input has ((List) null)
[
https://issues.apache.org/jira/browse/CALCITE-5137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17533383#comment-17533383
]
xiong duan commented on CALCITE-5137:
-
The CI has some failures, Please check that. Please remove the _colon_ in the
PR commit info Just stay the same as :
{code:java}
[CALCITE-345] AssertionError in RexToLixTranslator comparing to date
literal{code}
> EnumerableUncollect throws NPE if input has ((List) null)
> -
>
> Key: CALCITE-5137
> URL: https://issues.apache.org/jira/browse/CALCITE-5137
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Dmitry Sysolyatin
>Assignee: Dmitry Sysolyatin
>Priority: Major
> Labels: pull-request-available
>
> EnumerableUncollect throws NPE when an input has element = ((List) null)
> Example:
> {code}
> SELECT * FROM UNNEST(CAST(null AS INTEGER ARRAY))
> {code}
> In a real situation, it can appear if to use left join. For example:
> {code}
> SELECT ARRAY(SELECT * FROM UNNEST(t.x)) FROM (VALUES(1)) LEFT JOIN (SELECT
> ARRAY[1] as x, 2 as y) t ON t.y = 1
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Created] (CALCITE-5139) Improve Join print plan to add the CorrelationId info
xiong duan created CALCITE-5139:
---
Summary: Improve Join print plan to add the CorrelationId info
Key: CALCITE-5139
URL: https://issues.apache.org/jira/browse/CALCITE-5139
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.30.0
Reporter: xiong duan
When the Filter plan has CorrelationId info, The Filter output plan will be:
{noformat}
LogicalFilter(condition=[=($10, $SCALAR_QUERY({
LogicalAggregate(group=[{}], EXPR$0=[MAX($0)])
LogicalProject(NAME=[$1])
LogicalFilter(condition=[=($0, $cor0.DEPTNO0)])
LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
}))], variablesSet=[[$cor0]]){noformat}
This plan makes the user know this plan includes the variables set. But when
the Join include the CorrelationId info, it didn't output it.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (CALCITE-5138) Join on condition generates wrong plan when the condition is sub-query
[
https://issues.apache.org/jira/browse/CALCITE-5138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
xiong duan updated CALCITE-5138:
Description:
The SQL:
{code:java}
SELECT emp.deptno, emp.sal
FROM dept
LEFT JOIN emp ON (SELECT AVG(emp.sal) > 0 FROM emp) {code}
Calcite generates the wrong plan:
{noformat}
EnumerableCalc(expr#0..4=[{inputs}], DEPTNO=[$t3], SAL=[$t2])
EnumerableNestedLoopJoin(condition=[$0], joinType=[left])
EnumerableCalc(expr#0..2=[{inputs}], DEPTNO=[$t0])
EnumerableTableScan(table=[[scott, DEPT]])
EnumerableNestedLoopJoin(condition=[true], joinType=[left])
EnumerableCalc(expr#0..7=[{inputs}], EMPNO=[$t0], SAL=[$t5],
DEPTNO=[$t7])
EnumerableTableScan(table=[[scott, EMP]])
EnumerableAggregate(group=[{}], DUMMY=[COUNT()])
EnumerableAggregate(group=[{}], agg#0=[$SUM0($5)], agg#1=[COUNT($5)])
EnumerableTableScan(table=[[scott, EMP]]){noformat}
As above plan, the out NestedLoopJoin condition will be deptno column, not the
AVG(emp.sal) > 0 condition.
was:
The SQL:
{code:java}
SELECT emp.deptno, emp.sal
FROM dept
LEFT JOIN emp ON (SELECT AVG(emp.sal) > 0 FROM emp) {code}
Calcite generates the wrong plan:
{noformat}
EnumerableCalc(expr#0..4=[{inputs}], DEPTNO=[$t3], SAL=[$t2])
EnumerableNestedLoopJoin(condition=[$0], joinType=[left])
EnumerableCalc(expr#0..2=[{inputs}], DEPTNO=[$t0])
EnumerableTableScan(table=[[scott, DEPT]])
EnumerableNestedLoopJoin(condition=[true], joinType=[left])
EnumerableCalc(expr#0..7=[{inputs}], EMPNO=[$t0], SAL=[$t5],
DEPTNO=[$t7])
EnumerableTableScan(table=[[scott, EMP]])
EnumerableAggregate(group=[{}], DUMMY=[COUNT()])
EnumerableAggregate(group=[{}], agg#0=[$SUM0($5)], agg#1=[COUNT($5)])
EnumerableTableScan(table=[[scott, EMP]]){noformat}
As above plan, the out NestedLoopJoin condition will be deptno column, not the
AVG(emp.sal) > 0 condition.
> Join on condition generates wrong plan when the condition is sub-query
> --
>
> Key: CALCITE-5138
> URL: https://issues.apache.org/jira/browse/CALCITE-5138
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.30.0
>Reporter: xiong duan
>Priority: Major
>
> The SQL:
> {code:java}
> SELECT emp.deptno, emp.sal
> FROM dept
> LEFT JOIN emp ON (SELECT AVG(emp.sal) > 0 FROM emp) {code}
> Calcite generates the wrong plan:
> {noformat}
> EnumerableCalc(expr#0..4=[{inputs}], DEPTNO=[$t3], SAL=[$t2])
> EnumerableNestedLoopJoin(condition=[$0], joinType=[left])
> EnumerableCalc(expr#0..2=[{inputs}], DEPTNO=[$t0])
> EnumerableTableScan(table=[[scott, DEPT]])
> EnumerableNestedLoopJoin(condition=[true], joinType=[left])
> EnumerableCalc(expr#0..7=[{inputs}], EMPNO=[$t0], SAL=[$t5],
> DEPTNO=[$t7])
> EnumerableTableScan(table=[[scott, EMP]])
> EnumerableAggregate(group=[{}], DUMMY=[COUNT()])
> EnumerableAggregate(group=[{}], agg#0=[$SUM0($5)], agg#1=[COUNT($5)])
> EnumerableTableScan(table=[[scott, EMP]]){noformat}
> As above plan, the out NestedLoopJoin condition will be deptno column, not
> the AVG(emp.sal) > 0 condition.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Created] (CALCITE-5138) Join on condition generates wrong plan when the condition is sub-query
xiong duan created CALCITE-5138:
---
Summary: Join on condition generates wrong plan when the condition
is sub-query
Key: CALCITE-5138
URL: https://issues.apache.org/jira/browse/CALCITE-5138
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.30.0
Reporter: xiong duan
The SQL:
{code:java}
SELECT emp.deptno, emp.sal
FROM dept
LEFT JOIN emp ON (SELECT AVG(emp.sal) > 0 FROM emp) {code}
Calcite generates the wrong plan:
{noformat}
EnumerableCalc(expr#0..4=[{inputs}], DEPTNO=[$t3], SAL=[$t2])
EnumerableNestedLoopJoin(condition=[$0], joinType=[left])
EnumerableCalc(expr#0..2=[{inputs}], DEPTNO=[$t0])
EnumerableTableScan(table=[[scott, DEPT]])
EnumerableNestedLoopJoin(condition=[true], joinType=[left])
EnumerableCalc(expr#0..7=[{inputs}], EMPNO=[$t0], SAL=[$t5],
DEPTNO=[$t7])
EnumerableTableScan(table=[[scott, EMP]])
EnumerableAggregate(group=[{}], DUMMY=[COUNT()])
EnumerableAggregate(group=[{}], agg#0=[$SUM0($5)], agg#1=[COUNT($5)])
EnumerableTableScan(table=[[scott, EMP]]){noformat}
As above plan, the out NestedLoopJoin condition will be deptno column, not the
AVG(emp.sal) > 0 condition.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (CALCITE-5137) EnumerableUncollect throws NPE if input has ((List) null)
[
https://issues.apache.org/jira/browse/CALCITE-5137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dmitry Sysolyatin updated CALCITE-5137:
---
Component/s: (was: linq4j)
> EnumerableUncollect throws NPE if input has ((List) null)
> -
>
> Key: CALCITE-5137
> URL: https://issues.apache.org/jira/browse/CALCITE-5137
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Dmitry Sysolyatin
>Assignee: Dmitry Sysolyatin
>Priority: Major
> Labels: pull-request-available
>
> EnumerableUncollect throws NPE when an input has element = ((List) null)
> Example:
> {code}
> SELECT * FROM UNNEST(CAST(null AS INTEGER ARRAY))
> {code}
> In a real situation, it can appear if to use left join. For example:
> {code}
> SELECT ARRAY(SELECT * FROM UNNEST(t.x)) FROM (VALUES(1)) LEFT JOIN (SELECT
> ARRAY[1] as x, 2 as y) t ON t.y = 1
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (CALCITE-5137) EnumerableUncollect throws NPE if input has ((List) null)
[
https://issues.apache.org/jira/browse/CALCITE-5137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dmitry Sysolyatin updated CALCITE-5137:
---
Labels: pull-request-available (was: )
> EnumerableUncollect throws NPE if input has ((List) null)
> -
>
> Key: CALCITE-5137
> URL: https://issues.apache.org/jira/browse/CALCITE-5137
> Project: Calcite
> Issue Type: Bug
> Components: core, linq4j
>Reporter: Dmitry Sysolyatin
>Assignee: Dmitry Sysolyatin
>Priority: Major
> Labels: pull-request-available
>
> EnumerableUncollect throws NPE when an input has element = ((List) null)
> Example:
> {code}
> SELECT * FROM UNNEST(CAST(null AS INTEGER ARRAY))
> {code}
> In a real situation, it can appear if to use left join. For example:
> {code}
> SELECT ARRAY(SELECT * FROM UNNEST(t.x)) FROM (VALUES(1)) LEFT JOIN (SELECT
> ARRAY[1] as x, 2 as y) t ON t.y = 1
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (CALCITE-5137) EnumerableUncollect throws NPE if input has ((List) null)
[
https://issues.apache.org/jira/browse/CALCITE-5137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17533323#comment-17533323
]
Dmitry Sysolyatin commented on CALCITE-5137:
PR - https://github.com/apache/calcite/pull/2793
> EnumerableUncollect throws NPE if input has ((List) null)
> -
>
> Key: CALCITE-5137
> URL: https://issues.apache.org/jira/browse/CALCITE-5137
> Project: Calcite
> Issue Type: Bug
> Components: core, linq4j
>Reporter: Dmitry Sysolyatin
>Assignee: Dmitry Sysolyatin
>Priority: Major
>
> EnumerableUncollect throws NPE when an input has element = ((List) null)
> Example:
> {code}
> SELECT * FROM UNNEST(CAST(null AS INTEGER ARRAY))
> {code}
> In a real situation, it can appear if to use left join. For example:
> {code}
> SELECT ARRAY(SELECT * FROM UNNEST(t.x)) FROM (VALUES(1)) LEFT JOIN (SELECT
> ARRAY[1] as x, 2 as y) t ON t.y = 1
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Created] (CALCITE-5137) EnumerableUncollect throws NPE if input has ((List) null)
Dmitry Sysolyatin created CALCITE-5137:
--
Summary: EnumerableUncollect throws NPE if input has ((List) null)
Key: CALCITE-5137
URL: https://issues.apache.org/jira/browse/CALCITE-5137
Project: Calcite
Issue Type: Bug
Components: core, linq4j
Reporter: Dmitry Sysolyatin
Assignee: Dmitry Sysolyatin
EnumerableUncollect throws NPE when an input has element = ((List) null)
Example:
{code}
SELECT * FROM UNNEST(CAST(null AS INTEGER ARRAY))
{code}
In a real situation, it can appear if to use left join. For example:
{code}
SELECT ARRAY(SELECT * FROM UNNEST(t.x)) FROM (VALUES(1)) LEFT JOIN (SELECT
ARRAY[1] as x, 2 as y) t ON t.y = 1
{code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (CALCITE-5135) Planner#parse can't parse DAY() function
[
https://issues.apache.org/jira/browse/CALCITE-5135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17533291#comment-17533291
]
Mitsunori Komatsu commented on CALCITE-5135:
Hmm. DAY is a reserved keyword as well as YEAR and MONTH according to
https://calcite.apache.org/docs/reference.html#keywords.
> Planner#parse can't parse DAY() function
>
>
> Key: CALCITE-5135
> URL: https://issues.apache.org/jira/browse/CALCITE-5135
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.30.0
>Reporter: Mitsunori Komatsu
>Priority: Major
>
> Hi team,
> I might be missing something, but `Planner#parse` can't parse `DAY()`
> function while it can parse other YEAR(), MONTH(), HOUR() and so on.
>
> {code:java}
> import org.apache.calcite.sql.parser.SqlParseException;
> import org.apache.calcite.sql.validate.SqlValidator;
> import org.apache.calcite.tools.FrameworkConfig;
> import org.apache.calcite.tools.Frameworks;
> import org.apache.calcite.tools.Planner;
> public class Main {
> public static void main(String[] args) throws SqlParseException {
> FrameworkConfig config = Frameworks.newConfigBuilder().build();
> {
> Planner planner = Frameworks.getPlanner(config);
> System.out.println(planner.parse("select months(t) from tbl"));
> }
> {
> Planner planner = Frameworks.getPlanner(config);
> System.out.println(planner.parse("select month(t) from tbl"));
> }
> {
> Planner planner = Frameworks.getPlanner(config);
> System.out.println(planner.parse("select hours(t) from tbl"));
> }
> {
> Planner planner = Frameworks.getPlanner(config);
> System.out.println(planner.parse("select hour(t) from tbl"));
> }
> {
> Planner planner = Frameworks.getPlanner(config);
> System.out.println(planner.parse("select days(t) from tbl"));
> }
> {
> Planner planner = Frameworks.getPlanner(config);
> // This throws `org.apache.calcite.sql.parser.SqlParseException:
> Encountered "day" ...`
> System.out.println(planner.parse("select day(t) from tbl"));
> }
> }
> }
> {code}
> Is this a bug? Is there any way to parse DAY() function? Thanks.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (CALCITE-5135) Planner#parse can't parse DAY() function
[
https://issues.apache.org/jira/browse/CALCITE-5135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17533286#comment-17533286
]
Mitsunori Komatsu commented on CALCITE-5135:
> you'd need to use DAYOFYEAR / DAYOFMONTH / DAYOFWEEK ?
I need to parse and convert a SQL dialect that uses DAY(). What I want
Planner#parse to do is just not to throw the exception like for unknown
function.
{code:java}
Planner planner = Frameworks.getPlanner(config);
System.out.println(planner.parse("select unknownfunction(t) from tbl")); {code}
This code works fine without any exception
{code:java}
SELECT `UNKNOWNFUNCTION`(`T`)
FROM `TBL` {code}
> Planner#parse can't parse DAY() function
>
>
> Key: CALCITE-5135
> URL: https://issues.apache.org/jira/browse/CALCITE-5135
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.30.0
>Reporter: Mitsunori Komatsu
>Priority: Major
>
> Hi team,
> I might be missing something, but `Planner#parse` can't parse `DAY()`
> function while it can parse other YEAR(), MONTH(), HOUR() and so on.
>
> {code:java}
> import org.apache.calcite.sql.parser.SqlParseException;
> import org.apache.calcite.sql.validate.SqlValidator;
> import org.apache.calcite.tools.FrameworkConfig;
> import org.apache.calcite.tools.Frameworks;
> import org.apache.calcite.tools.Planner;
> public class Main {
> public static void main(String[] args) throws SqlParseException {
> FrameworkConfig config = Frameworks.newConfigBuilder().build();
> {
> Planner planner = Frameworks.getPlanner(config);
> System.out.println(planner.parse("select months(t) from tbl"));
> }
> {
> Planner planner = Frameworks.getPlanner(config);
> System.out.println(planner.parse("select month(t) from tbl"));
> }
> {
> Planner planner = Frameworks.getPlanner(config);
> System.out.println(planner.parse("select hours(t) from tbl"));
> }
> {
> Planner planner = Frameworks.getPlanner(config);
> System.out.println(planner.parse("select hour(t) from tbl"));
> }
> {
> Planner planner = Frameworks.getPlanner(config);
> System.out.println(planner.parse("select days(t) from tbl"));
> }
> {
> Planner planner = Frameworks.getPlanner(config);
> // This throws `org.apache.calcite.sql.parser.SqlParseException:
> Encountered "day" ...`
> System.out.println(planner.parse("select day(t) from tbl"));
> }
> }
> }
> {code}
> Is this a bug? Is there any way to parse DAY() function? Thanks.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (CALCITE-5136) Avatica build (or CI) must fail if there are deprecation warnings
[
https://issues.apache.org/jira/browse/CALCITE-5136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17533261#comment-17533261
]
Benchao Li commented on CALCITE-5136:
-
[~julianhyde] since CALCITE-5095 is introduced by me, I would like to take this.
> Avatica build (or CI) must fail if there are deprecation warnings
> -
>
> Key: CALCITE-5136
> URL: https://issues.apache.org/jira/browse/CALCITE-5136
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Priority: Major
> Fix For: avatica-1.22.0
>
>
> Avatica build (or CI) must fail if there are deprecation warnings.
> The build currently gives the following warnings on JDK 18:
> {noformat}
> core/src/main/java/org/apache/calcite/avatica/remote/DoAsAvaticaHttpClient.java:37:
> warning: [removal] doAs(Subject,PrivilegedAction) in Subject has been
> deprecated and marked for removal
> return Subject.doAs(kerberosUtil.getSubject(), new
> PrivilegedAction() {
> ^
> where T is a type-variable:
> T extends Object declared in method doAs(Subject,PrivilegedAction)
> Note: Some input files use or override a deprecated API.
> Note: Recompile with -Xlint:deprecation for details.
> Note: Some input files use unchecked or unsafe operations.
> Note: Recompile with -Xlint:unchecked for details.
> 1 warning
> > Task :server:compileJava
> server/src/main/java/org/apache/calcite/avatica/server/SubjectPreservingPrivilegedThreadFactory.java:19:
> warning: [removal] AccessController in java.security has been deprecated and
> marked for removal
> import java.security.AccessController;
> ^
> server/src/main/java/org/apache/calcite/avatica/server/HttpServer.java:204:
> warning: [removal] doAs(Subject,PrivilegedAction) in Subject has been
> deprecated and marked for removal
> Subject.doAs(subject, new PrivilegedAction() {
> ^
> where T is a type-variable:
> T extends Object declared in method doAs(Subject,PrivilegedAction)
> server/src/main/java/org/apache/calcite/avatica/server/SubjectPreservingPrivilegedThreadFactory.java:43:
> warning: [removal] AccessController in java.security has been deprecated and
> marked for removal
> Subject subject = Subject.getSubject(AccessController.getContext());
> ^
> server/src/main/java/org/apache/calcite/avatica/server/SubjectPreservingPrivilegedThreadFactory.java:43:
> warning: [removal] getSubject(AccessControlContext) in Subject has been
> deprecated and marked for removal
> Subject subject = Subject.getSubject(AccessController.getContext());
> ^
> server/src/main/java/org/apache/calcite/avatica/server/SubjectPreservingPrivilegedThreadFactory.java:46:
> warning: [removal] doAs(Subject,PrivilegedAction) in Subject has been
> deprecated and marked for removal
> return Subject.doAs(subject, new PrivilegedAction() {
> ^
> where T is a type-variable:
> T extends Object declared in method doAs(Subject,PrivilegedAction)
> server/src/main/java/org/apache/calcite/avatica/server/SubjectPreservingPrivilegedThreadFactory.java:44:
> warning: [removal] AccessController in java.security has been deprecated and
> marked for removal
> return AccessController.doPrivileged(new PrivilegedAction() {
> {noformat}
> I believe these warnings were introduced in CALCITE-5095.
> Must fix before 1.22. We can't allow broken windows in the build.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
