[jira] [Commented] (DRILL-8393) Allow parameters to be passed to headers through SQL in WHERE clause
[
https://issues.apache.org/jira/browse/DRILL-8393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17700914#comment-17700914
]
ASF GitHub Bot commented on DRILL-8393:
---
LYCJeff commented on PR #2747:
URL: https://github.com/apache/drill/pull/2747#issuecomment-1471106650
> @LYCJeff Thanks for making these changes. I have a few questions:
>
> 1. Are you certain that these filters are in fact being pushed down as
intended?
> 2. I'm really concerned about what would happen if a user aliased a data
source as `header` or `tail`.
>
> IE:
>
> ```sql
> SELECT ...
> FROM api.foo
> INNER JOIN dfs.`tail.csv` AS tail
> ON tail.id = foo.id
> WHERE tail.name = 'something'
> ```
>
> Do we know how this would be interpreted?
Well, we actually need to recognize `header.xxx` as a whole parameter name,
so we need to use back quotes. Only then can it be pushed normally, so these
prefixes are not confused with data source aliases.
If the `name` in your example above is an argument to the `foo` api, it
should be written as follows.
```sql
SELECT ...
FROM api.foo
INNER JOIN dfs.`tail.csv` AS tail
ON tail.id=foo.id
WHERE `tail.name` = 'something'
```
> Allow parameters to be passed to headers through SQL in WHERE clause
>
>
> Key: DRILL-8393
> URL: https://issues.apache.org/jira/browse/DRILL-8393
> Project: Apache Drill
> Issue Type: Improvement
> Components: Storage - HTTP
>Affects Versions: 1.20.0
>Reporter: Yuchen Liang
>Priority: Major
>
> Some APIs require parameters (e.g. digital signature) in the headers to be
> generated at access time.So I'm wondering if we can pass it in through filter
> statement.
> Perhaps we could design it like the params field in connections parameter.
> For example:
>
> Config:
> { "url": "https://api.sunrise-sunset.org/json;, "requireTail": false,
> "params": ["body.lat", "body.lng", "body.date", "header.header1"],
> "parameterLocation": "json_body" }
>
> SQL Query:
> SELECT * FROM api.sunrise
> WHERE `body.lat` = 36.7201600
> AND `body.lng` = -4.4203400
> AND `body.date` = '2019-10-02'
> AND `header.header1` = 'value1';
>
> Post body:
> { "lat": 36.7201600, "lng": -4.4203400, "date": "2019-10-02"}
>
> Headers:
> { "header1": "value1", ……}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (DRILL-8393) Allow parameters to be passed to headers through SQL in WHERE clause
[
https://issues.apache.org/jira/browse/DRILL-8393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17700907#comment-17700907
]
ASF GitHub Bot commented on DRILL-8393:
---
cgivre commented on PR #2747:
URL: https://github.com/apache/drill/pull/2747#issuecomment-1471002379
@LYCJeff Thanks for making these changes. I have a few questions:
1. Are you certain that these filters are in fact being pushed down as
intended?
2. I'm really concerned about what would happen if a user aliased a data
source as `header` or `tail`.
IE:
```sql
SELECT ...
FROM api.foo
INNER JOIN dfs.`tail.csv` AS tail
ON tail.id = foo.id
WHERE tail.name = 'something'
```
Do we know how this would be interpreted?
> Allow parameters to be passed to headers through SQL in WHERE clause
>
>
> Key: DRILL-8393
> URL: https://issues.apache.org/jira/browse/DRILL-8393
> Project: Apache Drill
> Issue Type: Improvement
> Components: Storage - HTTP
>Affects Versions: 1.20.0
>Reporter: Yuchen Liang
>Priority: Major
>
> Some APIs require parameters (e.g. digital signature) in the headers to be
> generated at access time.So I'm wondering if we can pass it in through filter
> statement.
> Perhaps we could design it like the params field in connections parameter.
> For example:
>
> Config:
> { "url": "https://api.sunrise-sunset.org/json;, "requireTail": false,
> "params": ["body.lat", "body.lng", "body.date", "header.header1"],
> "parameterLocation": "json_body" }
>
> SQL Query:
> SELECT * FROM api.sunrise
> WHERE `body.lat` = 36.7201600
> AND `body.lng` = -4.4203400
> AND `body.date` = '2019-10-02'
> AND `header.header1` = 'value1';
>
> Post body:
> { "lat": 36.7201600, "lng": -4.4203400, "date": "2019-10-02"}
>
> Headers:
> { "header1": "value1", ……}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (DRILL-8411) GoogleSheets Reader Will Not Read More than 1K Rows
[ https://issues.apache.org/jira/browse/DRILL-8411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17700797#comment-17700797 ] ASF GitHub Bot commented on DRILL-8411: --- cgivre merged PR #2774: URL: https://github.com/apache/drill/pull/2774 > GoogleSheets Reader Will Not Read More than 1K Rows > --- > > Key: DRILL-8411 > URL: https://issues.apache.org/jira/browse/DRILL-8411 > Project: Apache Drill > Issue Type: Bug > Components: Storage - GoogleSheets >Affects Versions: 1.21.0 >Reporter: Charles Givre >Assignee: Charles Givre >Priority: Major > Fix For: 1.21.1 > > > The GoogleSheets reader hits the batch limit from the GoogleSheets SDK of > 1000 rows and stops. This PR fixes that. > It also fixes a minor but annoying issue whereby the GoogleSheets reader > determines a column is a date/time, but is then unable to parse it because it > is in a non-standard format. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (DRILL-8410) Upgrade to Calcite 1.34
[ https://issues.apache.org/jira/browse/DRILL-8410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17700673#comment-17700673 ] ASF GitHub Bot commented on DRILL-8410: --- jnturton opened a new pull request, #2775: URL: https://github.com/apache/drill/pull/2775 # [DRILL-8410](https://issues.apache.org/jira/browse/DRILL-8410): Upgrade to Calcite 1.34 ## Description Calcite 1.34 includes 1.a fix for the currently broken date_trunc function in Drill 2. support for a new QUALIFY clause in windows functions 3. incompatible core parser grammar changes that break date_diff in Drill. Because of (3), Drill needs to make temporary use of a modified Parser.jj until Calcite backs out the mentioned parser changes. See the linked Calcite issues for more details. Normally it would be undesirable to backport the new QUALIFY clause but, short of setting up cherry picking from Calcite, getting the fix for the regression in DATE_TRUNC forces the addition of support for QUALIFY. Calcite does not do seperate bugfix releases. ## Documentation Document the new QUALIFY clause. ## Testing - Existing unit tests of DATE_TRUNC. - Existing unit tests of DATE_DIFF. - New unit test of QUALIFY. > Upgrade to Calcite 1.34 > --- > > Key: DRILL-8410 > URL: https://issues.apache.org/jira/browse/DRILL-8410 > Project: Apache Drill > Issue Type: Improvement > Components: library >Affects Versions: 1.21.0 >Reporter: James Turton >Assignee: James Turton >Priority: Major > Fix For: 1.21.1 > > > Calcite 1.34 includes > # a fix for the currently broken date_trunc function in Drill > # support for a new QUALIFY clause in windows functions > # incompatible core parser grammar changes that break date_diff in Drill. > Because of (3), Drill needs to make temporary use of a modified Parser.jj > until Calcite backs out the mentioned parser changes. See the linked Calcite > issues for more details. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8410) Upgrade to Calcite 1.34
[ https://issues.apache.org/jira/browse/DRILL-8410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8410: Description: Calcite 1.34 includes # a fix for the currently broken date_trunc function in Drill # support for a new QUALIFY clause in windows functions # incompatible core parser grammar changes that break date_diff in Drill. Because of (3), Drill needs to make temporary use of a modified Parser.jj until Calcite backs out the mentioned parser changes. See the linked Calcite issues for more details. was:Calcite 1.34 includes fixes for issues affecting Drill 1.21.0. > Upgrade to Calcite 1.34 > --- > > Key: DRILL-8410 > URL: https://issues.apache.org/jira/browse/DRILL-8410 > Project: Apache Drill > Issue Type: Improvement > Components: library >Affects Versions: 1.21.0 >Reporter: James Turton >Assignee: James Turton >Priority: Major > Fix For: 1.21.1 > > > Calcite 1.34 includes > # a fix for the currently broken date_trunc function in Drill > # support for a new QUALIFY clause in windows functions > # incompatible core parser grammar changes that break date_diff in Drill. > Because of (3), Drill needs to make temporary use of a modified Parser.jj > until Calcite backs out the mentioned parser changes. See the linked Calcite > issues for more details. -- This message was sent by Atlassian Jira (v8.20.10#820010)
