Re: Apache Drill/Calcite error on Date manipulation

[Julian Hyde]

I’ve added a comment to the JIRA case. Its a Calcite issue and it shouldn’t be 
too hard to develop a fix.

> On Dec 7, 2017, at 11:11 PM, Rahul Raj  wrote:
> 
> Hi All,
> 
> This is an issue related to Apache Drill and need some help to debug the
> problem.
> 
> Drill supports a feature to query a database directly. When querying a
> database directly, Drill throws an exception while manipulating a date
> field inside the 'where' clause as shown in the query below:
> 
> select * from config_1.APP.EXECUTIONSTEP
> where DATE_ADD(CAST(STARTED_AT as DATE),interval '1' second) <
> CAST(CURRENT_DATE as DATE)
> 
> I could see a calcite exception in the trace:
> Caused by: java.lang.AssertionError: 1000: INTERVAL_DAY_TIME
> at
> org.apache.calcite.adapter.jdbc.JdbcImplementor$Context.toSql(JdbcImplementor.java:179)
> ~[calcite-core-1.4.0-drill-r19.jar:1.4.0-drill-r19]
> 
> I have raised the issue and details at
> https://issues.apache.org/jira/browse/DRILL-5578.
> 
> The issue does not happen when querying parquet files.
> 
> Regards,
> Rahul

Re: JIRA Github integration

[Michael Mior]

Ah, I see. So maybe we just need to remove the square brackets on PR
titles. I'll try that next time I have code for review (or if someone else
gets a chance) and we'll see what happens. Thanks Josh!

--
Michael Mior
mm...@apache.org

2017-12-11 17:22 GMT-05:00 Josh Elser :

> I'm confused, we definitely have GH+JIRA integration enabled (at least, in
> part) already. See https://issues.apache.org/jira/browse/CALCITE-1922 for
> an example.
>
> Our commit message "structure" is at odds with the integration, though
> ([CALCITE-1234] would not link them, you have to change the PR title to
> just be CALCITE-1234). But, maybe it's just not set up for the
> github.com/apache/calcite repository, just the calcite-avatica repo. I
> don't recall how the configuration works on the ASF infra side.
>
>
> On 12/10/17 7:49 PM, Michael Mior wrote:
>
>> Was just reminded of this after creating a PR for the calcite-avatica repo
>> and seeing this in action. Since there are no objections, seems like it
>> would be good to get this set up.
>>
>> @Julian Unfortunately I'm currently ignorant as to how to request this
>> from
>> infra. Is it just a matter of creating an issue on the INFRA JIRA?
>>
>> --
>> Michael Mior
>> mm...@apache.org
>>
>> 2017-11-27 16:21 GMT-05:00 Julian Hyde :
>>
>> No objections from me, as long as it doesn’t cause excessive posts to the
>>> dev list.
>>>
>>> As a small project this is an area we have under-invested. Larger
>>> projects
>>> such as Spark seem to have very nice integration between GitHub, JIRA and
>>> various CIs.
>>>
>>> Julian
>>>
>>>
>>> On Nov 27, 2017, at 1:15 PM, Michael Mior  wrote:

 Sounds fine to me personally although perhaps others have some
 objections
 I'm not aware of.

 --
 Michael Mior
 mm...@apache.org

 2017-11-27 15:44 GMT-05:00 Enrico Olivelli :

 Hi,
> I see that there is no automatic link between PRs and JIRA.
> In Apache BookKeeper (I am committer there) we have such link: if you
>
 name
>>>
 a PR with for instance CALCITE-Xxx and you cite the issue id in commit
> messages Apache Github bot is able to copy every comment to JIRA
> automatically.
> I think this we be very useful.
> I think that just an infra ticket can enable it.
>
> Cheers
> Enrico
> --
>
>
> -- Enrico Olivelli
>
>
>>>
>>>
>>

Calcite-Master - Build # 160 - Still Failing

[Apache Jenkins Server]

The Apache Jenkins build system has built Calcite-Master (build #160)

Status: Still Failing

Check console output at https://builds.apache.org/job/Calcite-Master/160/ to 
view the results.

Re: About the Spatial functionality

[Julian Hyde]

Thanks for the review. I just committed the fix as 
http://git-wip-us.apache.org/repos/asf/calcite/commit/c1749ade 
. 

> On Dec 9, 2017, at 1:51 PM, Vamshi Krishna  wrote:
> 
> Hi Julian,
> 
> I have tested the fix posted for CALCITE-2072 and i was able to use the
> geometry functions.
> 
> Thanks for the fix.
> 
> Thanks,
> Vamshi.
> 
> On Fri, Dec 8, 2017 at 4:38 PM, Julian Hyde  > wrote:
> 
>> I don’t know. I don’t have time to look at the code right now.
>> 
>>> On Dec 8, 2017, at 12:53 PM, Vamshi Krishna 
>> wrote:
>>> 
>>> I think you meant the following:
>>> 
>>> 1. Create a user defined operator table for spatial functions (Say
>>> SpatialOperatorTable).
>>> 2. Honor the newly defined operator table (SpatialOperatorTable) when
>>> fun=spatial at CalciteConnectionConfiImpl.java#L102
>>> 
>>> Regarding Step  1: I don't see a way to convert or load the functions in
>>> GeoFunctions class into SqlFunction format other than manually adding
>> them
>>> similar to OracleSqlOperatorTable. Is my understanding right ?
>>> 
>>> 
>>> 
>>> On Fri, Dec 8, 2017 at 2:01 PM, Julian Hyde >>  > jh...@apache.org >> wrote:
>>> 
 You’re basically running into https://issues.apache.org/ 
  <
>> https://issues.apache.org/ >
 jira/browse/CALCITE-2072  <
>> https://issues.apache.org/ >
 jira/browse/CALCITE-2072>. The fix for that issue is straightforward - a
 couple of lines around https://github.com/apache/ 
  <
>> https://github.com/apache/ >
 calcite/blob/master/core/src/main/java/org/apache/calcite/config/
 CalciteConnectionConfigImpl.java#L102  <
>> https://github.com/apache/ >
 calcite/blob/master/core/src/main/java/org/apache/calcite/config/
 CalciteConnectionConfigImpl.java#L102> - so can you make that fix and
>> see
 whether it fixes the problem.
 
> On Dec 8, 2017, at 6:52 AM, Vamshi Krishna  
>>> 
 wrote:
> 
> Hi Christian,
> 
> Yes,  I have changed the conformance level to LENIENT.
> I was able test the create table syntax using geometry data type.
> 
> 
> Here's what i have in the code:
> //create root schema
> rootSchema = Frameworks.createRootSchema(true);
> 
> //add geo functions
> ModelHandler.addFunctions(rootSchema, null,
 ImmutableList.of(),
>   GeoFunctions.class.getName(), "*", true);
> 
> 
>  // Initialize default planner
>  FrameworkConfig calciteFrameworkConfig =
> Frameworks.newConfigBuilder()
>  .operatorTable(ChainedSqlOperatorTable.of(
> OracleSqlOperatorTable.instance(),SqlStdOperatorTable.instance()))
> 
> .parserConfig(SqlParser.configBuilder().setConformance(LENIENT)
>  // Lexical configuration defines how identifiers
> are quoted, whether they are converted to upper or lower
>  // case when they are read, and whether
 identifiers
> are matched case-sensitively.
>  .setParserFactory(SqlParserImpl.FACTORY)
>  .setLex(Lex.ORACLE)
>  .build())
>  // Sets the schema to use by the planner
>  .defaultSchema(rootSchema.add("CATALOG",schema))
>  .traitDefs(traitDefs)
>  // Context provides a way to store data within the
>> planner
> session that can be accessed in planner rules.
>  .context(Contexts.EMPTY_CONTEXT)
>  // Rule sets to use in transformation phases. Each
> transformation phase can use a different set of rules.
>  .ruleSets(RuleSets.ofList())
>  // Custom cost factory to use during optimization
>  .costFactory(null)
>  .typeSystem(RelDataTypeSystem.DEFAULT)
>  .build();
> 
>  this.planner = new CustomPlannerImpl(calciteFrameworkConfig);
> 
>  planner.parse();
> 
>  planner.validate(); <<-- reporting error with no match found.
> 
> Currently i am only looking for syntax and data type validation support
 and
> not the runtime implementation of the geo functions.
> 
> 
> 
> I am not sure if the GeoFunctions extension can be used for this
>> purpose
> similar to functions in OracleSqlOperator.
> 
> Thanks,
> Vamshi.
> 
> 

[jira] [Created] (CALCITE-2086) HTTP/413 in certain circumstances due to large Authorization headers

[Josh Elser (JIRA)]

Josh Elser created CALCITE-2086:
---

 Summary: HTTP/413 in certain circumstances due to large 
Authorization headers
 Key: CALCITE-2086
 URL: https://issues.apache.org/jira/browse/CALCITE-2086
 Project: Calcite
  Issue Type: Bug
  Components: avatica
Reporter: Josh Elser
Assignee: Josh Elser
Priority: Critical
 Fix For: avatica-1.11.0


Had a user case where a certain user's requests to an Avatica server were met 
with an HTTP/413 error because the Authorization header was larger than the 
default allowed 8KB. I'm not sure of the exact circumstances, but something is 
"special" about the user's Kerberos key (SPNEGO-based auth).

We should allow this to be configurable by the Avatica server and also increase 
the default (64KB should be more than enough for the present and would avoid 
more downstream changes).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: JIRA Github integration

[Josh Elser]

I'm confused, we definitely have GH+JIRA integration enabled (at least, 
in part) already. See https://issues.apache.org/jira/browse/CALCITE-1922 
for an example.


Our commit message "structure" is at odds with the integration, though 
([CALCITE-1234] would not link them, you have to change the PR title to 
just be CALCITE-1234). But, maybe it's just not set up for the 
github.com/apache/calcite repository, just the calcite-avatica repo. I 
don't recall how the configuration works on the ASF infra side.


On 12/10/17 7:49 PM, Michael Mior wrote:

Was just reminded of this after creating a PR for the calcite-avatica repo
and seeing this in action. Since there are no objections, seems like it
would be good to get this set up.

@Julian Unfortunately I'm currently ignorant as to how to request this from
infra. Is it just a matter of creating an issue on the INFRA JIRA?

--
Michael Mior
mm...@apache.org

2017-11-27 16:21 GMT-05:00 Julian Hyde :


No objections from me, as long as it doesn’t cause excessive posts to the
dev list.

As a small project this is an area we have under-invested. Larger projects
such as Spark seem to have very nice integration between GitHub, JIRA and
various CIs.

Julian



On Nov 27, 2017, at 1:15 PM, Michael Mior  wrote:

Sounds fine to me personally although perhaps others have some objections
I'm not aware of.

--
Michael Mior
mm...@apache.org

2017-11-27 15:44 GMT-05:00 Enrico Olivelli :


Hi,
I see that there is no automatic link between PRs and JIRA.
In Apache BookKeeper (I am committer there) we have such link: if you

name

a PR with for instance CALCITE-Xxx and you cite the issue id in commit
messages Apache Github bot is able to copy every comment to JIRA
automatically.
I think this we be very useful.
I think that just an infra ticket can enable it.

Cheers
Enrico
--


-- Enrico Olivelli

Calcite-Master - Build # 159 - Still Failing

[Apache Jenkins Server]

The Apache Jenkins build system has built Calcite-Master (build #159)

Status: Still Failing

Check console output at https://builds.apache.org/job/Calcite-Master/159/ to 
view the results.

[ANNOUNCE] Apache Calcite 1.15.0 released

[Julian Hyde]

The Apache Calcite team is pleased to announce the release of Apache
Calcite 1.15.0.

Calcite is a dynamic data management framework. Its cost-based
optimizer converts queries, represented in relational algebra, into
executable plans.  Calcite supports many front-end languages and
back-end data engines, and includes an SQL parser and, as a
sub-project, the Avatica JDBC driver.

In this release, three months after 1.14.0, 50 issues are fixed by 22
contributors. Some features of note are a "server" module with DDL
support; dynamic parameters in the LIMIT and OFFSET and clauses;
refactoring the JDBC adapter to make it easier to plug in a new SQL
dialect; and a data profiling algorithm, that efficiently analyzes
large data sets with many columns, estimating the number of distinct
values in columns and groups of columns.

You can start using it in Maven by simply updating your dependency to:

  
org.apache.calcite
calcite-core
1.15.0
  

If you'd like to download the source release, you can find it here:

  http://www.apache.org/dyn/closer.cgi/calcite/apache-calcite-1.15.0/

You can read more about the release (including release notes) here:

  http://calcite.apache.org/news/2017/12/11/release-1.15.0/

We welcome your help and feedback. For more information on how to
report problems, and to get involved, visit the project website at:

  http://calcite.apache.org/

Thanks to everyone involved!

Julian Hyde, on behalf of the Apache Calcite Team

Calcite-Master - Build # 158 - Still Failing

[Apache Jenkins Server]

The Apache Jenkins build system has built Calcite-Master (build #158)

Status: Still Failing

Check console output at https://builds.apache.org/job/Calcite-Master/158/ to 
view the results.