Re: Kerberos Authentication and Avatica

2017-07-10 Thread Josh Elser 
On Jul 10, 2017 20:18, "F21"  wrote:

Hey Josh,

Thanks for clearing things up. In Go, it is not idiomatic for a database
driver to reach out to environment variables. I think I will add an
additional parameter called `krb5Conf` for users to point the driver to the
location of `krb5.conf`. In the event that it is not provided, I plan to
search common locations listed here: https://www.ibm.com/support/kn
owledgecenter/en/SSAW57_8.5.5/com.ibm.websphere.nd.doc/ae/rs
ec_SPNEGO_config_krb5.html and https://docs.oracle.com/javase
/8/docs/technotes/guides/security/jgss/tutorials/KerberosReq.html


Sounds reasonable to me!

Regarding the use-case where the user performs authentication and passes
the ticket to Avatica, what does the driver configuration look like? In
particular, if I were using the Java driver, is it correct to assume that
I'd set `authentication` to `SPNEGO` and leave `keytab` and `principal` as
blank? In that case, I am assuming the Java Kerberos library would find the
cached ticket and set up the appropriate HTTP requests.


Exactly right. The user does nothing special, and then the underlying Java
security code provides it when the HTTP client library asks for the ticket.

Cheers,
Francis

On 11/07/2017 12:49 AM, Josh Elser wrote:

> Hey Francis,
>
> On 7/10/17 7:09 AM, F21 wrote:
>
>> Follow up questions:
>> - According to the client reference for the principal parameter [0], the
>> Java client is able to perform a Kerberos login before contacting the
>> Avatica server. There appears to be no way to set the KDC address into the
>> client. How does the Java client perform Kerberos logins?
>>
>
> This is convention for Java. There are expected locations at which a file,
> krb5.conf, is located on platforms. For Linux, this is /etc/krb5.conf.
>
> - There is also an option for the user to perform the login themselves. In
>> this case, how does the Java client pass the Kerberos ticket to the Avatica
>> server?
>>
>
> Again, convention. On Linux, the location of a user's ticket cache is
> defined to be /tmp/krb5cc_$(id -u $(whoami)). This location can be
> overriden by the environment variable KRB5CCNAME. All of this is handled by
> Java itself.
>
> This is definitely the common case for interactive users.
>
> [0] https://calcite.apache.org/avatica/docs/client_reference.htm
>> l#principal
>>
>> On 10/07/2017 3:57 PM, F21 wrote:
>>
>>> Recently, I came across a maintained pure-go kerberos client and server
>>> [0].
>>>
>>> I am now in the process of adding SPNEGO authentication to the Go
>>> avatica client [1].
>>>
>>> For the implementation, the plan is to make it as close to the official
>>> (java) client's implementation as possible. For SPNEGO, to Java client uses
>>> these 2 parameters: principal and keytab.
>>>
>>> The keytab parameter is easy to understand: a path to a keytab file.
>>>
>>> I'd like to confirm what a valid string for the principal looks like.
>>> - Is it a Service Principal Name?
>>> - What are the valid formats for the principal? A valid SPN looks like
>>> User1/User2@realm.
>>> - For the above example, I am assuming user2 can be optional.
>>> - Can the realm be optional?
>>>
>>
> See http://web.mit.edu/kerberos/krb5-1.5/krb5-1.5.4/doc/krb5-use
> r/What-is-a-Kerberos-Principal_003f.html. This page does a very good job
> at concisely expressing what a Kerberos principal is and what can be
> implied (based on krb5.conf).
>
> Let me know if you still have questions.
>
> Cheers,
>>> Francis
>>>
>>> [0] https://github.com/jcmturner/gokrb5
>>> [1] https://github.com/Boostport/avatica
>>>
>>
>>
>>

Re: Kerberos Authentication and Avatica

2017-07-10 Thread F21 

Hey Josh,

Thanks for clearing things up. In Go, it is not idiomatic for a database 
driver to reach out to environment variables. I think I will add an 
additional parameter called `krb5Conf` for users to point the driver to 
the location of `krb5.conf`. In the event that it is not provided, I 
plan to search common locations listed here: 
https://www.ibm.com/support/knowledgecenter/en/SSAW57_8.5.5/com.ibm.websphere.nd.doc/ae/rsec_SPNEGO_config_krb5.html 
and 
https://docs.oracle.com/javase/8/docs/technotes/guides/security/jgss/tutorials/KerberosReq.html 



Regarding the use-case where the user performs authentication and passes 
the ticket to Avatica, what does the driver configuration look like? In 
particular, if I were using the Java driver, is it correct to assume 
that I'd set `authentication` to `SPNEGO` and leave `keytab` and 
`principal` as blank? In that case, I am assuming the Java Kerberos 
library would find the cached ticket and set up the appropriate HTTP 
requests.


Cheers,
Francis

On 11/07/2017 12:49 AM, Josh Elser wrote:

Hey Francis,

On 7/10/17 7:09 AM, F21 wrote:

Follow up questions:
- According to the client reference for the principal parameter [0], 
the Java client is able to perform a Kerberos login before contacting 
the Avatica server. There appears to be no way to set the KDC address 
into the client. How does the Java client perform Kerberos logins?


This is convention for Java. There are expected locations at which a 
file, krb5.conf, is located on platforms. For Linux, this is 
/etc/krb5.conf.


- There is also an option for the user to perform the login 
themselves. In this case, how does the Java client pass the Kerberos 
ticket to the Avatica server?


Again, convention. On Linux, the location of a user's ticket cache is 
defined to be /tmp/krb5cc_$(id -u $(whoami)). This location can be 
overriden by the environment variable KRB5CCNAME. All of this is 
handled by Java itself.


This is definitely the common case for interactive users.

[0] 
https://calcite.apache.org/avatica/docs/client_reference.html#principal


On 10/07/2017 3:57 PM, F21 wrote:
Recently, I came across a maintained pure-go kerberos client and 
server [0].


I am now in the process of adding SPNEGO authentication to the Go 
avatica client [1].


For the implementation, the plan is to make it as close to the 
official (java) client's implementation as possible. For SPNEGO, to 
Java client uses these 2 parameters: principal and keytab.


The keytab parameter is easy to understand: a path to a keytab file.

I'd like to confirm what a valid string for the principal looks like.
- Is it a Service Principal Name?
- What are the valid formats for the principal? A valid SPN looks 
like User1/User2@realm.

- For the above example, I am assuming user2 can be optional.
- Can the realm be optional?


See 
http://web.mit.edu/kerberos/krb5-1.5/krb5-1.5.4/doc/krb5-user/What-is-a-Kerberos-Principal_003f.html. 
This page does a very good job at concisely expressing what a Kerberos 
principal is and what can be implied (based on krb5.conf).


Let me know if you still have questions.


Cheers,
Francis

[0] https://github.com/jcmturner/gokrb5
[1] https://github.com/Boostport/avatica

Re: Kerberos Authentication and Avatica

2017-07-10 Thread Josh Elser 

Hey Francis,

On 7/10/17 7:09 AM, F21 wrote:

Follow up questions:
- According to the client reference for the principal parameter [0], the 
Java client is able to perform a Kerberos login before contacting the 
Avatica server. There appears to be no way to set the KDC address into 
the client. How does the Java client perform Kerberos logins?


This is convention for Java. There are expected locations at which a 
file, krb5.conf, is located on platforms. For Linux, this is /etc/krb5.conf.


- There is also an option for the user to perform the login themselves. 
In this case, how does the Java client pass the Kerberos ticket to the 
Avatica server?


Again, convention. On Linux, the location of a user's ticket cache is 
defined to be /tmp/krb5cc_$(id -u $(whoami)). This location can be 
overriden by the environment variable KRB5CCNAME. All of this is handled 
by Java itself.


This is definitely the common case for interactive users.


[0] https://calcite.apache.org/avatica/docs/client_reference.html#principal

On 10/07/2017 3:57 PM, F21 wrote:
Recently, I came across a maintained pure-go kerberos client and 
server [0].


I am now in the process of adding SPNEGO authentication to the Go 
avatica client [1].


For the implementation, the plan is to make it as close to the 
official (java) client's implementation as possible. For SPNEGO, to 
Java client uses these 2 parameters: principal and keytab.


The keytab parameter is easy to understand: a path to a keytab file.

I'd like to confirm what a valid string for the principal looks like.
- Is it a Service Principal Name?
- What are the valid formats for the principal? A valid SPN looks like 
User1/User2@realm.

- For the above example, I am assuming user2 can be optional.
- Can the realm be optional?


See 
http://web.mit.edu/kerberos/krb5-1.5/krb5-1.5.4/doc/krb5-user/What-is-a-Kerberos-Principal_003f.html. 
This page does a very good job at concisely expressing what a Kerberos 
principal is and what can be implied (based on krb5.conf).


Let me know if you still have questions.


Cheers,
Francis

[0] https://github.com/jcmturner/gokrb5
[1] https://github.com/Boostport/avatica

[jira] [Created] (CALCITE-1883) HepPlanner should force garbage collect whenever a root registered

2017-07-10 Thread Ted Xu (JIRA) 
Ted Xu created CALCITE-1883:
---

 Summary: HepPlanner should force garbage collect whenever a root 
registered
 Key: CALCITE-1883
 URL: https://issues.apache.org/jira/browse/CALCITE-1883
 Project: Calcite
  Issue Type: Bug
Reporter: Ted Xu
Assignee: Julian Hyde


Currently HepPlanner#collectGarbage() will skip if there is no new transform 
since last GC. If HepPlanner is reused, it is common case that no transform is 
applied after last cheapest plan is built while before next round of optimize 
is kicked off.

This may break HepPlanner because the garbage is probably not a graph after 
HepPlanner#buildFinalPlan .

I'll try if I can create a reproduce case.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Questions about MonteCarloAlgorithm to compute tiles of Lattice

2017-07-10 Thread weijie tong 
additional remarks,
I mean if we do not follow the Star schema ( maybe two fact tables to join,
both with dimension columns) ,we only materialize the provided defined
tiles of a cube ( let algorithm option to be false), then a user's join
query will  correctly be transferred to the materialized cube's query ?

On Mon, Jul 10, 2017 at 7:22 PM, weijie tong 
wrote:

> @Julian thanks for your reply.
> Another question is about `Star schema` requirement. Does this
> precondition only affect `Lattice.computeTiles()` method to choose the
> right dimension group to be the candidate Tile ?
>
> On Fri, Jul 7, 2017 at 4:44 AM, Julian Hyde  wrote:
>
>>
>> > On Jun 28, 2017, at 10:58 PM, weijie tong 
>> wrote:
>> >
>> > HI all:
>> >   anyone can explain  the detail of the MonteCarlo algorithm to compute
>> > the tiles of a Lattice?
>> >It seems that  MonteCarlo algorithm will simulate every possible
>> query
>> > of all kind of  AggregateImpls ,and will choose the lowest cost's ( cost
>> > model determined by the estimateCost() method of LatticeImpl )
>> > AggregateImpl to be the titles.
>>
>> ExhaustiveLatticeAlgorithm will try every possible query (2^n if there
>> are n attributes), whereas MonteCarloAlgorithm tries a set of random
>> queries.
>>
>> Both algorithms are greedy algorithms. Each iteration, they assume that a
>> set of aggregates have been chosen, and choose the best aggregate to add to
>> it by calling getBenefit (which, despite its name, is a cost-benefit
>> ratio). Repeat until there are enough aggregates.
>>
>>
>> >I also find that the cost benefits of the choose AggregateImpls don't
>> > play any role to the final output AggregateImpl.
>>
>> If you’re referring to the list of CostBenefit objects created at the end
>> of the algorithm; yes, they are just info to put on the screen and prove
>> that the algorithm has done a great job.
>>
>> But you’ll see that getBenefit is called in the inner loop.
>>
>>
>> >please correct my opinion and show me the mathematical theory of the
>> > MonteCarlo algorithm to  choose the best  aggregates .
>>
>> MonteCarloAlgorithm could be improved by taking into account historic
>> queries, but I think it does a good job for the case where no previous
>> queries are not known.
>>
>> The biggest problem with the algorithm is the amount of time spent
>> gathering statistics. My work on data profiling [1] [2] will speed up
>> getBenefit hugely because it will be able to answer
>> aggregate.estimateRowCount() without executing a query.
>>
>> Julian
>>
>> [1] https://issues.apache.org/jira/browse/CALCITE-1616 <
>> https://issues.apache.org/jira/browse/CALCITE-1616>
>>
>> [2] https://www.slideshare.net/julianhyde/data-profiling-with-
>> apache-calcite > lianhyde/data-profiling-with-apache-calcite>
>>
>>
>

Re: Kerberos Authentication and Avatica

2017-07-10 Thread F21 

Follow up questions:
- According to the client reference for the principal parameter [0], the 
Java client is able to perform a Kerberos login before contacting the 
Avatica server. There appears to be no way to set the KDC address into 
the client. How does the Java client perform Kerberos logins?


- There is also an option for the user to perform the login themselves. 
In this case, how does the Java client pass the Kerberos ticket to the 
Avatica server?


[0] https://calcite.apache.org/avatica/docs/client_reference.html#principal

On 10/07/2017 3:57 PM, F21 wrote:
Recently, I came across a maintained pure-go kerberos client and 
server [0].


I am now in the process of adding SPNEGO authentication to the Go 
avatica client [1].


For the implementation, the plan is to make it as close to the 
official (java) client's implementation as possible. For SPNEGO, to 
Java client uses these 2 parameters: principal and keytab.


The keytab parameter is easy to understand: a path to a keytab file.

I'd like to confirm what a valid string for the principal looks like.
- Is it a Service Principal Name?
- What are the valid formats for the principal? A valid SPN looks like 
User1/User2@realm.

- For the above example, I am assuming user2 can be optional.
- Can the realm be optional?

Cheers,
Francis

[0] https://github.com/jcmturner/gokrb5
[1] https://github.com/Boostport/avatica

[jira] [Created] (CALCITE-1882) Can't obtain the user defined aggregate function such as sum,avg by calcite

2017-07-10 Thread yuemeng (JIRA) 
yuemeng created CALCITE-1882:


 Summary: Can't obtain the user defined aggregate function such as 
sum,avg by calcite
 Key: CALCITE-1882
 URL: https://issues.apache.org/jira/browse/CALCITE-1882
 Project: Calcite
  Issue Type: Bug
Affects Versions: 1.12.0
Reporter: yuemeng
Assignee: yuemeng
Priority: Critical


If we want to register a sum or avg aggregate function to deal with different 
data type such as sum(double) ,we implement a SqlUserDefinedAggFunction and 
register with name sum,but when we execute a sql like:
{code}
select id,sum(payment) from table test group by id
{/code}
in fact,it always give the SqlSumAggFunction function which buidin by 
calcite,never find the exactly function which we register by ourself.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Kerberos Authentication and Avatica

2017-07-10 Thread F21 

Recently, I came across a maintained pure-go kerberos client and server [0].

I am now in the process of adding SPNEGO authentication to the Go 
avatica client [1].


For the implementation, the plan is to make it as close to the official 
(java) client's implementation as possible. For SPNEGO, to Java client 
uses these 2 parameters: principal and keytab.


The keytab parameter is easy to understand: a path to a keytab file.

I'd like to confirm what a valid string for the principal looks like.
- Is it a Service Principal Name?
- What are the valid formats for the principal? A valid SPN looks like 
User1/User2@realm.

- For the above example, I am assuming user2 can be optional.
- Can the realm be optional?

Cheers,
Francis

[0] https://github.com/jcmturner/gokrb5
[1] https://github.com/Boostport/avatica