Re: Phoenix ODBC driver limitations

[Francis Chuang]

Namespace mapping is something you need to enable on the server (it's 
off by default).


See documentation for enabling it here: 
http://phoenix.apache.org/namspace_mapping.html


Francis

On 24/05/2018 5:23 AM, Stepan Migunov wrote:

Thanks you for response, Josh!

I got something like "Inconsistent namespace mapping properties" and thought
it was because it's impossible to set "isNamespaceMappingEnabled" for the
ODBC driver (client side). After your explanation I understood that the
"client" in this case is queryserver but not ODBC driver. And now I need to
check why queryserver doesn't apply this property.

-Original Message-
From: Josh Elser [mailto:els...@apache.org]
Sent: Wednesday, May 23, 2018 6:52 PM
To: user@phoenix.apache.org
Subject: Re: Phoenix ODBC driver limitations

I'd be surprised to hear that the ODBC driver would need to know anything
about namespace-mapping.

Do you have an error? Steps to reproduce an issue which you see?

The reason I am surprised is that namespace mapping is an implementation
detail of the JDBC driver which lives inside of PQS -- *not* the ODBC
driver. The trivial thing you can check would be to validate that the
hbase-site.xml which PQS references is up to date and that PQS was restarted
to pick up the newest version of hbase-site.xml

On 5/22/18 4:16 AM, Stepan Migunov wrote:

Hi,

Is the ODBC driver from Hortonworks the only way to access Phoenix from
.NET code now?
The problem is that driver has some critical limitations - it seems,
driver doesn't support Namespace Mapping (it couldn't be able to connect
to Phoenix if phoenix.schema.isNamespaceMappingEnabled=true) and doesn't
support query hints.

Regards,
Stepan.

RE: Phoenix ODBC driver limitations

[Stepan Migunov]

Thanks you for response, Josh!

I got something like "Inconsistent namespace mapping properties" and thought
it was because it's impossible to set "isNamespaceMappingEnabled" for the
ODBC driver (client side). After your explanation I understood that the
"client" in this case is queryserver but not ODBC driver. And now I need to
check why queryserver doesn't apply this property.

-Original Message-
From: Josh Elser [mailto:els...@apache.org]
Sent: Wednesday, May 23, 2018 6:52 PM
To: user@phoenix.apache.org
Subject: Re: Phoenix ODBC driver limitations

I'd be surprised to hear that the ODBC driver would need to know anything
about namespace-mapping.

Do you have an error? Steps to reproduce an issue which you see?

The reason I am surprised is that namespace mapping is an implementation
detail of the JDBC driver which lives inside of PQS -- *not* the ODBC
driver. The trivial thing you can check would be to validate that the
hbase-site.xml which PQS references is up to date and that PQS was restarted
to pick up the newest version of hbase-site.xml

On 5/22/18 4:16 AM, Stepan Migunov wrote:
> Hi,
>
> Is the ODBC driver from Hortonworks the only way to access Phoenix from
> .NET code now?
> The problem is that driver has some critical limitations - it seems,
> driver doesn't support Namespace Mapping (it couldn't be able to connect
> to Phoenix if phoenix.schema.isNamespaceMappingEnabled=true) and doesn't
> support query hints.
>
> Regards,
> Stepan.
>

Re: Cannot access from jdbc

[Nicolas Paris]

I solved the problem.
I added the hbase-site.xml file found on the cluster in my java classpath,
added few lines for kerberos access also.

Both helped.

Thanks,

2018-05-23 19:56 GMT+02:00 Josh Elser :

> Try enabling DEBUG logging for HBase and take a look at the RegionServer
> log identified by the hostname in the log message.
>
> Most of the time when you see this error, it's a result of HBase rejecting
> the incoming request for a Kerberos authentication issue.
>
>
> On 5/23/18 12:10 PM, Nicolas Paris wrote:
>
>> hi team
>>
>> I am having this error while trying to connect from a java program to a
>> working phoenix 4.7 (hdp build) instance:
>>
>>
>>   Wed May 23 16:40:56 CEST 2018, null, java.net.SocketTimeoutException:
>> callTimeout=6, callDuration=69037:
>>   Call to /: failed on local exception:
>> java.io.IOException: Connection reset by peer row 'SYSTEM:CATALOG,,'
>>   on table 'hbase:meta' at region=hbase:meta,,1.1588230740,
>> hostname=,,1526661975419, seqNum=0
>>
>> The hbase is securized with kerberos, and the whole is managed by
>> zookeeper.
>>
>> Any insight on this behavior ?
>>
>> Thanks a lot,
>>
>>
>>
>>
>>

Re: Hash aggregation

[Gerald Sangudi]

Hi Maryann,

I filed PHOENIX-4751 .

Is this likely to be reviewed soon (say next few weeks), or should I look
at the Phoenix source to estimate the scope / impact?

Thanks,
Gerald

On Tue, May 22, 2018 at 11:12 AM, Maryann Xue  wrote:

> Since the performance running a group-by aggregation on client side is
> most likely bad, it’s usually not desired. The original implementation was
> for functionality completeness only so it chose the easiest way, which
> reused some existing classes. In some cases, though, the client group-by
> can still be tolerable if there aren’t many distinct keys. So yes, please
> open a JIRA for implementing hash aggregation on client side. Thank you!
>
>
> Thanks,
> Maryann
>
> On Tue, May 22, 2018 at 10:50 AM Gerald Sangudi 
> wrote:
>
>> Hello,
>>
>> Any guidance or thoughts on the thread below?
>>
>> Thanks,
>> Gerald
>>
>>
>> On Fri, May 18, 2018 at 11:39 AM, Gerald Sangudi 
>> wrote:
>>
>>> Maryann,
>>>
>>> Can Phoenix provide hash aggregation on the client side? Are there
>>> design / implementation reasons not to, or should I file a ticket for this?
>>>
>>> Thanks,
>>> Gerald
>>>
>>> On Fri, May 18, 2018 at 11:29 AM, Maryann Xue 
>>> wrote:
>>>
 Hi Gerald,

 Phoenix does have hash aggregation. The reason why sort-based
 aggregation is used in your query plan is that the aggregation happens on
 the client side. And that is because sort-merge join is used (as hinted)
 which is a client driven join, and after that join stage all operations can
 only be on the client-side.


 Thanks,
 Marynn

 On Fri, May 18, 2018 at 10:57 AM, Gerald Sangudi 
 wrote:

> Hello,
>
> Does Phoenix provide hash aggregation? If not, is it on the roadmap,
> or should I file a ticket? We have aggregation queries that do not require
> sorted results.
>
> For example, this EXPLAIN plan shows a CLIENT SORT.
>
> *CREATE TABLE unsalted (   keyA BIGINT NOT NULL,   keyB BIGINT
> NOT NULL,   val SMALLINT,   CONSTRAINT pk PRIMARY KEY (keyA,
> keyB));*
>
>
> *EXPLAINSELECT /*+ USE_SORT_MERGE_JOIN */ t1.val v1, t2.val v2,
> COUNT(*) c FROM unsalted t1 JOIN unsalted t2 ON (t1.keyA = t2.keyA) GROUP
> BY t1.val,
> t2.val;++-++--+|
>PLAN   | EST_BYTES_READ | EST_ROWS_READ  |
> |++-++--+|
> SORT-MERGE-JOIN (INNER) TABLES | null | null |
> || CLIENT 1-CHUNK PARALLEL 1-WAY FULL SCAN OVER UNSALTED  | null | 
> null
> | || AND| null |
> null | || CLIENT 1-CHUNK PARALLEL 1-WAY FULL SCAN OVER UNSALTED  | 
> null
> | null | || CLIENT SORTED BY [TO_DECIMAL(T1.VAL), T2.VAL]  |
> null | null | || CLIENT AGGREGATE INTO DISTINCT ROWS BY [T1.VAL, T2.VAL]
>| null | null |
> |++-++--+*
> Thanks,
> Gerald
>


>>>
>>

Re: Cannot access from jdbc

[Josh Elser]

Try enabling DEBUG logging for HBase and take a look at the RegionServer 
log identified by the hostname in the log message.


Most of the time when you see this error, it's a result of HBase 
rejecting the incoming request for a Kerberos authentication issue.


On 5/23/18 12:10 PM, Nicolas Paris wrote:

hi team

I am having this error while trying to connect from a java program to a working 
phoenix 4.7 (hdp build) instance:


  Wed May 23 16:40:56 CEST 2018, null, java.net.SocketTimeoutException: 
callTimeout=6, callDuration=69037:
  Call to /: failed on local exception: java.io.IOException: 
Connection reset by peer row 'SYSTEM:CATALOG,,'
  on table 'hbase:meta' at region=hbase:meta,,1.1588230740, 
hostname=,,1526661975419, seqNum=0

The hbase is securized with kerberos, and the whole is managed by zookeeper.

Any insight on this behavior ?

Thanks a lot,

Re: Cannot access from jdbc

[Nicolas Paris]

Agreed James

Thanks !

2018-05-23 18:40 GMT+02:00 James Taylor :

> Hi Nicolas,
> I don't think it's fair to ask the Apache Phoenix community to investigate
> a vendor specific issue from a more than two year old forked repo. I have
> no idea which of the 912 JIRAs since Phoenix 4.7.0 made it into that fork.
>
> Thanks,
> James
>
> On Wed, May 23, 2018 at 9:26 AM, Nicolas Paris 
> wrote:
>
>> Hi James.
>>
>> I tried to make the update myself but without success.
>> This actualy a custom 4.7 version (here the commit log
>> https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.1/bk_
>> release-notes/content/patch_phoenix.html)
>> The commit log looks more recent than 2 years
>>
>>
>> 2018-05-23 18:16 GMT+02:00 James Taylor :
>>
>>> The 4.7 release is more than two years old. That's seven releases back
>>> from the current release we're voting on 4.14. I'd recommend working with
>>> your vendor and urging them to upgrade to a newer, supportable version.
>>>
>>> Thanks,
>>> James
>>>
>>> On Wed, May 23, 2018 at 9:10 AM, Nicolas Paris 
>>> wrote:
>>>
 hi team

 I am having this error while trying to connect from a java program to a 
 working phoenix 4.7 (hdp build) instance:


  Wed May 23 16:40:56 CEST 2018, null, java.net.SocketTimeoutException: 
 callTimeout=6, callDuration=69037:
  Call to /: failed on local exception: 
 java.io.IOException: Connection reset by peer row 'SYSTEM:CATALOG,,'
  on table 'hbase:meta' at region=hbase:meta,,1.1588230740, 
 hostname=,,1526661975419, seqNum=0

 The hbase is securized with kerberos, and the whole is managed by 
 zookeeper.

 Any insight on this behavior ?

 Thanks a lot,





>>>
>>
>

Re: Cannot access from jdbc

[James Taylor]

Hi Nicolas,
I don't think it's fair to ask the Apache Phoenix community to investigate
a vendor specific issue from a more than two year old forked repo. I have
no idea which of the 912 JIRAs since Phoenix 4.7.0 made it into that fork.

Thanks,
James

On Wed, May 23, 2018 at 9:26 AM, Nicolas Paris  wrote:

> Hi James.
>
> I tried to make the update myself but without success.
> This actualy a custom 4.7 version (here the commit log
> https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.1/
> bk_release-notes/content/patch_phoenix.html)
> The commit log looks more recent than 2 years
>
>
> 2018-05-23 18:16 GMT+02:00 James Taylor :
>
>> The 4.7 release is more than two years old. That's seven releases back
>> from the current release we're voting on 4.14. I'd recommend working with
>> your vendor and urging them to upgrade to a newer, supportable version.
>>
>> Thanks,
>> James
>>
>> On Wed, May 23, 2018 at 9:10 AM, Nicolas Paris 
>> wrote:
>>
>>> hi team
>>>
>>> I am having this error while trying to connect from a java program to a 
>>> working phoenix 4.7 (hdp build) instance:
>>>
>>>
>>>  Wed May 23 16:40:56 CEST 2018, null, java.net.SocketTimeoutException: 
>>> callTimeout=6, callDuration=69037:
>>>  Call to /: failed on local exception: java.io.IOException: 
>>> Connection reset by peer row 'SYSTEM:CATALOG,,'
>>>  on table 'hbase:meta' at region=hbase:meta,,1.1588230740, 
>>> hostname=,,1526661975419, seqNum=0
>>>
>>> The hbase is securized with kerberos, and the whole is managed by zookeeper.
>>>
>>> Any insight on this behavior ?
>>>
>>> Thanks a lot,
>>>
>>>
>>>
>>>
>>>
>>
>

Re: Cannot access from jdbc

[Nicolas Paris]

Hi James.

I tried to make the update myself but without success.
This actualy a custom 4.7 version (here the commit log
https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.1/bk_release-notes/content/patch_phoenix.html
)
The commit log looks more recent than 2 years


2018-05-23 18:16 GMT+02:00 James Taylor :

> The 4.7 release is more than two years old. That's seven releases back
> from the current release we're voting on 4.14. I'd recommend working with
> your vendor and urging them to upgrade to a newer, supportable version.
>
> Thanks,
> James
>
> On Wed, May 23, 2018 at 9:10 AM, Nicolas Paris 
> wrote:
>
>> hi team
>>
>> I am having this error while trying to connect from a java program to a 
>> working phoenix 4.7 (hdp build) instance:
>>
>>
>>  Wed May 23 16:40:56 CEST 2018, null, java.net.SocketTimeoutException: 
>> callTimeout=6, callDuration=69037:
>>  Call to /: failed on local exception: java.io.IOException: 
>> Connection reset by peer row 'SYSTEM:CATALOG,,'
>>  on table 'hbase:meta' at region=hbase:meta,,1.1588230740, 
>> hostname=,,1526661975419, seqNum=0
>>
>> The hbase is securized with kerberos, and the whole is managed by zookeeper.
>>
>> Any insight on this behavior ?
>>
>> Thanks a lot,
>>
>>
>>
>>
>>
>

Re: Cannot access from jdbc

[James Taylor]

The 4.7 release is more than two years old. That's seven releases back from
the current release we're voting on 4.14. I'd recommend working with your
vendor and urging them to upgrade to a newer, supportable version.

Thanks,
James

On Wed, May 23, 2018 at 9:10 AM, Nicolas Paris  wrote:

> hi team
>
> I am having this error while trying to connect from a java program to a 
> working phoenix 4.7 (hdp build) instance:
>
>
>  Wed May 23 16:40:56 CEST 2018, null, java.net.SocketTimeoutException: 
> callTimeout=6, callDuration=69037:
>  Call to /: failed on local exception: java.io.IOException: 
> Connection reset by peer row 'SYSTEM:CATALOG,,'
>  on table 'hbase:meta' at region=hbase:meta,,1.1588230740, 
> hostname=,,1526661975419, seqNum=0
>
> The hbase is securized with kerberos, and the whole is managed by zookeeper.
>
> Any insight on this behavior ?
>
> Thanks a lot,
>
>
>
>
>

Cannot access from jdbc

[Nicolas Paris]

hi team

I am having this error while trying to connect from a java program to
a working phoenix 4.7 (hdp build) instance:


 Wed May 23 16:40:56 CEST 2018, null, java.net.SocketTimeoutException:
callTimeout=6, callDuration=69037:
 Call to /: failed on local exception:
java.io.IOException: Connection reset by peer row 'SYSTEM:CATALOG,,'
 on table 'hbase:meta' at region=hbase:meta,,1.1588230740,
hostname=,,1526661975419, seqNum=0

The hbase is securized with kerberos, and the whole is managed by zookeeper.

Any insight on this behavior ?

Thanks a lot,

Re: Phoenix ODBC driver limitations

[Josh Elser]

I'd be surprised to hear that the ODBC driver would need to know 
anything about namespace-mapping.


Do you have an error? Steps to reproduce an issue which you see?

The reason I am surprised is that namespace mapping is an implementation 
detail of the JDBC driver which lives inside of PQS -- *not* the ODBC 
driver. The trivial thing you can check would be to validate that the 
hbase-site.xml which PQS references is up to date and that PQS was 
restarted to pick up the newest version of hbase-site.xml


On 5/22/18 4:16 AM, Stepan Migunov wrote:

Hi,

Is the ODBC driver from Hortonworks the only way to access Phoenix from .NET 
code now?
The problem is that driver has some critical limitations - it seems, driver 
doesn't support Namespace Mapping (it couldn't be able to connect to Phoenix if 
phoenix.schema.isNamespaceMappingEnabled=true) and doesn't support query hints.

Regards,
Stepan.

Re: Decrease HTTP chattiness?

[Josh Elser]

Yeah, as Francis says, this should already be exposed via the expected 
JDBC APIs.


Kevin -- can you share more details about what version(s) you're 
running? A sample program?


If you're running a new enough version, you can set the following log 
level via Log4j


org.apache.calcite.avatica.remote.ProtobufTranslationImpl=TRACE

which will give you a client-side dump of all of the "RPC data" being 
sent and received. This will help you see the frame_max_size that 
Francis alluded to :)


On 5/21/18 6:55 PM, Francis Chuang wrote:
I am not familiar with the JDBC driver, but Phoenix uses Avatica[1] 
under the hood. The protobuf documentation does state that it's possible 
to control the number of rows returned in each response. See 
frame_max_size under the FetchRequest[2] message. This may be something 
that you can set in the JDBC driver.


Francis

[1] https://calcite.apache.org/avatica
[2] 
https://calcite.apache.org/avatica/docs/protobuf_reference.html#fetchrequest 



On 22/05/2018 8:39 AM, Kevin Minder wrote:
It is possible to decrease the chattiness of the Phoenix JDBC driver 
operating in HTTP mode?

We've tried using stmt.setFetchSize() but this appears to be ignored.
As it stands new we appear to be getting about 100 rows per POST which 
presents a number of throughput issues when the results can be 100,000 
rows.