Re: Multiple upserts via JDBC

[Sergey Soldatov]

Zack,
Actually command line and GUI tools are using the same JDBC layer. It
would be nice if you provide more information about the application
itself.
Meanwhile you may try to set autocommit to false for the connection
and use .commit when all upserts were done.

Thanks,
Sergey

On Fri, Feb 19, 2016 at 4:02 AM, Riesland, Zack
 wrote:
> Thanks Sergey,
>
> The upserts are much faster via command line or a GUI tool like Aquadata 
> Studio.
>
> Table structure is below.
>
> I'm creating a new user with 8 permissions. So 9 total upserts.
>
> Individually, via command line, this is almost instantaneous. But via JDBC, 
> it takes tens of seconds to minutes.
>
> create table user (
>   user_id varchar (40) not null,
>   password_hash varchar (200) ,
>   user_full_name varchar (40),
>   user_email_address varchar (60),
>   token varchar ( 36),
>   expiration date
>   CONSTRAINT pk_user PRIMARY KEY (user_id)
> );
>
> create table user_access(
>user_id varchar(30) not null ,
>screen_id tinyint not null, --key to sda.screen
>access_id tinyint --key to sda.screen_access
>CONSTRAINT pk_user_access PRIMARY KEY (user_id, screen_id)
> );
>
> -Original Message-
> From: sergey.solda...@gmail.com [mailto:sergey.solda...@gmail.com] On Behalf 
> Of Sergey Soldatov
> Sent: Friday, February 19, 2016 3:01 AM
> To: user@phoenix.apache.org
> Subject: Re: Multiple upserts via JDBC
>
> Hi Zack,
>
> Have you tried to use sqlline to manually do those upserts to check the 
> performance? Information about the tables structures would be useful as well.
>
> Thanks,
> Sergey
>
> On Tue, Feb 16, 2016 at 8:10 AM, Riesland, Zack  
> wrote:
>> I have a handful of VERY small phoenix tables (< 100 entries).
>>
>>
>>
>> I wrote some javascript to interact with the tables via servlet + JDBC.
>>
>>
>>
>> I can query the data almost instantaneously, but upserting is
>> extremely slow – on the order of tens of seconds to several minutes.
>>
>>
>>
>> The main write operation does 10 upserts. Is there a better way to do
>> this than 10 separate statement.execute() commands?
>>
>>
>>
>> Is there a way to pass all 10 at once?
>>
>>
>>
>> Any tips on why these upserts might be so slow? I see that the tables
>> are backed by one region, so the overhead should be minimal.
>>
>>
>>
>> Thanks!
>>
>>

question about a config property

[ashish tapdiya]

Hi,

Is phoenix.query.maxGlobalMemoryPercentage a server side property?

Thanks,
~Ashish

Re: Spark Phoenix Plugin

[Benjamin Kim]

Hi Josh,

When I run the following code in spark-shell for spark 1.6:

import org.apache.phoenix.spark._
val df = sqlContext.load("org.apache.phoenix.spark", Map("table" -> 
"TEST.MY_TEST", "zkUrl" -> “zk1,zk2,zk3:2181"))
df.select(df("ID")).show()

I get this error:

java.lang.IllegalStateException: unread block data

Thanks,
Ben


> On Feb 19, 2016, at 11:12 AM, Josh Mahonin  wrote:
> 
> What specifically doesn't work for you?
> 
> I have a Docker image that I used to do some basic testing on it with and 
> haven't run into any problems:
> https://github.com/jmahonin/docker-phoenix/tree/phoenix_spark 
> 
> 
> On Fri, Feb 19, 2016 at 12:40 PM, Benjamin Kim  > wrote:
> All,
> 
> Thanks for the help. I have switched out Cloudera’s HBase 1.0.0 with the 
> current Apache HBase 1.1.3. Also, I installed Phoenix 4.7.0, and everything 
> works fine except for the Phoenix Spark Plugin. I wonder if it’s a version 
> incompatibility issue with Spark 1.6. Has anyone tried compiling 4.7.0 using 
> Spark 1.6?
> 
> Thanks,
> Ben
> 
>> On Feb 12, 2016, at 6:33 AM, Benjamin Kim > > wrote:
>> 
>> Anyone know when Phoenix 4.7 will be officially released? And what Cloudera 
>> distribution versions will it be compatible with?
>> 
>> Thanks,
>> Ben
>> 
>>> On Feb 10, 2016, at 11:03 AM, Benjamin Kim >> > wrote:
>>> 
>>> Hi Pierre,
>>> 
>>> I am getting this error now.
>>> 
>>> Error: org.apache.phoenix.exception.PhoenixIOException: 
>>> org.apache.hadoop.hbase.DoNotRetryIOException: 
>>> SYSTEM.CATALOG,,1453397732623.8af7b44f3d7609eb301ad98641ff2611.: 
>>> org.apache.hadoop.hbase.client.Delete.setAttribute(Ljava/lang/String;[B)Lorg/apache/hadoop/hbase/client/Delete;
>>> 
>>> I even tried to use sqlline.py to do some queries too. It resulted in the 
>>> same error. I followed the installation instructions. Is there something 
>>> missing?
>>> 
>>> Thanks,
>>> Ben
>>> 
>>> 
 On Feb 9, 2016, at 10:20 AM, Ravi Kiran >>> > wrote:
 
 Hi Pierre,
 
   Try your luck for building the artifacts from 
 https://github.com/chiastic-security/phoenix-for-cloudera 
 . Hopefully it 
 helps.
 
 Regards
 Ravi .
 
 On Tue, Feb 9, 2016 at 10:04 AM, Benjamin Kim >>> > wrote:
 Hi Pierre,
 
 I found this article about how Cloudera’s version of HBase is very 
 different than Apache HBase so it must be compiled using Cloudera’s repo 
 and versions. But, I’m not having any success with it.
 
 http://stackoverflow.com/questions/31849454/using-phoenix-with-cloudera-hbase-installed-from-repo
  
 
 
 There’s also a Chinese site that does the same thing.
 
 https://www.zybuluo.com/xtccc/note/205739 
 
 
 I keep getting errors like the one’s below.
 
 [ERROR] 
 /opt/tools/phoenix/phoenix-core/src/main/java/org/apache/hadoop/hbase/regionserver/LocalIndexMerger.java:[110,29]
  cannot find symbol
 [ERROR] symbol:   class Region
 [ERROR] location: class 
 org.apache.hadoop.hbase.regionserver.LocalIndexMerger
 …
 
 Have you tried this also?
 
 As a last resort, we will have to abandon Cloudera’s HBase for Apache’s 
 HBase.
 
 Thanks,
 Ben
 
 
> On Feb 8, 2016, at 11:04 PM, pierre lacave  > wrote:
> 
> Havent met that one.
> 
> According to SPARK-1867, the real issue is hidden.
> 
> I d process by elimination, maybe try in local[*] mode first
> 
> https://issues.apache.org/jira/plugins/servlet/mobile#issue/SPARK-1867 
> 
> On Tue, 9 Feb 2016, 04:58 Benjamin Kim  > wrote:
> Pierre,
> 
> I got it to work using phoenix-4.7.0-HBase-1.0-client-spark.jar. But, 
> now, I get this error:
> 
> org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 
> in stage 0.0 failed 4 times, most recent failure: Lost task 0.3 in stage 
> 0.0 (TID 3, prod-dc1-datanode151.pdc1i.gradientx.com 
> ): 
> java.lang.IllegalStateException: unread block data
> 
> It happens when I do:
> 
> df.show()
> 
> Getting closer…
> 
> Thanks,
> Ben
> 
> 
> 
>> On Feb 8, 2016, at 2:57 PM, pierre lacave > > wrote:
>> 
>> This is the wrong client jar try with the one named 
>> phoenix-4.7.0-HBase-1.1-client-spark.jar 
>> 
>> 
>> On Mon, 8 Feb 20

Re: Spark Phoenix Plugin

[Josh Mahonin]

What specifically doesn't work for you?

I have a Docker image that I used to do some basic testing on it with and
haven't run into any problems:
https://github.com/jmahonin/docker-phoenix/tree/phoenix_spark

On Fri, Feb 19, 2016 at 12:40 PM, Benjamin Kim  wrote:

> All,
>
> Thanks for the help. I have switched out Cloudera’s HBase 1.0.0 with the
> current Apache HBase 1.1.3. Also, I installed Phoenix 4.7.0, and everything
> works fine except for the Phoenix Spark Plugin. I wonder if it’s a version
> incompatibility issue with Spark 1.6. Has anyone tried compiling 4.7.0
> using Spark 1.6?
>
> Thanks,
> Ben
>
> On Feb 12, 2016, at 6:33 AM, Benjamin Kim  wrote:
>
> Anyone know when Phoenix 4.7 will be officially released? And what
> Cloudera distribution versions will it be compatible with?
>
> Thanks,
> Ben
>
> On Feb 10, 2016, at 11:03 AM, Benjamin Kim  wrote:
>
> Hi Pierre,
>
> I am getting this error now.
>
> Error: org.apache.phoenix.exception.PhoenixIOException:
> org.apache.hadoop.hbase.DoNotRetryIOException:
> SYSTEM.CATALOG,,1453397732623.8af7b44f3d7609eb301ad98641ff2611.:
> org.apache.hadoop.hbase.client.Delete.setAttribute(Ljava/lang/String;[B)Lorg/apache/hadoop/hbase/client/Delete;
>
> I even tried to use sqlline.py to do some queries too. It resulted in the
> same error. I followed the installation instructions. Is there something
> missing?
>
> Thanks,
> Ben
>
>
> On Feb 9, 2016, at 10:20 AM, Ravi Kiran  wrote:
>
> Hi Pierre,
>
>   Try your luck for building the artifacts from
> https://github.com/chiastic-security/phoenix-for-cloudera. Hopefully it
> helps.
>
> Regards
> Ravi .
>
> On Tue, Feb 9, 2016 at 10:04 AM, Benjamin Kim  wrote:
>
>> Hi Pierre,
>>
>> I found this article about how Cloudera’s version of HBase is very
>> different than Apache HBase so it must be compiled using Cloudera’s repo
>> and versions. But, I’m not having any success with it.
>>
>>
>> http://stackoverflow.com/questions/31849454/using-phoenix-with-cloudera-hbase-installed-from-repo
>>
>> There’s also a Chinese site that does the same thing.
>>
>> https://www.zybuluo.com/xtccc/note/205739
>>
>> I keep getting errors like the one’s below.
>>
>> [ERROR]
>> /opt/tools/phoenix/phoenix-core/src/main/java/org/apache/hadoop/hbase/regionserver/LocalIndexMerger.java:[110,29]
>> cannot find symbol
>> [ERROR] symbol:   class Region
>> [ERROR] location: class
>> org.apache.hadoop.hbase.regionserver.LocalIndexMerger
>> …
>>
>> Have you tried this also?
>>
>> As a last resort, we will have to abandon Cloudera’s HBase for Apache’s
>> HBase.
>>
>> Thanks,
>> Ben
>>
>>
>> On Feb 8, 2016, at 11:04 PM, pierre lacave  wrote:
>>
>> Havent met that one.
>>
>> According to SPARK-1867, the real issue is hidden.
>>
>> I d process by elimination, maybe try in local[*] mode first
>>
>> https://issues.apache.org/jira/plugins/servlet/mobile#issue/SPARK-1867
>>
>> On Tue, 9 Feb 2016, 04:58 Benjamin Kim  wrote:
>>
>>> Pierre,
>>>
>>> I got it to work using phoenix-4.7.0-HBase-1.0-client-spark.jar. But,
>>> now, I get this error:
>>>
>>> org.apache.spark.SparkException: Job aborted due to stage failure: Task
>>> 0 in stage 0.0 failed 4 times, most recent failure: Lost task 0.3 in stage
>>> 0.0 (TID 3, prod-dc1-datanode151.pdc1i.gradientx.com):
>>> java.lang.IllegalStateException: unread block data
>>>
>>> It happens when I do:
>>>
>>> df.show()
>>>
>>> Getting closer…
>>>
>>> Thanks,
>>> Ben
>>>
>>>
>>>
>>> On Feb 8, 2016, at 2:57 PM, pierre lacave  wrote:
>>>
>>> This is the wrong client jar try with the one named
>>> phoenix-4.7.0-HBase-1.1-client-spark.jar
>>>
>>> On Mon, 8 Feb 2016, 22:29 Benjamin Kim  wrote:
>>>
 Hi Josh,

 I tried again by putting the settings within the spark-default.conf.


 spark.driver.extraClassPath=/opt/tools/phoenix/phoenix-4.7.0-HBase-1.0-client.jar

 spark.executor.extraClassPath=/opt/tools/phoenix/phoenix-4.7.0-HBase-1.0-client.jar

 I still get the same error using the code below.

 import org.apache.phoenix.spark._
 val df = sqlContext.load("org.apache.phoenix.spark", Map("table" ->
 "TEST.MY_TEST", "zkUrl" -> “zk1,zk2,zk3:2181"))

 Can you tell me what else you’re doing?

 Thanks,
 Ben


 On Feb 8, 2016, at 1:44 PM, Josh Mahonin  wrote:

 Hi Ben,

 I'm not sure about the format of those command line options you're
 passing. I've had success with spark-shell just by setting the
 'spark.executor.extraClassPath' and 'spark.driver.extraClassPath' options
 on the spark config, as per the docs [1].

 I'm not sure if there's anything special needed for CDH or not though.
 I also have a docker image I've been toying with which has a working
 Spark/Phoenix setup using the Phoenix 4.7.0 RC and Spark 1.6.0. It might be
 a useful reference for you as well [2].

 Good luck,

 Josh

 [1] https://phoenix.apache.org/phoenix_spark.html
 [2] https://github.com/jmahonin/d

Re: YCSB with Phoenix?

[Andrew Purtell]

Never mind that last question. Check the email thread and ye shall find the 
answer... 

> On Feb 19, 2016, at 11:10 AM, Andrew Purtell  wrote:
> 
> The brianfrankcooper github repo (https://github.com/brianfrankcooper/YCSB) 
> is still the "official" home of YCSB as far as I know. YCSB ships with a JDBC 
> client driver. Just use that with the Phoenix client jar on the classpath and 
> the correct connection string for Phoenix.
> 
> What patch is this James? Does it change or improve the YCSB JDBC driver?
> 
>> On Feb 19, 2016, at 10:58 AM, James Taylor  wrote:
>> 
>> I believe some folks over in the HBase community have revived YCSB, but I'm 
>> not sure where it's new home is. Also, not sure if they applied Mujtaba's 
>> patch. I'd recommend asking over on the HBase dev or user list.
>> 
>> FWIW, we developed Pherf to enable Phoenix users to compare various Phoenix 
>> configurations and schemas for a given use case against each other. It's 
>> also super useful to catch performance and functional regressions with 
>> representative data sizes.
>> 
>> Thanks,
>> 
>> James
>> 
>>> On Fri, Feb 19, 2016 at 10:38 AM, Mujtaba Chohan  wrote:
>>> You can apply this patch on YCSB to test out Phoenix with variable number 
>>> of VARCHAR fields as well as test out combination of single/multiple CFs, 
>>> compression and salt buckets. See usage details here. 
>>> 
>>> You can also use Pherf which is a Phoenix specific performance test suite 
>>> that uses XML scenario to define how synthetic data is generated and 
>>> queried. It's launcher scripts are in phoenix/bin directory.
>>> 
>>> HTH
>>> 
 On Fri, Feb 19, 2016 at 9:54 AM, Gaurav Kanade  
 wrote:
 Hi All
 
 I am relatively new to Phoenix and was working on some performance 
 tuning/benchmarking experiments and tried to search online for whether 
 there exists YCSB client to go through Phoenix.
 
 I came across this https://github.com/brianfrankcooper/YCSB/pull/178 and 
 some related links but it seems this didnt seem to have moved forward.
 
 Could someone help me know where to look for the latest status on this ? 
 Does there exist an easy way to test Phoenix via YCSB and if not what are 
 the other existing options?
 
 Best,
 Gaurav
 
 
>>> 
>> 

Re: YCSB with Phoenix?

[Andrew Purtell]

The brianfrankcooper github repo (https://github.com/brianfrankcooper/YCSB) is 
still the "official" home of YCSB as far as I know. YCSB ships with a JDBC 
client driver. Just use that with the Phoenix client jar on the classpath and 
the correct connection string for Phoenix.

What patch is this James? Does it change or improve the YCSB JDBC driver?

> On Feb 19, 2016, at 10:58 AM, James Taylor  wrote:
> 
> I believe some folks over in the HBase community have revived YCSB, but I'm 
> not sure where it's new home is. Also, not sure if they applied Mujtaba's 
> patch. I'd recommend asking over on the HBase dev or user list.
> 
> FWIW, we developed Pherf to enable Phoenix users to compare various Phoenix 
> configurations and schemas for a given use case against each other. It's also 
> super useful to catch performance and functional regressions with 
> representative data sizes.
> 
> Thanks,
> 
> James
> 
>> On Fri, Feb 19, 2016 at 10:38 AM, Mujtaba Chohan  wrote:
>> You can apply this patch on YCSB to test out Phoenix with variable number of 
>> VARCHAR fields as well as test out combination of single/multiple CFs, 
>> compression and salt buckets. See usage details here. 
>> 
>> You can also use Pherf which is a Phoenix specific performance test suite 
>> that uses XML scenario to define how synthetic data is generated and 
>> queried. It's launcher scripts are in phoenix/bin directory.
>> 
>> HTH
>> 
>>> On Fri, Feb 19, 2016 at 9:54 AM, Gaurav Kanade  
>>> wrote:
>>> Hi All
>>> 
>>> I am relatively new to Phoenix and was working on some performance 
>>> tuning/benchmarking experiments and tried to search online for whether 
>>> there exists YCSB client to go through Phoenix.
>>> 
>>> I came across this https://github.com/brianfrankcooper/YCSB/pull/178 and 
>>> some related links but it seems this didnt seem to have moved forward.
>>> 
>>> Could someone help me know where to look for the latest status on this ? 
>>> Does there exist an easy way to test Phoenix via YCSB and if not what are 
>>> the other existing options?
>>> 
>>> Best,
>>> Gaurav
> 

Re: YCSB with Phoenix?

[James Taylor]

I believe some folks over in the HBase community have revived YCSB, but I'm
not sure where it's new home is. Also, not sure if they applied Mujtaba's
patch. I'd recommend asking over on the HBase dev or user list.

FWIW, we developed Pherf to enable Phoenix users to compare various Phoenix
configurations and schemas for a given use case against each other. It's
also super useful to catch performance and functional regressions with
representative data sizes.

Thanks,

James

On Fri, Feb 19, 2016 at 10:38 AM, Mujtaba Chohan  wrote:

> You can apply this patch on YCSB to test out Phoenix with variable number
> of VARCHAR fields as well as test out combination of single/multiple CFs,
> compression and salt buckets. See usage details here
> .
>
>
> You can also use Pherf
> 
> which is a Phoenix specific performance test suite that uses XML scenario
> to define how synthetic data is generated and queried. It's launcher
> scripts are in phoenix/bin directory.
>
> HTH
>
> On Fri, Feb 19, 2016 at 9:54 AM, Gaurav Kanade 
> wrote:
>
>> Hi All
>>
>> I am relatively new to Phoenix and was working on some performance
>> tuning/benchmarking experiments and tried to search online for whether
>> there exists YCSB client to go through Phoenix.
>>
>> I came across this https://github.com/brianfrankcooper/YCSB/pull/178 and
>> some related links but it seems this didnt seem to have moved forward.
>>
>> Could someone help me know where to look for the latest status on this ?
>> Does there exist an easy way to test Phoenix via YCSB and if not what are
>> the other existing options?
>>
>> Best,
>> Gaurav
>>
>>
>>
>

Re: YCSB with Phoenix?

[Mujtaba Chohan]

You can apply this patch on YCSB to test out Phoenix with variable number
of VARCHAR fields as well as test out combination of single/multiple CFs,
compression and salt buckets. See usage details here
.


You can also use Pherf

which is a Phoenix specific performance test suite that uses XML scenario
to define how synthetic data is generated and queried. It's launcher
scripts are in phoenix/bin directory.

HTH

On Fri, Feb 19, 2016 at 9:54 AM, Gaurav Kanade 
wrote:

> Hi All
>
> I am relatively new to Phoenix and was working on some performance
> tuning/benchmarking experiments and tried to search online for whether
> there exists YCSB client to go through Phoenix.
>
> I came across this https://github.com/brianfrankcooper/YCSB/pull/178 and
> some related links but it seems this didnt seem to have moved forward.
>
> Could someone help me know where to look for the latest status on this ?
> Does there exist an easy way to test Phoenix via YCSB and if not what are
> the other existing options?
>
> Best,
> Gaurav
>
>
>

YCSB with Phoenix?

[Gaurav Kanade]

Hi All

I am relatively new to Phoenix and was working on some performance
tuning/benchmarking experiments and tried to search online for whether
there exists YCSB client to go through Phoenix.

I came across this https://github.com/brianfrankcooper/YCSB/pull/178 and
some related links but it seems this didnt seem to have moved forward.

Could someone help me know where to look for the latest status on this ?
Does there exist an easy way to test Phoenix via YCSB and if not what are
the other existing options?

Best,
Gaurav

Re: Spark Phoenix Plugin

[Benjamin Kim]

All,

Thanks for the help. I have switched out Cloudera’s HBase 1.0.0 with the 
current Apache HBase 1.1.3. Also, I installed Phoenix 4.7.0, and everything 
works fine except for the Phoenix Spark Plugin. I wonder if it’s a version 
incompatibility issue with Spark 1.6. Has anyone tried compiling 4.7.0 using 
Spark 1.6?

Thanks,
Ben

> On Feb 12, 2016, at 6:33 AM, Benjamin Kim  wrote:
> 
> Anyone know when Phoenix 4.7 will be officially released? And what Cloudera 
> distribution versions will it be compatible with?
> 
> Thanks,
> Ben
> 
>> On Feb 10, 2016, at 11:03 AM, Benjamin Kim > > wrote:
>> 
>> Hi Pierre,
>> 
>> I am getting this error now.
>> 
>> Error: org.apache.phoenix.exception.PhoenixIOException: 
>> org.apache.hadoop.hbase.DoNotRetryIOException: 
>> SYSTEM.CATALOG,,1453397732623.8af7b44f3d7609eb301ad98641ff2611.: 
>> org.apache.hadoop.hbase.client.Delete.setAttribute(Ljava/lang/String;[B)Lorg/apache/hadoop/hbase/client/Delete;
>> 
>> I even tried to use sqlline.py to do some queries too. It resulted in the 
>> same error. I followed the installation instructions. Is there something 
>> missing?
>> 
>> Thanks,
>> Ben
>> 
>> 
>>> On Feb 9, 2016, at 10:20 AM, Ravi Kiran >> > wrote:
>>> 
>>> Hi Pierre,
>>> 
>>>   Try your luck for building the artifacts from 
>>> https://github.com/chiastic-security/phoenix-for-cloudera 
>>> . Hopefully it 
>>> helps.
>>> 
>>> Regards
>>> Ravi .
>>> 
>>> On Tue, Feb 9, 2016 at 10:04 AM, Benjamin Kim >> > wrote:
>>> Hi Pierre,
>>> 
>>> I found this article about how Cloudera’s version of HBase is very 
>>> different than Apache HBase so it must be compiled using Cloudera’s repo 
>>> and versions. But, I’m not having any success with it.
>>> 
>>> http://stackoverflow.com/questions/31849454/using-phoenix-with-cloudera-hbase-installed-from-repo
>>>  
>>> 
>>> 
>>> There’s also a Chinese site that does the same thing.
>>> 
>>> https://www.zybuluo.com/xtccc/note/205739 
>>> 
>>> 
>>> I keep getting errors like the one’s below.
>>> 
>>> [ERROR] 
>>> /opt/tools/phoenix/phoenix-core/src/main/java/org/apache/hadoop/hbase/regionserver/LocalIndexMerger.java:[110,29]
>>>  cannot find symbol
>>> [ERROR] symbol:   class Region
>>> [ERROR] location: class 
>>> org.apache.hadoop.hbase.regionserver.LocalIndexMerger
>>> …
>>> 
>>> Have you tried this also?
>>> 
>>> As a last resort, we will have to abandon Cloudera’s HBase for Apache’s 
>>> HBase.
>>> 
>>> Thanks,
>>> Ben
>>> 
>>> 
 On Feb 8, 2016, at 11:04 PM, pierre lacave >>> > wrote:
 
 Havent met that one.
 
 According to SPARK-1867, the real issue is hidden.
 
 I d process by elimination, maybe try in local[*] mode first
 
 https://issues.apache.org/jira/plugins/servlet/mobile#issue/SPARK-1867 
 
 On Tue, 9 Feb 2016, 04:58 Benjamin Kim >>> > wrote:
 Pierre,
 
 I got it to work using phoenix-4.7.0-HBase-1.0-client-spark.jar. But, now, 
 I get this error:
 
 org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 
 in stage 0.0 failed 4 times, most recent failure: Lost task 0.3 in stage 
 0.0 (TID 3, prod-dc1-datanode151.pdc1i.gradientx.com 
 ): 
 java.lang.IllegalStateException: unread block data
 
 It happens when I do:
 
 df.show()
 
 Getting closer…
 
 Thanks,
 Ben
 
 
 
> On Feb 8, 2016, at 2:57 PM, pierre lacave  > wrote:
> 
> This is the wrong client jar try with the one named 
> phoenix-4.7.0-HBase-1.1-client-spark.jar 
> 
> 
> On Mon, 8 Feb 2016, 22:29 Benjamin Kim  > wrote:
> Hi Josh,
> 
> I tried again by putting the settings within the spark-default.conf.
> 
> spark.driver.extraClassPath=/opt/tools/phoenix/phoenix-4.7.0-HBase-1.0-client.jar
> spark.executor.extraClassPath=/opt/tools/phoenix/phoenix-4.7.0-HBase-1.0-client.jar
> 
> I still get the same error using the code below.
> 
> import org.apache.phoenix.spark._
> val df = sqlContext.load("org.apache.phoenix.spark", Map("table" -> 
> "TEST.MY_TEST", "zkUrl" -> “zk1,zk2,zk3:2181"))
> 
> Can you tell me what else you’re doing?
> 
> Thanks,
> Ben
> 
> 
>> On Feb 8, 2016, at 1:44 PM, Josh Mahonin > > wrote:
>> 
>> Hi Ben,
>> 
>> I'm not sure about the format of those command line options you're 
>> passing. I've had success with spark-shell just by se

RE: Multiple upserts via JDBC

[Riesland, Zack]

Thanks Sergey,

The upserts are much faster via command line or a GUI tool like Aquadata Studio.

Table structure is below.

I'm creating a new user with 8 permissions. So 9 total upserts.

Individually, via command line, this is almost instantaneous. But via JDBC, it 
takes tens of seconds to minutes.

create table user (
  user_id varchar (40) not null,
  password_hash varchar (200) ,
  user_full_name varchar (40),
  user_email_address varchar (60),
  token varchar ( 36),
  expiration date
  CONSTRAINT pk_user PRIMARY KEY (user_id)
);

create table user_access(
   user_id varchar(30) not null ,
   screen_id tinyint not null, --key to sda.screen
   access_id tinyint --key to sda.screen_access
   CONSTRAINT pk_user_access PRIMARY KEY (user_id, screen_id)
);

-Original Message-
From: sergey.solda...@gmail.com [mailto:sergey.solda...@gmail.com] On Behalf Of 
Sergey Soldatov
Sent: Friday, February 19, 2016 3:01 AM
To: user@phoenix.apache.org
Subject: Re: Multiple upserts via JDBC

Hi Zack,

Have you tried to use sqlline to manually do those upserts to check the 
performance? Information about the tables structures would be useful as well.

Thanks,
Sergey

On Tue, Feb 16, 2016 at 8:10 AM, Riesland, Zack  
wrote:
> I have a handful of VERY small phoenix tables (< 100 entries).
>
>
>
> I wrote some javascript to interact with the tables via servlet + JDBC.
>
>
>
> I can query the data almost instantaneously, but upserting is 
> extremely slow – on the order of tens of seconds to several minutes.
>
>
>
> The main write operation does 10 upserts. Is there a better way to do 
> this than 10 separate statement.execute() commands?
>
>
>
> Is there a way to pass all 10 at once?
>
>
>
> Any tips on why these upserts might be so slow? I see that the tables 
> are backed by one region, so the overhead should be minimal.
>
>
>
> Thanks!
>
>

Re: Multiple upserts via JDBC

[Sergey Soldatov]

Hi Zack,

Have you tried to use sqlline to manually do those upserts to check
the performance? Information about the tables structures would be
useful as well.

Thanks,
Sergey

On Tue, Feb 16, 2016 at 8:10 AM, Riesland, Zack
 wrote:
> I have a handful of VERY small phoenix tables (< 100 entries).
>
>
>
> I wrote some javascript to interact with the tables via servlet + JDBC.
>
>
>
> I can query the data almost instantaneously, but upserting is extremely slow
> – on the order of tens of seconds to several minutes.
>
>
>
> The main write operation does 10 upserts. Is there a better way to do this
> than 10 separate statement.execute() commands?
>
>
>
> Is there a way to pass all 10 at once?
>
>
>
> Any tips on why these upserts might be so slow? I see that the tables are
> backed by one region, so the overhead should be minimal.
>
>
>
> Thanks!
>
>