[
https://issues.apache.org/jira/browse/PHOENIX-5082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16729548#comment-16729548
]
Lars Hofhansl edited comment on PHOENIX-5082 at 12/27/18 1:39 PM:
------------------------------------------------------------------
Hi [~yonigo], thanks for looking.
Here's how I am testing:
# Single node setup. I run a single Namenode, Datanode, HMaster,
HRegionServer, ZK QuorumPeer, and TSO on a single machine (since it's almost
only the region server using CPU and IOPS this should be a valid setup).
# Set up a table in Phoenix like this: {{create table test (pk1 integer not
null, pk2 integer not null, pk3 integer not null, v1 float, v2 float, v3
integer CONSTRAINT pk PRIMARY KEY (pk1, pk2, pk3)) DISABLE_WAL=true,
TRANSACTIONAL=true;}}
# Seed the table: {{upsert into test values(rand()*10000000, rand()*10000000,
rand()*10000000, rand(), rand(), rand()*1000000);}}
# The repeatedly: {{upsert into test select rand()*10000000, rand()*10000000,
rand()*10000000, rand(), rand(), rand()*1000000 from test;}}
# Config for the TSO: {{waitStrategy: LOW_CPU}}
I filed OMID-122, OMID-123, OMID-124. Feel like having a look? :)
In order to isolate this to the actual commit, I turned autocommit off in
SQLLine (!autocommit off). Then issued a manual commit, it seems the time is
spent writing the shadow columns, but I am not 100% sure.
The behavior is interesting. It takes 1.2s to upsert 256 rows, and 2.4s to
upsert 512 rows, so something does not scale right in the number of rows in
Phoenix/Omid.
Oh, profiling... I used jvisualvm ([https://visualvm.github.io/] if your JDK
doesn't have it.)
was (Author: lhofhansl):
Hi [~yonigo], thanks for looking.
Here's how I am testing:
# Single node setup. I run a single Namenode, Datanode, HMaster,
HRegionServer, ZK QuorumPeer, and TSO on a single machine (since it's almost
only the region server using CPU and IOPS this should be a valid setup).
# Set up a table in Phoenix like this: {{create table test (pk1 integer not
null, pk2 integer not null, pk3 integer not null, v1 float, v2 float, v3
integer CONSTRAINT pk PRIMARY KEY (pk1, pk2, pk3)) DISABLE_WAL=true,
TRANSACTIONAL=true;}}
# Seed the table: {{upsert into test values(rand()*10000000, rand()*10000000,
rand()*10000000, rand(), rand(), rand()*1000000);}}
# The repeatedly: {{upsert into test select rand()*10000000, rand()*10000000,
rand()*10000000, rand(), rand(), rand()*1000000 from test;}}
# Config for the TSO: {{waitStrategy: LOW_CPU}}
I filed OMID-122, OMID-123, OMID-124. Feel like having a look? :)
In order to isolate this to the actual commit, I turned autocommit off in
SQLLine (!autocommit off). Then issued a manual commit, it seems the time is
spent writing the shadow columns, but I am not 100% sure.
The behavior is interesting. It takes 1.2s to upsert 256 rows, and 2.4s to
upsert 512 rows, so something does not scale right in the number of rows in
Phoenix/Omid.
Oh, profiling... I used jvisualvm ([https://visualvm.github.io/] if your JVM
doesn't have it.)
> Observations about the Phoenix-Omid integration
> -----------------------------------------------
>
> Key: PHOENIX-5082
> URL: https://issues.apache.org/jira/browse/PHOENIX-5082
> Project: Phoenix
> Issue Type: Task
> Reporter: Lars Hofhansl
> Assignee: Yonatan Gottesman
> Priority: Major
> Attachments: Omid-profile.png, Omid-sample.png
>
>
> Filing this issue to track my observations and perhaps some fixes.
> [~ohads]
> [~jamestaylor]
> [~yonigo]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
