[jira] [Commented] (PHOENIX-1081) CPU usage 100% With phoenix
[
https://issues.apache.org/jira/browse/PHOENIX-1081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14107865#comment-14107865
]
James Taylor commented on PHOENIX-1081:
---
I suspect this is fixed - would you mind trying this in our 3.1/4.1 RCs?
CPU usage 100% With phoenix
Key: PHOENIX-1081
URL: https://issues.apache.org/jira/browse/PHOENIX-1081
Project: Phoenix
Issue Type: Bug
Affects Versions: 3.0.0
Reporter: yang ming
Priority: Critical
Attachments: JMX.jpg, jstat.jpg, the jstack of all threads, the
jstack of thread 12725.jpg, the jstack of thread 12748.jpg, the threads of
regionserver process.jpg
The concurrent of the system is not high,but CPU usage often up to 100%.
I had stopped the system,but regionserver's CPU usage is still high.
what can case this problem?
table row count：6000 million
table ddl：
create table if not exists summary
(
videoid integer not null,
date date not null,
platform varchar not null,
device varchar not null,
systemgroup varchar not null,
system varchar not null,
vv bigint,
ts bigint,
up bigint,
down bigint,
comment bigint,
favori bigint,
favord bigint,
quote bigint,
reply bigint
constraint pk primary key (videoid, date,platform, device, systemgroup,system)
)salt_buckets = 30,versions=1,compression='snappy';
query 1：
select sum(vv) as sumvv,sum(comment) as sumcomment,sum(up) as sumup,sum(down)
as sumdown,sum(reply) as sumreply,count(*) as count from summary(reply
bigint) where videoid
in(137102991,151113895,171559204,171559439,171573932,171573932,171573932,171574082,171574082,171574164,171677219,171794335,171902734,172364368,172475141,172700554,172700554,172700554,172716705,172784258,172835778,173112067,173165316,173165316,173379601,173448315,173503961,173692664,173911358,174077089,174099017,174349633,174349877,174651474,174651474,174759297,174883566,174883566,174987670,174987670,175131298)
and date=to_date('2013-09-01','-MM-dd') and
date=to_date('2014-07-07','-MM-dd')
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (PHOENIX-1081) CPU usage 100% With phoenix
[
https://issues.apache.org/jira/browse/PHOENIX-1081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14059493#comment-14059493
]
yang ming commented on PHOENIX-1081:
IPC Server handler 69 on 60020 daemon prio=10 tid=0x4def5000
nid=0x31ed runnable [0x48c36000]
java.lang.Thread.State: RUNNABLE
at
org.apache.phoenix.filter.SkipScanFilter.navigate(SkipScanFilter.java:288)
at
org.apache.phoenix.filter.SkipScanFilter.filterKeyValue(SkipScanFilter.java:112)
at
org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:354)
{color:red}
at
org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:390)
{color}
at
org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:143)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:4047)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:4123)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:3990)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:3980)
at
org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver.scanUnordered(GroupedAggregateRegionObserver.java:384)
at
org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver.doPostScannerOpen(GroupedAggregateRegionObserver.java:133)
at
org.apache.phoenix.coprocessor.BaseScannerRegionObserver.postScannerOpen(BaseScannerRegionObserver.java:66)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postScannerOpen(RegionCoprocessorHost.java:1316)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.internalOpenScanner(HRegionServer.java:2588)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:2556)
at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:323)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1434)
CPU usage 100% With phoenix
Key: PHOENIX-1081
URL: https://issues.apache.org/jira/browse/PHOENIX-1081
Project: Phoenix
Issue Type: Bug
Affects Versions: 3.0.0
Reporter: yang ming
Priority: Critical
Attachments: JMX.jpg, jstat.jpg, the jstack of all threads, the
jstack of thread 12725.jpg, the jstack of thread 12748.jpg, the threads of
regionserver process.jpg
The concurrent of the system is not high,but CPU usage often up to 100%.
I had stopped the system,but regionserver's CPU usage is still high.
what can case this problem?
table row count：6000 million
table ddl：
create table if not exists summary
(
videoid integer not null,
date date not null,
platform varchar not null,
device varchar not null,
systemgroup varchar not null,
system varchar not null,
vv bigint,
ts bigint,
up bigint,
down bigint,
comment bigint,
favori bigint,
favord bigint,
quote bigint,
reply bigint
constraint pk primary key (videoid, date,platform, device, systemgroup,system)
)salt_buckets = 30,versions=1,compression='snappy';
query 1：
select sum(vv) as sumvv,sum(comment) as sumcomment,sum(up) as sumup,sum(down)
as sumdown,sum(reply) as sumreply,count(*) as count from summary(reply
bigint) where videoid
in(137102991,151113895,171559204,171559439,171573932,171573932,171573932,171574082,171574082,171574164,171677219,171794335,171902734,172364368,172475141,172700554,172700554,172700554,172716705,172784258,172835778,173112067,173165316,173165316,173379601,173448315,173503961,173692664,173911358,174077089,174099017,174349633,174349877,174651474,174651474,174759297,174883566,174883566,174987670,174987670,175131298)
and date=to_date('2013-09-01','-MM-dd') and
date=to_date('2014-07-07','-MM-dd')
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (PHOENIX-1081) CPU usage 100% With phoenix
[
https://issues.apache.org/jira/browse/PHOENIX-1081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14059499#comment-14059499
]
yang ming commented on PHOENIX-1081:
Hbase version is 0.94.19.I had added some debug log,may be the endless loop is
here.
LOOP: while((kv = this.heap.peek()) != null) {
// Check that the heap gives us KVs in an increasing order.
assert prevKV == null || comparator == null ||
comparator.compare(prevKV, kv) = 0 :
Key + prevKV + followed by a + smaller key + kv + in cf
+ store;
prevKV = kv;
{color:red}
ScanQueryMatcher.MatchCode qcode = matcher.match(kv);
{color}
switch(qcode) {
case INCLUDE:
case INCLUDE_AND_SEEK_NEXT_ROW:
case INCLUDE_AND_SEEK_NEXT_COL:
Filter f = matcher.getFilter();
outResult.add(f == null ? kv : f.transform(kv));
count++;
CPU usage 100% With phoenix
Key: PHOENIX-1081
URL: https://issues.apache.org/jira/browse/PHOENIX-1081
Project: Phoenix
Issue Type: Bug
Affects Versions: 3.0.0
Reporter: yang ming
Priority: Critical
Attachments: JMX.jpg, jstat.jpg, the jstack of all threads, the
jstack of thread 12725.jpg, the jstack of thread 12748.jpg, the threads of
regionserver process.jpg
The concurrent of the system is not high,but CPU usage often up to 100%.
I had stopped the system,but regionserver's CPU usage is still high.
what can case this problem?
table row count：6000 million
table ddl：
create table if not exists summary
(
videoid integer not null,
date date not null,
platform varchar not null,
device varchar not null,
systemgroup varchar not null,
system varchar not null,
vv bigint,
ts bigint,
up bigint,
down bigint,
comment bigint,
favori bigint,
favord bigint,
quote bigint,
reply bigint
constraint pk primary key (videoid, date,platform, device, systemgroup,system)
)salt_buckets = 30,versions=1,compression='snappy';
query 1：
select sum(vv) as sumvv,sum(comment) as sumcomment,sum(up) as sumup,sum(down)
as sumdown,sum(reply) as sumreply,count(*) as count from summary(reply
bigint) where videoid
in(137102991,151113895,171559204,171559439,171573932,171573932,171573932,171574082,171574082,171574164,171677219,171794335,171902734,172364368,172475141,172700554,172700554,172700554,172716705,172784258,172835778,173112067,173165316,173165316,173379601,173448315,173503961,173692664,173911358,174077089,174099017,174349633,174349877,174651474,174651474,174759297,174883566,174883566,174987670,174987670,175131298)
and date=to_date('2013-09-01','-MM-dd') and
date=to_date('2014-07-07','-MM-dd')
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (PHOENIX-1081) CPU usage 100% With phoenix
[
https://issues.apache.org/jira/browse/PHOENIX-1081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14057581#comment-14057581
]
yang ming commented on PHOENIX-1081:
[~jamestaylor]
CPU usage 100% With phoenix
Key: PHOENIX-1081
URL: https://issues.apache.org/jira/browse/PHOENIX-1081
Project: Phoenix
Issue Type: Bug
Affects Versions: 3.0.0
Reporter: yang ming
Priority: Critical
Attachments: JMX.jpg, jstat.jpg, the jstack of all threads, the
jstack of thread 12725.jpg, the jstack of thread 12748.jpg, the threads of
regionserver process.jpg
The concurrent of the system is not high,but CPU usage often up to 100%.
I had stopped the system,but regionserver's CPU usage is still high.
what can case this problem?
table row count：6000 million
table ddl：
create table if not exists summary
(
videoid integer not null,
date date not null,
platform varchar not null,
device varchar not null,
systemgroup varchar not null,
system varchar not null,
vv bigint,
ts bigint,
up bigint,
down bigint,
comment bigint,
favori bigint,
favord bigint,
quote bigint,
reply bigint
constraint pk primary key (videoid, date,platform, device, systemgroup,system)
)salt_buckets = 30,versions=1,compression='snappy';
query 1：
select sum(vv) as sumvv,sum(comment) as sumcomment,sum(up) as sumup,sum(down)
as sumdown,sum(reply) as sumreply,count(*) as count from summary(reply
bigint) where videoid
in(137102991,151113895,171559204,171559439,171573932,171573932,171573932,171574082,171574082,171574164,171677219,171794335,171902734,172364368,172475141,172700554,172700554,172700554,172716705,172784258,172835778,173112067,173165316,173165316,173379601,173448315,173503961,173692664,173911358,174077089,174099017,174349633,174349877,174651474,174651474,174759297,174883566,174883566,174987670,174987670,175131298)
and date=to_date('2013-09-01','-MM-dd') and
date=to_date('2014-07-07','-MM-dd')
--
This message was sent by Atlassian JIRA
(v6.2#6252)
