Query optimization

2019-06-19 Thread Alexander Batyrshin
Hello, We have 2 tables: Table1 - big one (2000M+ rows): CREATE TABLE table1 ( pk varchar PRIMARY KEY, col varchar ); Table2 - small one (300K rows): CREATE TABLE table2 ( pk varchar PRIMARY KEY, other varchar ); Query like this work fast (~ 30sec): SELECT table1.pk,

java.io.IOException: Added a key not lexically larger than previous

2019-06-19 Thread Alexander Batyrshin
Hello, Are there any ideas where this problem comes from and how to fix? Jun 18 21:38:05 prod022 hbase[148581]: 2019-06-18 21:38:05,348 WARN [MemStoreFlusher.0] regionserver.HStore: Failed flushing store file, retrying num=9 Jun 18 21:38:05 prod022 hbase[148581]: java.io.IOException: Added a

Re: Query optimization

2019-06-19 Thread Alexander Batyrshin
he explain plan to see if it's working as expected > > On Wed, Jun 19, 2019 at 7:43 AM Alexander Batyrshin <0x62...@gmail.com > <mailto:0x62...@gmail.com>> wrote: > Hello, > We have 2 tables: > > Table1 - big one (2000M+ rows): > > CREATE TABLE ta

Re: java.io.IOException: Added a key not lexically larger than previous

2019-08-15 Thread Alexander Batyrshin
Is is possible that Phoenix is the reason of this problem? > On 20 Jun 2019, at 04:16, Alexander Batyrshin <0x62...@gmail.com> wrote: > > Hello, > Are there any ideas where this problem comes from and how to fix? > > Jun 18 21:38:05 prod022 hbase[148581]: 2019

Re: java.io.IOException: Added a key not lexically larger than previous

2019-08-15 Thread Alexander Batyrshin
t seeing if you've shared this previously on this or another thread. > Sorry if you have. > > Short-answer, it's possible that something around secondary indexing in > Phoenix causes this but not possible to definitively say in a vaccuum. > > On 8/15/19 1:19 PM, Alexander Batyrshin w

Re: java.io.IOException: Added a key not lexically larger than previous

2019-08-15 Thread Alexander Batyrshin
> On 15 Aug 2019, at 21:27, Josh Elser wrote: > > Short-answer, it's possible that something around secondary indexing in > Phoenix causes this but not possible to definitively say in a vaccuum. As I see region-server crashes on main table (not index) memstore flush. How can I help to

Re: java.io.IOException: Added a key not lexically larger than previous

2019-08-15 Thread Alexander Batyrshin
> Since you're using a global index, which stores the index data in a separate > table (and hence different regions, each of which has a different MemStore / > set of HFiles), and the error's happening to the base table, I'd be surprised > if Phoenix indexing is related. AFAIK Phoenix handle

Re: java.io.IOException: Added a key not lexically larger than previous

2019-08-26 Thread Alexander Batyrshin
Looks like problem begun with 4.14.2 Maybe somehow https://issues.apache.org/jira/browse/PHOENIX-5266 <https://issues.apache.org/jira/browse/PHOENIX-5266> can re-apply mutations to main table with bugs? > On 20 Jun 2019, at 04:16, Alexander Batyrshin <0x62...@gmail.com> wr

Any reason for so small phoenix.mutate.batchSize by default?

2019-09-03 Thread Alexander Batyrshin
Hello all, 1) There is bug in documentation - http://phoenix.apache.org/tuning.html phoenix.mutate.batchSize is not 1000, but only 100 by default

Re: Any reason for so small phoenix.mutate.batchSize by default?

2019-09-03 Thread Alexander Batyrshin
pt statistics > * How much data ends up being sent to a RegionServer in one RPC? Where I can get this metric? > On 3 Sep 2019, at 17:19, Josh Elser wrote: > > Hey Alexander, > > Was just poking at the code for this: it looks like this is really just > determining the num

Re: Any reason for so small phoenix.mutate.batchSize by default?

2019-09-04 Thread Alexander Batyrshin
> On 3 Sep 2019, at 19:45, Alexander Batyrshin <0x62...@gmail.com> wrote: > > I observer that there is some extra mutations in batch for every my UPSERTs > For example if app call executeUpdate() only 5 times then on commit there > will be "DEBUG MutationState:1046 -

Re: Why we change index state to PENDING_DISABLE on RegionMovedException

2019-09-10 Thread Alexander Batyrshin
if the next write succeeds, it > can flip back to "ACTIVE". > > On Tue, Sep 10, 2019 at 2:29 PM Alexander Batyrshin <0x62...@gmail.com > <mailto:0x62...@gmail.com>> wrote: > As I know RegionMovedException is not a problem at all, its just notification > th

Why we change index state to PENDING_DISABLE on RegionMovedException

2019-09-10 Thread Alexander Batyrshin
As I know RegionMovedException is not a problem at all, its just notification that we need to update meta information about table regions and retry. Why we do extra work with changing state of index? 2019-09-10 22:35:00,764 WARN [hconnection-0x4a63b6ea-shared--pool10-t961] client.AsyncProcess:

Re: Phoenix Index Scrutiny Tool

2019-08-09 Thread Alexander Batyrshin
I have familiar question - how to partially rebuild indexes by timestamps interval like many MapReduce has —starttime/—endttime > On 9 Aug 2019, at 17:21, Aleksandr Saraseka wrote: > > Hello community! > I'm testing scrutiny tool to check index consistency. > I hard-deleted from HBase a couple

RS Index Handler got regionserver.MultiVersionConcurrencyControl: STUCK

2019-07-17 Thread Alexander Batyrshin
Hello all, Any ideas how to avoid situations like this: Jul 18 01:55:30 prod005 hbase[227531]: 2019-07-18 01:55:30,042 WARN [RpcServer.Index.handler=4,queue=0,port=60020] regionserver.MultiVersionConcurrencyControl: STUCK: MultiVersionConcurrencyControl{readPoint=55042096,

Alter Table throws java.lang.NullPointerException

2019-07-23 Thread Alexander Batyrshin
Hello all, Got this: alter table TEST_TABLE SET APPEND_ONLY_SCHEMA=true; java.lang.NullPointerException at org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:3240) at org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:3221) at

Re: Alter Table throws java.lang.NullPointerException

2019-07-25 Thread Alexander Batyrshin
nix that you're using. > > Did you search Jira to see if there was someone else who also reported this > issue? > > On 7/23/19 4:24 PM, Alexander Batyrshin wrote: >> Hello all, >> Got this: >> alter table TEST_TABLE SET APPEND_ONL