Query optimization

Hello, We have 2 tables: Table1 - big one (2000M+ rows): CREATE TABLE table1 ( pk varchar PRIMARY KEY, col varchar ); Table2 - small one (300K rows): CREATE TABLE table2 ( pk varchar PRIMARY KEY, other varchar ); Query like this work fast (~ 30sec): SELECT table1.pk,

java.io.IOException: Added a key not lexically larger than previous

Hello, Are there any ideas where this problem comes from and how to fix? Jun 18 21:38:05 prod022 hbase[148581]: 2019-06-18 21:38:05,348 WARN [MemStoreFlusher.0] regionserver.HStore: Failed flushing store file, retrying num=9 Jun 18 21:38:05 prod022 hbase[148581]: java.io.IOException: Added a

Re: Query optimization

he explain plan to see if it's working as expected > > On Wed, Jun 19, 2019 at 7:43 AM Alexander Batyrshin <0x62...@gmail.com > <mailto:0x62...@gmail.com>> wrote: > Hello, > We have 2 tables: > > Table1 - big one (2000M+ rows): > > CREATE TABLE ta

Re: java.io.IOException: Added a key not lexically larger than previous

Is is possible that Phoenix is the reason of this problem? > On 20 Jun 2019, at 04:16, Alexander Batyrshin <0x62...@gmail.com> wrote: > > Hello, > Are there any ideas where this problem comes from and how to fix? > > Jun 18 21:38:05 prod022 hbase[148581]: 2019

Re: java.io.IOException: Added a key not lexically larger than previous

t seeing if you've shared this previously on this or another thread. > Sorry if you have. > > Short-answer, it's possible that something around secondary indexing in > Phoenix causes this but not possible to definitively say in a vaccuum. > > On 8/15/19 1:19 PM, Alexander Batyrshin w

Re: java.io.IOException: Added a key not lexically larger than previous

> On 15 Aug 2019, at 21:27, Josh Elser wrote: > > Short-answer, it's possible that something around secondary indexing in > Phoenix causes this but not possible to definitively say in a vaccuum. As I see region-server crashes on main table (not index) memstore flush. How can I help to

Re: java.io.IOException: Added a key not lexically larger than previous

> Since you're using a global index, which stores the index data in a separate > table (and hence different regions, each of which has a different MemStore / > set of HFiles), and the error's happening to the base table, I'd be surprised > if Phoenix indexing is related. AFAIK Phoenix handle

Re: java.io.IOException: Added a key not lexically larger than previous

Looks like problem begun with 4.14.2 Maybe somehow https://issues.apache.org/jira/browse/PHOENIX-5266 <https://issues.apache.org/jira/browse/PHOENIX-5266> can re-apply mutations to main table with bugs? > On 20 Jun 2019, at 04:16, Alexander Batyrshin <0x62...@gmail.com> wr

Any reason for so small phoenix.mutate.batchSize by default?

Hello all, 1) There is bug in documentation - http://phoenix.apache.org/tuning.html phoenix.mutate.batchSize is not 1000, but only 100 by default

Re: Any reason for so small phoenix.mutate.batchSize by default?

pt statistics > * How much data ends up being sent to a RegionServer in one RPC? Where I can get this metric? > On 3 Sep 2019, at 17:19, Josh Elser wrote: > > Hey Alexander, > > Was just poking at the code for this: it looks like this is really just > determining the num

Re: Any reason for so small phoenix.mutate.batchSize by default?

> On 3 Sep 2019, at 19:45, Alexander Batyrshin <0x62...@gmail.com> wrote: > > I observer that there is some extra mutations in batch for every my UPSERTs > For example if app call executeUpdate() only 5 times then on commit there > will be "DEBUG MutationState:1046 -

Re: Why we change index state to PENDING_DISABLE on RegionMovedException

if the next write succeeds, it > can flip back to "ACTIVE". > > On Tue, Sep 10, 2019 at 2:29 PM Alexander Batyrshin <0x62...@gmail.com > <mailto:0x62...@gmail.com>> wrote: > As I know RegionMovedException is not a problem at all, its just notification > th

Why we change index state to PENDING_DISABLE on RegionMovedException

As I know RegionMovedException is not a problem at all, its just notification that we need to update meta information about table regions and retry. Why we do extra work with changing state of index? 2019-09-10 22:35:00,764 WARN [hconnection-0x4a63b6ea-shared--pool10-t961] client.AsyncProcess:

Re: Phoenix Index Scrutiny Tool

I have familiar question - how to partially rebuild indexes by timestamps interval like many MapReduce has —starttime/—endttime > On 9 Aug 2019, at 17:21, Aleksandr Saraseka wrote: > > Hello community! > I'm testing scrutiny tool to check index consistency. > I hard-deleted from HBase a couple

RS Index Handler got regionserver.MultiVersionConcurrencyControl: STUCK

Hello all, Any ideas how to avoid situations like this: Jul 18 01:55:30 prod005 hbase[227531]: 2019-07-18 01:55:30,042 WARN [RpcServer.Index.handler=4,queue=0,port=60020] regionserver.MultiVersionConcurrencyControl: STUCK: MultiVersionConcurrencyControl{readPoint=55042096,

Alter Table throws java.lang.NullPointerException

Hello all, Got this: alter table TEST_TABLE SET APPEND_ONLY_SCHEMA=true; java.lang.NullPointerException at org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:3240) at org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:3221) at

Re: Alter Table throws java.lang.NullPointerException

nix that you're using. > > Did you search Jira to see if there was someone else who also reported this > issue? > > On 7/23/19 4:24 PM, Alexander Batyrshin wrote: >> Hello all, >> Got this: >> alter table TEST_TABLE SET APPEND_ONL

When Phoenix 5.x for HBase-2.x will be updated?

As I see there are many bug fixes and updates (consistent indexes) for 4.x Phoenix branch. So im curios when this will be available for 5.x branch?

FAQ page is blank now

Looks like FAQ page at http://phoenix.apache.org/faq.html is blank now

Optimisation for SELECT ... WHERE pk IN (pk1, pk2...,pkN)

Hello all, I have I quite big mutable table and need to select rows by primary keys. The problem is that "SELECT … WHERE pk IN (pk1, pk2...,pkN)" (ROUND ROBIN POINT LOOKUP ON $num KEYS OVER) do not use HBase bloom filter ‘ROW’. This leads to performance degradation when more than one StoreFile

Re: Speedup initial index creation

; the new index design using IndexUpgradeTool? I am asking this to make sure > that your index actually uses the new index design. You can verify this using > the HBase shell by describing the data table and checking if the > IndexRegionObserver coproc is loaded on your base table. >

Re: Speedup initial index creation

> 3) No >>> >>> It will be a good improvement to have an option to support (3) by just >>> creating indexes using the last data row versions. Please feel free to >>> create an improvement Jira for this. >>> >>> Did you create your base tabl

Re: Speedup initial index creation

I tried on phoenix-4.16.0 > On 31 Mar 2021, at 00:54, Alexander Batyrshin <0x62...@gmail.com> wrote: > > Hello, > I tried to create new consistent index on mutable table and found out that > IndexTool MapReduce works 3-5 times slower compared to old indexes on 4.14.2 >

Speedup initial index creation

Hello, I tried to create new consistent index on mutable table and found out that IndexTool MapReduce works 3-5 times slower compared to old indexes on 4.14.2 So I have some question; 1) Is it possible to create index old way via intermediate HFiles and bulk-loading? 2) Is it possible to