12 index tables * 256 region per table = ~3K regions for index tables assuming we are talking of covered index which implies 200+ regions/region server on a 15 node cluster.
On Mon, Jul 11, 2016 at 1:58 PM, James Taylor <jamestay...@apache.org> wrote: > Hi Simon, > > I might be missing something, but with 12 separate index tables or 1 index > table, the amount of data will be the same. Won't there be the same number > of regions either way? > > Thanks, > James > > On Sun, Jul 10, 2016 at 10:50 PM, Simon Wang <simon.w...@airbnb.com> > wrote: > >> Hi James, >> >> Thanks for the response. >> >> In our use case, there is a 256 region table, and we want to build ~12 >> indexes on it. We have 15 region servers. If each index is in its own >> table, that would be a total of 221 regions per region server of this >> single table. I think the extra write time cost is okay. But the number of >> regions is too high for us. >> >> Best, >> Simon >> >> >> On Jul 9, 2016, at 1:18 AM, James Taylor <jamestay...@apache.org> wrote: >> >> Hi Simon, >> The reason we've taken this approach with views is that it's possible >> with multi-tenancy that the number of views would grow unbounded since you >> might end up with a view per tenant (100K or 1M views or more - clearly too >> many for HBase to handle as separate tables). >> >> With secondary indexes directly on physical tables, you're somewhat >> bounded by the hit you're willing to take on the write side, as the cost of >> maintaining the index is similar to the cost of the write to the data >> table. So the extra number of physical tables for indexes seems within the >> bounds of what HBase could handle. >> >> How many secondary indexes are you creating and are you ok with the extra >> write-time cost? >> >> From a code consistency standpoint, using the same approach across local, >> global, and view indexes might simplify things, though. Please file a JIRA >> with a bit more detail on your use case. >> >> Thanks, >> James >> >> >> >> On Fri, Jul 8, 2016 at 8:59 PM, Simon Wang <simon.w...@airbnb.com> wrote: >> >>> Hi all, >>> >>> I am writing to ask if there is a way to let Phoenix store all indexes >>> on a single table in the same HBase table. If each index must be stored in >>> a separate table, creating more than a few indexes on table with a large >>> number of regions will not scale well. >>> >>> From what I have learned, when Phoenix builds indexes on a view, it >>> stores all indexes in a table associated with the underlying table of the >>> view. e.g. if V1 is a view of T1, all indexes on V1 will be stored in >>> _IDX_T1. It would be great if this behavior can be optionally turned on for >>> indexes on tables. >>> >>> Best, >>> Simon >> >> >> >> >
