Re: Number of partitions for sharded table

2012-10-30 Thread Krishmin Rai
I should clarify that I've been pre-splitting tables at each shard so that each tablet consists of a single row. On Oct 30, 2012, at 3:06 PM, Krishmin Rai wrote: Hi All, We're working with an index table whose row is a shardId (an integer, like the wiki-search or IndexedDoc examples). I

Re: Number of partitions for sharded table

2012-10-30 Thread dlmarion
table and that all queries will have an infinite range. Dave Marion - Original Message - From: Krishmin Rai kr...@missionfoc.us To: user@accumulo.apache.org Sent: Tuesday, October 30, 2012 3:28:15 PM Subject: Re: Number of partitions for sharded table I should clarify that I've

Re: Number of partitions for sharded table

2012-10-30 Thread Krishmin Rai
: Krishmin Rai kr...@missionfoc.us To: user@accumulo.apache.org Sent: Tuesday, October 30, 2012 3:28:15 PM Subject: Re: Number of partitions for sharded table I should clarify that I've been pre-splitting tables at each shard so that each tablet consists of a single row. On Oct 30, 2012, at 3:06

Re: Number of partitions for sharded table

2012-10-30 Thread Adam Fuchs
Krishmin, There are a few extremes to keep in mind when choosing a manual partitioning strategy: 1. Parallelism and balance at ingest time. You need to find a happy medium between too few partitions (not enough parallelism) and too many partitions (tablet server resource contention and

Re: Number of partitions for sharded table

2012-10-30 Thread Krishmin Rai
Thanks, Adam… that's exactly what I was looking for, and gives me a lot to think about. -Krishmin On Oct 30, 2012, at 4:08 PM, Adam Fuchs wrote: Krishmin, There are a few extremes to keep in mind when choosing a manual partitioning strategy: 1. Parallelism and balance at ingest time.