>From Vidhyashankar @ 20/May/11 05:53: Hence there might be a case when all regions are indeed fully assigned in META but it is just that the master is yet to populate META with the rest of the regions.
The above means scanning .META. alone wouldn't guarantee that all the regions of the table are assigned. On Mon, Jun 20, 2011 at 2:05 PM, Todd Lipcon <[email protected]> wrote: > Why not the following pseudo-code for isTableAvailable: > > scanMeta(table, Set<RegionId> online, Set<RegionId> offline) { > scan meta, inserting into the above sets any regions found for the > table in question > } > > Set<RegionId> everSeenOnline = new Set(); > while (true) { > Set<RegionId> online = new Set(); > Set<RegionId> offline = new Set(); > scanMeta(table, online, offline); > everSeenOnline.addAll(online); > Set<RegionId> stillWaiting = Sets.setDifference(offline, everSeenOnline) > if (stillWaiting.isEmpty()) return; > sleep; > } > > i.e we wait until we have no offline regions that we've never seen online > > On Mon, Jun 20, 2011 at 10:27 AM, Stack <[email protected]> wrote: > > Yeah, its not intuitive but failing your passing isTableAvailable the > > knowledge it needs, then its going to have to do heuristics to figure > > table deploy. We can work on what these will look like -- region is > > open but not split, no region opened in last N minutes or seconds -- > > but I think it will always be an imprecise especially when you are up > > in Vidhya's 70k regions range. > > > > St.Ack > > > > On Sun, Jun 19, 2011 at 9:02 PM, Ted Yu <[email protected]> wrote: > >> I put patch v5 on 3904. > >> When user calls HBaseAdmin.createTableAsync() to create the table, > >> HCM.isTableAvailable() is still vulnerable because currently it doesn't > >> known the desired number of regions. > >> We can add a new parameter to HCM.isTableAvailable() but > >> 1. HConnection needs to be modified to accommodate this change > >> 2. I personally think it is not intuitive. > >> > >> Comments are welcome. > >> > >> On Fri, Jun 17, 2011 at 8:48 AM, Stack <[email protected]> wrote: > >> > >>> On Thu, Jun 16, 2011 at 5:43 PM, Ted Yu <[email protected]> wrote: > >>> > HBA.createTable() takes too long to execute. Client receives Socket > >>> timeout > >>> > exception. > >>> > >>> This is because the create is running in the master? Can we have > >>> waiting happen client-side over in HBaseAdmin? > >>> > >>> > Client calls HCM.isTableAvailable() which would report inaccurate > status. > >>> > I want to get other developers' opinion on whether > HCM.isTableAvailable() > >>> > should be accurate. > >>> > > >>> > >>> Do we need two forms? An isTableAvailable that will looks at table > >>> and if all regions are online reports true and then another > >>> isTableAvailable where you know how many regions there are supposed to > >>> be in the table and it returns true if that count of regions is open > >>> and deployed? > >>> > >>> St.Ack > >>> > >> > > > > > > -- > Todd Lipcon > Software Engineer, Cloudera >
