Can we simplify the following with this: 1st iteration of meta scanning, calculate numRegs = number of offline regions + number of online regions (for underlying table) subsequent iterations of meta scanning, if number of online regions (for underlying table) reaches numRegs, break.
Cheers On Mon, Jun 20, 2011 at 2:05 PM, Todd Lipcon <[email protected]> wrote: > Why not the following pseudo-code for isTableAvailable: > > scanMeta(table, Set<RegionId> online, Set<RegionId> offline) { > scan meta, inserting into the above sets any regions found for the > table in question > } > > Set<RegionId> everSeenOnline = new Set(); > while (true) { > Set<RegionId> online = new Set(); > Set<RegionId> offline = new Set(); > scanMeta(table, online, offline); > everSeenOnline.addAll(online); > Set<RegionId> stillWaiting = Sets.setDifference(offline, everSeenOnline) > if (stillWaiting.isEmpty()) return; > sleep; > } > > i.e we wait until we have no offline regions that we've never seen online > > On Mon, Jun 20, 2011 at 10:27 AM, Stack <[email protected]> wrote: > > Yeah, its not intuitive but failing your passing isTableAvailable the > > knowledge it needs, then its going to have to do heuristics to figure > > table deploy. We can work on what these will look like -- region is > > open but not split, no region opened in last N minutes or seconds -- > > but I think it will always be an imprecise especially when you are up > > in Vidhya's 70k regions range. > > > > St.Ack > > > > On Sun, Jun 19, 2011 at 9:02 PM, Ted Yu <[email protected]> wrote: > >> I put patch v5 on 3904. > >> When user calls HBaseAdmin.createTableAsync() to create the table, > >> HCM.isTableAvailable() is still vulnerable because currently it doesn't > >> known the desired number of regions. > >> We can add a new parameter to HCM.isTableAvailable() but > >> 1. HConnection needs to be modified to accommodate this change > >> 2. I personally think it is not intuitive. > >> > >> Comments are welcome. > >> > >> On Fri, Jun 17, 2011 at 8:48 AM, Stack <[email protected]> wrote: > >> > >>> On Thu, Jun 16, 2011 at 5:43 PM, Ted Yu <[email protected]> wrote: > >>> > HBA.createTable() takes too long to execute. Client receives Socket > >>> timeout > >>> > exception. > >>> > >>> This is because the create is running in the master? Can we have > >>> waiting happen client-side over in HBaseAdmin? > >>> > >>> > Client calls HCM.isTableAvailable() which would report inaccurate > status. > >>> > I want to get other developers' opinion on whether > HCM.isTableAvailable() > >>> > should be accurate. > >>> > > >>> > >>> Do we need two forms? An isTableAvailable that will looks at table > >>> and if all regions are online reports true and then another > >>> isTableAvailable where you know how many regions there are supposed to > >>> be in the table and it returns true if that count of regions is open > >>> and deployed? > >>> > >>> St.Ack > >>> > >> > > > > > > -- > Todd Lipcon > Software Engineer, Cloudera >
