On Mon, Mar 28, 2016 at 11:12 AM, Jean-Daniel Cryans <[email protected]> wrote:
> Hey Mike, > > If you fix 1380, I'll let you get 1377 in :) > Sweet. More seriously, when do think 1377 can get in? Also, looks like Adar > already took a look, but do you need more eyes on it? What about cluster > testing, are the unit tests enough? > I spoke w/ Adar about this bug fix on Slack, as well as on the WIP patch on Gerrit @ http://gerrit.cloudera.org/#/c/2595/. What we think needs to happen there is we truncate the metadata file if the last record is corrupt *and* if it is the last record in the metadata file. I think the patch will be fine as-is, assuming it gets reviewed and has tests. However it exposes a hole in our testing, since we don't currently simulate failing disks or full disks. But I think that's something that results in relatively rare bugs such as this one and doesn't require a solution for this patch. The reason I'd like to get this in is that we had one user hit this issue mutliple times (due to full disks) and I'd like to remove that pain point for them. For Flume, I'm not an expert so can't comment on the patch, but do you > think what's there offers the right interfaces/idioms? Also, should we mark > it as "experimental"? > I think it's starting to shape up, but yeah we could mark it as experimental for the first release. It seems to be starting to shape up. It would be even better if we had UPSERT support, since right now if there is a duplicate key error (possibly due to a retry on an insert) we plan to just drop the event (this will be configurable). Mike > > Thanks, > > J-D > > On Mon, Mar 28, 2016 at 11:01 AM, Mike Percy <[email protected]> wrote: > > > Hey JD, > > I'll start taking a look at KUDU-1380. I'd also like to get the Flume > sink > > committed (it's under review but almost there) and get KUDU-1377 into 0.8 > > as well (I almost have a fix ready but it's not quite done). > > > > Mike > > > > On Mon, Mar 28, 2016 at 10:31 AM, Jean-Daniel Cryans < > [email protected]> > > wrote: > > > > > Hey Todd, > > > > > > Thanks for all the testing! > > > > > > @Dan B, you handling KUDU-1379? > > > > > > @Mike P, since you were in that part of the code recently, can you make > > > some time for KUDU-1380? > > > > > > And regarding the list of things I said I wanted to pull in, partition > > > pruning made it in and the Java client's new KuduPredicate looks almost > > > ready. The Get API didn't see any action since the 11th, so I'm > inclined > > to > > > drop it from 0.8.0. > > > > > > I'm writing the release notes, should be up soon. > > > > > > J-D > > > > > > On Mon, Mar 28, 2016 at 8:57 AM, Todd Lipcon <[email protected]> > wrote: > > > > > > > I discovered a couple bad issues over the weekend that I think we > > should > > > > prioritize to get in for 0.8: > > > > > > > > - KUDU-1379 - this is a regression blocker where column range > > predicates > > > on > > > > float/double/bool columns no longer work. We should either fix this > or > > > > revert the predicate changes that Dan committed last week (preferably > > > just > > > > fix it, since I dont think it's too hard) > > > > - KUDU-1380 - this is not a regression but it's a case where we can > get > > > > silently wrong results (duplicated rows) from the C++ scanner. Since > > > it's a > > > > correctness issue I marked it as blocker. Does anyone have time to > look > > > at > > > > this today/tomorrow? If not I will try to take a look. > > > > > > > > -Todd > > > > > > > > On Fri, Mar 25, 2016 at 2:12 PM, Jean-Daniel Cryans < > > [email protected] > > > > > > > > wrote: > > > > > > > > > I just went through the Blocker and Critical jiras targeted for > 0.8.0 > > > and > > > > > moved a bunch that I knew wouldn't get done in time. Once I branch > > I'll > > > > > bulk move the rest. In most cases it's fine to bring them back into > > > > 0.8.0, > > > > > as long as it can be done by midweek next week. In doubt, poke me > in > > > the > > > > > jira. > > > > > > > > > > J-D > > > > > > > > > > On Fri, Mar 25, 2016 at 8:29 AM, Jean-Daniel Cryans < > > > [email protected] > > > > > > > > > > wrote: > > > > > > > > > > > Hey devs, > > > > > > > > > > > > It's almost time for 0.8.0, I'd like to get a first RC out in a > > week > > > on > > > > > > April 1st (but more realistically it might happen the 4th). I > will > > > > create > > > > > > the branch on March 30th. > > > > > > > > > > > > The release note-worthy items that have been committed since > 0.7.0 > > > that > > > > > > aren't in 0.7.1: > > > > > > - RPC and application feature flags > > > > > > - KUDU-1337 Avoid spurious remote bootstraps on DeleteTablet() > > > > > > - Fixed the Spark shutdown bug > > > > > > - KUDU-1322 failover writes on tablet not found in the C++ > client > > > > > > - KUDU-969. Fix handling of crashes just before tablet metadata > > > flush > > > > > > - KUDU-1354. Writes should not release locks before committing > > MVCC > > > > > > - master: don't expose intermediate CreateTable() state to > > consumers > > > > > > > > > > > > The things that are still in flight that I really want to get in: > > > > > > - Partition pruning in the C++ client and improvements on the > > > > > > server-side. http://gerrit.cloudera.org:8080/#/c/2413/ > > > > > > - [java-client] implement ColumnPredicate API > > > > > > http://gerrit.cloudera.org:8080/#/c/2591/ > > > > > > - KUDU-1235. Add Get API > > http://gerrit.cloudera.org:8080/#/c/2519/ > > > > > > > > > > > > Anything else that comes in is gravy, but I think the scan token > > API > > > > > > integration in the Java client needs more thinking. I think Dan > is > > > > > supposed > > > > > > to send an email to dev@ about it. > > > > > > > > > > > > What do you devs think? > > > > > > > > > > > > J-D > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > Todd Lipcon > > > > Software Engineer, Cloudera > > > > > > > > > >
