Re: Projection schema drops key

Agreed, I considered this, however I decided to consider it in the same vein as the key issue. Especially considering that those attributes currently aren't accessible from the KuduColumnSchema class. Im thinking that I'll just note it in the method comments that compression and encoding will need

Re: Projection schema drops key

On Fri, Sep 9, 2016 at 2:26 PM, Jordan Birdsell wrote: > Threw in a helper function in the SchemaBuilder to at least make it a > little cleaner to construct a new schema with a key from the projection > schema > > ex: > builder = kudu.schema_builder() > for col in scanner.get_proj

Re: Projection schema drops key

Threw in a helper function in the SchemaBuilder to at least make it a little cleaner to construct a new schema with a key from the projection schema ex: builder = kudu.schema_builder() for col in scanner.get_projection_schema(): builder.copy_column(col) builder.

Re: Projection schema drops key

I follow now, thanks On Fri, Sep 9, 2016 at 12:48 PM Todd Lipcon wrote: > Agreed with what David wrote. The dropping of 'key' status in projections > is intentional. It's the same as you get in MySQL if you have something > like: > > CREATE TABLE t ( > x integer primary key, > y integer > );

Re: Flume tests failing

yea, that's probably it. Maven's the absolute worst. Will give it a try. -Todd On Fri, Sep 9, 2016 at 11:20 AM, Mike Percy wrote: > Maybe a Maven thing? Try mvn clean install > > Mike > > Sent from my iPhone > > > On Sep 9, 2016, at 6:51 AM, William Berkeley > wrote: > > > > I'm not seeing tha

Re: Flume tests failing

Maybe a Maven thing? Try mvn clean install Mike Sent from my iPhone > On Sep 9, 2016, at 6:51 AM, William Berkeley wrote: > > I'm not seeing that. It must be related to fdfcdba, which changed the API > for the EventProducer and renamed it to OperationsProducer. But that commit > changed all th

Re: Projection schema drops key

Agreed with what David wrote. The dropping of 'key' status in projections is intentional. It's the same as you get in MySQL if you have something like: CREATE TABLE t ( x integer primary key, y integer ); CREATE TABLE t2 AS SELECT * FROM t; the 'select' results don't carry any notion of whic

Re: Projection schema drops key

Oh I see what you mean. It's not that the keys are getting dropped, its that they're not marked as keys. This arguably makes sense on a projection: for instance you might want the keys returns in the end of the projection, while table schemas (at least for now) require that they are present at the

Re: Projection schema drops key

I meant a gist with a example of client API usage where the keys are getting dropped, so that we can reproduce what you're seeing. -david On Fri, Sep 9, 2016 at 8:51 AM, Jordan Birdsell wrote: > Right, what i'm saying is, if i do include the key in my projection, the > schema does not maintain

Re: Projection schema drops key

Right, what i'm saying is, if i do include the key in my projection, the schema does not maintain it as a key. The issue isnt so much that i cant apply predicates to the key column, its that if i wanted to create a projection and then want to use that projection to create a table based on that pro

Re: Projection schema drops key

Wait, If you _do_ set a projection on the scanner that does not include the keys, then they won't be returned (and won't appear on the projection's schema). Note that this does not mean that you can't set predicates on the key, it's just that they'll be evaluated server side, but the key won't actu

Re: Projection schema drops key

Hey David, Yep, i'm sure, taking a look at the scan_configuration class, the issue seems to be here: Status ScanConfiguration::SetProjectedColumnIndexes(const vector& col_indexes) { RETURN_NOT_OK*(s->Reset(cols, 0));* In the SetProjectedColumnIndexes method (which is also used by Set

Re: Projection schema drops key

Hi Jordan KuduScanner::GetProjectionSchema returns the schema of the projection that was previously set on the scanner. If you don't a projection it should indeed return all the columns. Are you sure you didn't set a projection (with SetProjectedColumnNames or SetProjectedColumnIndexes) that e

Re: Flume tests failing

I'm not seeing that. It must be related to fdfcdba, which changed the API for the EventProducer and renamed it to OperationsProducer. But that commit changed all the existing EventProducers to be OperationsProducers, so I'm not sure how that problem is happening. -Will On Fri, Sep 9, 2016 at 4:56

Projection schema drops key

Hey folks, i was doing some work on KUDU-854 and when testing the KuduScanner::GetProjectionSchema method call, found that the key was being dropped, which makes this much more challenging to test. Any ideas if it is intended to drop the key informa

Flume tests failing

Anyone seen this on their boxes? Running org.apache.kudu.flume.sink.KeyedKuduEventProducerTest Tests run: 7, Failures: 0, Errors: 7, Skipped: 0, Time elapsed: 2.768 sec <<< FAILURE! - in org.apache.kudu.flume.sink.KeyedKuduEventProducerTest testDuplicateRowsWithUpsert(org.apache.kudu.flume.sink.Key