Pig filter against flatten column

2011-04-07 Thread Badrinarayanan S
Hi, I am trying to run a filter against a column which is the result of a flatten operation. But the filter clause throws an exception as org.apache.pig.data.DataByteArray cannot be cast to java.lang.String. The exception is against the line doing the matches filter. If I change matches to

Dereferencing columns of nested bags

2011-04-08 Thread Badrinarayanan S
Is it possible to dereference a column part of a nested bag. In the schema given below, I am trying to dereference the columns Key and Value which is part of visit bag which is part of visits bag. (id, visits:bag{visittuple:tuple(timestamp, visit:bag{details:tuple(Key:chararray,

RE: Dereferencing columns of nested bags

2011-04-08 Thread Badrinarayanan S
of foreach. The reference to $1.$1.($0, $1) results in an error like out of bound access. Thanks, badri -Original Message- From: Mridul Muralidharan [mailto:mrid...@yahoo-inc.com] Sent: Friday, April 08, 2011 4:32 PM To: user@pig.apache.org Cc: Badrinarayanan S Subject: Re: Dereferencing

RE: Dereferencing columns of nested bags

2011-04-09 Thread Badrinarayanan S
the mechanism is different than tuple. If $1 is a tuple, $1.$1 does reduce one level and get nested item out. To reduce the level of a bag, you can only flatten the bag. Daniel On 04/08/2011 09:41 AM, Mridul Muralidharan wrote: On Friday 08 April 2011 08:16 PM, Badrinarayanan S wrote: I am trying

RE: Dereferencing columns of nested bags

2011-04-09 Thread Badrinarayanan S
)})}. Note the mechanism is different than tuple. If $1 is a tuple, $1.$1 does reduce one level and get nested item out. To reduce the level of a bag, you can only flatten the bag. Daniel On 04/08/2011 09:41 AM, Mridul Muralidharan wrote: On Friday 08 April 2011 08:16 PM, Badrinarayanan S

NullPointerException at POProject.consumeInputBag

2011-04-09 Thread Badrinarayanan S
Hi, I am working with below schema against the latest trunk of Pig. Visits = LOAD 'cassandra://MyTest/MyCF' USING CassandraStorage() as (Id:chararray, Details:bag{DetailsTuple:tuple(VisitTimestamp:chararray, VisitDetails:bag{VisitColumns:tuple(ColumnName:chararray, ColumnValue:chararray)})});

PIG Cassandra - IPs of nodes in a ring

2011-05-06 Thread Badrinarayanan S
Hi, I got a cluster with seven Cassandra nodes. The ring is formed using the private ips of each of the nodes. The rpc_address of the nodes is set to private and listen_address of the nodes set to public mainly to facilitate cross data centre ring. When I ring the nodes, it shows all nodes are

RE: PIG Cassandra - IPs of nodes in a ring

2011-05-06 Thread Badrinarayanan S
local machine. I've never done a different address for rpc_address and listen_address for that configuration, so there might be peculiarities there that I wouldn't have seen. On May 6, 2011, at 11:37 AM, Badrinarayanan S wrote: Hi, I got a cluster with seven Cassandra nodes. The ring

RE: PIG Cassandra - IPs of nodes in a ring

2011-05-10 Thread Badrinarayanan S
. He might know what implications there are for that setup. On May 6, 2011, at 1:13 PM, Badrinarayanan S wrote: Hi, I am running from one of the nodes in the cluster. I too believe it is something to do with different address for rpc_address and listen_address but not sure what

RE: PIG Cassandra Consistency Level

2011-06-14 Thread Badrinarayanan S
, Badrinarayanan S wrote: Hi, can anyone let me know the configuration steps required to read data from Cassandra using PIG with consistency level of LOCAL_QUORUM. Thanks, badri