[mailto:jeremy.hanna1...@gmail.com]
Sent: Thursday, March 24, 2011 11:34 AM
To: user@cassandra.apache.org
Subject: Re: pig counting question
The limit defaults to 1024 but you can set it when you use CassandraStorage
in pig, like so:
rows = LOAD 'cassandra://Keyspace/ColumnFamily' USING CassandraStorage(4096
@cassandra.apache.org
Subject: Re: pig counting question
One thing I wonder though - if your columns are the thing that are increasing
your heap size and eating up a lot of memory, and you're reading the data
structure out as a bag of columns, why isn't pig spilling to disk instead of
growing in memory
-
From: Jeffrey Wang [mailto:jw...@palantir.com]
Sent: Friday, March 25, 2011 11:42 AM
To: user@cassandra.apache.org
Subject: RE: pig counting question
I don't think it's Pig running out of memory, but rather Cassandra itself (the
data doesn't even make it to Pig). get_range_slices() is called
: Friday, March 25, 2011 11:42 AM
To: user@cassandra.apache.org
Subject: RE: pig counting question
I don't think it's Pig running out of memory, but rather Cassandra itself
(the data doesn't even make it to Pig). get_range_slices() is called with a
row batch size of 4096, the default, and it's
On Fri, Mar 25, 2011 at 1:41 PM, Jeffrey Wang jw...@palantir.com wrote:
I don't think it's Pig running out of memory, but rather Cassandra itself
(the data doesn't even make it to Pig). get_range_slices() is called with a
row batch size of 4096, the default, and it's fetching all of the
The limit defaults to 1024 but you can set it when you use CassandraStorage in
pig, like so:
rows = LOAD 'cassandra://Keyspace/ColumnFamily' USING CassandraStorage(4096);
or whatever value you wish.
Give that a try and see if it gives you more of what you're looking for.
On Mar 24, 2011, at
this streaming/paged?
-Jeffrey
-Original Message-
From: Jeremy Hanna [mailto:jeremy.hanna1...@gmail.com]
Sent: Thursday, March 24, 2011 11:34 AM
To: user@cassandra.apache.org
Subject: Re: pig counting question
The limit defaults to 1024 but you can set it when you use CassandraStorage in
pig
?
-Jeffrey
-Original Message-
From: Jeremy Hanna [mailto:jeremy.hanna1...@gmail.com]
Sent: Thursday, March 24, 2011 11:34 AM
To: user@cassandra.apache.org
Subject: Re: pig counting question
The limit defaults to 1024 but you can set it when you use CassandraStorage
in pig, like so
?
-Jeffrey
-Original Message-
From: Jeremy Hanna [mailto:jeremy.hanna1...@gmail.com]
Sent: Thursday, March 24, 2011 11:34 AM
To: user@cassandra.apache.org
Subject: Re: pig counting question
The limit defaults to 1024 but you can set it when you use CassandraStorage
in pig, like
