Re: Thrift Perl API Timeout Issues

2009-10-16 Thread Anthony Molinaro
Yes, I use perl to do ETL type stuff, dump mysql, postgres tables and load them into cassandra. Although I also see these errors in front end PHP systems which connect to cassandra. -Anthony On Thu, Oct 15, 2009 at 01:48:14PM -0500, Jonathan Ellis wrote: > Are you also using Perl? > > On Thu, O

Re: Thrift Perl API Timeout Issues

2009-10-16 Thread Eric Lubow
Simon, I understand what you're saying and tend to agree with that philosophy. I think the issue has more to do with the undocumentedness (if thats not a word, it should be) of the Perl Thrift/Cassandra API in general. That is something I hope to change in the near future. Timeouts are defini

Re: Thrift Perl API Timeout Issues

2009-10-16 Thread Jake Luciani
Perhaps we should add this to the thrift/Cassandra FAQ? On Oct 16, 2009, at 9:36 AM, Simon Smith wrote: I don't have an opinion on the default timeout. But in my experience with other applications, you want to consciously make a choice about what your timeout, based on your architecture and

Re: Thrift Perl API Timeout Issues

2009-10-16 Thread Simon Smith
I don't have an opinion on the default timeout. But in my experience with other applications, you want to consciously make a choice about what your timeout, based on your architecture and performance requirements. You're much better off explicitly setting a timeout that will cause your transactio

Re: Thrift Perl API Timeout Issues

2009-10-15 Thread Eric Lubow
So I ran the tests again twice with a huge timeout and it managed to run in just under 3 hours both times. So this issue is definitely related to the timeouts. It might be worth changing the default timeouts for Perl to match the infinite timeouts for Python. Thanks for the quick responses. -e

Re: Thrift Perl API Timeout Issues

2009-10-15 Thread Jonathan Ellis
Are you also using Perl? On Thu, Oct 15, 2009 at 1:38 PM, Anthony Molinaro wrote: > I see a similar thing happening all the time.  I get around it by closing > the current connection and reconnecting after a sleep.  Although I am able > to do quite a few inserts between errors, so I'm not sure if

Re: Thrift Perl API Timeout Issues

2009-10-15 Thread Anthony Molinaro
I see a similar thing happening all the time. I get around it by closing the current connection and reconnecting after a sleep. Although I am able to do quite a few inserts between errors, so I'm not sure if it's the exact problem. -Anthony On Thu, Oct 15, 2009 at 11:26:08AM -0400, Eric Lubow w

Re: Thrift Perl API Timeout Issues

2009-10-15 Thread Jake Luciani
What happens if you set it to 10? On Oct 15, 2009, at 11:48 AM, Eric Lubow wrote: My connection section of the script is here: # Connect to the database my $socket = new Thrift::Socket('localhost',9160); $socket->setSendTimeout(2500); $socket->setRecvTimeout(7500); my $transpo

Re: Thrift Perl API Timeout Issues

2009-10-15 Thread Jonathan Ellis
On Thu, Oct 15, 2009 at 10:51 AM, Simon Smith wrote: > While on the topic, I'm using the python Thrift interface - if I > wanted to, how would I change the timeout? I currently do: > >  socket = TSocket.TSocket(host,port) > > If I wanted to change the timeout would I do something like: > > socket.

Re: Thrift Perl API Timeout Issues

2009-10-15 Thread Simon Smith
While on the topic, I'm using the python Thrift interface - if I wanted to, how would I change the timeout? I currently do: socket = TSocket.TSocket(host,port) If I wanted to change the timeout would I do something like: socket.setTimeout(timeout) or...? Sorry if I should be able to see this

Re: Thrift Perl API Timeout Issues

2009-10-15 Thread Eric Lubow
My connection section of the script is here: # Connect to the database my $socket = new Thrift::Socket('localhost',9160); $socket->setSendTimeout(2500); $socket->setRecvTimeout(7500); my $transport = new Thrift::BufferedTransport($socket,2048,2048); my $protocol = new Thrift::BinaryProto

Re: Thrift Perl API Timeout Issues

2009-10-15 Thread Jake Luciani
I think it's 100ms. I need to increase it to match python I guess. Sent from my iPhone On Oct 15, 2009, at 11:40 AM, Jonathan Ellis wrote: What is the default? On Thu, Oct 15, 2009 at 10:37 AM, Jake Luciani wrote: You need to call $socket->setRecvTimeout() With a higher number in ms. O

Re: Thrift Perl API Timeout Issues

2009-10-15 Thread Jonathan Ellis
What is the default? On Thu, Oct 15, 2009 at 10:37 AM, Jake Luciani wrote: > You need to call > $socket->setRecvTimeout() > With a higher number in ms. > > > On Oct 15, 2009, at 11:26 AM, Eric Lubow wrote: > > Using the Thrift Perl API into Cassandra, I am running into what is > endearingly refe

Re: Thrift Perl API Timeout Issues

2009-10-15 Thread Jake Luciani
You need to call $socket->setRecvTimeout() With a higher number in ms. On Oct 15, 2009, at 11:26 AM, Eric Lubow wrote: Using the Thrift Perl API into Cassandra, I am running into what is endearingly referred to as the 4 bytes of doom: TSocket: timed out reading 4 bytes from localhost:916

Thrift Perl API Timeout Issues

2009-10-15 Thread Eric Lubow
Using the Thrift Perl API into Cassandra, I am running into what is endearingly referred to as the 4 bytes of doom: TSocket: timed out reading 4 bytes from localhost:9160 The script I am using is fairly simple. I have a text file that has about 3.6 million lines that are formatted like: f...@ba