Re: SSTable structure

2015-03-30 Thread Jacob Rhoden
Yes updating code and documentation can sometimes be annoying, you would only ever maintain both if it were important. It comes down or is having the format of the data files documented for everyone to understand an important thing? __ Sent from iPhone > On 31 Mar 2

What are the reasons for holding off on 2.1.x at this point?

2015-03-09 Thread Jacob Rhoden
I notice some of the discussion about rolling back and avoiding upgrading. I wonder if people can elaborate on their pain points?  We are in a situation where there are some use cases we wish to implement that appear to be much simpler to implement using indexed sets. So it has me wondering ab

Re: C* 2.1.2 invokes oom-killer

2015-02-18 Thread Jacob Rhoden
I neglected to mention, I also adjust the oom score of cassandra, to tell the kernel to kill something else other than cassandra. (Like if one of your dev’s runs a script that uses a lot of memory, so it kills your dev’s script instead). http://lwn.net/Articles/317814/

Re: C* 2.1.2 invokes oom-killer

2015-02-18 Thread Jacob Rhoden
Are you tweaking the "nice" priority on Cassandra? (Type: man nice) if you don't know much about it. Certainly improving cassandra's nice score becomes important when you have other things running on the server like scheduled jobs of people logging in to the server and doing things. ___

Re: Getting errors while starting DataStax Cassandra enterprise service

2015-01-23 Thread Jacob Rhoden
What does this show? ls $JAVA_HOME __ Sent from iPhone > On 24 Jan 2015, at 2:18 pm, anujacharya11 . > wrote: > > I had installed DataStax Enterprise Cassandra on my Ubuntu Linux Desktop > which is having Oracle/Sun JDK1.8.x. Setup JAVA_HOME correctly but when i

Re: Storing large files for later processing through hadoop

2015-01-02 Thread Jacob Rhoden
If it's for auditing, if recommend pushing the files out somewhere reasonably external, Amazon S3 works well for this type of thing, and you don't have to worry too much about backups and the like. __ Sent from iPhone > On 3 Jan 2015, at 5:07 pm, Srinivasa T N wrote

Removing commit log files

2014-11-18 Thread Jacob Rhoden
Hi Guys, Is it correct to assume that if you do a “nodetool drain” on a node and then shutdown a node, you can safely remove all commit logs on that node as long as all nodes are up? I have some VPS’s with low amounts of disk space that could do with it being recovered, I also assume this mea

Cassandra patterns/design for setting up a history/version/change log table?

2014-11-12 Thread Jacob Rhoden
Hi Guys, Assuming you have, for example, an “account” table, and an “account_history” table which simply tracks older versions of what a persons account looks like when an administrator edits a customer account. Given that we don’t have the luxury of a safe transaction to update the account re

Why is one query 10 times slower than the other?

2014-11-05 Thread Jacob Rhoden
Hi Guys, I have two cassandra 2.0.5 nodes, RF=2. When I do a: select * from table1 where clustercolumn=‘something' The trace indicates that it only needs to talk to one node, which I would have expected. However when I do a: select * from table2 Which is a small table with only has 20

Re: Connection reset by peer error

2014-07-17 Thread Jacob Rhoden
opping > services on app server. > > On Mon, Jun 30, 2014 at 3:29 PM, Jacob Rhoden wrote: > How are the two machines connected? Direct cable? Via a hub, router, > firewall, wan? > > On 1 Jul 2014, at 6:01 am, cass savy wrote: >> We use Datastax Java driver version 1

Re: Connection reset by peer error

2014-06-30 Thread Jacob Rhoden
How are the two machines connected? Direct cable? Via a hub, router, firewall, wan? __ Sent from iPhone > On 1 Jul 2014, at 6:01 am, cass savy wrote: > > We use Datastax Java driver version 1.0.6. Application is running into issues > connecting to the 3 node cluste

Re: Anyone using Astyanax in production besides Netflix itself?

2014-05-29 Thread Jacob Rhoden
Not long ago a vote was organised to get the developers to agree to stop work on the thrift API. New Cassandra features from this point are intended only for CQL. You probably want to make the effort to switch to CQL now rather than later. __ Sent from iPhone > On 3

null date bug? Not sure if its cassandra 2.0.5 or the gocql (golang) driver.

2014-05-16 Thread Jacob Rhoden
Im noticing the following strange behaviour when I do a query on a table: cqlsh:mykeyspace> select uuid, discontinued_from from mytable; uuid | discontinued_from --+-- b838a632-dd61-11e3-a32e-b8f6b11b196

nodetool hangs

2014-04-24 Thread Jacob Rhoden
I’ve done an install on an amazon instance, and for some strange reason I can telnet to the JMX port, but node tool just hangs and doesn’t do anything. I am hoping I’m overlooking something simple that someone could help me point out? Thanks (: cassandra@t1:/cassandra/db$ telnet 127.0.0.1 7199

cqlsh very strange query results behaviour (Cassandra 2.0.6)

2014-04-20 Thread Jacob Rhoden
This just happened, is this fixed in 2.0.7? cqlsh:tap> select * from setting; Bad Request: unconfigured columnfamily settings cqlsh:tap> select * from settings; name | value --+-- ld

Re: List support in Net::Async::CassandraCQL ?

2014-03-06 Thread Jacob Rhoden
t will have to wait until I get to a computer). Thanks, Jacob __ Sent from iPhone > On 7 Mar 2014, at 3:06 am, "Paul \"LeoNerd\" Evans" > wrote: > > On Sun, 23 Feb 2014 13:06:54 +1100 > Jacob Rhoden wrote: > >> Thank

Re: Cassandra Internal error processing get_slice java.lang.AssertionError

2014-03-05 Thread Jacob Rhoden
If this is a _new_ 8 node cluster, is there a specific reason you starting with 1.0.x, not 1.2.x? (Upgrading later will be more work) __ Sent from iPhone > On 6 Mar 2014, at 9:22 am, "Alaa Zubaidi (PDF)" wrote: > > Cassandra 1.0.10 with default settings.. > > >> O

Re: Commit logs building up

2014-03-05 Thread Jacob Rhoden
I've been seeing commit logs grow much much larger than the actual data under 2.0.5, even after flushing and draining a node. Especially so on the node doing bulk data loads. I've been doing intermittent drain/shutdown/delete commit logs to keep things in check. I suspect a either a bug, or a

Re: List support in Net::Async::CassandraCQL ?

2014-02-22 Thread Jacob Rhoden
Hi Paul, On 23 Feb 2014, at 4:15 am, Paul LeoNerd Evans wrote: > On Sat, 22 Feb 2014 14:03:06 +1100 Jacob Rhoden wrote: >>my $q = $cass->prepare("update contact set name=?, address=? >> where uuid=?")->get; push @f, $q->execute([$name, @address, $uuid]);

List support in Net::Async::CassandraCQL ?

2014-02-21 Thread Jacob Rhoden
This perl library has been extremely useful for scripting up data migrations. I wonder if anyone knows of the easiest way to use lists with this driver? Throwing a perl array in as a parameter doesn’t work as is: my $q = $cass->prepare("update contact set name=?, address=? where uuid=?")->g

Re: Where to I start to get to the bottom of this WriteTimeout issue?

2014-02-16 Thread Jacob Rhoden
Hi Guys, On 17 Feb 2014, at 3:13 pm, Thunder Stumpges wrote: > If you are looking for write throughput and running on a VM you could likely > have IO issues with your virtual disks.. Best practices are to put the write > ahead log on a separate disk from the data folder(s). Not sure if you have

Re: Where to I start to get to the bottom of this WriteTimeout issue?

2014-02-16 Thread Jacob Rhoden
Hi Erick, On 17 Feb 2014, at 1:19 pm, Erick Ramirez wrote: > Are you able to post log snippets around the time that the timeouts occur? > > I have a suspicion you may be running out of heap memory and might need to > tune your environment. The INFO entries in the log should indicate this. Im k

Where to I start to get to the bottom of this WriteTimeout issue?

2014-02-16 Thread Jacob Rhoden
Hi Guys, While doing a bulk load of data, incrementing counters based on log data, I am encountering a timeout exception. Im using the Datastax 2.0-rc2 java driver. The driver is set up with both nodes as contact points. The keyspace is setup with replication factor = 2. Is this an issue with

Re: Expired column showing up

2014-02-14 Thread Jacob Rhoden
It is my understanding that rows with TTLs don't mix well with rows that don't have TTLs. ie they should all have TTL or all not have TTL. That said if you can create a small java class (test case) that demonstrates the problem, I'm happy to try it out on 2.0.5. This code can be attached to a

Is it possible to set a counter to zero using CQL?

2014-02-12 Thread Jacob Rhoden
Hi Guys, My question is probably best described by example. Is it possible to indicate that on a particular day, the total number of “events” were zero. // Three events today update mytable count=count+1 where day=“20140101” update mytable count=count+1 where day=“20140101” update mytable count=c

Re: Using "IN" with the Datastax driver (2.0-??)

2014-02-10 Thread Jacob Rhoden
Perfect, thanks! I wonder if this is documented anywhere? Certainly I have no idea how to search google using the keyword “in” :D String[] words = TagsToArray.tagsToArray(keyword.toLowerCase()); PreparedStatement p = api.getCassandraSession().prepare("select log_entry from log_index wher

Using "IN" with the Datastax driver (2.0-??)

2014-02-10 Thread Jacob Rhoden
Hi Guys, Im experimenting with using IN to reduce the number of quires I have to execute. The following works in CQL: i.e select log_entry from log_index where keyword in (‘keyword1’, ‘keyword2’, ‘keyword3’, etc…); So I now want to work out how to convert this: PreparedStateme

Re: Update cassandra from perl 5.8

2014-02-03 Thread Jacob Rhoden
doing nightly dumps and transforms of the mysql database, but its no where near as good as having real time cassandra updates) Thanks! Jacob On 3 Feb 2014, at 10:43 pm, Paul LeoNerd Evans wrote: > On Mon, 03 Feb 2014 19:44:47 +1100 > Jacob Rhoden wrote: > >> We’ve got a legacy

Update cassandra from perl 5.8

2014-02-03 Thread Jacob Rhoden
Hi, We’ve got a legacy application on Perl 5.8, don’t want to upgrade the server for fear of something breaking. However for redundancy, It would be great if I could update the app so it writes to both the existing database, and cassandra. The Perl drivers appear to require perl 5.10+ http

Tracking and throttling failed sign in attempts in cassandra.

2014-01-24 Thread Jacob Rhoden
Hi Guys, If anyone has some wisdom to contribute to the following, I would very much appreciate it. Our existing small web application has some _basic_ mechanisims to prevent abuses such as repeated password guessing, sending spam, and so on, using a MySQL table. Im looking at if it is practic

Re: bad interaction between CompositeTypes and Secondary index

2014-01-21 Thread Jacob Rhoden
Was the original table created, or created then altered? It makes a difference as I have seen this type of thing occur on tables I first created then updated. Not sure if that issue was fixed in 2.0.4, I'm avoiding altering tables completely for now. __ Sent from iPh

Re: Are triggers production ready?

2014-01-13 Thread Jacob Rhoden
I would use them if I really needed them and had done comprehensive testing with real life usage patterns to ensure nothing unexpected happens. I would call it "beta" in the google sense of the word "beta". __ Sent from iPhone > On 14 Jan 2014, at 5:29 am, Robert Col

Re: What are the Pros and Cons to upgrade Cassandra from 1.2.10 to 2.0.X ?

2014-01-09 Thread Jacob Rhoden
On 10 Jan 2014, at 3:46 am, Miguel Angel Martin junquera wrote: > We have a Cassandra 1.2.10 pre-production ring integrated with Hadoop and > Pig and we are thinking to upgrade the system from 1.2.10 to 2.0.X ( to > 2.0.4 for example) before we pass to production. > > What are the Pros a

Cassandra and "bug track" type number sequencing.

2013-12-16 Thread Jacob Rhoden
Hi Guys, As per the subject, is there any way at all to easily associate small numbers in systems where users traditionally associate “bug/request” tickets with short numbers? In this use case I imagine the requirements would be as follows: The numbers don’t necessary need to be sequential, jus

Re: One big table/cf or many small ones?

2013-12-13 Thread Jacob Rhoden
Hi Tinus, > On 12 Dec 2013, at 6:59 pm, Tinus Sky wrote: > > My service does have users who can add a message to a list. The list of > message is sorted by date and displayed. When a user changes the message the > date is changed and the message moves to the top of the list. > > A possible so

Re: calculating sizes on disk

2013-12-06 Thread Jacob Rhoden
Not sure what your end setup will be, but I would probably just spin up a cluster and fill it with typical data to and measure the size on disk. __ Sent from iPhone > On 7 Dec 2013, at 6:08 am, John Sanda wrote: > > I am trying to do some disk capacity planning. I h

Re: How to measure data transfer between data centers?

2013-12-04 Thread Jacob Rhoden
http://unix.stackexchange.com/questions/41765/traffic-stats-per-network-port __ Sent from iPhone > On 5 Dec 2013, at 5:44 am, Tom van den Berge wrote: > > Hi Chris, > > I think streaming is used for repair tasks, bulk loading and that kind of > things, but not for

Cassandra issue on startup (2.0.1 +?)

2013-11-29 Thread Jacob Rhoden
I know I need to get around to upgrading. Is this (exception on startup) an issue fixed in 2.0.3? Caused by: java.lang.IndexOutOfBoundsException: index (1) must be less than size (1) at com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:306) at com.google

cassandra-driver-core-2.0.0-rc1.jar issue

2013-11-18 Thread Jacob Rhoden
Testing this out switching over to the new driver. Its mostly working except for one particular query (or code?) is causing the following: com.datastax.driver.core.exceptions.DriverInternalError: Tried to execute unknown prepared query 0x67dfcaa71c14d42a0a7f62406b41ea3e com.datastax.driver.co

Re: Efficient IP address location lookup

2013-11-15 Thread Jacob Rhoden
013, at 4:36 PM, Jacob Rhoden wrote: >> >> Hi Guys, >> >> It occurs to me that someone may have done this before and be willing to >> share, or may just be interested in helping work out it. >> >> Assuming a database table where the partition key is

Efficient IP address location lookup

2013-11-15 Thread Jacob Rhoden
Hi Guys, It occurs to me that someone may have done this before and be willing to share, or may just be interested in helping work out it. Assuming a database table where the partition key is the first component of a users IPv4 address, i.e. (ip=100.0.0.1, part=100) and the remaining three part

Best data structure for tracking most recent updates.

2013-11-08 Thread Jacob Rhoden
I need to be able to show the most recent changes that have occurred in a system, I understand inserting every update into a tracking table and deleting old updates may not be great, as I may end up creating millions of tombstones. i.e. don't do this:create table recent_updates(uuid timeuuid primar

Re: Create an index on CQL table

2013-11-07 Thread Jacob Rhoden
> On 8 Nov 2013, at 12:54 pm, Techy Teck wrote: > > I am using the below table in our use case - > > create table testing1 ( > employee_id text, > employee_name text, > value text, > last_modified_date timeuuid, > primary key (employee_name,last_modified_date) >); Bef

Recovering from mystical cassandra errors.

2013-10-29 Thread Jacob Rhoden
Hi Guys, Im testing out Cassandra with a 2.0.1 install, just like with 1.2.8 I last tested, from time to time I am coming across errors that prevent Cassandra starting up. For example: Yesterday I installed this instance, created and removed some “tables” using cql3 and inserted/updated/trunca

Example Cassandra 2.0 trigger classes?

2013-10-28 Thread Jacob Rhoden
Hi Guys, 1. Before I slog my way through the documentation on using triggers? Is anyone aware of any real world usage sample code or blogs with examples on this? 2. My use case is, if an update would result in changing (for example) the the “name” field of the “person” table, storing a copy of