Hi,
I am Jan Algermissen (REST-head, freelance programmer/consultant) and
Cassandra-newbie.
I am looking at Cassandra for an application I am working on. There will be a
max. of 10 Million items (Texts and attributes of a retailer's products) in the
database. There will occasional writes (e.g
Hi,
second question:
is it recommended to set up Cassandra using 'RAID-ed' disks for per-node
reliability or do people usually just rely on having the multiple nodes anyway
- why bother with replicated disks?
Jan
Hi,
thanks for the helpful replies last week.
It looks as if I will deploy Cassandra on a bunch of VMs and I am now in the
process of understanding what the dimensions of the VMS should be.
So far, I understand the following:
- I need at least 3 VMs for a minimal Cassandra setup
- I should
(plus some overhead of
course).??
Jan
Any memory not allocated to a process will generally be put to good use
serving as page cache. See here: http://en.wikipedia.org/wiki/Page_cache
Jon
On Tue, Jul 30, 2013 at 10:51 PM, Jan Algermissen
jan.algermis...@nordsc.com wrote:
Hi
cause Problems because the way
Cassandra works is likely to cause the nodes to compete for IO (unless each
node has dedicated storage)
Jan
What do you mean by that?
Thanks,
Shahab
On Wed, Jul 31, 2013 at 3:10 AM, Jan Algermissen jan.algermis...@nordsc.com
wrote:
Jon
Hi,
I plan on using Cassandra in a Java EE7 (Glassfish) project and I wonder which
driver I should pick.
Can anyone make a recommendation which of the available drivers fits best into
a Java EE environment?
(I looking for avoiding thread pool management issues mostly. I am not so
interested
, but got compile
errors. Do you know what the quality of cassandra-jdbc is?
Jan
Just my .2 cents worth
Andy
On 3 Aug 2013, at 08:28, Jan Algermissen jan.algermis...@nordsc.com wrote:
Hi,
I plan on using Cassandra in a Java EE7 (Glassfish) project and I wonder
which driver I
. IMHO
it's really down to the functionality you require in your application. The
Java Driver form datastax has the lowest learning curve for any java
programmer familiar with JDBC and I believe deals with pool management.
Just my .2 cents worth
Andy
On 3 Aug 2013, at 08:28, Jan
. The
Java Driver form datastax has the lowest learning curve for any java
programmer familiar with JDBC and I believe deals with pool management.
Just my .2 cents worth
Andy
On 3 Aug 2013, at 08:28, Jan Algermissen jan.algermis...@nordsc.com wrote:
Hi,
I plan on using Cassandra
Hi,
I think it does not fit the model of how C* does writes, but just to verify:
Is there an update-in-place possibility on maps? That is, could I do an atomic
increment on a value in a map?
Jan
Hi,
after seeing Patrick's truly excellent 3-part series on modeling, this question
pops up:
When I do an update on a collection, using a TTL in the update statement (like
Patrick does in the example with the login-location time series example), does
the TTL apply to the update only, or to
, I wanted to make sure I do not read
anything into the docs that isn't there.
As for the atomic increment, I take the answer is 'no, there is no atomic
increment, I have to pull the value to the client and send an update with the
new value'.
Jan
Alain
2013/8/6 Jan Algermissen
On 06.08.2013, at 11:36, Andy Twigg andy.tw...@gmail.com wrote:
Store pointers to counters as map values?
Sorry, but this fits into nothing I know about C* so far - can you explain?
Jan
Hi,
I have a specific use case to address with Cassandra and I can't get my head
around whether using Hadoop on top creates any significant benefit or not.
Situation:
I have product data and each product 'contains' a number of articles (100 /
product), representing individual colors/sizes
to ...).
Makes sense?
Jan
Cheers
-
Aaron Morton
Cassandra Consultant
New Zealand
@aaronmorton
http://www.thelastpickle.com
On 10/08/2013, at 8:15 PM, Jan Algermissen jan.algermis...@nordsc.com wrote:
Hi,
I have a specific use case to address with Cassandra and I
HI,
ok, so I found token() [1], and that it is an option for paging through
randomly partitioned data.
I take it that combining token() and LIMIT is the CQL3 idiom for paging (set
aside the fact that one shouldn't raelly want to page and use C*)
Now, when I page through a CF with wide rows,
Hi,
I have a use case, where I periodically need to apply updates to a wide row
that should replace the whole row.
The straight-forward insert/update only replace values that are present in the
executed statement, keeping remaining data around.
Is there a smooth way to do a replace with C* or
Hi Dawood,
On 02.09.2013, at 16:36, dawood abdullah muhammed.daw...@gmail.com wrote:
Hi
I have a requirement of versioning to be done in Cassandra.
Following is my column family definition
create table file_details(id text primary key, fname text, version int,
mimetype text);
I have
, Sep 2, 2013 at 10:47 PM, Jan Algermissen jan.algermis...@nordsc.com
wrote:
Hi Dawood,
On 02.09.2013, at 16:36, dawood abdullah muhammed.daw...@gmail.com wrote:
Hi
I have a requirement of versioning to be done in Cassandra.
Following is my column family definition
create table
Baskar,
On 03.09.2013, at 23:11, Baskar Duraikannu baskar.duraika...@outlook.com
wrote:
I have a similar use case but only need to update portion of the row. We
basically perform single write (with old and new columns) with very low value
of ttl for old columns.
I found out that using
Hi,
I have set up C* in a very limited environment: 3 VMs at digitalocean with 2GB
RAM and 40GB SSDs, so my expectations about overall performance are low.
Keyspace uses replication level of 2.
I am loading 1.5 Mio rows (each 60 columns of a mix of numbers and small texts,
300.000 wide rows
The subject line isn't appropriate - the servers do not crash but shut down.
Since the log messages appear several lines before the end of the log file, I
only saw afterwards. Excuse the confusion.
Jan
On 04.09.2013, at 10:44, Jan Algermissen jan.algermis...@nordsc.com wrote:
Hi,
I have
Romain,
On 04.09.2013, at 11:11, Romain HARDOUIN romain.hardo...@urssaf.fr wrote:
Maybe you should include the end of Cassandra logs.
There is nothing that seems interesting in cassandra.log. Below you find
system.log.
What comes to my mind when I read your first post is OOM killer.
But
[CompactionExecutor:5] 2013-09-06 11:02:28,685 CassandraDaemon.java (line
192) Exception in thread Thread[CompactionExecutor:
On the other hosts the log looks similar, but these keep running, desipte the
OutOfMemory Errors.
Jan
On Wed, Sep 4, 2013 at 9:44 AM, Jan Algermissen
Trying to approach this in a bit more structured way to make it more helpful
for others.
AFAIU my problem seems to be the combination of heavy write load and very
limited RAM (2GB).
C* design seems to cause nodes to run out of heap space instead of reducing
processing of incoming writes.
Hi,
I keep seeing the error message below in my log files. Can someone explain what
it means and how to prevent it?
INFO [OptionalTasks:1] 2013-09-07 13:46:27,160 MeteredFlusher.java (line 58)
flushing high-traffic column family CFS(Keyspace='pr
oducts', ColumnFamily='product') (estimated
On 06.09.2013, at 17:07, Jan Algermissen jan.algermis...@nordsc.com wrote:
On 06.09.2013, at 13:12, Alex Major al3...@gmail.com wrote:
Have you changed the appropriate config settings so that Cassandra will run
with only 2GB RAM? You shouldn't find the nodes go down.
Check out
I have a strange pattern: In a cluster with three equally dimensioned and
configured nodes I keep loosing one because apparently it fails to flush its
memtables:
http://twitpic.com/dcrtel
It is a different node every time.
So far I understand that I should expect to see the chain-saw graph
, but the minimal
memory I need for that cannot be tuned down but depends on the size of the
stuff written to C*. (Due to C* doing its memtable magic) to save using
sequential IO.
It is an interesting trade off. (if I get it right by now :-)
Jan
On Friday, September 6, 2013, Jan Algermissen wrote
PM, Jan Algermissen jan.algermis...@nordsc.com
wrote:
I have a strange pattern: In a cluster with three equally dimensioned and
configured nodes I keep loosing one because apparently it fails to flush its
memtables:
http://twitpic.com/dcrtel
It is a different node every time.
So
Based on my tuning work with C* over the last days, I guess I reached the
following insights.
Maybe someone can confirm whether they make sense:
The more heap I give to Cassandra (up to the GC tipping point of ~8GB) the more
writes it can accumulate in memtables before doing IO.
The more
On 10.09.2013, at 19:37, Robert Coli rc...@eventbrite.com wrote:
Cassandra does not prevent a given node from writing to RAM faster than it
can flush to disk?
Yes, that is what I meant.
What remains unclear to me is what the oprational strategy is towards handling
an increase in writes or
Tim,
On 23.09.2013, at 18:17, Tim Dunphy bluethu...@gmail.com wrote:
Hello,
I am running Cassandra 2.0 on a 2gb memory 10 gb HD in a virtual cloud
environment. It's supporting a php application running on the same node.
I have played with C* (1.2 and 2.0) in a low-RAM environment the last
Seems I am impacted by a paging bug in C* 2.0.0 and need to go to 2.0.1.
Is there any estimate for when the (Centos 6/64bit) RPM will be updated?
I'd rather wait than change installation procedure - but it depends on the
timeline for me.
Jan
Jimmy,
On 01.10.2013, at 17:26, Jimmy Lin y2klyf+w...@gmail.com wrote:
i have a table like the following:
CREATE TABLE log (
mykey timeuuid,
type text,
msg text,
primary key(mykey, type)
);
I want to page through all the results from the table using
Have you considered the new
for token function?
On Tue, Oct 1, 2013 at 8:57 AM, David Ward da...@shareablee.com wrote:
2.0 has a lot of really exciting stuff, unfortunately 2.0 has a lot of really
exciting stuff that may increase the risk of updating to 2.0 just yet.
On Tue, Oct 1, 2013 at 9:30 AM, Jan Algermissen
Hi,
my rows consist of ~70 columns each, some containing small values, some
containing larger amounts of content (think small documents).
My data is occasionally updated and read several times per day as complete
paging through all rows.
The updates usually affect only about 10% of the small
Petter,
On 24.10.2013, at 14:38, Petter von Dolwitz (Hem)
petter.von.dolw...@gmail.com wrote:
Hi,
I have a table that (in simplified version) looks like this:
CREATE TABLE mytable (
a varchar,
b varchar,
c varchar
d timstamp,
e varchar,
PRIMARY KEY (a, b, c, d)
);
am on Cassandra 2.0.1
/Petter
2013/10/24 Jan Algermissen jan.algermis...@nordsc.com
Petter,
On 24.10.2013, at 14:38, Petter von Dolwitz (Hem)
petter.von.dolw...@gmail.com wrote:
Hi,
I have a table that (in simplified version) looks like this:
CREATE TABLE mytable
with the
data or a problem with Cassandra.
/Petter
Den torsdagen den 24:e oktober 2013 skrev Jan Algermissen:
On 24.10.2013, at 15:13, Petter von Dolwitz (Hem)
petter.von.dolw...@gmail.com wrote:
You cannot use a part in a where clause unless you specify the preceeding
parts also
Hi all,
after upgrading C* and the java-driver I am running into problems with paging.
Maybe someone can provide a quick clue.
Upgrading was
C* from 2.0.1 to 2.0.3
Java Driver from 2.0.0-rc1 to 2.0.0-rc2
Client side, I get the following messages (apparently during a call to
me with the faulty situation the assertion is there to
detect?
Jan
On 19.12.2013, at 11:39, Sylvain Lebresne sylv...@datastax.com wrote:
https://issues.apache.org/jira/browse/CASSANDRA-6447
On Thu, Dec 19, 2013 at 11:16 AM, Jan Algermissen
jan.algermis...@nordsc.com wrote:
Hi all
branch.
As I am on Debian packages, I'd rather not :-)
Thanks for the quick insights!
Jan
--
Sylvain
On 19.12.2013, at 11:39, Sylvain Lebresne sylv...@datastax.com wrote:
https://issues.apache.org/jira/browse/CASSANDRA-6447
On Thu, Dec 19, 2013 at 11:16 AM, Jan Algermissen
On 09.01.2014, at 03:36, Sanjeeth Kumar sanje...@exotel.in wrote:
Hi all,
What is the latest stable version of cassandra you have in production ? We
are migrating a large chunk of our mysql database to cassandra. I see a lot
of discussions regarding 1.* versions, but I have not seen /
Hi,
I am designing a larger system that is composed of several independent smaller
systems (think services). Each of the smaller systems has its own
funtionality, logical database, and data access patterns.
My plan is to use a Cassandra cluster as the (cloud) database solution for
all/most of
Hi,
does anyone know of a C-driver that can be / has been used with nginx?
I am afraid that the C++ drivers[1] threading and connection pooling approach
interferes with nginx's threading model.
Doe anyone have any ideas?
Jan
[1] https://github.com/datastax/cpp-driver
On 15 Mar 2014, at 19:23, Joe Stein crypt...@gmail.com wrote:
Here is an example wrapper how to use the DataStax java driver in scala
https://github.com/stealthly/scala-cassandra
Thanks Joe,
yes, better to just use the java-driver than to create a generic scala-driver
it seems.
Jan
Hi Theo,
On 19 Mar 2014, at 17:23, Theo Hultberg t...@iconara.net wrote:
And cql-rb is full featured when it comes to CQL3. It supports all features
of Cassandra 1.2. For some of the Cassandra 2.0 features you have to wait for
a final version of 2.0, but the current prerelease is stable and
Hi,
maybe someone knows a nice solution to the following problem:
I have N worker processes that are intentionally masterless and do not know
about each other - they are stateless and independent instances of a given
service system.
These workers need to poll an event feed, say about every 10
* cluster that can (potentially) do CAS.
Why would I set up a MySQL cluster to solve that problem?
And yeah, I could use a queue or redis or whatnot, but I want to avoid yet
another moving part :-)
Jan
On Thu, Apr 3, 2014 at 11:44 PM, Jan Algermissen jan.algermis...@nordsc.com
wrote:
Hi
ipremya...@gmail.com wrote:
Oh ok. I thought you did not have a cassandra cluster already. Sorry about
that.
On Fri, Apr 4, 2014 at 11:42 AM, Jan Algermissen jan.algermis...@nordsc.com
wrote:
On 04 Apr 2014, at 11:18, prem yadav ipremya...@gmail.com wrote:
Though cassandra can work
Hi,
can anyone point me to recommendations for hosting and configuration
requirements when running a Production Cassandra Cluster at Rackspace?
Are there reference projects that document the suitability of Rackspace for
running a production Cassandra cluster?
Jan
Hi,
in our project, we apparently have a problem or misunderstanding of the
relationship between schema changes and data updates.
One team is doing automated tests during build and deployment that executes
data migration tests on a development cluster. In those migrations there will
be schema
of the change using nodetool
or by querying the schema_cf or migrations_cf tables?
http://wiki.apache.org/cassandra/LiveSchemaUpdates
Jan
--
Colin Clark
+1-320-221-9531
On May 18, 2014, at 3:30 AM, Jan Algermissen jan.algermis...@nordsc.com
wrote:
Hi,
in our project, we
On 18 May 2014, at 10:30, Jan Algermissen jan.algermis...@nordsc.com wrote:
Hi,
in our project, we apparently have a problem or misunderstanding of the
relationship between schema changes and data updates.
One team is doing automated tests during build and deployment that executes
Hi,
I just came across this recipe by Netflix, that addresses the impact of
tombstones in queue access patterns with a time based rolling shard to allow
compaction to happen in one shard while the other is ‘busy’. (At least this is
what understand from the intro)
Hi Gary,
On 31 Aug 2014, at 07:19, Gary Zhao garyz...@gmail.com wrote:
Hi
Could you recommend a Scala driver and share your experiences of using it. Im
thinking if i use java driver in Scala directly
I am using Martin’s approach without any problems:
On 17 Sep 2014, at 20:55, Sávio S. Teles de Oliveira savio.te...@cuia.com.br
wrote:
I'm using the Cassandra 2.0.9 with JAVA datastax driver.
I'm running the tests in a cluster with 3 nodes, RF=3 and CL=ALL for each
operation.
I have a Column family filled with some keys (for example 'a'
Hi,
I have put together some thoughts on realizing simple queues with Cassandra.
https://github.com/algermissen/cassandra-ruby-queue
The design is inspired by (the much more sophisticated) Netfilx approach[1] but
very reduced.
Given that I am still a C* newbie, I’d be very glad to hear some
- but since performance isn’t our concern, CAS should do fine, I
guess(?)
Thanks again,
Jan
---
Chris Lohfink
On Oct 5, 2014, at 6:03 PM, Jan Algermissen jan.algermis...@nordsc.com
wrote:
Hi,
I have put together some thoughts on realizing simple queues with Cassandra.
https
the job in the queue and another insert to mark the
job as done or in process
or whatever. This would also give you the benefit of being able to replay the
state of the queue.
Thanks, I’ll try that, too.
Jan
On Mon, Oct 6, 2014 at 12:57 AM, Jan Algermissen jan.algermis...@nordsc.com
Robert,
On 06 Oct 2014, at 17:50, Robert Coli rc...@eventbrite.com wrote:
In theory they can also be designed such that history is not infinite, which
mitigates the buildup of old queue state.
Hmm, I was under the impression that issues with old queue state disappear
after
2014, at 22:50, Robert Coli rc...@eventbrite.com wrote:
On Mon, Oct 6, 2014 at 1:40 PM, Jan Algermissen jan.algermis...@nordsc.com
wrote:
Hmm, I was under the impression that issues with old queue state disappear
after gc_grace_seconds and that the goal primarily is to keep the rows
‘short
Hi,
suppose I have two data centers and want to coordinate a bunch of services in
each data center (for example to load data into a per-DC system that is not
DC-aware (Solr)).
Does it make sense to use CAS functionality with explicit LOCAL_SERIAL to
'elect' a leader per data center to do the
Hi,
when I replace the content of a map-valued column (when I replace the complete
map), will this create tombstones for those map entries that are not present in
the new map?
My expectation is 'yes', because the map is laid out as normal columns
internally so keys not in the new map should
Hi,
I am looking for a driver for the Rust language. I found some projects
which seem quite abandoned.
Can someone point me to the driver that makes the most sense to look at
or help working on?
Cheers,
Jan
Hi,
I just read [1] which describes a lease implementation using CAS
queries. It applies a TTL to the lease which needs to be refreshed
periodically by the lease holder.
I use such a pattern myself since a couple of years, so no surprise
there.
However, the article uses CAS queries not
Jonathan,
On 26 May 2017, at 17:00, Jonathan Haddad wrote:
If you have a small amount of hot data, enable the row cache. The
memtable
is not designed to be a cache. You will not see a massive performance
impact of writing one to disk. Sstables will be in your page cache,
meaning
you won't
Hi,
is it possible to extract from repair logs the writetime of the writes
that needed to be repaired?
I have some processes I would like to re-trigger from a time point if
repair found problems.
Is that useful? Possible?
Jan
mples
http://techblog.outbrain.com/2011/07/leader-election-with-zookeeper/
https://www.tutorialspoint.com/zookeeper/zookeeper_leader_election.htm
Its more/new work, but should be an elegant solution.
Hope that helps.
Jayesh
On 5/25/17, 9:19 AM, "Jan Algermissen" <algermissen1...@icloud.com>
wrote:
Hi,
I am using a updates to a column with a ttl to represent a lock. The
owning process keeps updating the lock's TTL as long as it is running.
If the process crashes, the lock will timeout and be deleted. Then
another process can take over.
I have used this pattern very successfully over
AM, Jan Algermissen
<algermissen1...@icloud.com
wrote:
Hi,
is it possible to extract from repair logs the writetime of the
writes
that needed to be repaired?
I have some processes I would like to re-trigger from a time point if
repair found problems.
Is that useful? Possible?
Hi,
my understanding is that
- for writes using any of the quorum CLs will not put more overall load
on the cluster because writes will be sent to all nodes responsible for
a partition anyhow. So quorum only increases response time of the
coordinator, not cluster load.
Correct?
- for
73 matches
Mail list logo
