date:20171117

Re: How quickly we can bootstrap

2017-11-17 Thread Ben Slater

Hi Anshu

For quick scaling, we’ve had success with an approach of scaling up the
compute capacity (attached to EBS) rather than scaling out with more nodes
in order to provide relatively quick scale up/down capability. The approach
is implemented as part of our managed service but the concept is generic
enough to work in any virtualised environment. You can find more detail
here if interested:
https://www.instaclustr.com/instaclustr-dynamic-resizing-for-apache-cassandra/

Cheers
Ben

On Sat, 18 Nov 2017 at 05:02 Anshu Vajpayee 
wrote:

> Cassandra supports elastic scalability  - meaning on demand we can
> increase or decrease #of nodes as per scaling demand from the application.
>
> Let's consider we have 5 node cluster and each node has data pressure of
> about 3 TB.
>
> Now as per sudden load, we want to add 1 node in the cluster  as quick as
> possible.
>
> Please suggest what would be the fastest method to add the new node on
> cluster? Normal bootstrapping will definitely take time because it needs to
> stream at least 2.5 TB ( 5*3TB/6 nodes) from 5 nodes.  Please
> consider multi-core machines & 10 Gpbs card .
>
> Streaming throughput can help but not much.
>
> The similar requirement can come when we want to replace the failed node
> due to any hardware.
>
> Please suggest any best practice or scenarios to deal with above
> situations.
>
> Scaling is good but how quickly we can scale is another thing to consider.
>
>
>
>
>
>
>
>
> --
> *C*heers,*
> *Anshu V*
>
>
> --

*Ben Slater*

*Chief Product Officer *

Read our latest technical blog posts here
.

This email has been sent on behalf of Instaclustr Pty. Limited (Australia)
and Instaclustr Inc (USA).

This email and any attachments may contain confidential and legally
privileged information.  If you are not the intended recipient, do not copy
or disclose its content, but please reply to this email immediately and
highlight the error to the sender and then immediately delete the message.

Re: Reaper 1.0

2017-11-17 Thread Jon Haddad

Reaper uses log4j.  You can set up your logging configuration in the logging 
section of your yaml.  

Please see the section in the documentation under logging: 
http://cassandra-reaper.io/docs/configuration/reaper_specific/ 



> On Nov 17, 2017, at 10:36 AM, Harika Vangapelli -T (hvangape - AKRAYA INC at 
> Cisco)  wrote:
> 
> Jon,
>  
> I built a docker image and deployed stage environment for testing. I started 
> process as java –jar way.
>  
> Where do the logs generate?
>  
> 
>  
> Harika Vangapelli
> Engineer - IT
> hvang...@cisco.com 
> Tel:
> Cisco Systems, Inc.
> 
> 
> 
> United States
> cisco.com  
> Think before you print.
> This email may contain confidential and privileged material for the sole use 
> of the intended recipient. Any review, use, distribution or disclosure by 
> others is strictly prohibited. If you are not the intended recipient (or 
> authorized to receive for the recipient), please contact the sender by reply 
> email and delete all copies of this message.
> Please click here 
>  for 
> Company Registration Information.
>  
> From: Jon Haddad [mailto:jonathan.had...@gmail.com] On Behalf Of Jon Haddad
> Sent: Wednesday, November 15, 2017 9:23 AM
> To: user 
> Subject: Re: Reaper 1.0
>  
> Apache 2 Licensed, just like Cassandra.  
> https://github.com/thelastpickle/cassandra-reaper/blob/master/LICENSE.txt 
> 
>  
> Feel free to modify, put in prod, fork or improve. 
>  
> Unfortunately I had to re-upload the Getting Started video, we had 
> accidentally uploaded a first cut.  Correctly link is here: 
> https://www.youtube.com/watch?v=0dub29BgwPI 
> 
>  
> Jon
> 
> 
> On Nov 15, 2017, at 9:14 AM, Harika Vangapelli -T (hvangape - AKRAYA INC at 
> Cisco) > wrote:
>  
> Open source, free to use in production? Any License constraints, Please let 
> me know.
>  
> I experimented with it yesterday, really liked it.
>  
> 
>  
> Harika Vangapelli
> Engineer - IT
> hvang...@cisco.com 
> Tel:
> Cisco Systems, Inc.
> 
> 
> 
> United States
> cisco.com  
> Think before you print.
> This email may contain confidential and privileged material for the sole use 
> of the intended recipient. Any review, use, distribution or disclosure by 
> others is strictly prohibited. If you are not the intended recipient (or 
> authorized to receive for the recipient), please contact the sender by reply 
> email and delete all copies of this message.
> Please click here 
>  for 
> Company Registration Information.
>  
> From: Jon Haddad [mailto:jonathan.had...@gmail.com 
> ] On Behalf Of Jon Haddad
> Sent: Tuesday, November 14, 2017 2:18 PM
> To: user >
> Subject: Reaper 1.0
>  
> We’re excited to announce the release of the 1.0 version of Reaper for Apache 
> Cassandra!  We’ve made a lot of improvements to the flexibility of managing 
> repairs and simplified the UI based on feedback we’ve received. 
>  
> We’ve written a blog post discussing the changes in detail here: 
> http://thelastpickle.com/blog/2017/11/14/reaper-10-announcement.html 
> 
>  
> We also have a new YouTube video to help folks get up and running quickly: 
> https://www.youtube.com/watch?v=YKJRRFa22T4 
> 
>  
> The reaper site has all the docs should you have any questions: 
> http://cassandra-reaper.io/ 
>  
> Thanks all,
> Jon

RE: Reaper 1.0

2017-11-17 Thread Harika Vangapelli -T (hvangape - AKRAYA INC at Cisco)

Jon,

I built a docker image and deployed stage environment for testing. I started 
process as java –jar way.

Where do the logs generate?

[http://wwwin.cisco.com/c/dam/cec/organizations/gmcc/services-tools/signaturetool/images/logo/logo_gradient.png]



Harika Vangapelli
Engineer - IT
hvang...@cisco.com
Tel:

Cisco Systems, Inc.



United States
cisco.com


[http://www.cisco.com/assets/swa/img/thinkbeforeyouprint.gif]Think before you 
print.

This email may contain confidential and privileged material for the sole use of 
the intended recipient. Any review, use, distribution or disclosure by others 
is strictly prohibited. If you are not the intended recipient (or authorized to 
receive for the recipient), please contact the sender by reply email and delete 
all copies of this message.
Please click 
here for 
Company Registration Information.


From: Jon Haddad [mailto:jonathan.had...@gmail.com] On Behalf Of Jon Haddad
Sent: Wednesday, November 15, 2017 9:23 AM
To: user 
Subject: Re: Reaper 1.0

Apache 2 Licensed, just like Cassandra.  
https://github.com/thelastpickle/cassandra-reaper/blob/master/LICENSE.txt

Feel free to modify, put in prod, fork or improve.

Unfortunately I had to re-upload the Getting Started video, we had accidentally 
uploaded a first cut.  Correctly link is here: 
https://www.youtube.com/watch?v=0dub29BgwPI

Jon


On Nov 15, 2017, at 9:14 AM, Harika Vangapelli -T (hvangape - AKRAYA INC at 
Cisco) > wrote:

Open source, free to use in production? Any License constraints, Please let me 
know.

I experimented with it yesterday, really liked it.





Harika Vangapelli
Engineer - IT
hvang...@cisco.com
Tel:

Cisco Systems, Inc.



United States
cisco.com


Think before you print.

This email may contain confidential and privileged material for the sole use of 
the intended recipient. Any review, use, distribution or disclosure by others 
is strictly prohibited. If you are not the intended recipient (or authorized to 
receive for the recipient), please contact the sender by reply email and delete 
all copies of this message.
Please click 
here for 
Company Registration Information.


From: Jon Haddad [mailto:jonathan.had...@gmail.com] On Behalf Of Jon Haddad
Sent: Tuesday, November 14, 2017 2:18 PM
To: user >
Subject: Reaper 1.0

We’re excited to announce the release of the 1.0 version of Reaper for Apache 
Cassandra!  We’ve made a lot of improvements to the flexibility of managing 
repairs and simplified the UI based on feedback we’ve received.

We’ve written a blog post discussing the changes in detail here: 
http://thelastpickle.com/blog/2017/11/14/reaper-10-announcement.html

We also have a new YouTube video to help folks get up and running quickly: 
https://www.youtube.com/watch?v=YKJRRFa22T4

The reaper site has all the docs should you have any questions: 
http://cassandra-reaper.io/

Thanks all,
Jon

How quickly we can bootstrap

2017-11-17 Thread Anshu Vajpayee

Cassandra supports elastic scalability  - meaning on demand we can increase
or decrease #of nodes as per scaling demand from the application.

Let's consider we have 5 node cluster and each node has data pressure of
about 3 TB.

Now as per sudden load, we want to add 1 node in the cluster  as quick as
possible.

Please suggest what would be the fastest method to add the new node on
cluster? Normal bootstrapping will definitely take time because it needs to
stream at least 2.5 TB ( 5*3TB/6 nodes) from 5 nodes.  Please
consider multi-core machines & 10 Gpbs card .

Streaming throughput can help but not much.

The similar requirement can come when we want to replace the failed node
due to any hardware.

Please suggest any best practice or scenarios to deal with above
situations.

Scaling is good but how quickly we can scale is another thing to consider.








-- 
*C*heers,*
*Anshu V*

Re: Reaper 1.0

2017-11-17 Thread Anshu Vajpayee

Sure, I will update this thread.

On Sat, Nov 18, 2017 at 12:26 AM, Jonathan Haddad  wrote:

> It should work with DSE, but we don’t explicitly test it.
>
> Mind testing it and posting your results? If you could include the DSE
> version it would be great.
> On Thu, Nov 16, 2017 at 11:57 PM Anshu Vajpayee 
> wrote:
>
>> Thanks John for your efforts and nicley putting it on website & youtube .
>>
>> Just quick question - Is  it compactiable with DSE  versions? I know
>> under the hood they have  cassandra only , but just wanted to listen your
>> thoughts.
>>
>> On Thu, Nov 16, 2017 at 1:23 AM, Jon Haddad  wrote:
>>
>>> Apache 2 Licensed, just like Cassandra.  https://github.com/
>>> thelastpickle/cassandra-reaper/blob/master/LICENSE.txt
>>>
>>> Feel free to modify, put in prod, fork or improve.
>>>
>>> Unfortunately I had to re-upload the Getting Started video, we had
>>> accidentally uploaded a first cut.  Correctly link is here:
>>> https://www.youtube.com/watch?v=0dub29BgwPI
>>>
>>> Jon
>>>
>>> On Nov 15, 2017, at 9:14 AM, Harika Vangapelli -T (hvangape - AKRAYA INC
>>> at Cisco)  wrote:
>>>
>>> Open source, free to use in production? Any License constraints, Please
>>> let me know.
>>>
>>> I experimented with it yesterday, really liked it.
>>>
>>> 
>>>
>>> *Harika Vangapelli*
>>> Engineer - IT
>>> hvang...@cisco.com
>>> Tel:
>>> *Cisco Systems, Inc.*
>>>
>>>
>>>
>>> United States
>>> cisco.com
>>>
>>> Think before you print.
>>> This email may contain confidential and privileged material for the sole
>>> use of the intended recipient. Any review, use, distribution or disclosure
>>> by others is strictly prohibited. If you are not the intended recipient (or
>>> authorized to receive for the recipient), please contact the sender by
>>> reply email and delete all copies of this message.
>>> Please click here
>>>  for
>>> Company Registration Information.
>>>
>>> *From:* Jon Haddad [mailto:jonathan.had...@gmail.com
>>> ] *On Behalf Of *Jon Haddad
>>> *Sent:* Tuesday, November 14, 2017 2:18 PM
>>> *To:* user 
>>> *Subject:* Reaper 1.0
>>>
>>> We’re excited to announce the release of the 1.0 version of Reaper for
>>> Apache Cassandra!  We’ve made a lot of improvements to the flexibility of
>>> managing repairs and simplified the UI based on feedback we’ve received.
>>>
>>> We’ve written a blog post discussing the changes in detail here:
>>> http://thelastpickle.com/blog/2017/11/14/reaper-10-announcement.html
>>>
>>> We also have a new YouTube video to help folks get up and running
>>> quickly: https://www.youtube.com/watch?v=YKJRRFa22T4
>>>
>>> The reaper site has all the docs should you have any questions:
>>> http://cassandra-reaper.io/
>>>
>>> Thanks all,
>>> Jon
>>>
>>>
>>>
>>
>>
>> --
>> *C*heers,*
>> *Anshu V*
>>
>>
>>


-- 
*C*heers,*
*Anshu V*

Re: Reaper 1.0

2017-11-17 Thread Jonathan Haddad

It should work with DSE, but we don’t explicitly test it.

Mind testing it and posting your results? If you could include the DSE
version it would be great.
On Thu, Nov 16, 2017 at 11:57 PM Anshu Vajpayee 
wrote:

> Thanks John for your efforts and nicley putting it on website & youtube .
>
> Just quick question - Is  it compactiable with DSE  versions? I know under
> the hood they have  cassandra only , but just wanted to listen your
> thoughts.
>
> On Thu, Nov 16, 2017 at 1:23 AM, Jon Haddad  wrote:
>
>> Apache 2 Licensed, just like Cassandra.
>> https://github.com/thelastpickle/cassandra-reaper/blob/master/LICENSE.txt
>>
>> Feel free to modify, put in prod, fork or improve.
>>
>> Unfortunately I had to re-upload the Getting Started video, we had
>> accidentally uploaded a first cut.  Correctly link is here:
>> https://www.youtube.com/watch?v=0dub29BgwPI
>>
>> Jon
>>
>> On Nov 15, 2017, at 9:14 AM, Harika Vangapelli -T (hvangape - AKRAYA INC
>> at Cisco)  wrote:
>>
>> Open source, free to use in production? Any License constraints, Please
>> let me know.
>>
>> I experimented with it yesterday, really liked it.
>>
>> 
>>
>> *Harika Vangapelli*
>> Engineer - IT
>> hvang...@cisco.com
>> Tel:
>> *Cisco Systems, Inc.*
>>
>>
>>
>> United States
>> cisco.com
>>
>> Think before you print.
>> This email may contain confidential and privileged material for the sole
>> use of the intended recipient. Any review, use, distribution or disclosure
>> by others is strictly prohibited. If you are not the intended recipient (or
>> authorized to receive for the recipient), please contact the sender by
>> reply email and delete all copies of this message.
>> Please click here
>>  for
>> Company Registration Information.
>>
>> *From:* Jon Haddad [mailto:jonathan.had...@gmail.com
>> ] *On Behalf Of *Jon Haddad
>> *Sent:* Tuesday, November 14, 2017 2:18 PM
>> *To:* user 
>> *Subject:* Reaper 1.0
>>
>> We’re excited to announce the release of the 1.0 version of Reaper for
>> Apache Cassandra!  We’ve made a lot of improvements to the flexibility of
>> managing repairs and simplified the UI based on feedback we’ve received.
>>
>> We’ve written a blog post discussing the changes in detail here:
>> http://thelastpickle.com/blog/2017/11/14/reaper-10-announcement.html
>>
>> We also have a new YouTube video to help folks get up and running
>> quickly: https://www.youtube.com/watch?v=YKJRRFa22T4
>>
>> The reaper site has all the docs should you have any questions:
>> http://cassandra-reaper.io/
>>
>> Thanks all,
>> Jon
>>
>>
>>
>
>
> --
> *C*heers,*
> *Anshu V*
>
>
>

Re: Running Cassandra on FreeBSD

2017-11-17 Thread Michael Shuler

Looks simlar to:
https://lists.freebsd.org/pipermail/freebsd-stable/2014-October/080834.html

That ML post was linked on:
https://issues.apache.org/jira/browse/CASSANDRA-8325

That JIRA has a couple patches you might look at for examples.

-- 
Kind regards,
Michael


On 11/17/2017 04:06 AM, Jens Grassel wrote:
> Hi,
> 
> we downloaded the latest release and tried to run it on a FreeBSD 11
> machine which resultet in a JVM crash. Here is the output:
> 
> <---snip--->
> ./bin/cassandra -f  ? 134 
> CompilerOracle: dontinline 
> org/apache/cassandra/db/Columns$Serializer.deserializeLargeSubset 
> (Lorg/apache/cassandra/io/util/DataInputPlus;Lorg/apache/cassandra/db/Columns;I)Lorg/apache/cassandra/db/Columns;
> CompilerOracle: dontinline 
> org/apache/cassandra/db/Columns$Serializer.serializeLargeSubset 
> (Ljava/util/Collection;ILorg/apache/cassandra/db/Columns;ILorg/apache/cassandra/io/util/DataOutputPlus;)V
> CompilerOracle: dontinline 
> org/apache/cassandra/db/Columns$Serializer.serializeLargeSubsetSize 
> (Ljava/util/Collection;ILorg/apache/cassandra/db/Columns;I)I
> CompilerOracle: dontinline 
> org/apache/cassandra/db/commitlog/AbstractCommitLogSegmentManager.advanceAllocatingFrom
>  (Lorg/apache/cassandra/db/commitlog/CommitLogSegment;)V
> CompilerOracle: dontinline 
> org/apache/cassandra/db/transform/BaseIterator.tryGetMoreContents ()Z
> CompilerOracle: dontinline 
> org/apache/cassandra/db/transform/StoppingTransformation.stop ()V
> CompilerOracle: dontinline 
> org/apache/cassandra/db/transform/StoppingTransformation.stopInPartition ()V
> CompilerOracle: dontinline 
> org/apache/cassandra/io/util/BufferedDataOutputStreamPlus.doFlush (I)V
> CompilerOracle: dontinline 
> org/apache/cassandra/io/util/BufferedDataOutputStreamPlus.writeExcessSlow ()V
> CompilerOracle: dontinline 
> org/apache/cassandra/io/util/BufferedDataOutputStreamPlus.writeSlow (JI)V
> CompilerOracle: dontinline 
> org/apache/cassandra/io/util/RebufferingInputStream.readPrimitiveSlowly (I)J
> CompilerOracle: inline 
> org/apache/cassandra/db/rows/UnfilteredSerializer.serializeRowBody 
> (Lorg/apache/cassandra/db/rows/Row;ILorg/apache/cassandra/db/SerializationHeader;Lorg/apache/cassandra/io/util/DataOutputPlus;)V
> CompilerOracle: inline org/apache/cassandra/io/util/Memory.checkBounds (JJ)V
> CompilerOracle: inline org/apache/cassandra/io/util/SafeMemory.checkBounds 
> (JJ)V
> CompilerOracle: inline 
> org/apache/cassandra/utils/AsymmetricOrdering.selectBoundary 
> (Lorg/apache/cassandra/utils/AsymmetricOrdering/Op;II)I
> CompilerOracle: inline 
> org/apache/cassandra/utils/AsymmetricOrdering.strictnessOfLessThan 
> (Lorg/apache/cassandra/utils/AsymmetricOrdering/Op;)I
> CompilerOracle: inline org/apache/cassandra/utils/BloomFilter.indexes 
> (Lorg/apache/cassandra/utils/IFilter/FilterKey;)[J
> CompilerOracle: inline org/apache/cassandra/utils/BloomFilter.setIndexes 
> (JJIJ[J)V
> CompilerOracle: inline org/apache/cassandra/utils/ByteBufferUtil.compare 
> (Ljava/nio/ByteBuffer;[B)I
> CompilerOracle: inline org/apache/cassandra/utils/ByteBufferUtil.compare 
> ([BLjava/nio/ByteBuffer;)I
> CompilerOracle: inline 
> org/apache/cassandra/utils/ByteBufferUtil.compareUnsigned 
> (Ljava/nio/ByteBuffer;Ljava/nio/ByteBuffer;)I
> CompilerOracle: inline 
> org/apache/cassandra/utils/FastByteOperations$UnsafeOperations.compareTo 
> (Ljava/lang/Object;JILjava/lang/Object;JI)I
> CompilerOracle: inline 
> org/apache/cassandra/utils/FastByteOperations$UnsafeOperations.compareTo 
> (Ljava/lang/Object;JILjava/nio/ByteBuffer;)I
> CompilerOracle: inline 
> org/apache/cassandra/utils/FastByteOperations$UnsafeOperations.compareTo 
> (Ljava/nio/ByteBuffer;Ljava/nio/ByteBuffer;)I
> CompilerOracle: inline org/apache/cassandra/utils/vint/VIntCoding.encodeVInt 
> (JI)[B
> INFO  [main] 2017-11-17 11:03:12,020 YamlConfigurationLoader.java:89 - 
> Configuration location: 
> file:/usr/home/jens/apache-cassandra-3.11.1/conf/cassandra.yaml
> INFO  [main] 2017-11-17 11:03:12,366 Config.java:481 - Node 
> configuration:[allocate_tokens_for_keyspace=null; 
> authenticator=AllowAllAuthenticator; authorizer=AllowAllAuthorizer; 
> auto_bootstrap=true; auto_snapshot=true; back_pressure_enabled=false; 
> back_pressure_strategy=org.apache.cassandra.net.RateBasedBackPressure{high_ratio=0.9,
>  factor=5, flow=FAST}; batch_size_fail_threshold_in_kb=50; 
> batch_size_warn_threshold_in_kb=5; batchlog_replay_throttle_in_kb=1024; 
> broadcast_address=null; broadcast_rpc_address=null; 
> buffer_pool_use_heap_if_exhausted=true; cas_contention_timeout_in_ms=1000; 
> cdc_enabled=false; cdc_free_space_check_interval_ms=250; 
> cdc_raw_directory=null; cdc_total_space_in_mb=0; 
> client_encryption_options=; cluster_name=Test Cluster; 
> column_index_cache_size_in_kb=2; column_index_size_in_kb=64; 
> commit_failure_policy=stop; commitlog_compression=null; 
> commitlog_directory=null; commitlog_max_compression_buffers_in_pool=3; 
>

Re: Running Cassandra on FreeBSD

2017-11-17 Thread Johnny Miller

Jens,

I don't believe it is supported on FreeBSD - I would recommend  you stick
to the list here:
https://docs.datastax.com/en/landing_page/doc/landing_page/supportedPlatforms.html
. Also, I would suggest using Oracle JDK.

Regards,

Johnny


On 17 November 2017 at 11:06, Jens Grassel  wrote:

> Hi,
>
> we downloaded the latest release and tried to run it on a FreeBSD 11
> machine which resultet in a JVM crash. Here is the output:
>
> <---snip--->
> ./bin/cassandra -f  ? 134
> CompilerOracle: dontinline 
> org/apache/cassandra/db/Columns$Serializer.deserializeLargeSubset
> (Lorg/apache/cassandra/io/util/DataInputPlus;Lorg/
> apache/cassandra/db/Columns;I)Lorg/apache/cassandra/db/Columns;
> CompilerOracle: dontinline 
> org/apache/cassandra/db/Columns$Serializer.serializeLargeSubset
> (Ljava/util/Collection;ILorg/apache/cassandra/db/Columns;
> ILorg/apache/cassandra/io/util/DataOutputPlus;)V
> CompilerOracle: dontinline 
> org/apache/cassandra/db/Columns$Serializer.serializeLargeSubsetSize
> (Ljava/util/Collection;ILorg/apache/cassandra/db/Columns;I)I
> CompilerOracle: dontinline org/apache/cassandra/db/commitlog/
> AbstractCommitLogSegmentManager.advanceAllocatingFrom
> (Lorg/apache/cassandra/db/commitlog/CommitLogSegment;)V
> CompilerOracle: dontinline 
> org/apache/cassandra/db/transform/BaseIterator.tryGetMoreContents
> ()Z
> CompilerOracle: dontinline 
> org/apache/cassandra/db/transform/StoppingTransformation.stop
> ()V
> CompilerOracle: dontinline org/apache/cassandra/db/transform/
> StoppingTransformation.stopInPartition ()V
> CompilerOracle: dontinline org/apache/cassandra/io/util/
> BufferedDataOutputStreamPlus.doFlush (I)V
> CompilerOracle: dontinline org/apache/cassandra/io/util/
> BufferedDataOutputStreamPlus.writeExcessSlow ()V
> CompilerOracle: dontinline org/apache/cassandra/io/util/
> BufferedDataOutputStreamPlus.writeSlow (JI)V
> CompilerOracle: dontinline org/apache/cassandra/io/util/
> RebufferingInputStream.readPrimitiveSlowly (I)J
> CompilerOracle: inline 
> org/apache/cassandra/db/rows/UnfilteredSerializer.serializeRowBody
> (Lorg/apache/cassandra/db/rows/Row;ILorg/apache/cassandra/db/
> SerializationHeader;Lorg/apache/cassandra/io/util/DataOutputPlus;)V
> CompilerOracle: inline org/apache/cassandra/io/util/Memory.checkBounds
> (JJ)V
> CompilerOracle: inline org/apache/cassandra/io/util/SafeMemory.checkBounds
> (JJ)V
> CompilerOracle: inline 
> org/apache/cassandra/utils/AsymmetricOrdering.selectBoundary
> (Lorg/apache/cassandra/utils/AsymmetricOrdering/Op;II)I
> CompilerOracle: inline 
> org/apache/cassandra/utils/AsymmetricOrdering.strictnessOfLessThan
> (Lorg/apache/cassandra/utils/AsymmetricOrdering/Op;)I
> CompilerOracle: inline org/apache/cassandra/utils/BloomFilter.indexes
> (Lorg/apache/cassandra/utils/IFilter/FilterKey;)[J
> CompilerOracle: inline org/apache/cassandra/utils/BloomFilter.setIndexes
> (JJIJ[J)V
> CompilerOracle: inline org/apache/cassandra/utils/ByteBufferUtil.compare
> (Ljava/nio/ByteBuffer;[B)I
> CompilerOracle: inline org/apache/cassandra/utils/ByteBufferUtil.compare
> ([BLjava/nio/ByteBuffer;)I
> CompilerOracle: inline 
> org/apache/cassandra/utils/ByteBufferUtil.compareUnsigned
> (Ljava/nio/ByteBuffer;Ljava/nio/ByteBuffer;)I
> CompilerOracle: inline 
> org/apache/cassandra/utils/FastByteOperations$UnsafeOperations.compareTo
> (Ljava/lang/Object;JILjava/lang/Object;JI)I
> CompilerOracle: inline 
> org/apache/cassandra/utils/FastByteOperations$UnsafeOperations.compareTo
> (Ljava/lang/Object;JILjava/nio/ByteBuffer;)I
> CompilerOracle: inline 
> org/apache/cassandra/utils/FastByteOperations$UnsafeOperations.compareTo
> (Ljava/nio/ByteBuffer;Ljava/nio/ByteBuffer;)I
> CompilerOracle: inline org/apache/cassandra/utils/vint/VIntCoding.encodeVInt
> (JI)[B
> INFO  [main] 2017-11-17 11:03:12,020 YamlConfigurationLoader.java:89 -
> Configuration location: file:/usr/home/jens/apache-cassandra-3.11.1/conf/
> cassandra.yaml
> INFO  [main] 2017-11-17 11:03:12,366 Config.java:481 - Node
> configuration:[allocate_tokens_for_keyspace=null; 
> authenticator=AllowAllAuthenticator;
> authorizer=AllowAllAuthorizer; auto_bootstrap=true; auto_snapshot=true;
> back_pressure_enabled=false; back_pressure_strategy=org.
> apache.cassandra.net.RateBasedBackPressure{high_ratio=0.9, factor=5,
> flow=FAST}; batch_size_fail_threshold_in_kb=50;
> batch_size_warn_threshold_in_kb=5; batchlog_replay_throttle_in_kb=1024;
> broadcast_address=null; broadcast_rpc_address=null; 
> buffer_pool_use_heap_if_exhausted=true;
> cas_contention_timeout_in_ms=1000; cdc_enabled=false;
> cdc_free_space_check_interval_ms=250; cdc_raw_directory=null;
> cdc_total_space_in_mb=0; client_encryption_options=;
> cluster_name=Test Cluster; column_index_cache_size_in_kb=2;
> column_index_size_in_kb=64; commit_failure_policy=stop;
> commitlog_compression=null; commitlog_directory=null;
> commitlog_max_compression_buffers_in_pool=3;
> commitlog_periodic_queue_size=-1;

Running Cassandra on FreeBSD

2017-11-17 Thread Jens Grassel

Hi,

we downloaded the latest release and tried to run it on a FreeBSD 11
machine which resultet in a JVM crash. Here is the output:

<---snip--->
./bin/cassandra -f  ? 134 
CompilerOracle: dontinline 
org/apache/cassandra/db/Columns$Serializer.deserializeLargeSubset 
(Lorg/apache/cassandra/io/util/DataInputPlus;Lorg/apache/cassandra/db/Columns;I)Lorg/apache/cassandra/db/Columns;
CompilerOracle: dontinline 
org/apache/cassandra/db/Columns$Serializer.serializeLargeSubset 
(Ljava/util/Collection;ILorg/apache/cassandra/db/Columns;ILorg/apache/cassandra/io/util/DataOutputPlus;)V
CompilerOracle: dontinline 
org/apache/cassandra/db/Columns$Serializer.serializeLargeSubsetSize 
(Ljava/util/Collection;ILorg/apache/cassandra/db/Columns;I)I
CompilerOracle: dontinline 
org/apache/cassandra/db/commitlog/AbstractCommitLogSegmentManager.advanceAllocatingFrom
 (Lorg/apache/cassandra/db/commitlog/CommitLogSegment;)V
CompilerOracle: dontinline 
org/apache/cassandra/db/transform/BaseIterator.tryGetMoreContents ()Z
CompilerOracle: dontinline 
org/apache/cassandra/db/transform/StoppingTransformation.stop ()V
CompilerOracle: dontinline 
org/apache/cassandra/db/transform/StoppingTransformation.stopInPartition ()V
CompilerOracle: dontinline 
org/apache/cassandra/io/util/BufferedDataOutputStreamPlus.doFlush (I)V
CompilerOracle: dontinline 
org/apache/cassandra/io/util/BufferedDataOutputStreamPlus.writeExcessSlow ()V
CompilerOracle: dontinline 
org/apache/cassandra/io/util/BufferedDataOutputStreamPlus.writeSlow (JI)V
CompilerOracle: dontinline 
org/apache/cassandra/io/util/RebufferingInputStream.readPrimitiveSlowly (I)J
CompilerOracle: inline 
org/apache/cassandra/db/rows/UnfilteredSerializer.serializeRowBody 
(Lorg/apache/cassandra/db/rows/Row;ILorg/apache/cassandra/db/SerializationHeader;Lorg/apache/cassandra/io/util/DataOutputPlus;)V
CompilerOracle: inline org/apache/cassandra/io/util/Memory.checkBounds (JJ)V
CompilerOracle: inline org/apache/cassandra/io/util/SafeMemory.checkBounds (JJ)V
CompilerOracle: inline 
org/apache/cassandra/utils/AsymmetricOrdering.selectBoundary 
(Lorg/apache/cassandra/utils/AsymmetricOrdering/Op;II)I
CompilerOracle: inline 
org/apache/cassandra/utils/AsymmetricOrdering.strictnessOfLessThan 
(Lorg/apache/cassandra/utils/AsymmetricOrdering/Op;)I
CompilerOracle: inline org/apache/cassandra/utils/BloomFilter.indexes 
(Lorg/apache/cassandra/utils/IFilter/FilterKey;)[J
CompilerOracle: inline org/apache/cassandra/utils/BloomFilter.setIndexes 
(JJIJ[J)V
CompilerOracle: inline org/apache/cassandra/utils/ByteBufferUtil.compare 
(Ljava/nio/ByteBuffer;[B)I
CompilerOracle: inline org/apache/cassandra/utils/ByteBufferUtil.compare 
([BLjava/nio/ByteBuffer;)I
CompilerOracle: inline 
org/apache/cassandra/utils/ByteBufferUtil.compareUnsigned 
(Ljava/nio/ByteBuffer;Ljava/nio/ByteBuffer;)I
CompilerOracle: inline 
org/apache/cassandra/utils/FastByteOperations$UnsafeOperations.compareTo 
(Ljava/lang/Object;JILjava/lang/Object;JI)I
CompilerOracle: inline 
org/apache/cassandra/utils/FastByteOperations$UnsafeOperations.compareTo 
(Ljava/lang/Object;JILjava/nio/ByteBuffer;)I
CompilerOracle: inline 
org/apache/cassandra/utils/FastByteOperations$UnsafeOperations.compareTo 
(Ljava/nio/ByteBuffer;Ljava/nio/ByteBuffer;)I
CompilerOracle: inline org/apache/cassandra/utils/vint/VIntCoding.encodeVInt 
(JI)[B
INFO  [main] 2017-11-17 11:03:12,020 YamlConfigurationLoader.java:89 - 
Configuration location: 
file:/usr/home/jens/apache-cassandra-3.11.1/conf/cassandra.yaml
INFO  [main] 2017-11-17 11:03:12,366 Config.java:481 - Node 
configuration:[allocate_tokens_for_keyspace=null; 
authenticator=AllowAllAuthenticator; authorizer=AllowAllAuthorizer; 
auto_bootstrap=true; auto_snapshot=true; back_pressure_enabled=false; 
back_pressure_strategy=org.apache.cassandra.net.RateBasedBackPressure{high_ratio=0.9,
 factor=5, flow=FAST}; batch_size_fail_threshold_in_kb=50; 
batch_size_warn_threshold_in_kb=5; batchlog_replay_throttle_in_kb=1024; 
broadcast_address=null; broadcast_rpc_address=null; 
buffer_pool_use_heap_if_exhausted=true; cas_contention_timeout_in_ms=1000; 
cdc_enabled=false; cdc_free_space_check_interval_ms=250; 
cdc_raw_directory=null; cdc_total_space_in_mb=0; 
client_encryption_options=; cluster_name=Test Cluster; 
column_index_cache_size_in_kb=2; column_index_size_in_kb=64; 
commit_failure_policy=stop; commitlog_compression=null; 
commitlog_directory=null; commitlog_max_compression_buffers_in_pool=3; 
commitlog_periodic_queue_size=-1; commitlog_segment_size_in_mb=32; 
commitlog_sync=periodic; commitlog_sync_batch_window_in_ms=NaN; 
commitlog_sync_period_in_ms=1; commitlog_total_space_in_mb=null; 
compaction_large_partition_warning_threshold_mb=100; 
compaction_throughput_mb_per_sec=16; concurrent_compactors=null; 
concurrent_counter_writes=32; concurrent_materialized_view_writes=32; 
concurrent_reads=32; concurrent_replicates=null; concurrent_writes=32; 
counter_cache_keys_to_save=2147483647;

Re: Executing a check before replication / manual replication

2017-11-17 Thread Abdelkrim Fitouri

Hi, and thanks for all your replies,

in some ways, i see that my idea is so complicated to implement and is not
very "clean" ...

I will try to use trigger to do that with one account per application with
the good privileges, but in some ways i am trusting all the external
components like OS or other application layers, because like you mentioned
if some one have a root access to one node he can put down all the cluster
integrity.

As i know replication process is called after each write to the DB, i think
it is not a bad idea to plug a public interface at this level, because for
some project and due to some business constraints, a not reachable database
is better than a "corrupted" one ..

Kind regards.



2017-11-17 9:58 GMT+01:00 Jacques-Henri Berthemet <
jacques-henri.berthe...@genesys.com>:

> In the trigger API I mentioned you’ll get the data that is about to get
> inserted, you can decode that data and check that it is compliant to your
> security. If you want to kill the node, just call System.exit() or
> CassandraDaemon.stop(); The thing is that if you have RF=4 with 4 nodes,
> they will all receive the same update and this will kill your whole
> cluster. If instead you throw an exception, you’ll prevent the rogue
> write and your client will get an error.
>
>
>
> As far as I know there is no public interface to plug your code at the
> replication level, and even if there was one, it would only work when you
> have different DCs. One DC would stay with rogue data, and the other one
> would shutdown.
>
>
>
> *--*
>
> *Jacques-Henri Berthemet*
>
>
>
> *From:* Abdelkrim Fitouri [mailto:abdou@gmail.com]
> *Sent:* jeudi 16 novembre 2017 22:33
> *To:* user@cassandra.apache.org
> *Subject:* Re: Executing a check before replication / manual replication
>
>
>
> ok please find bellow an example:
>
> Lets suppose that i have a cassandra cluster of 4 nodes / one DC /
> replication factor = 4, So in this architecture i have on full copy of the
> data on each node.
>
>
>
> Imagine now that one node have been hacked and in some way with full
> access to cqlsh session, if data is changed on that node, data will be
> changed on the three other, am i right ?
>
> imagine now that i am able to know (using cryptographic bases) if one
> column was modified by my API ( => normal way) or not ( => suspicious way),
> and i want to execute this check function just before any replication of a
> keyspace to avoid that all the replica will be affected by that and so a
> rollback will be not easy and the integrity of all the system will be down,
> the check will for example kill the local cassandra service ...
>
> Hope that my question is more clear now.
>
> Many thanks for any help.
>
>
>
>
>
> 2017-11-16 22:01 GMT+01:00 Oliver Ruebenacker :
>
>
>
>  Hello,
>
>   If I understand the OP right, he wants an automated response one node
> displays suspicious activity.
>
>   I suppose in that case, one would want the node to be removed from the
> cluster or shut down or both.
>
>  Best, Oliver
>
>
>
> On Thu, Nov 16, 2017 at 3:40 PM, kurt greaves 
> wrote:
>
> What's the purpose here? If they have access to cqlsh, they have access to
> every nodes data, not just the one they are on. An attacker modifying RF
> would be the least of your worries. If you manage to detect that some node
> is compromise you should isolate it immediately.
>
>
>
>
>
> On 16 Nov. 2017 07:33, "Abdelkrim Fitouri"  wrote:
>
> Hi,
>
>
> I know that cassandra handel properly data replication between cluster
> nodes, but for some security reasons I am wonderning how to avoid data
> replication after a server node have been compromised and someone is
> executing modification via cqlsh ?
>
>
> is there a posibility on Cassandra to execute a custom check / Hook
> before replication ?
>
>
> is there a posibilty to execute a manual replication between node ?
>
>
>
>
> --
>
> Best Regards.
>
> *Abdelkarim FITOURI*
>
> System And Security Engineer
>
>
>
>
>
>
>
> --
>
> Oliver Ruebenacker
>
> Senior Software Engineer, Diabetes Portal
> , Broad Institute
> 
>
>
>
>
>
>
>
>

RE: Executing a check before replication / manual replication

2017-11-17 Thread Jacques-Henri Berthemet

In the trigger API I mentioned you’ll get the data that is about to get 
inserted, you can decode that data and check that it is compliant to your 
security. If you want to kill the node, just call System.exit() or 
CassandraDaemon.stop(); The thing is that if you have RF=4 with 4 nodes, they 
will all receive the same update and this will kill your whole cluster. If 
instead you throw an exception, you’ll prevent the rogue write and your client 
will get an error.

As far as I know there is no public interface to plug your code at the 
replication level, and even if there was one, it would only work when you have 
different DCs. One DC would stay with rogue data, and the other one would 
shutdown.

--
Jacques-Henri Berthemet

From: Abdelkrim Fitouri [mailto:abdou@gmail.com]
Sent: jeudi 16 novembre 2017 22:33
To: user@cassandra.apache.org
Subject: Re: Executing a check before replication / manual replication

ok please find bellow an example:
Lets suppose that i have a cassandra cluster of 4 nodes / one DC / replication 
factor = 4, So in this architecture i have on full copy of the data on each 
node.

Imagine now that one node have been hacked and in some way with full access to 
cqlsh session, if data is changed on that node, data will be changed on the 
three other, am i right ?
imagine now that i am able to know (using cryptographic bases) if one column 
was modified by my API ( => normal way) or not ( => suspicious way), and i want 
to execute this check function just before any replication of a keyspace to 
avoid that all the replica will be affected by that and so a rollback will be 
not easy and the integrity of all the system will be down, the check will for 
example kill the local cassandra service ...
Hope that my question is more clear now.
Many thanks for any help.

2017-11-16 22:01 GMT+01:00 Oliver Ruebenacker 
>:

 Hello,
  If I understand the OP right, he wants an automated response one node 
displays suspicious activity.
  I suppose in that case, one would want the node to be removed from the 
cluster or shut down or both.
 Best, Oliver

On Thu, Nov 16, 2017 at 3:40 PM, kurt greaves 
> wrote:
What's the purpose here? If they have access to cqlsh, they have access to 
every nodes data, not just the one they are on. An attacker modifying RF would 
be the least of your worries. If you manage to detect that some node is 
compromise you should isolate it immediately.

On 16 Nov. 2017 07:33, "Abdelkrim Fitouri" 
> wrote:
Hi,

I know that cassandra handel properly data replication between cluster nodes, 
but for some security reasons I am wonderning how to avoid data replication 
after a server node have been compromised and someone is executing modification 
via cqlsh ?

is there a posibility on Cassandra to execute a custom check / Hook  before 
replication ?

is there a posibilty to execute a manual replication between node ?

--

Best Regards.

Abdelkarim FITOURI

System And Security Engineer

--
Oliver Ruebenacker
Senior Software Engineer, Diabetes 
Portal, Broad 
Institute

Re: How quickly we can bootstrap

Re: Reaper 1.0

RE: Reaper 1.0

How quickly we can bootstrap

Re: Reaper 1.0

Re: Reaper 1.0

Re: Running Cassandra on FreeBSD

Re: Running Cassandra on FreeBSD

Running Cassandra on FreeBSD

Re: Executing a check before replication / manual replication

RE: Executing a check before replication / manual replication

11 matches

Site Navigation

Mail list logo

Footer information