RE: A question to updatesstables

[Lu, Boying]

Thanks a lot.

I’m a little bit of confusing.  If the ‘nodetool updatesstable’ doesn’t work 
without Cassandra server running,
and Cassandra server failed to start due to the incompatible SSTable format,  
how to resolve this dilemma?



From: Carlos Alonso [mailto:i...@mrcalonso.com]
Sent: 2016年8月18日 18:44
To: user@cassandra.apache.org
Subject: Re: A question to updatesstables

Replies inline

Carlos Alonso | Software Engineer | @calonso

On 18 August 2016 at 11:56, Lu, Boying 
> wrote:
Hi, All,

We use Cassandra in our product. I our early release we use Cassandra 1.2.10 
whose SSTable is ‘ic’ format.
We upgrade Cassandra to 2.0.10 in our product release. But the Cassandra server 
failed to start due to the
incompatible SSTable format and the log message told us to use ‘nodetool 
updatesstables’ to upgrade SSTable files.

To make sure that no negative impact on our data, I want to confirm following 
things about this command before trying it:

1.   Does it work without Cassandra server running?
No, it won't.

2.   Will it cause data lost with this command?
It shouldn't if you followed the upgrade instructions properly

3.   What’s the best practice to void this error occurs again (e.g. 
upgrading Cassandra next time)?
Upgrading SSTables is required or not depending on the upgrade you're running, 
basically if the SSTables layout changes you'll need to run it and not 
otherwise so there's nothing you can do to avoid it

Thanks

Boying

Re: How to create a TupleType/TupleValue in a UDF

[Tyler Hobbs]

The logback-related error is due to
https://issues.apache.org/jira/browse/CASSANDRA-11033, which is fixed in
3.0.4 and 3.4.

I'm not sure about the CodecNotFoundException, can you reproduce that one
reliably?

On Thu, Aug 18, 2016 at 10:52 AM, Drew Kutcharian  wrote:

> Hi All,
>
> I have a UDF/UDA that returns a map of date -> TupleValue.
>
> CREATE OR REPLACE FUNCTION min_max_by_timestamps_udf(state map frozen>>, flake blob)
> RETURNS NULL ON NULL INPUT
> RETURNS map>>
> LANGUAGE java
>
> CREATE OR REPLACE AGGREGATE min_max_by_timestamps(blob)
> SFUNC min_max_by_timestamps_udf
> STYPE map>>
> INITCOND {};
>
> I’ve been using the following syntax to build the TupleType/TupleValue in
> my UDF:
>
> TupleType tupleType = TupleType.of(com.datastax.
> driver.core.ProtocolVersion.NEWEST_SUPPORTED, CodecRegistry.DEFAULT_INSTANCE,
> DataType.timestamp(), DataType.timestamp());
> tupleType.newValue(new java.util.Date(timestamp), new
> java.util.Date(timestamp)));
>
> But “randomly" I get errors like the following:
> FunctionFailure: code=1400 [User Defined Function failure]
> message="execution of ’testdb.min_max_by_timestamps_udf[map frozen>>, blob]' failed: 
> java.security.AccessControlException:
> access denied ("java.io.FilePermission" "/etc/cassandra/logback.xml"
> "read”)"
>
> Or CodecNotFoundException for Cassandra not being able to find a codec for
> "map>>”.
>
> Is this a bug or I’m doing something wrong?
>
>
> Thanks,
>
> Drew
>



-- 
Tyler Hobbs
DataStax 

Re: How to create a TupleType/TupleValue in a UDF

[Drew Kutcharian]

I’m running 3.0.8, so it probably wasn’t fixed? ;)
[cqlsh 5.0.1 | Cassandra 3.0.8 | CQL spec 3.4.0 | Native protocol v4]

The CodecNotFoundException is very random, when I get it, if I re-run the same 
exact query then it works! I’ll see if I can reproduce it more consistently. 
BTW, is there a way to get the CodecRegistry and the ProtocolVersion from the 
UDF environment so I don’t have to create them?

- Drew


> On Aug 18, 2016, at 10:10 AM, Tyler Hobbs  wrote:
> 
> The logback-related error is due to 
> https://issues.apache.org/jira/browse/CASSANDRA-11033 
> , which is fixed in 
> 3.0.4 and 3.4.
> 
> I'm not sure about the CodecNotFoundException, can you reproduce that one 
> reliably?
> 
> On Thu, Aug 18, 2016 at 10:52 AM, Drew Kutcharian  > wrote:
> Hi All,
> 
> I have a UDF/UDA that returns a map of date -> TupleValue.
> 
> CREATE OR REPLACE FUNCTION min_max_by_timestamps_udf(state map frozen>>, flake blob)
> RETURNS NULL ON NULL INPUT
> RETURNS map>>
> LANGUAGE java
> 
> CREATE OR REPLACE AGGREGATE min_max_by_timestamps(blob)
> SFUNC min_max_by_timestamps_udf
> STYPE map>>
> INITCOND {};
> 
> I’ve been using the following syntax to build the TupleType/TupleValue in my 
> UDF:
> 
> TupleType tupleType = 
> TupleType.of(com.datastax.driver.core.ProtocolVersion.NEWEST_SUPPORTED, 
> CodecRegistry.DEFAULT_INSTANCE, DataType.timestamp(), DataType.timestamp());
> tupleType.newValue(new java.util.Date(timestamp), new 
> java.util.Date(timestamp)));
> 
> But “randomly" I get errors like the following:
> FunctionFailure: code=1400 [User Defined Function failure] message="execution 
> of ’testdb.min_max_by_timestamps_udf[map timestamp>>>, blob]' failed: java.security.AccessControlException: access 
> denied ("java.io.FilePermission" "/etc/cassandra/logback.xml" "read”)"
> 
> Or CodecNotFoundException for Cassandra not being able to find a codec for 
> "map>>”.
> 
> Is this a bug or I’m doing something wrong?
> 
> 
> Thanks,
> 
> Drew
> 
> 
> 
> -- 
> Tyler Hobbs
> DataStax 

lack of space for replicas

[jean paul]

Hi,


The distributed replicas take space. Is there a scenario when a new
replicas cannot be stored on a node due to lack of space? If we choose a
replication factor =3, and we can find a space for the third replicas, how
cassandra treats this case. Please, where can i find the algorithm ?


Thank you so much for help.
Kind regards.

Re: lack of space for replicas

[Carlos Alonso]

It will then possibly fail with a no disk space error. The replicas
placement algorithm is determined via de Replication Strategy given when
you created your keyspace (SimpleStrategy or NetworkTopologyStrategy).
https://docs.datastax.com/en/cassandra/2.0/cassandra/architecture/architectureDataDistributeReplication_c.html

Carlos Alonso | Software Engineer | @calonso 

On 18 August 2016 at 12:15, jean paul  wrote:

> Hi,
>
>
> The distributed replicas take space. Is there a scenario when a new
> replicas cannot be stored on a node due to lack of space? If we choose a
> replication factor =3, and we can find a space for the third replicas, how
> cassandra treats this case. Please, where can i find the algorithm ?
>
>
> Thank you so much for help.
> Kind regards.
>

Re: A question to updatesstables

[Ryan Svihla]

It hasn't ever prevented me from starting unless there was something else
going on. Can you share the log message that's preventing you from starting.

On Thu, Aug 18, 2016, 5:44 AM Carlos Alonso  wrote:

> Replies inline
>
> Carlos Alonso | Software Engineer | @calonso 
>
> On 18 August 2016 at 11:56, Lu, Boying  wrote:
>
>> Hi, All,
>>
>>
>>
>> We use Cassandra in our product. I our early release we use Cassandra
>> 1.2.10 whose SSTable is ‘ic’ format.
>>
>> We upgrade Cassandra to 2.0.10 in our product release. But the Cassandra
>> server failed to start due to the
>>
>> incompatible SSTable format and the log message told us to use ‘nodetool
>> updatesstables’ to upgrade SSTable files.
>>
>>
>>
>> To make sure that no negative impact on our data, I want to confirm
>> following things about this command before trying it:
>>
>> 1.   Does it work without Cassandra server running?
>>
> No, it won't.
>
>> 2.   Will it cause data lost with this command?
>>
> It shouldn't if you followed the upgrade instructions properly
>
>> 3.   What’s the best practice to void this error occurs again (e.g.
>> upgrading Cassandra next time)?
>>
> Upgrading SSTables is required or not depending on the upgrade you're
> running, basically if the SSTables layout changes you'll need to run it and
> not otherwise so there's nothing you can do to avoid it
>
>>
>>
>> Thanks
>>
>>
>>
>> Boying
>>
>
> --
Regards,

Ryan Svihla

A question to updatesstables

[Lu, Boying]

Hi, All,

We use Cassandra in our product. I our early release we use Cassandra 1.2.10 
whose SSTable is 'ic' format.
We upgrade Cassandra to 2.0.10 in our product release. But the Cassandra server 
failed to start due to the
incompatible SSTable format and the log message told us to use 'nodetool 
updatesstables' to upgrade SSTable files.

To make sure that no negative impact on our data, I want to confirm following 
things about this command before trying it:

1.   Does it work without Cassandra server running?

2.   Will it cause data lost with this command?

3.   What's the best practice to void this error occurs again (e.g. 
upgrading Cassandra next time)?

Thanks

Boying

Re: A question to updatesstables

[Carlos Alonso]

Replies inline

Carlos Alonso | Software Engineer | @calonso 

On 18 August 2016 at 11:56, Lu, Boying  wrote:

> Hi, All,
>
>
>
> We use Cassandra in our product. I our early release we use Cassandra
> 1.2.10 whose SSTable is ‘ic’ format.
>
> We upgrade Cassandra to 2.0.10 in our product release. But the Cassandra
> server failed to start due to the
>
> incompatible SSTable format and the log message told us to use ‘nodetool
> updatesstables’ to upgrade SSTable files.
>
>
>
> To make sure that no negative impact on our data, I want to confirm
> following things about this command before trying it:
>
> 1.   Does it work without Cassandra server running?
>
No, it won't.

> 2.   Will it cause data lost with this command?
>
It shouldn't if you followed the upgrade instructions properly

> 3.   What’s the best practice to void this error occurs again (e.g.
> upgrading Cassandra next time)?
>
Upgrading SSTables is required or not depending on the upgrade you're
running, basically if the SSTables layout changes you'll need to run it and
not otherwise so there's nothing you can do to avoid it

>
>
> Thanks
>
>
>
> Boying
>

How to create a TupleType/TupleValue in a UDF

[Drew Kutcharian]

Hi All,

I have a UDF/UDA that returns a map of date -> TupleValue.

CREATE OR REPLACE FUNCTION min_max_by_timestamps_udf(state map>>, flake blob)
RETURNS NULL ON NULL INPUT
RETURNS map>>
LANGUAGE java

CREATE OR REPLACE AGGREGATE min_max_by_timestamps(blob)
SFUNC min_max_by_timestamps_udf
STYPE map>>
INITCOND {};

I’ve been using the following syntax to build the TupleType/TupleValue in my 
UDF:

TupleType tupleType = 
TupleType.of(com.datastax.driver.core.ProtocolVersion.NEWEST_SUPPORTED, 
CodecRegistry.DEFAULT_INSTANCE, DataType.timestamp(), DataType.timestamp());
tupleType.newValue(new java.util.Date(timestamp), new 
java.util.Date(timestamp)));

But “randomly" I get errors like the following:
FunctionFailure: code=1400 [User Defined Function failure] message="execution 
of ’testdb.min_max_by_timestamps_udf[map>>, blob]' failed: java.security.AccessControlException: access 
denied ("java.io.FilePermission" "/etc/cassandra/logback.xml" "read”)"

Or CodecNotFoundException for Cassandra not being able to find a codec for 
"map>>”.

Is this a bug or I’m doing something wrong?


Thanks,

Drew